Skip to content

feat[next]: embedded jit backends for jax.jit and torch.compile - #3027

Draft
havogt wants to merge 144 commits into
GridTools:mainfrom
havogt:jax-jit
Draft

havogt wants to merge 144 commits into
GridTools:mainfrom
havogt:jax-jit

Conversation

@havogt

@havogt havogt commented Nov 20, 2026 •

Copy link
Copy Markdown
Contributor

Description

Two embedded backends that run gt4py programs and field operators, unchanged, inside the tracing compiler of their array library (ADR 0030):

  • gt4py.next.program_processors.runners.jax_jit.jax_jit: under jax.jit.
  • gt4py.next.program_processors.runners.torch_compile.torch_compile: under torch.compile(fullgraph=True, dynamic=False), through torch.compiler.nonstrict_trace.
prog.with_backend(jax_jit.jax_jit)(*fields, offset_provider=...)  # fields allocated with allocator=jnp
fo.with_backend(torch_compile.torch_compile)(*fields, out=out, domain=..., offset_provider=...)

Fields and connectivities are arguments of the compiled function, so tables are runtime arguments (ADR 0029), and domain inference narrows as it does eagerly. Retracing is per field domain, per table buffer and per value of a static parameter.

Design

  • EmbeddedJitBackend (gt4py.next.backend): a Backend that does not compile. Its jit(entry, signature, static_params) wraps the embedded entry point of a program or field operator once.
  • Dispatch: an entry point with such a backend takes the embedded path and calls its cached jitted entry. Whether an entry point is compiled is an attribute set in __post_init__, so a compiled backend's call costs the same as before (measured; a cached property was slower). .compile() raises.
  • Static vs traced:
    • All scalars are traced, on both backends.
    • The parameters in static_params, the field operator's domain=, and the Dimensions of the offset provider are static.
    • Named collections reach the backend as tuples and are rebuilt inside, as on the compiled path. No user type is registered as a pytree.
  • out:
    • jax: the jitted function returns the new buffers of the fields it assigned to, and the caller's fields get them. A field passed more than once is a single field in the trace.
    • torch: assigns in place.
  • Embedded changes:
    • Embedded accepts 0-d arrays wherever it accepts a scalar (array_ops.is_scalar_or_0d), and the scalar-only builtins dispatch them to their namespace.
    • ArrayOps.scalar keeps a traced element as a 0-d array, which scans need.
    • embedded/__init__.py no longer imports nd_array_field/array_libraries, which removes the fbuiltins ↔ embedded import cycle and its function-local imports.
  • torch specifics (all forced by Dynamo):
    • nonstrict_trace takes neither NumPy scalars nor frozen dataclasses. So scalars become 0-d tensors, and the static arguments are closed over, with one region per combination of their values.
    • Each region gets its own code object, since the recompile limit is per code object.
    • Errors of the embedded code are re-raised as themselves.

Tests

  • unit_tests/program_processor_tests/runners_tests/test_embedded_jit.py, on both backends, CPU and CUDA:
    • writes into the caller's out, inputs left untouched;
    • static params of programs and field operators;
    • domain=;
    • a dataclass that appears only as out;
    • aliased fields;
    • error propagation;
    • no retrace for new scalar values, with torch set to fail on any recompile;
    • jax: retrace per table buffer, and the table as a runtime argument.
  • Integration matrix: new entries jax_jit, torch_compile and torch_compile_cuda.
    • Tests that need a concrete scalar are marked uses_scalar_in_control_flow, uses_scalar_in_domain or uses_scan_carry_in_control_flow, and xfailed for both backends.
    • cases.runs_embedded(case) replaces case.backend is None where a test only makes sense compiled.
  • Local results (CPU and an NVIDIA GPU, jax 0.11 CPU-only, torch 2.15 CUDA):
    • -k jax_jit: 271 passed, 131 xfailed
    • -k "torch_compile and not torch_compile_cuda": 283 passed, 119 xfailed
    • -k torch_compile_cuda: 283 passed, 119 xfailed
    • the other embedded backends: 1350 passed
    • the new unit tests: 31 passed
    • roundtrip and gtfn samples of the compiled path: passed
    • 0 failed in every run

Open

  • jax on GPU not run (CPU-only jaxlib in the test environment).
  • An embedded scan loops per column and level, so its trace grows with the domain.

Requirements

  • All fixes and/or new features come with corresponding tests.
  • Important design decisions have been documented in the appropriate ADR inside the docs/development/ADRs/ folder.

A skip entry wrapped around to a real element of the buffer, which is
masked later but still receives a cotangent under jax.grad.

Claude-Session: https://claude.ai/code/session_013m5PzT4FxbaKPxfXqiQLr5
A reduction after a narrowing premap broadcast the full table against the
narrowed field. The mask now uses the connectivity's own skip value.

Claude-Session: https://claude.ai/code/session_013m5PzT4FxbaKPxfXqiQLr5
The neighbor table travels as the traced child, so it is a runtime
argument of the compiled program rather than an inlined constant. The
aux data carries a handle to the same buffer, compared by identity, through
which `inverse_image` reads the concrete table at trace time, so domain
inference narrows under `jax.jit` exactly as it does eagerly.

`inverse_image` also answers covering image ranges from eagerly computed
table bounds instead of scanning the table.

Claude-Session: https://claude.ai/code/session_013m5PzT4FxbaKPxfXqiQLr5
… bounds on write

Also reuse the gather's index-array layout for the reduction mask, which
handles transposed table dimensions, sanitise skip indices before the
broadcast, and treat an empty table as an empty image.
…ax trace

Restricting a concrete jax connectivity inside jax.jit (e.g. for
field(E2C2V[0])) produced a traced buffer, so the later domain inference
on it failed with a ConcretizationTypeError. Slice concrete tables with
jax.ensure_compile_time_eval.
A connectivity whose table is computed inside a jax trace (e.g. from
as_offset) has no concrete values for domain inference. Assume all its
targets lie in the field and clip the gather indices, so out-of-range
targets read the boundary value. Eager execution is unchanged.
jax.jit does not need it: domain inference already reads concrete tables
through `_hyperslice`. On tables with skip values computing the bounds costs
more than a covering scan, and under jit they are recomputed per trace.
…ivity base class

The jax connectivity overrides `inverse_image` instead of an `_image_slices`
indirection, and adjusts gather indices through a `_gather_index` hook, so
shared code no longer knows about values computed in a trace. Test the clamp
on negative targets: positive ones are already clamped by jax indexing.
It is an implementation detail of premap: nothing outside the embedded
implementations calls it.
All callers pass a UnitRange, so the NamedRange branch and its dimension check
were unreachable.
- Drop the cache in FieldOffset.as_connectivity_field: it returned its input,
  and kept traced connectivities alive after the trace.
- Align the table for skip masks without broadcasting, so reductions work on
  fields with an infinite range again.
- Keep as_offset of concrete offsets concrete under a trace, so jit narrows
  like eager. Allow subclasses to override a registered builtin for this.
- Cover the f89 fix with a narrowing case, custom skip values, non-zero domain
  starts and nested jit, and fix the check that a table is not inlined.
…ssert in the inverse-image test

Also state in the ADR that writing into a connectivity buffer through
`.ndarray` is not supported.
…via ArrayOps

NdArrayField and NdArrayConnectivityField no longer carry the array
namespace as a class variable. Each field takes its array operations from
an 'ArrayOps' object resolved from the type of its buffer: the namespace
array_api_compat returns for the buffer selects a registered ArrayOps
subclass, or the default one, which is written against the Array API
standard only.

The per-library field classes (NumPyArrayField, CuPyArrayField,
JaxArrayField and their connectivity counterparts) are removed; NumPy,
CuPy and JAX differ only in NumPyOps, CuPyOps and JaxOps (gamma, scalar
operands, item assignment for JAX, conversion to NumPy). 'common._field'
and 'common._connectivity' construct an NdArrayField from any Array API
object instead of a per-type registration.

'NdArrayField.array_ns' stays the library's own module (numpy, cupy,
jax.numpy). An ArrayOps can replace a whole fbuiltins implementation via
'builtin_overrides'.
array-api-strict is added to the namespace matrix of the embedded unit
tests, which shows that NdArrayField uses only the Array API standard.
Test inputs are built with each namespace's own dtype objects. The
non-standard builtins fmod, cbrt and gamma raise NotImplementedError on
strict.

The constructors read the dtype of the given array through ArrayOps and
test for skip values with the namespace's 'any' instead of the NumPy
method.
torch tensors work through array_api_compat with one ArrayOps subclass
and no field class: scalar operands of elementwise functions become 0-d
tensors (Python floats as float64, as NumPy promotes them), cbrt and
gamma have torch implementations, assigned values become tensors and
conversion to NumPy detaches and copies to the host. The embedded unit
tests run on torch when it is installed.
…PI specifies

- slicing a size-1 broadcast axis of a longer range reads its only
  element instead of relying on NumPy clamping out-of-bounds slices;
- a relative index shorter than the buffer gets a trailing ellipsis;
- assigned arrays are converted to the target's namespace, device and
  dtype by the default ArrayOps (NumPy and CuPy assign directly);
- the identity index array takes its length from the domain range.
…'s creators, kinds instead of None and a sentinel
…on-2976

# Conflicts:
#	tests/next_tests/unit_tests/embedded_tests/test_nd_array_field.py
# Conflicts:
#	src/gt4py/next/embedded/nd_array_field.py
@havogt

havogt commented Nov 20, 2026

Copy link
Copy Markdown
Contributor Author

🤖 Written by an agent on behalf of @havogt

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant