Repository navigation
Conversation
…en both branches cover the result When both branches are defined on the whole result range along the condition's dimension, the result is a masked select of the two, which avoids slicing and concatenating copies. Other cases keep the concatenation.
Contributor
Author
|
🤖 Written by an agent on behalf of @havogt |
havogt
commented
Nov 20, 2026
| return _concat(*f_slices, *t_slices, dim=domain_dim) | ||
|
|
||
|
|
||
| def _where_on_range( |
Contributor
Author
There was a problem hiding this comment.
let's make it a torch-only override as the performance results suggest..
Eagerly the select reads both branches over the whole result and is slower than the concatenation; under jax.jit it was a loss on the largest measured grid. The choice is the ArrayOps hook concat_where_selects, which only TorchOps answers for traced tensors.
Contributor
Author
|
Measured on santis (one GH200) with icon4py's
Caveat: both rows were measured with an earlier revision of this PR that selected under every backend, on a torch.compile development branch that already contained #2989 and #2895, not on 🤖 Written by an agent on behalf of @havogt |
…erride of TorchOps Replaces the ArrayOps.concat_where_selects switch: the generic concat_where is main's again, with its slicing split out for the torch override.
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Under
torch.compile, embeddedconcat_wherealong one dimension now evaluates as an elementwise select over the result range whenever both (broadcast) branches cover that range, instead of slicing the branches and concatenating the pieces. Inductor fuses the select into its consumers, while the concatenation materializes the result. Eager execution and JAX keep the concatenation: executed eagerly the select reads both branches over the whole result and is slower, and underjax.jitit lost on the largest grid measured. The select is theconcat_whereentry ofTorchOps.builtin_overrides; outside a trace and where a branch does not cover the result it uses the generic implementation.