Repository navigation
ggml kernels: GQ_FUSED_ROWS in every build, the CUDA chain's host side checked without AVX2 - #2585
Merged
Merged
Conversation
…e checked without AVX2 qwen38's CUDA chain groups a verify's pooled slots of one expert by GQ_FUSED_ROWS, defined only with the AVX2 kernels: qwen38 with CUDA=1 did not compile on an ARM host with a card. The paths without those kernels give a row its own bits too, so the grouping holds there. make cuda-host-syntax compiles the CUDA engines' host side; CI runs it at ARCH=x86-64.
JustVugg
force-pushed
the
fix/gq-fused-rows-portable
branch
from
November 4, 2026 13:18
dc960e5 to
acff318
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
qwen38's CUDA chain groups a verify's pooled slots of one expert by
GQ_FUSED_ROWS(#2559's "a verify's pooled slots of one expert together").ggml_quants.hdefined it only inside the AVX2 kernels' block, somake qwen38 CUDA=1stopped with'GQ_FUSED_ROWS' undeclaredon a host without them: an ARM host with a card (DGX Spark, GH200, Jetson), or an x86 one built for a baseline arch. CI never built that combination.Change
GQ_FUSED_ROWSis defined before the AVX2 block, in every build. The grouping holds there: without the fused kernels a row goes throughgq_dot_i8(or the Q8_0 dot) on its own, so its result does not depend on how many rows are grouped.make cuda-host-syntax:-fsyntax-only -DCOLI_CUDAof the engines with a CUDA chain (qwen38, qwen36, olmoe, qwen36_tier, backend_loader), no toolkit needed. The Linux CPU job runs it atARCH=x86-64, so a name only the AVX2 kernels define shows on an x86 runner.Checked
make cuda-host-syntax ARCH=x86-64fails (two errors in qwen38_cuda_chain.h); this branch passes.-march=x86-64,x86-64-v2andnativewith-DCOLI_CUDA: no error or warning.