[yijingl-nvidia]

Description

This PR adds get_env_flag() to tensorrt_llm/_utils.py and uses it for most on/off environment switches that only TensorRT-LLM's Python code reads.

Before this PR, each module parsed its switches by hand. Most turned on only for the exact string 1, so X=true meant off. Some used != "0", so X=false meant on. A typo silently picked one of the two paths. Some switches were read again for every request or step, so a typo surfaced only after the server was up.

get_env_flag(name, default) reads a switch the same way wherever it is used:

• Unset or blank returns default.
• 1/true/yes/on return True, and 0/false/no/off return False, in any case.
• Any other value raises a ValueError that names the variable.

The PR reads 183 switches in 122 files with get_env_flag. Every changed module already imports tensorrt_llm._utils when it loads, so this adds no import time. A switch is converted only when all of these hold:

• No C++ code, shell script or other library also reads it, so all readers accept the same values.
• It is not in vendored code.
• It is not read inside code that falls back to another path on any error, which would hide the ValueError.

Switches that used to be read per request, per executor iteration, per refit or per forward step are now read once at startup, so a bad value stops the launch:

• At import: TRTLLM_KIMI_K3_STRICT_TOOL_GRAMMAR, TRTLLM_ALLOW_UNUSED_CHAT_TEMPLATE_KWARGS, TRTLLM_ENABLE_PYAV, TLLM_VISUALGEN_CPU_RNG, TLLM_VISUAL_GEN_LOG_CUDA_MEMORY, TRTLLM_PP_REQ_SEND_ASYNC, TLLM_REFIT_DEDUP_MODULES, TLLM_REFIT_SKIP_WRAPPER_CHECK, the DeepSeek-V4 per-step switches and TLLM_DISABLE_ALLREDUCE_AUTOTUNE.
• When the serving extensions load at server start: TRTLLM_KIMI_PARAM_POLICY.
• When OpenAIServer is constructed: TRTLLM_DISALLOW_LOCAL_MEDIA_PATH, which it passes to parse_visual_gen_params, so a value an embedding application sets after import still applies.
• In PyExecutor._initialize_execution: TLLM_METRICS_ALL_RANKS.

DISABLE_HARMONY_ADAPTER is converted in all three of its readers, so the workers, the disaggregated router and the standalone renderer agree. The router reads it per request, but the workers read it at startup, so a bad value stops the launch.

What changes for someone who sets a converted switch:

• true, yes and on now turn it on. For TRTLLM_DISALLOW_LOCAL_MEDIA_PATH, true used to log a warning and keep server-side paths allowed; it now refuses them.
• false, no and off now turn it off. These switches used to treat false as on: TRTLLM_<MODEL>_EAGER_FUSION_DISABLED for DeepSeek, GLM, Llama, MiniMax-M3 and Qwen3 (false disabled the fusion), TRTLLM_FP4_MLA_FOLD_Q, TLLM_CUDA_OBJECT_COLLECTIVES, TRTLLM_LATE_COMPILE_DETECT, TRTLLM_USE_MAMBA_REPLAY, TRTLLM_RECLAIM_WORKSPACE, TRTLLM_DEEP_GEMM_CACHE_PER_PROCESS, TLLM_LLMAPI_ZMQ_DEBUG and TLLM_LLMAPI_ZMQ_PAIR. The TRTLLM_V41_* replay switches, TRTLLM_DWDP_PREFETCH_GATHER and TRTLLM_LTX2_AUDIO_CONDITIONAL_SHARD already treated false as off, but not no or off.
• A blank value now means the default.
• Any other value now raises when the process starts. No test, example or script in the repo sets a converted switch to another value.
• Switches read at import do not see env_overrides, which workers apply after import (its documentation notes this). TRTLLM_ENABLE_PDL, which 128 shipped configs set through env_overrides, is still read on each call.

Reads that keep their current parsing

• Also read by C++ code or shell scripts: TLLM_DISABLE_MPI, TLLM_FAULT_TOLERANCE_MODE, TLLM_USE_FINE_GRAINED_SYNC, TRTLLM_DISABLE_KV_CACHE_TRANSFER_OVERLAP, TRTLLM_DISAGG_BENCHMARK_GEN_ONLY, TRTLLM_DISAGG_LAYERWISE, TRTLLM_PRINT_SKIP_SOFTMAX_STAT, TRTLLM_USE_{NIXL,UCX,MOONCAKE,MPI}_KVCACHE and FORCE_DETERMINISTIC.
• Another library's variables or vendored code: FLASHINFER_CUTE_DSL_DISABLE_CACHE, the FLA_* variables in modules/fla/, and MEGA_DIAG_SKIP_TOKEN_BACK in cutedsl_megamoe/kernel_src.
• Read inside code that falls back on any error: TRTLLM_MOE_POST_QUANT_ALLTOALLV, TRTLLM_DEEP_EP_DISABLE_P2P_FOR_LOW_LATENCY_MODE, TRTLLM_FORCE_MNNVL_AR and TLLM_AUTOTUNER_DISABLE_SHORT_PROFILE.
• Tests require exactly 1 to turn them on: FORCE_MOE_KERNEL_DETERMINISTIC, FORCE_ALL_REDUCE_DETERMINISTIC, TLLM_CUDA_GRAPH_STRICT_BUFFERS, TLLM_EVAL_SPEC_STATS, TRTLLM_DISABLE_FUSED_DSA_METADATA, TRTLLM_GPU_KEEPALIVE, TRTLLM_MEDIA_ALLOW_PRIVATE_URLS and TRTLLM_USE_GDN_REPLAY.
• Their own documented parsing: the GVR top-k switches (any value other than 0 turns them on) and TRTLLM_ENABLE_KV_TRANSFER_LIFECYCLE_ENFORCEMENT (accepts only 0 or 1).
• Modules that do not import tensorrt_llm._utils (_bootstrap.py, usage/usage_lib.py), and four reads inside CuTe DSL kernel functions.

Other changes:

• get_env_enable_pdl() moves to _utils.py (still importable from flashinfer_utils) and parses TRTLLM_ENABLE_PDL like the C++ getEnvEnablePDL(): on unless the variable is exactly 0. All 35 Python readers call it, so Python and native kernels agree. Before, most Python readers turned PDL off for any value other than 1 (such as true), and the GLM-5.2 TEP4 MoE kernels turned it off for false, no and off.
• The eight copies of the MNNVL fabric-handle check in the GLM-5.2 and Kimi K3 kernels call use_mnnvl_fabric_handle().
• Duplicate reads call their shared helpers: a new nvtx_debug_enabled() for the two NVTX debug switches, _is_mm_disagg() and use_py_mamba_cache_manager().
• _bootstrap.py re-raises a ValueError raised while the custom ops import, so a bad switch shows its own message instead of a request to rebuild TensorRT-LLM.
• CODING_GUIDELINES.md and AGENTS.md ask new on/off switches to use get_env_flag, and the 1.4 release notes describe the new parsing.
• llm_args.py changes only one env read, not the LLM API, so this PR has the api-compatible label.

The Kimi K3 merge-back PRs that follow read their switches with get_env_flag. The first commit is @hvagadia's from feat/k3_26.08.

For reviewers of the first round: the commits after the first conversion commit narrow the conversion and address the comments, https://github.com/NVIDIA/TensorRT-LLM/pull/21212/files/d925f23bbc78b631c2d30d05d214e2d73df6bb65..070b06268271ec40c5f4f128b0e2f1ff785093f2

Test Coverage

1 test file added, 14 test files changed:

• tests/unittest/utils/test_get_env_flag.py: added, CPU only, listed in l0_cpu.yml.
• tests/unittest/others/test_nemo_skills_eval.py: new test that the NeMo-Skills sandbox refuses a non-loopback host unless NEMO_SKILLS_SANDBOX_ALLOW_REMOTE is on.
• tests/unittest/disaggregated/test_router.py: the router's DISABLE_HARMONY_ADAPTER test also runs with true.
• tests/unittest/_torch/modeling_v2/test_modeling_v2_claims.py: get_env_flag joins the imports that compute nothing, so modeling-v2 targets can read switches with it.
• tests/unittest/_torch/moe/test_megamoe_shared_fc12_autotune.py: the test that runs AutoTuner._prime_cached_tactics in its own namespace now gives it get_env_flag.
• Tests that changed a switch now read at startup set its module-level value instead of the environment: tests/unittest/llmapi/apps/test_kimi_serve_extensions.py, test_chat_template_kwargs_guard.py, test_render_callers.py, test_render_core.py, test_serving_extensions.py, tests/unittest/llmapi/test_kv_hints.py, tests/unittest/grpc/openengine/test_multimodal.py, tests/unittest/_torch/pyexecutor/test_model_loader_compile_guard.py, tests/unittest/_torch/visual_gen/test_visual_gen_utils.py and test_trtllm_serve_endpoints.py. The visual-gen test that expected TRTLLM_DISALLOW_LOCAL_MEDIA_PATH=true to warn and stay allowed is removed.

The converted modules run in the existing pre-merge tests.

PR Checklist

Please review the following before submitting your PR:

• PR description clearly explains what and why. If using CodeRabbit's summary, please make sure it makes sense.

• PR Follows TRT-LLM CODING GUIDELINES to the best of your knowledge.

• Test cases are provided for new code paths (see test instructions)

• If PR introduces API changes, an appropriate PR label is added - either api-compatible or api-breaking. For api-breaking, include BREAKING in the PR title.

• Any new dependencies have been scanned for license and vulnerabilities

• CODEOWNERS updated if ownership changes

• Documentation updated as needed

• Update tava architecture diagram if there is a significant design change in PR.

• The reviewers assigned automatically/manually are appropriate for the PR.

• Please check this after reviewing the above items as appropriate for this PR.

GitHub Bot Help

To see a list of available CI bot commands, please comment /bot help.

Comprehensible Explanation

This PR adds get_env_flag(name, default) to tensorrt_llm/_utils.py. It is one strict parser for the on/off environment variables ("switches") that TensorRT-LLM's Python code reads. 1, true, yes and on turn a switch on, and 0, false, no and off turn it off. Unset or blank gives the default. Any other value raises a ValueError that names the variable. 183 existing switches now use it.

Before this PR, each module parsed its own switches. Most checked == "1", so true meant off. A few checked != "0", so false meant on. A typo such as ture silently picked one of the two paths, so a benchmark or a deployment could run with a feature off while its owner thought it was on. The Kimi K3 PRs that port code from feat/k3_26.08 to main add many new switches, so this PR fixes the parsing first and those PRs read their switches with get_env_flag.

A strict parser should fail at launch, before the server takes traffic. Some switches were read per request, per executor iteration, per refit or per forward step, where a typo would fail every request. This PR reads those switches once, at import or when the server or executor starts, so a bad value stops the launch.

Reading at import has one limit. env_overrides is an LLM API field that sets environment variables in each worker after the worker has imported TensorRT-LLM, so an import-time read misses it. 128 shipped configs set TRTLLM_ENABLE_PDL through env_overrides, so get_env_enable_pdl() still reads it on each call. It now parses the value like the C++ getEnvEnablePDL() (on unless the value is exactly 0), so kernels launched from Python and from C++ make the same PDL choice.

Some switches keep their old parsing:

• Switches that C++ code, shell scripts or another library also read, because all readers of one switch must accept the same values.
• Switches read inside code that falls back to another path on any error, because the fallback would hide the ValueError.

The Description section lists them. New on/off switches should use get_env_flag, and CODING_GUIDELINES.md now says so.

🤖 Generated with Claude Code

[yijingl-nvidia]

/bot run

[tensorrt-cicd]

PR_Github #81726 [ run ] triggered by Bot. Commit: 23046d3 Link to invocation

[tensorrt-cicd]

PR_Github #81726 [ run ] completed with state FAILURE. Commit: 23046d3
/LLM/main/L0_MergeRequest_PR pipeline #67386 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

• Please check the failed tests and fix your PR
• If you cannot view the failures, ask the CI triggerer to share details
• Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

[tensorrt-cicd]

PR_Github #82881 [ run ] triggered by Bot. Commit: f2c706d Link to invocation

[tensorrt-cicd]

PR_Github/21212-edcdd68 #82879 was force-killed by a newer pipeline run.
LLM/main/L0_MergeRequest_PR #68444 (Blue Ocean) completed with status: ABORTED

Link to superseding invocation

[sunnyqgg]

The Blockers from my review of 2026-11-05 are resolved or answered at f2c706d:

• test_megamoe_shared_fc12_autotune.py:476 now passes get_env_flag into the exec namespace for _prime_cached_tactics (5ad8f26; import moved to :22 in dfb8714).
• Switches with C++ or script readers (FT mode, DISABLE_MPI, USE_*_KVCACHE, GEN_ONLY, LAYERWISE, transfer overlap, skip-softmax stat) keep their exact-"1" reads (3011446), and no switch read with get_env_flag at H has a native or script reader.
• serve/render/resources.py:31 now reads DISABLE_HARMONY_ADAPTER with get_env_flag like the server and router, and the router test covers true (5158365).
• TRTLLM_GVR_TIERS_DISABLE is back on its documented parse (anything but unset/""/"0" engages it) at gvr_topk_decode_dispatch.py:102 (3011446).
• prims_ts_precompile.py:110 keeps FlashInfer's exact-"1" check for FLASHINFER_CUTE_DSL_DISABLE_CACHE (3011446).
• All Python TRTLLM_ENABLE_PDL reads now go through _utils.get_env_enable_pdl() (on unless exactly 0), matching getEnvEnablePDL and the GLM-5.2 TP8 csrc ops (edcdd68).

No CI result on f2c706d yet: PR_Github #82881 had not reported, and the runs on the intermediate commits were aborted or force-killed. The autotuner test fix and the CPU and multi-GPU stages are verified only by reading the code.

No one else reviewed edcdd68 (PDL and MNNVL helpers, 54 files) or f2c706d (local media-path plumbing); they came after the last CodeRabbit pass, and I checked only the parts tied to the Blockers.

[yijingl-nvidia]

/bot run --disable-fail-fast

[tensorrt-cicd]

PR_Github #82908 [ run ] triggered by Bot. Commit: 070b062 Link to invocation

[yijingl-nvidia]

All review comments so far are addressed: each of the 56 review threads has a reply and is resolved. The current head, 070b062, is rebased on main, and CI is running on it.

[tensorrt-cicd]

PR_Github #82881 [ run ] completed with state ABORTED. Commit: f2c706d

Link to invocation

[tensorrt-cicd]

PR_Github #82908 [ run ] completed with state SUCCESS. Commit: 070b062
/LLM/main/L0_MergeRequest_PR pipeline #68471 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

• Please check the failed tests and fix your PR
• If you cannot view the failures, ask the CI triggerer to share details
• Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

[yuxianq]

Attention part LGTM

[yijingl-nvidia]

/bot run --disable-fail-fast

[tensorrt-cicd]

PR_Github #82997 [ run ] triggered by Bot. Commit: d227dd2 Link to invocation

[erictsai-nv]

modeling_v2 changes LGTM; they are mechanical and the unset path is unchanged. Nit: qwen4_exp still parses TRTLLM_MODELING_V2_PLE_HOST_OFFLOAD and TRTLLM_MODELING_V2_MOE_PREFILL_COPY by hand.

[tensorrt-cicd]

PR_Github #82997 [ run ] completed with state SUCCESS. Commit: d227dd2
/LLM/main/L0_MergeRequest_PR pipeline #68552 completed with status: 'UNSTABLE'

CI Report

⚠️ Multi-GPU Label Required:
Multi-GPU tests require the ci: full pre-merge approved label on this PR. Either:

• Wait for the PR to be fully approved — the label is added automatically once approval is complete. Having unresolved open comments is fine, or
• If needed, ask a member of NVIDIA/trt-llm-ci-approvers to add the label manually.
  Then re-trigger CI with the same bot command (no rebase needed).

⚠️ Action Required:

• Please check the failed tests and fix your PR
• If you cannot view the failures, ask the CI triggerer to share details
• Once fixed, request an NVIDIA team member to trigger CI again

Link to invocation