#6647 [Engine] 0.1.65 outputs random multilingual tokens on NVFP4 Qwen3.8-Flash-Next with the same config that works on 0.1.64
open · @enkynakamura · 3 commentaires · Sur GitHub
Server & APINVIDIA / CUDAModels & quantsWindows
Description
## Problem
With engine 0.1.65, the NVFP4 packs of Qwen3.8-Flash-Next generate garbage:
random tokens in many languages, `content` empty, the text ends up in
`reasoning_content`. Text-only and image requests are both affected.
With the same pack and the identical launch command, engine 0.1.64 works.
## Environment
- Windows, RTX 5080 16 GB, Xeon w7-2475X, 128 GB RAM
- NVIDIA driver 610.88, CUDA UMD 13.3 (nvidia-smi)
- Qwen engine 0.1.65 (build 26350e4387b8+cuda13.0) and 0.1.64 (build f609f862fa9e+cuda13.0), both cuda13
- Packs: qwen3.8-flash-next-nvfp4-nvfp4 (NVFP4) and qwen3.8-flash-next-nvfp4-nvfp4-nvidia (NVFP4-NVIDIA)
- Engine switched with `strata components use 0.1.64 --family qwen`
(0.1.64 exists for two families, so the family flag is required)
## Reproduce
1. strata components use 0.1.65 --family qwen
2. strata launch --family nvfp4 --model NVFP4-NVIDIA --context 512k --drafting mtp
--set STRATA_PREFILL_NVFP4=w4a4 --set STRATA_ROUTE_TAIL_SKIP=6
--set rope-scaling=yarn --set rope-scale=2
3. POST http://127.0.0.1:8080/v1/chat/completions
{"model":"qwen3.8-flash-next-nvfp4-nvfp4-nvidia","max_tokens":300,"temperature":1.0,
"messages":[{"role":"user","content":"Scrivi due frasi su una villa con piscina."}]}
4. Repeat with `strata components use 0.1.64 --family qwen` and the same launch.
## Expected
Coherent Italian text, as on 0.1.64.
## Actual
0.1.65: finish_reason "length", prompt_tokens 63, completion_tokens 300, content "",
reasoning_content = random multilingual tokens.
0.1.64: finish_reason "stop", 108 tokens, two coherent Italian sentences.
## Evidence
- Same pack (NVFP4-NVIDIA), identical launch, /v1/models showed context_length 524288 in both runs:
0.1.65 garbage, 0.1.64 coherent. Attached: nvfp4-nvidia-0.1.65.json, nvfp4-nvidia-0.1.64.json
- NVFP4 pack on 0.1.65, same daily config: text-only garbage 4 times (one attached as
garbage-0.1.65.json). One image request (image_url, 269 prompt tokens) also garbage:
{"content": "", "reasoning_content": "aliesenz-signedBy.Eulerrakeableputbdemet doingillersongEncoded few FoobelGradientianoerzia gegrAYZY/DDvoudagrid fokusidaçãoPRSusa走弱Ir查sarukanGilla Debekl到记分tuics", "finish_reason": "stop"}
usage: prompt_tokens 269, completion_tokens 45
- NVFP4 pack on 0.1.64, plain launch (32K, no YaRN, tail-skip 0): text and image OK.
- On 0.1.65, other quants are fine: Q8_0 (256K, --drafting mtp, tail-skip 0) text and image OK;
Q2_0 image description OK.
- NVFP4-NVIDIA was freshly downloaded, so a corrupted pack is unlikely.
- Server log: not attached.
## Reproduction status
Reproduced on 0.1.65 with two NVFP4 packs. Not reproduced on 0.1.64 with the identical command,
so the engine version is the only change between the broken and the working run.
Not tested: which 0.1.65 change is responsible, or which option (w4a4 prefill, MTP, YaRN,
tail-skip) is needed to trigger it.
[garbage-0.1.65.json](https://github.com/user-attachments/files/35239879/garbage-0.1.65.json)
[nvfp4-nvidia-0.1.65.json](https://github.com/user-attachments/files/35243732/nvfp4-nvidia-0.1.65.json)
[nvfp4-nvidia-0.1.64.json](https://github.com/user-attachments/files/35243731/nvfp4-nvidia-0.1.64.json)Sur le site
Liens install, modèles, releases.