Issues / #974

#974 Missing --kv-resident in generated args when --resident-budget-gib is set (tools/test_setup_unsloth.py)

closed · @AmjedMVP · 3 评论 · 在 GitHub 查看

Models & quantsLinux

描述

## Summary

On `main` (6f32ec0), two `tools/test_setup_unsloth.py` tests fail: when `--resident-budget-gib` is set, the generated engine args do **not** include `--kv-resident`, although the KV-streaming flag should be kept/implied (per DETAILS.md, KV streaming in WSL-capable setups comes out of the resident budget).

## Environment

- Ubuntu 26.04 LTS (WSL2), x86_64, Python 3.14.4, fresh clone @ `6f32ec0`

## Reproduce

```sh
python3 -m unittest tools/test_setup_unsloth.py -v
```

```
FAIL: test_budget_flag_with_kv_streaming_is_kept (__main__.Main.test_budget_flag_with_kv_streaming_is_kept)
    self.assertIn("--kv-resident", args)
AssertionError: '--kv-resident' not found in ['--pack', '/tmp/tmp39gne95o/data/packs/unsloth-ud-q4_k_xl', '--native', '/tmp/tmp39gne95o/models/unsloth-UD-Q4_K_XL/Qwen3.8-Flash-Next-UD-Q4_K_XL-00001-of-00004.gguf', '--expert-profile', '/tmp/tmp39gne95o/data/expert-profile.bin', '--expert-cache', 'auto', '--prefill', 'auto', '--spec', '4', '--spec-min-p', '0.5', '--mtp', '/tmp/tmp39gne95o/data/mtp/rt', '--max-context', '131072', '--kv', 'int8', '--resident-budget-gib', '40']
FAIL: test_kv_streaming_comes_out_of_the_budget (__main__.Main.test_kv_streaming_comes_out_of_the_budget)
    self.assertIn("--kv-resident", args)
```

Two failures total; the rest of the suite (37 tests) passes.

## Notes

The generated args do include `--kv int8` and `--resident-budget-gib 40`, so only the `--kv-resident` flag goes missing in this path. Looks like a regression in the budget-flag handling (the flag was likely dropped when the budget override was wired in). Related context: on WSL we observed startup correctly prints `WSL: KV streaming off (the driver pins only about 1 GB of RAM)` — so the WSL off-path works; this failure is the non-WSL on-path.

站内延伸阅读

链到安装、模型与版本说明,便于 SEO/GEO,非官方 issue 正文。