问题 / #2897

#2897 After updating to 0.1.45 I face out of memory!

open · @Coderx7 · 1 评论 · 去 GitHub 看

BenchmarksSetup & installServer & APINVIDIA / CUDAModels & quantsWindows

说明

I was doing great up until the last update. after I updated, I can no longer use my previous configs! 
I get this error which signifies I'm out of vram! 

```
❯ ./run-iq3_s.sh 

Available configs for: iq3_s

1) strata-iq3_s_int8_186k_cpu_vision.json
2) strata-iq3_s.json
3) strata-iq3_s_q4_128k_cpu_vision.json
4) strata-iq3_s_q4_262k_cpu_vision.json
5) strata-iq3_s_q4_313k_cpu_vision.json

Choose config [1]: 5
Starting with: strata-iq3_s_q4_313k_cpu_vision.json
loading the vision encoder ...
loading the model (the first start takes a minute or two) ...
[strata] starting the engine: reading the model's weights ...
[strata] loading the experts into RAM (about 55 GB) and locking part of them for the GPU.
         YOUR PC CAN BE SLOW OR STOP RESPONDING FOR 1-3 MINUTES NOW - this is normal.
         Please wait and don't close this window; the browser opens when it is ready.
[strata] still starting (26 s) - please wait ...
[strata] still starting (46 s) - please wait ...
[strata] still starting (66 s) - please wait ...
[strata] experts loaded: 46.84 GiB at 0.79 GiB/s (75 s so far)
[strata] filling the GPU's expert cache (304 experts, 0.59 GiB of VRAM) ...
[strata] almost ready ...
Traceback (most recent call last):
  File "/media/hossein/SSD/strarta_stuff/Strata/serve/server.py", line 7159, in <module>
    sys.exit(main())
             ^^^^^^
  File "/media/hossein/SSD/strarta_stuff/Strata/serve/server.py", line 6933, in main
    engine = StrataEngine(exe, engine_args(cfg) + (effort_end or []), cwd=cfg.get("cwd"), log=cfg.get("log"),
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/media/hossein/SSD/strarta_stuff/Strata/serve/server.py", line 912, in __init__
    raise EngineDied(why + (f" (see {log})" if log else "") + note +
EngineDied: the engine exited before it was ready (see /media/hossein/SSD/strarta_stuff/Strata/strata-iq3_s.log)
the engine process ended: exit code 1. mtp: the draft head does not fit. the draft head over 106299 tokens needs 213 MiB of VRAM and 176 MiB is free. hint: a smaller draft vocabulary needs less VRAM: --draft-vocab cyrillic (English, code and the Cyrillic script, ~118 MiB) or --draft-vocab en (English and code, ~81 MiB). Start once with it - START-HERE.bat --draft-vocab en (Windows) or ./setup.sh --draft-vocab en - and the model keeps it ("draft_vocab" in its strata-*.json config); or a smaller --context in setup.
the engine log's last lines:
  strata aux cpus: threads that are neither pool workers nor the host go to CPUs 1 (off at start; a request's aux_cpus=1 turns it on)
  strata generate: expert cache auto: 1.49 GiB free, 700 MiB reserved (+218 MiB for the draft head) -> 238 slots
  strata generate: expert cache 304 slots, 0.59 GiB of VRAM; policy is
  strata generate: the GPU computes the experts in the cache; it rounds differently from the CPU,
                   so a reply can differ slightly from a run without the cache (same quality:
                   bench/results/2026-09-27-cache-parity).
                   PROFILE, ranked by routing frequency, no eviction.
  strata generate: pre-filled 304 of 304 slots from the profile in 0.1 s (11006 MB/s); slot 0 verified
  strata generate: R4 hit path ON - resident experts are computed on the GPU
  strata generate: 11 expert-pool workers + the host thread
  strata generate: session is up (engine 0.1.45)
  strata generate: token graph hit path: 304 resident experts, decided on the device
  prefill: the CPU share is ON by default for prompt chunks below 1024 tokens (the idle CPU takes some of the experts the GPU would stream; answers can differ slightly from 0.1.40.3, mean KL ~0.004). STRATA_PREFILL_CPU_SHARE=0 turns it off.
  strata serve: the prompt path allocates its own buffers (too few cache slots to borrow)
  strata serve: prefill: device buffers for a chunk of 1024 tokens do not fit (197 of 9877 MiB free at the failure; 530 MiB taken, the failing buffer wanted 975 MiB): trying a 512-token chunk
  strata hc: CUDA0: the hyper-connection read runs as staged (the norm per token and stream, the down projection's activations staged ahead by cp.async); checked bit for bit against the plain read on this card (STRATA_HC_SPLIT=1 or 0 for the earlier ones)
  strata verify: window up to 6 tokens, 76.1 MiB of device buffers
  strata mtp: the draft head over 106299 tokens needs 213 MiB of VRAM and 176 MiB is free.
  strata mtp: hint: a smaller draft vocabulary needs less VRAM: --draft-vocab cyrillic (English, code and the Cyrillic script, ~118 MiB) or --draft-vocab en (English and code, ~81 MiB). Start once with it - START-HERE.bat --draft-vocab en (Windows) or ./setup.sh --draft-vocab en - and the model keeps it ("draft_vocab" in its strata-*.json config); or a smaller --context in setup.

```
Is there a quick way to revert back to the last working version? or should I manually revert back to 0.1.44.1 which was working well? 
sth like :  
```
git fetch --tags
git switch --detach v0.1.44.1
./setup.sh
```

本站相关内容

相关页面的快捷入口。