#6609 [Engine] Inkling-2-Mini on RTX 3090 24 GB: long prompts OOM at ink_cuda.cuh:54 unless --max-context is small (SQ-20 and SQ-15)

open · @warc0s · 1 comentarios · En GitHub

BenchmarksSetup & installServer & APINVIDIA / CUDAModels & quantsLinux

Descripción

### Area

[Engine]  (speed, kernels, models, the server and API)

### Desktop app, Terminal or source?

Terminal (the `strata` command)

### What happened

On an RTX 3090 24 GB, Inkling-2-Mini only reads long prompts when `--max-context` is small. With a larger `--max-context`, a prompt of ~7k tokens or more makes the engine run out of memory at `ink_cuda.cuh:54` (or hang in "reading the prompt"), even though the same pack handles a 28k prompt at 32K context. It looks like the engine sizes its buffers from the free VRAM for the full `--max-context` at start, leaving no headroom for the prefill workspace.

Related: #6605 (prefill hang on an RTX 5090, split ignored) and #6587 (OOM at `ink_cuda.cuh:54`). This adds data from an sm_86 card, with both SQ-20 and SQ-15 and several context sizes.

All runs: `strata launch --family inkling-2-mini --model <pack> --context <ctx> --vision gpu --parallel 1`, DSpark on, one request at a time, prompts sent straight to `/v1/chat/completions` (no agent client), `max_tokens` 64.

| Pack | `--max-context` | VRAM in use after load | Longest prompt that worked | First failure |
|---|---|---|---|---|
| SQ-20 | 262144 | — | — | OOM at load: `ink_cuda.cuh:54` (auto-sizing line: `max_ctx 262144 -> 135296 for parallel=1 (4.84 GiB free VRAM)`) |
| SQ-20 | 131072 | — | — | OOM at load: `ink_moe_cuda.cuh:117` (with `--parallel 3` chosen by setup: `ink_cuda.cuh:54`) |
| SQ-20 | 65536 | 23.6 GB | 7.2k tokens | 28k-token prompt: stuck at "reading the prompt: 7,168 of 28,449 tokens" with GPU at 0 %, engine log ends with OOM at `ink_cuda.cuh:54` |
| SQ-20 | 32768 | 22.4 GB | 7.2k tokens | 28k-token prompt: OOM at `ink_cuda.cuh:54` (VRAM went 22.4 -> 24.1 GB) |
| SQ-15 | 131072 | 22.8 GB | 7.2k tokens | 28k-token prompt: OOM at `ink_cuda.cuh:54` |
| SQ-15 | 32768 | 17.7 GB | **28.4k tokens, OK** (8.1k, 9.8k, 14.2k and 28.4k all fine, prefill 552–591 tok/s, VRAM peak 19.9 GB) | — |

Expected: either the engine keeps enough headroom for prefill at the context it accepted, or it starts with a smaller context / refuses the prompt with an error.

Other things seen on the same machine:

1. **The VRAM/RAM split is ignored for SQ-20** (as in #6605). Setup passes `--vram-experts-gb 12.49 --ram-experts-gb 3.19` at 256K, but the engine loads `experts=3072 VRAM / 0 host`.
2. **`--vram-reserve-mib` changes backend.** With `--set vram-reserve-mib=3000` the engine switches to `inkling2-ep` and does honour the split (`experts: 2683 VRAM (GPU0=12.49 GiB), 389 host`), starts at 262144 and reports `READY 262144`. But decode drops to ~3.5–6.4 tok/s, a short chat answer comes back with empty `content`, and a 7.2k-token prompt still ends in OOM at `ink_cuda.cuh:54` (exit code -11), after which the server restarts the engine.
3. **The server does not notice the dead engine in the hang case.** After the 28k prompt above, `/v1/models` still answered, `strata launch --status` kept saying `ready`, a new request never got a token (GPU idle), and `strata launch --stop --preset <name>` returned `Could not stop <name>: server_stop_failed`. The `strata-inkling` process had to be killed by hand.
4. Setup's fit for Inkling seems to leave the vision shard out of the VRAM budget: in `strata_control/inkling_fit.py`, `compute()` calls `non_expert_gb(ctx, parallel, vision=False)`, so `--vision gpu` still gets all 15.68 GB of SQ-20 experts placed in VRAM at 128K. (I read this in the code; I did not test whether changing it fixes the OOM.)

Note on speed figures: this card runs with a 160 W power limit, so decode speeds here (~37 tok/s SQ-20, ~29 tok/s SQ-15 after a long prompt) are not representative of a stock 3090. The failures above do not depend on speed.

### Smallest steps or command that reproduce it

1. `strata launch --model sq-15 --family inkling-2-mini --context 128k --vision gpu --parallel 1 --no-tui`
2. Send one chat completion with a ~28k-token user message (varied filler text) and `max_tokens` 64.
3. The engine exits with `strata-inkling CUDA error 2 (out of memory) at .../inkling2/cuda/ink_cuda.cuh:54`.
4. Same steps with `--context 32k`: the same prompt is answered.

### Reproduction status

Reproduced locally

### Strata version

Strata 0.2.19 · engine 0.1.65 (`strata-inkling` 0.1.65 linux-x86_64-cuda-cuda13)

### Graphics card, memory and system

RTX 3090 24 GB (sm_86, PCIe 3.0 x8, 160 W power limit, driver 595.91.07) · 256 GB RAM · Ubuntu 26.04.1 LTS (kernel 7.0)

### Model, size and settings

Inkling-2-Mini SQ-20 and SQ-15 · 32K–256K context · pictures on (GPU reader) · parallel 1 · DSpark

### Logs

Engine log, SQ-20 at 256K (default placement):

```text
[InkCUDA] Loaded Inkling-2-Mini-SQ-20.strata (SQ-20): trunk=3.67 GB (1.22s), experts=3072 VRAM / 0 host (5.08s, hyb=0, promo=0, pf_buf=0MB), sm120_tc=0, screen=1
[InkCUDA] Auto-sizing max_ctx 262144 -> 135296 for parallel=1 (4.84 GiB free VRAM)
strata-inkling CUDA error 2 (out of memory) at /data/split-pkg/rc219-r2/src/engine/src/inkling2/cuda/ink_cuda.cuh:54
```

Engine log, SQ-20 at 256K with `--set vram-reserve-mib=3000`:

```text
[inkling2-ep] Loaded strata (SQ-20) | gpus=1 (p2p=pinned, ep_split=0.50) | experts: 2683 VRAM (GPU0=12.49 GiB, GPU1=0.00 GiB), 389 host (doorbell=off, numa=1, thr=0) | kv=fp16
INFO engine=strata-inkling-0.1.65 arch=inkling2_loop backend=cuda-ep1 model=Inkling-2-Mini-SQ-20.strata layers=24 visits=48 vocab=200058 chunk=512 batch_slots=1 slot_read=1 kv_dtype=fp16 drafter=dspark images=1 image_pad=200054 parallel=64
READY 262144 stop hold holdb
```

Server log, SQ-20 at 64K, 28k-token prompt (no further progress; GPU utilization 0 %):

```text
[strata] reading the prompt: 7,168 of 28,449 tokens, 433 s so far
...
[strata] reading the prompt: 7,168 of 28,449 tokens, 543 s so far
```

Server log, SQ-20 at 256K with the `inkling2-ep` backend, 7.2k-token prompt:

```text
[strata] the engine stopped unexpectedly (exit code -11). The engine stopped because the graphics card ran out of memory (its log says: strata-inkling CUDA error 2 (out of memory) at /data/split-pkg/rc219-r2/src/engine/src/inkling2/cuda/ink_cuda.cuh:54).
[strata] the engine had stopped (exit code -11); starting it again (a minute or two) ...
```

En el sitio

Enlaces a install, modelos, releases.