Issues / #3493
#3493 [Benchmark] With Deepseek Harness Prefix cache never hits: Reused = 0 on 12/12 requests, long-context prefill consumes 73% of wall time
open · @shiftonetothree · 3 comments · View on GitHub
Description
this is log. | Time | Status | Prompt | Reused | Output | Prefill t/s | Decode t/s | VRAM hit rate | Duration | | :--- | :---: | ---: | ---: | ---: | ---: | ---: | :---: | ---: | | 02:20:28 | Done | 35,963 | 0 | 80 | 1,491 | 41.4 | 55.8% +10.8% PCIe | 26.1 s | | 02:19:11 | Done | 100,582 | 0 | 459 | 1,471 | 53.1 | 69.1% +7.9% PCIe | 77.0 s | | 02:18:41 | Done | 35,516 | 0 | 265 | 1,474 | 45.6 | 66.7% +8.2% PCIe | 29.9 s | | 02:16:58 | Done | 98,206 | 0 | 2,316 | 1,542 | 58.6 | 82.9% +4.0% PCIe | 103.2 s | | 02:16:21 | Done | 35,341 | 0 | 664 | 1,470 | 60.1 | 81.2% +4.3% PCIe | 35.1 s | | 02:15:09 | Done | 97,673 | 0 | 460 | 1,530 | 55.6 | 71.5% +7.4% PCIe | 72.2 s | | 02:14:33 | Done | 34,697 | 0 | 482 | 1,446 | 54.2 | 75.4% +5.8% PCIe | 32.9 s | | 02:13:02 | Done | 95,967 | 0 | 1,538 | 1,519 | 55.9 | 79.4% +5.0% PCIe | 90.8 s | | 02:11:03 | Done | 34,404 | 0 | 3,000 | 1,435 | 66.5 | 84.7% +3.2% PCIe | 69.1 s | | 02:09:54 | Done | 94,214 | 0 | 257 | 1,486 | 52.4 | 62.9% +10.2% PCIe | 68.3 s | | 02:09:29 | Done | 33,545 | 0 | 85 | 1,458 | 38.8 | 53.8% +12.0% PCIe | 25.2 s | | 02:07:50 | Done | 92,123 | 0 | 2,029 | 1,458 | 57.4 | 81.3% +4.3% PCIe | 98.5 s |
Related on this site
Quick links to related pages.