#2789 Benchmark 48GB RAM + 2xP100 IQ3_XSS

本文

I came here just to say that IQ3_XSS is actually possible with lowRAM + 48GB of RAM + 2x P100 GPUs. From what I've seen, with a 192K context window, the speed fluctuates between 27 and 50 tokens/s (I'm using my custom agent).

<img width="1919" height="1079" alt="Image" src="https://github.com/user-attachments/assets/c5025186-374a-4d05-a0be-1d05589afe0b" />

<img width="915" height="423" alt="Image" src="https://github.com/user-attachments/assets/e3489de1-e476-4fd4-83cf-f65eb6955c99" />

Specs of the server

I7 12700
48GB DDR4 2133mhz
240GB SSD (Iq3 SSD)
2x 500GB HDD
1x 120GB SSD
2x P100 16GB with custom cooler

Ubuntu 26.04 headless