Pull requests / #2891
#2891 Community benchmark on 4x Radeon Instinct MI50 32GB
open · @matusnovak · 0 comentarios · En GitHub
Descripción
## Title Adding a community benchmark for 4x Radeon Instinct MI50 32GB running IQ4_XS ## Summary | Configuration | Actual prompt tokens | Reused tokens | Generated tokens | Runs | Prompt tok/s median and range | Decode tok/s median and range | TTFT seconds median and range | | --- | ---: | ---: | ---: | ---: | --- | --- | --- | | short chat | 46 (43-51) | 0 | 100 (100-100) | 9 | not measured | 33.7 (33.3-35.5) | 0.61 (0.56-0.68) | | 4K prompt, fresh | 4019 (4019-4019) | 0 (0-0) | 24 cap | 3 | 490.2 (489.8-490.7) | not measured | 8.24 (8.23-8.25) | | 16K prompt, fresh | 16058 (16058-16058) | 0 (0-0) | 24 cap | 3 | 778.3 (770.2-788.5) | not measured | 20.69 (20.43-20.91) | | 32K prompt, fresh | 32049 (32049-32049) | 0 (0-0) | 24 cap | 3 | 1010.4 (995.8-1045.9) | not measured | 31.80 (30.73-32.26) | | agent session, follow-up turns | 14598 (14589-14608) at the last turn | see below | up to 64 | 3 | see below | 33.6 (30.2-35.4) | 7.00 (4.06-7.95) |
En el sitio
Enlaces a install, modelos, releases.