Mistral
Mistral Small 4 119B
Mistral Small family, 119B parameters, recommended as Q4_K_M for first-pass local usage in V1.
Params
119B
Context
256K
License
Mistral Research License
Best runtime
vLLM
Recommended hardware
First-pass fit across priority GPUs
| Hardware | Fit | Decode | Safe ctx |
|---|---|---|---|
| NVIDIA A10 24GB | Too heavy | 10.9 tok/s | 5K |
| NVIDIA A100 40GB | Too heavy | 28.3 tok/s | 8K |
| NVIDIA A100 80GB | Runs with offload | 37.2 tok/s | 16K |
| NVIDIA A16 64GB | Too heavy | 10.9 tok/s | 13K |