Mistral
Devstral Small 2 24B Instruct
Devstral Small family, 24B parameters, recommended as Q4_K_M for first-pass local usage in V1.
Params
24B
Context
256K
License
Mistral Research License
Best runtime
Ollama
Recommended hardware
First-pass fit across priority GPUs
| Hardware | Fit | Decode | Safe ctx |
|---|---|---|---|
| NVIDIA A10 24GB | Tight fit | 23.5 tok/s | 18K |
| NVIDIA A100 40GB | Runs well | 84.5 tok/s | 28K |
| NVIDIA A100 80GB | Runs well | 110.7 tok/s | 48K |
| NVIDIA A16 64GB | Runs well | 32.6 tok/s | 41K |