Question 1

Can RTX 5090 32GB run mistral small 3.1 24b instruct 2503 hf?

Accepted Answer

Yes, RTX 5090 32GB can run mistral small 3.1 24b instruct 2503 hf with a B grade (Runs well). Expected decode speed: 88.1 tok/s.

Question 2

How much VRAM does mistral small 3.1 24b instruct 2503 hf need?

Accepted Answer

mistral small 3.1 24b instruct 2503 hf (24B parameters) requires approximately 22.8 GB of memory with Q4_K_M quantization.

Question 3

What is the best quantization for mistral small 3.1 24b instruct 2503 hf?

Accepted Answer

The recommended quantization for mistral small 3.1 24b instruct 2503 hf is Q4_K_M, which balances quality and memory efficiency.

Workload	Grade	Fit	Decode	TTFT	Context
Agentic Coding	C	Tight fit	82.0 tok/s	3434 ms	39K
Chat	B	Runs well	82.0 tok/s	1288 ms	12K
Coding	B	Runs well	88.1 tok/s	2197 ms	22K
RAG	C	Tight fit	82.0 tok/s	4292 ms	39K
Reasoning	B	Runs well	82.0 tok/s	2790 ms	22K

Quant	Bits	VRAM	Quality	Fit
Q2_K	2	9.4 GB	Low	D35
Q3_K_S	3	11.8 GB	Low	D37
NVFP4	4

Can RTX 5090 32GB run mistral small 3.1 24b instruct 2503 hf?

Memory breakdown