Question 1

Can RX 7900 XTX 24GB run mistral small 3.1 24b instruct 2503 hf?

Accepted Answer

Yes, RX 7900 XTX 24GB can run mistral small 3.1 24b instruct 2503 hf with a C grade (Tight fit). Expected decode speed: 47.2 tok/s.

Question 2

How much VRAM does mistral small 3.1 24b instruct 2503 hf need?

Accepted Answer

mistral small 3.1 24b instruct 2503 hf (24B parameters) requires approximately 22.0 GB of memory with Q4_K_M quantization.

Question 3

What is the best quantization for mistral small 3.1 24b instruct 2503 hf?

Accepted Answer

The recommended quantization for mistral small 3.1 24b instruct 2503 hf is Q4_K_M, which balances quality and memory efficiency.

Workload	Grade	Fit	Decode	TTFT	Context
Agentic Coding	C	Runs with offload (needs ~1 GB host RAM)	44.5 tok/s	6328 ms	30K
Chat	C	Tight fit	47.2 tok/s	2237 ms	10K
Coding	C	Tight fit	47.2 tok/s	4101 ms	17K
RAG	C	Runs with offload (needs ~1 GB host RAM)	44.5 tok/s	7910 ms	30K
Reasoning	C	Tight fit	47.2 tok/s	4846 ms

Quant	Bits	VRAM	Quality	Fit
Q2_K	2	9.4 GB	Low	D38
Q3_K_S	3	11.8 GB	Low	D40
NVFP4	4

Can RX 7900 XTX 24GB run mistral small 3.1 24b instruct 2503 hf?

Memory breakdown