How much VRAM does InternLM 20B need?

InternLM 20B (20B parameters) requires approximately 21.9 GB of memory with Q5_K_M quantization.

What is the best quantization for InternLM 20B?

The recommended quantization for InternLM 20B is Q5_K_M, which balances quality and memory efficiency.

Can it run?

Yes, MacBook Pro M4 32GB can run InternLM 20B with a C grade (Tight fit). Expected decode speed: 5.5 tok/s.

CUsable

Tight fit

Using Q5_K_M in llama.cpp

Capabilities:

Fit status

Tight fit

Decode

5.5 tok/s

TTFT

35041 ms

Safe context

Memory

21.9 GB / 23.0 GB

Weights14.4 GB

KV Cache3.1 GB

Runtime0.9 GB

Headroom3.5 GB

Workload	Grade	Fit	Decode	TTFT	Context
Agentic Coding	D	Very compromised (needs ~1.1 GB host RAM)	6.0 tok/s	47125 ms	8K
Chat	C	Tight fit	6.1 tok/s	17255 ms	8K
Coding	C	Tight fit	5.5 tok/s	35041 ms	8K
RAG	D	Very compromised (needs ~1.1 GB host RAM)	6.0 tok/s	58906 ms	8K
Reasoning	C	Tight fit	6.1 tok/s	37386 ms	8K

How InternLM 20B (20B params) fits at each quantization level on MacBook Pro M4 32GB (23.0 GB usable).

Quant	Bits	VRAM	Quality	Fit
Q2_K	2	7.8 GB	Low	D36
Q3_K_S	3	9.8 GB	Low	D38
NVFP4	4

HuggingFace

huggingface-cli download internlm-20b

Upgrade options

RX 7900 XTX 24GBBudget pick

C49 tok/s decode

~$999 MSRP

RTX 3090 24GBBest value

C46.4 tok/s decode

~$1,499 MSRP

RTX 5090 32GBBiggest leap

B85 tok/s decode

~$1,999 MSRP