How much VRAM does BGE M3 need?

BGE M3 (0.5680000185966492B parameters) requires approximately 3.5 GB of memory with F16 quantization.

What is the best quantization for BGE M3?

The recommended quantization for BGE M3 is F16, which balances quality and memory efficiency.

Can it run?

Yes, Intel Arc B580 12GB can run BGE M3 with a C grade (Runs well). Expected decode speed: 328.3 tok/s.

CUsable

Runs well

Using F16 in Ollama

Capabilities:

Fit status

Runs well

Decode

328.3 tok/s

TTFT

590 ms

Safe context

Memory

3.5 GB / 12.0 GB

Weights0.3 GB

KV Cache0.8 GB

Runtime1.2 GB

Headroom1.2 GB

Workload	Grade	Fit	Decode	TTFT	Context
Agentic Coding	C	Runs well	218.9 tok/s	1287 ms	8K
Chat	C	Runs well	218.9 tok/s	482 ms	8K
Coding	C	Runs well	328.3 tok/s	590 ms	8K
RAG	C	Runs well	218.9 tok/s	1608 ms	8K
Reasoning	C	Runs well	218.9 tok/s	1045 ms	8K

How BGE M3 (0.5680000185966492B params) fits at each quantization level on Intel Arc B580 12GB (12.0 GB usable).

Quant	Bits	VRAM	Quality	Fit
Q2_K	2	0.2 GB	Low	D30
Q3_K_S	3	0.3 GB	Low	D30
NVFP4	4

Ollama

ollama run bge-m3

HuggingFace

huggingface-cli download bge-m3

Upgrade options

MacBook Pro M3 Pro 18GBBudget pick

C109.5 tok/s decode

~$1,999 MSRP