Will It Run AI
CalculatorModelsHardwareCompare
Product
  • Calculator
  • Compare
  • Tier List
Browse
  • Models
  • Hardware
  • Docs
About
  • Why It Works
  • What's New
  • Legal Notice
  • Privacy Policy

All estimates are approximations based on mathematical models and public specifications. Actual performance may vary. Do not make purchasing decisions based solely on these estimates.

Data sourced from Hugging Face, Ollama, and official model documentation. Model names and logos are trademarks of their respective owners.

© 2026 Will It Run AI — Fase Consulting Ibiza, S.L. (NIF: B57969656)

Home/Hardware/GPUs/NVIDIA A100 40GB
NVIDIA

NVIDIA

NVIDIA A100 40GB

Ampere DatacenterDatacenterAmperePCIe 4CUDA
40GB
VRAM
1.6kGB/s
Bandwidth
312TFLOPS
FP16 Compute
624TOPS
INT8 Inference
$10,000 MSRP
VRAM40 GBBandwidth1.6k GB/sCompute312 TFInference624 TOPSValue3.12 TF/$k
NVIDIA A100 40GBCategory AvgMacBook Pro M1 Max 64GB

Specifications

Compute
FP16312 TFLOPS
INT8624 TOPS
ArchitectureAmpere
Memory
VRAM40 GB
Bandwidth1555 GB/s
General
FamilyAmpere Datacenter
SegmentDatacenter
InterconnectPCIe 4
Compute PlatformCUDA
MSRP$10,000

Architecture

Ampere

Ampere is NVIDIA's second-generation RTX architecture, built on Samsung's 8nm process. It introduced 3rd-generation Tensor Cores with support for sparsity-accelerated INT8 operations and improved FP16 throughput over Turing.

AI Relevance

Sparsity-aware Tensor Cores can effectively double throughput for structured sparse workloads. However, the lack of FP8 support means quantized inference is less efficient than Ada Lovelace or Blackwell.

Process: Samsung 8nmPlatform: CUDATensor Cores: Gen 3Precisions: FP32, FP16, BF16, INT8, INT4

Recommendations by Workload

Agentic Coding

B

Devstral Small 2 24B Instruct

This model is still usable for agentic-coding, but it is not the most specialized pick. It belongs to a current frontier family for local AI. It fits natively with comfortable headroom. Known channels: huggingface, ollama, lm-studio.

Decode 89.2 tok/s · 47K ctx · llama.cpp
27.0 GB / 40.0 GB VRAM

Chat

B

Qwen 3 30B A3B

This model is a direct match for chat. It belongs to a current frontier family for local AI. It fits natively with comfortable headroom. Known channels: huggingface, ollama, lm-studio.

Decode 181.6 tok/s · 13K ctx · llama.cpp
24.3 GB / 40.0 GB VRAM

Coding

C

Devstral Small 2 24B Instruct

This model is a direct match for coding. It belongs to a current frontier family for local AI. It fits natively with comfortable headroom. Known channels: huggingface, ollama, lm-studio.

Decode 89.2 tok/s · 27K ctx · llama.cpp
23.3 GB / 40.0 GB VRAM

RAG

C

Codestral 21B Pruned i1

This model is a direct match for rag. It sits in the middle of the current model mix. It fits natively with comfortable headroom.

Decode 102.0 tok/s · 53K ctx · llama.cpp
24.3 GB / 40.0 GB VRAM

Reasoning

C

Devstral Small 2 24B Instruct

This model is a direct match for reasoning. It belongs to a current frontier family for local AI. It fits natively with comfortable headroom. Known channels: huggingface, ollama, lm-studio.

Decode 89.2 tok/s · 27K ctx · llama.cpp
23.3 GB / 40.0 GB VRAM

Full Model Compatibility

AlibabaQwen 2.5 Coder 32B
B56
32B29.4 GB67 tok/s22K ctx
dense
UnslothQwen3.5 35B A3B
B56
35B31.7 GB61 tok/s20K ctx
dense
Lmstudio-communityLQwen3.5 35B A3B
B56
35B31.7 GB61 tok/s20K ctx
dense
UnslothQwen3.5 27B
B56
27B25.6 GB79 tok/s25K ctx
dense
Unslothgemma 3 27b it
B55
27B25.6 GB79 tok/s25K ctx
dense
AlibabaQwen3-Coder 30B A3B Instruct
B55
30.5B24.3 GB182 tok/s26K ctx
moe
AlibabaQwen3-VL 30B A3B Instruct
B55
30B24.0 GB188 tok/s27K ctx
moe
MistralDevstral Small 2 24B Instruct
C55
24B23.3 GB89 tok/s27K ctx
dense
MistralDevstral Small 1.1
C54
24B23.3 GB89 tok/s27K ctx
dense
MistralCodestral 2 25.08
C54
22B21.8 GB97 tok/s29K ctx
dense
UnslothQwen3.5 9B
C49
9B11.8 GB238 tok/s54K ctx
dense
HauhauCSHQwen3.5 9B Uncensored HauhauCS Aggressive
C49
9B11.8 GB238 tok/s54K ctx
dense
Lmstudio-communityLQwen3.5 9B
C48
9B11.8 GB238 tok/s54K ctx
dense
BartowskiBMeta Llama 3.1 8B Instruct
C48
8B11.0 GB268 tok/s58K ctx
dense
XtunerXllava llama 3 8b v1 1
C48
8B11.0 GB268 tok/s58K ctx
dense
UnslothDeepSeek R1 0528 Qwen3 8B
C48
8B11.0 GB268 tok/s58K ctx
dense
MaziyarPanahiMMeta Llama 3 8B Instruct
C48
8B11.0 GB268 tok/s58K ctx
dense
TheBlokeTLlama 2 7B Chat
C48
7B10.3 GB306 tok/s62K ctx
dense
TheBlokeTMistral 7B Instruct v0.2
C48
7B10.3 GB306 tok/s62K ctx
dense
MaziyarPanahiMMistral 7B Instruct v0.3
C48
7B10.3 GB306 tok/s62K ctx
dense
UnslothQwen3.5 4B
C47
4B8.1 GB535 tok/s79K ctx
dense
BartowskiBLlama 3.2 3B Instruct
C46
3B7.9 GB617 tok/s81K ctx
dense
Lmstudio-communityLgemma 3 4b it
C46
4B8.1 GB535 tok/s79K ctx
dense
QwenQwen2.5 3B Instruct
C46
3B7.5 GB714 tok/s85K ctx
dense
BartowskiBgemma 2 2b it
C46
2B7.3 GB836 tok/s87K ctx
dense
Googlegemma 2b
C46
2B6.9 GB1071 tok/s92K ctx
dense
TheDrummerTGemmasutra Mini 2B v1
C46
2B6.9 GB1071 tok/s92K ctx
dense
QwenQwen2.5 1.5B Instruct
C46
1.5B6.6 GB1306 tok/s97K ctx
dense
Hugging-quantsHLlama 3.2 1B Instruct Q8 0
C46
1B6.5 GB1372 tok/s98K ctx
dense
TheBlokeTTinyLlama 1.1B Chat v1.0
C46
1.1B6.4 GB1306 tok/s100K ctx
dense
Ggml-orgGSmolVLM 500M Instruct
C45
0.5B6.1 GB1372 tok/s105K ctx
dense
Ggml-orgGembeddinggemma 300M
C45
0.3B5.9 GB1372 tok/s108K ctx
dense
DeepSeekDeepSeek R1 671B
F0
671B420.0 GB9 tok/s4K ctx
moe
MistralDevstral 2 123B Instruct
F0
123B99.1 GB17 tok/s6K ctx
dense
Z.aiGLM-5
F0
744B465.0 GB8 tok/s4K ctx
moe
Moonshot AIKimi K2.5
F0
1000B619.9 GB7 tok/s4K ctx
moe
MistralMistral Large 3
F0
675B423.1 GB9 tok/s4K ctx
+1moe
MistralMistral Small 4 119B
F0
119B78.5 GB52 tok/s8K ctx
moe
AlibabaQwen3-Coder 480B A35B Instruct
F0
480B303.2 GB12 tok/s4K ctx
moe
AlibabaQwen3-Coder-Next
F0
80B54.5 GB81 tok/s12K ctx
moe
UnslothQwen3.5 122B A10B
F0
122B83.7 GB20 tok/s8K ctx
dense
DeepSeekDeepSeek V3 671B
F0
671B420.0 GB9 tok/s4K ctx
moe
MistralMixtral 8x22B
F0
141B97.0 GB29 tok/s7K ctx
moe
AlibabaQwen 2.5 72B
F0
72B60.1 GB30 tok/s11K ctx
dense
AlibabaQwen 3 235B A22B
F0
235B151.7 GB24 tok/s4K ctx
moe
UnslothQwen3.5 397B A17B
F0
397B309.1 GB5 tok/s4K ctx
dense
MetaLlama 3.3 70B
F0
70B58.5 GB31 tok/s11K ctx
dense
MetaLlama 4 Maverick 17B 128E
F0
400B251.6 GB16 tok/s4K ctx
moe
CohereCommand A 111B
F0
111B90.0 GB19 tok/s7K ctx
dense
AlibabaQwen 2.5 VL 72B
F0
72B60.1 GB30 tok/s11K ctx
dense

Just out of reach

Models you could run with an upgrade

High-quality models that need a bit more memory

DeepSeekDeepSeek R1 671B
671BTier 5Needs ~425.8 GB
MistralDevstral 2 123B Instruct
123BTier 5Needs ~118.4 GB
Runs on Mac Studio M3 Ultra 256GB
Z.aiGLM-5
744BTier 5Needs ~471.2 GB
Moonshot AIKimi K2.5
1000BTier 5Needs ~624.9 GB
MistralMistral Large 3
675BTier 5Needs ~429.5 GB

Upgrade paths

Upgrade from NVIDIA A100 40GB

See what you unlock with more powerful hardware

Upgrade options

Upgrade options

AppleMacBook Pro M1 Max 64GBNext step up
64 GB Unified (+24)
A
Unlocks Qwen3 48B A4B Savant Commander Distill 12X Closed Open Heretic Uncensored

~$2,499 MSRP

NVIDIARTX A6000 48GBNVIDIA upgrade
48 GB VRAM (+8)
A
Unlocks Qwen3-Coder-Next, Qwen3 48B A4B Savant Commander Distill 12X Closed Open Heretic Uncensored

~$4,650 MSRP

AppleMacBook Pro M3 Max 128GBBest value
128 GB Unified (+88)
B
Unlocks Mistral Small 4 119B, Qwen3-Coder-Next, Qwen 2.5 72B+14 more

~$2,499 MSRP

AMDAMD Instinct MI350X 288GBBiggest leap
288 GB VRAM (+248)8000 GB/s (+6445)
A
Unlocks Devstral 2 123B Instruct, Mistral Small 4 119B, Qwen3-Coder 480B A35B Instruct+28 more · +307% faster avg

~$8,000 MSRP

Compare this GPU