NVIDIA
The RTX 3060 12GB is one of the most popular entry points for local AI inference. Its generous 12 GB of GDDR6 VRAM — more than the RTX 3060 Ti — allows it to run 7B parameter models at full precision and 13B models with Q4 quantization. While its compute throughput is modest, the VRAM capacity makes it a budget-friendly option for getting started with local LLMs.
Official product page ↗Architecture
Ampere is NVIDIA's second-generation RTX architecture, built on Samsung's 8nm process. It introduced 3rd-generation Tensor Cores with support for sparsity-accelerated INT8 operations and improved FP16 throughput over Turing.
AI Relevance
Sparsity-aware Tensor Cores can effectively double throughput for structured sparse workloads. However, the lack of FP8 support means quantized inference is less efficient than Ada Lovelace or Blackwell.
Ampere is NVIDIA's second-generation RTX architecture, built on Samsung's 8nm process. It introduced 3rd-generation Tensor Cores with support for sparsity-accelerated INT8 operations and improved FP16 throughput over Turing.
The RTX 3060 uses the GA106 GPU die with 28 Streaming Multiprocessors, each containing 128 CUDA cores. While its 112 Tensor Cores are modest, they provide meaningful acceleration for quantized inference workloads.
Notably, the RTX 3060 uses a wider 192-bit memory bus than the RTX 3060 Ti (256-bit), but compensates with more VRAM chips to reach 12 GB total. For AI workloads, this VRAM advantage is significant — it determines which models can run entirely on-GPU versus requiring slower CPU offloading.
Agentic Coding
CThis model is still usable for agentic-coding, but it is not the most specialized pick. It sits in the middle of the current model mix. It fits natively with comfortable headroom. Known channels: huggingface, ollama.
Chat
CThis model is a direct match for chat. It belongs to a current frontier family for local AI. It fits natively with comfortable headroom. Known channels: huggingface, ollama, lm-studio.
Coding
CThis model is still usable for coding, but it is not the most specialized pick. It sits in the middle of the current model mix. It fits natively with comfortable headroom. Known channels: huggingface, ollama.
RAG
CThis model is a direct match for rag. It sits in the middle of the current model mix. It fits natively with comfortable headroom.
Reasoning
CThis model is a direct match for reasoning. It belongs to a current frontier family for local AI. It fits natively with comfortable headroom. Known channels: huggingface, ollama, lm-studio.
Qwen3.5 9BDeepSeek R1 0528 Qwen3 8BQwen3.5 4B
Qwen2.5 3B Instruct
gemma 2bQwen2.5 1.5B Instruct
DeepSeek R1 671B
Devstral 2 123B Instruct
GLM-5Qwen3.5 27BQwen3.5 35B A3B
Kimi K2.5Mistral Large 3Mistral Small 4 119BQwen3-Coder 30B A3B InstructQwen3-Coder 480B A35B InstructQwen3-Coder-NextQwen3.5 122B A10BDeepSeek V3 671BMixtral 8x22BQwen 2.5 72BQwen 3 235B A22BQwen3-VL 30B A3B InstructQwen3.5 397B A17BDevstral Small 2 24B Instruct
Llama 3.3 70BLlama 4 Maverick 17B 128E
Command A 111BQwen 2.5 Coder 32BQwen 2.5 VL 72Bgemma 3 27b itCodestral 2 25.08Devstral Small 1.1Just out of reach
High-quality models that need a bit more memory
Upgrade paths
See what you unlock with more powerful hardware
Upgrade options
~$1,999 MSRP
~$329 MSRP
RTX A4000 16GBNVIDIA upgrade~$1,000 MSRP
~$8,000 MSRP
