With 12–16 GB,
here is what really runs.
Experiences reported by people using their machines every day. The figure is the reported setup capacity; multi-GPU and unified-memory layouts can behave differently.
in this range
Local is for tinkering; Claude still does the real work
On a VRAM-constrained 5070 Ti, this person keeps local strictly in the 'learning and playing around' lane, and routes actual work through Claude with local models as a validation step at most.
The first local model this person actually trusts with real work
On a 12GB 5070, a heavily quantized Qwen 3.8 27B through a custom Hermes backend is, in their words, the first local model that's felt confidently productive.
Good enough for 80% of coding, honest about the other 20%
Daily local coding on a 4070 is explicitly not as good as Claude for complex refactoring — but the privacy and iteration speed still make it worth keeping.
Iterative coding with Qwen 27B on a 4070 Ti Super
A developer uses Qwen 27B as a carefully supervised coding partner inside Cline, with explicit planning and changelog-based context management.
Building a personal academic RAG pipeline on a 16GB 5060 Ti
A researcher in applied mathematics is assembling a fully local pipeline to process roughly 20,000 PDFs, comparing PDF extractors, chunking strategies and small summarisation models.
Local models as checker and tracker for a text adventure engine
A game developer on an RTX 5080 uses Gemma 4 26B for background passes in a text adventure engine, while larger models handle knowledge-heavy generation.
Going fully local on a home server for under $3,000
Using a gifted 600GB server and a cheap used P100, this owner reports that a capable fully local setup is doable for under $3,000 today.
Qwen 3.8 27B at 100K context on a single RTX 4080 16GB (NInfer 4080)
A purpose-built engine (NInfer 4080) runs ISTA-DASLab's Qwen 3.8 27B GSQ at 100K+ context on one RTX 4080 16GB, measuring up to ~2,720 tok/s prefill and ~262 tok/s decode through DFlash2 speculative decoding.
A Hermes agent coordinating hobbies, inventories and price tracking
On an RTX 4070 and 128GB of RAM, a Hermes agent is being set up to query a private SQL database of hobby equipment and watch prices on the owner's behalf.
Qwen 3.8 Flash Next on 12GB of VRAM via RAM+VRAM offload (Strata engine)
Four quants of a vision model measured on a 12GB RTX 5070 SFF with 64GB DDR5, through a purpose-built inference engine, at 37 to 55 GB combined RAM+VRAM.
Qwen3.8 Flash Next 177B at 11-15 tok/s on a single RTX 5070 12GB with 32GB DDR4-2400
A llama.cpp expert-streaming build runs the 177B Qwen 3.8 Flash Next MoE at roughly 11.5 tok/s on a 12GB RTX 5070 and 32GB DDR4-2400 by streaming experts from SSD.
Running Qwen3.8 Flash Next 176B on a 16GB RTX 3080 Laptop with 32GB RAM and SSD
TensorSharp runs the 176B Qwen 3.8 Flash Next MoE on a 16GB RTX 3080 Laptop with 32GB RAM, matching Strata on decode while finishing the benchmark process far faster.
Qwen 3.8 27B at a true 128K context with vision on a single 16GB RTX 5080 (NInfer)
A purpose-built engine (NInfer v1.3) fits Qwen 3.8 27B at a real 131,072-token context and KV capacity, plus vision, on one RTX 5080 16GB, qualifying ~71.6 tok/s decode and ~1,381 tok/s prefill on a 118,001-token prompt.
Turning math-heavy PDFs into clean Markdown and Word with Qwen vision
Daily OCR of math- and table-heavy PDFs into LaTeX-correct Markdown and Word, using Qwen vision models on llama.cpp instead of a smaller model that misreads numbers.
An agent that plans a wedding from your inbox
The same low-key homelab setup also runs a semi-agentic wedding planner that reads email, keeps a running database and document trail, and answers questions about where things stand — plus a separate loop that just deletes marketing mail.
Background coding at 10-15 tok/s on an Intel Arc laptop GPU
A NUC12 with a 16GB Intel Arc A770M handles a 262K-context Qwen build entirely in VRAM, coding in the background while the owner does something else.
Compare with another budget.
Every capacity guide, or start from your own machine and a task.