A gaming PC with an RTX 4060 8GB and 32GB DDR5 runs a 17.5GB Qwen 3.8 27B by splitting it between VRAM and system RAM and adding speculative decoding, then drives it as a remote coding agent; the same machine also runs Qwen3-VL for invoice extraction.
Start with your machine.
Capacity is the reported total for this setup. Multi-GPU and unified-memory layouts can behave differently.
A Framework 13 with a Ryzen 7 7840U and 64GB of unified memory runs privacy-first coding and homelab work through llama.cpp and hermes, where the larger model is too slow to use interactively.
A llama.cpp expert-streaming build runs the 177B Qwen 3.8 Flash Next MoE at roughly 11.5 tok/s on a 12GB RTX 5070 and 32GB DDR4-2400 by streaming experts from SSD.
Two Radeon Instinct MI50 cards limited to 145W each are benchmarked with llama.cpp Vulkan across dense and MoE models, with MoE models far ahead of dense ones.
A single RTX 3090 running Qwen 3.8 27B Q4 via llama.cpp, orchestrated by GPT-6.1 Sol in the cloud, finished three small 3D games in 43.4 minutes and $0.17 total.
Personal life/project management agent with persistent memory, reminders, smart home duties (timers lights etc), computer use.
A documented, measured local coding setup: Qwen 3.8 27B on one RTX 4090 with llama.cpp and OpenCode, holding a 128k context and reporting around 92 tokens/s generation with speculative decoding.
The same project's measured run of a ternary-quantised 27B model: about 92 tokens/s generation with no speculative decoding, from 6.7 GB of weights, at a documented cost in agentic coding ability.
Daily OCR of math- and table-heavy PDFs into LaTeX-correct Markdown and Word, using Qwen vision models on llama.cpp instead of a smaller model that misreads numbers.
A laptop workflow combines Qwen Flash Next, llama.cpp and a DeepSeek harness, using system memory and SSD-backed loading to make a MoE model practical on 12GB of VRAM.
A contractor whose clients forbid commercial providers from reading their source code runs unit tests and code reviews locally on a 3090, with a long-context llama.cpp setup.
A custom harness layers a written skill and an orchestrator agent that dispatches tasks to five subagents, each starting from a clean context with hard test gates.