Can I run
this locally?
Choose your GPU to see setups using that card, or a memory budget to find single-card setups. This is evidence from contributors, not a guarantee that a model fits. Explore multi-GPU setups by GPU model in the directory.
Single-card memory at or below 97 GB+. 56 setups match.
A purpose-built engine (NInfer 4080) runs ISTA-DASLab's Qwen 3.8 27B GSQ at 100K+ context on one RTX 4080 16GB, measuring up to ~2,720 tok/s prefill and ~262 tok/s decode through DFlash2 speculative decoding.
Qwen 3.8 27B · nInferA teacher runs an entire exam-correction pipeline on a single 3090, from generating answer sheets to scoring hundreds of students automatically.
Qwen 3.8 Flash NextA single RTX 3090 running Qwen 3.8 27B Q4 via llama.cpp, orchestrated by GPT-6.1 Sol in the cloud, finished three small 3D games in 43.4 minutes and $0.17 total.
Qwen 3.8 27B · llama.cppA cloned five-store e-commerce operation — cart, mailings, WhatsApp login, payment gateways — runs day to day on a single 7900XT, turning a real monthly profit.
Qwen 3.8 27BA 3090 owner runs Qwen 3.8 27B daily in Cline at a 96K context by quantising the KV cache to q8_0, reporting about 43 tok/s once the model stops thinking.
Qwen 3.8 27B · OllamaClient confidentiality rules out cloud APIs for this freelancer, who spreads different model sizes across three separate machines.
Kimi K3A developer uses Qwen 27B as a carefully supervised coding partner inside Cline, with explicit planning and changelog-based context management.
Qwen 3.8 27BA local Qwen model maintains a website, answers email and runs recurring jobs on a dedicated Radeon workstation.
Qwen 3.8 27BRunning Qwen 3.8 27B on the same Mac they work on, this developer credits per-feature domain glossaries and tight scoping — not the model — for making local coding reliable.
Qwen 3.8 27BA legal user has Claude build task plans while a local Qwen 27B executes them, with anonymisation, OCR for difficult documents and a RAG pipeline using nomic embeddings.
Qwen 3.8 27BA developer with 20 years of experience runs three concurrent coding sessions at about 150 tok/s each on a single RTX 5090 and has been cloud-free for three months.
Stack not specifiedA four-bit Qwen 3.8 quant at 200K context handles complete app builds through OpenCode, left running until it hits a decision point.
Qwen 3.8 27BThe same low-key homelab setup also runs a semi-agentic wedding planner that reads email, keeps a running database and document trail, and answers questions about where things stand — plus a separate loop that just deletes marketing mail.
Qwen 3.6 35B-A3BTwo 12GB 3060s handle both code review and customer-support lookups, reasoning read-only over live account data and internal docs.
Qwen 3.6 35B-A3BOn a VRAM-constrained 5070 Ti, this person keeps local strictly in the 'learning and playing around' lane, and routes actual work through Claude with local models as a validation step at most.
Stack not specifiedBeyond personal daily use, this person stood up a Spark cluster for a client so their employees get an internal, compliance-friendly LLM platform.
Stack not specifiedOn a 12GB 5070, a heavily quantized Qwen 3.8 27B through a custom Hermes backend is, in their words, the first local model that's felt confidently productive.
Qwen 3.8 27BAn unusual layered setup: the local model isn't the coding agent — it's the thing an outer coding agent uses to build and stress-test a separate 'inner' agent.
Qwen 3.8 27BA vision model on a modest 3070 turns raw detection events from a home-built surveillance system into plain-language descriptions.
Qwen3-VL-4BThe same Qwen 3.8 27B instance that drives this person's Hermes agent also sits as an MCP tool for Codex, reviewing its proposals and its final output — and the task simply stops if the two can't agree.
Qwen 3.8 27BReferral triage that extracts details, standardizes them, and suggests categories and tests — with the final call always left to a doctor, and nothing ever leaving the network.
Stack not specifiedAn RTX 5090 runs recurring agents for project maintenance, site contributions and personalized news monitoring.
Qwen 3.8 27BA game developer on an RTX 5080 uses Gemma 4 26B for background passes in a text adventure engine, while larger models handle knowledge-heavy generation.
Gemma 4 26BA large enterprise deployment: eight A100 80GBs running Qwen 3.8 under SGLang with tensor parallelism, while the company brings newer B200s online for bigger models.
Qwen 3.8 27B · SGLangA contractor whose clients forbid commercial providers from reading their source code runs unit tests and code reviews locally on a 3090, with a long-context llama.cpp setup.
Qwen 3.8 27B · llama.cppOn an RTX 4070 and 128GB of RAM, a Hermes agent is being set up to query a private SQL database of hobby equipment and watch prices on the owner's behalf.
LM StudioA custom harness layers a written skill and an orchestrator agent that dispatches tasks to five subagents, each starting from a clean context with hard test gates.
llama.cppThis company believes uploading user data to a cloud model would be illegal for their use case, so email and user-text automation runs on a Gemma 4 build on a single Radeon Pro AI R9700.
Gemma 4Proving out AI tagging and logging for a media archive on one of the smallest cards in the thread, then building toward a self-hosted digital asset manager.
Stack not specifiedA NUC12 with a 16GB Intel Arc A770M handles a 262K-context Qwen build entirely in VRAM, coding in the background while the owner does something else.
Qwen 3.6 35B-A3B · llama.cppA Python collector pulls read-only configs, firewall rules, and network health data every day and feeds it into a local RAG pipeline, so the advisor always knows the current state of the homelab without ever being able to touch it.
Qwen 3.8 27BDaily local coding on a 4070 is explicitly not as good as Claude for complex refactoring — but the privacy and iteration speed still make it worth keeping.
Qwen 3.8 27BA small research group built a deterministic local pipeline that chunks project data, finds it again with a multimodal RAG and matches it into each project's own report structure.
Qwen 3.8 9BA single Radeon Pro AI R9700 runs coding work using Lucebox and a recommended 4-bit quant, relying on custom optimized kernels for the card.
LuceboxA researcher in applied mathematics is assembling a fully local pipeline to process roughly 20,000 PDFs, comparing PDF extractors, chunking strategies and small summarisation models.
Gemma 4 12BA custom llama.cpp fork for Ampere cards pushes Qwen 3.8 27B to 90+ tok/s through a 100K context, aimed at caching-heavy agentic work.
Qwen 3.8 27B · llama.cpp (llamAmpere fork)A 3080 churned for about 18 hours to produce roughly 60,000 synthetic helpdesk tickets, complete with email threads and time entries, for testing a ticketing system.
Stack not specifiedUsing a gifted 600GB server and a cheap used P100, this owner reports that a capable fully local setup is doable for under $3,000 today.
Qwen 3.8 27BRather than chatting with a local model, this user runs local document Q&A, summarisation, translation and grammar correction on an M4 Pro, then built a desktop app to tie the pieces together.
Gemma 4 12B · MLXA custom Linux workstation with an RTX Pro 6000 and 48GB of RAM runs programming, copy editing and summarisation through vLLM, with no cloud use all year.
Qwen 3.8 Flash Next · vLLMA daily local coder on an RTX 5060 8GB and 32GB of RAM finds responsiveness under memory pressure matters more than benchmark speed.
Qwen · OllamaQwen 3.8 Flash Next on an RTX Pro 6000 takes implementation and review tasks delegated from Codex or Claude, replacing Sonnet and Opus for that slice.
Qwen 3.8 Flash Next · DeepSeek HarnessA Radeon 7900XTX with 24GB VRAM and 64GB DDR5 runs Qwen 3.8 Flash Next through Strata at about 60 tok/s up to 250k context while leaving VRAM for gaming.
Qwen 3.8 Flash Next · StrataA llama.cpp expert-streaming build runs the 177B Qwen 3.8 Flash Next MoE at roughly 11.5 tok/s on a 12GB RTX 5070 and 32GB DDR4-2400 by streaming experts from SSD.
Qwen 3.8 Flash Next · llama.cppTensorSharp runs the 176B Qwen 3.8 Flash Next MoE on a 16GB RTX 3080 Laptop with 32GB RAM, matching Strata on decode while finishing the benchmark process far faster.
Qwen 3.8 Flash Next · TensorSharpThe author runs Qwen 3.8 27B at Q6 with a 160k context and vision on an RTX 5090, using the Hermes agent harness and SearxNG in Docker, and reports around 80 tok/s but about 15 minutes for a one-shot website.
Qwen 3.8 27BA measured benchmark on a modded 48GB RTX 4090 shows Strata's n-gram table belongs on SSD while conversation parking delivers the large speedup.
StrataDaily OCR of math- and table-heavy PDFs into LaTeX-correct Markdown and Word, using Qwen vision models on llama.cpp instead of a smaller model that misreads numbers.
Qwen 3.8 27B · llama.cppA hybrid personal-assistant setup for creative writing, quick searches and Discord bot experiments on an 8GB RTX 4060, where 16GB of system RAM is the real ceiling
Gemma 4 26B A4B QAT · Unsloth StudioA contributor experiments with agents, tools and MCP on an 8GB laptop, using lightweight Gemma models to turn constrained hardware into a practical coding playground.
Gemma 4 · LM StudioFour quants of a vision model measured on a 12GB RTX 5070 SFF with 64GB DDR5, through a purpose-built inference engine, at 37 to 55 GB combined RAM+VRAM.
Qwen 3.8 Flash Next · StrataA gaming PC with an RTX 4060 8GB and 32GB DDR5 runs a 17.5GB Qwen 3.8 27B by splitting it between VRAM and system RAM and adding speculative decoding, then drives it as a remote coding agent; the same machine also runs Qwen3-VL for invoice extraction.
Qwen 3.8 27B · llama.cppA purpose-built engine (NInfer v1.3) fits Qwen 3.8 27B at a real 131,072-token context and KV capacity, plus vision, on one RTX 5080 16GB, qualifying ~71.6 tok/s decode and ~1,381 tok/s prefill on a 118,001-token prompt.
Qwen 3.8 27B · nInferA documented, measured local coding setup: Qwen 3.8 27B on one RTX 4090 with llama.cpp and OpenCode, holding a 128k context and reporting around 92 tokens/s generation with speculative decoding.
Qwen 3.8 27B · llama.cppThe same project's measured run of a ternary-quantised 27B model: about 92 tokens/s generation with no speculative decoding, from 6.7 GB of weights, at a documented cost in agentic coding ability.
Ternary Bonsai 2 27B · llama.cppA Radeon 6650 XT runs quantized Qwen models through vLLM and Ollama, while a custom three-layer harness handles private, deterministic automation around the clock.
Qwen 2.5 7B · vLLM (ROCm 6.x), Ollama fallbackChecking your budget is step one. Open the full 97 GB+ guide, or cut the catalog another way.
The lists here are evidence from other people's rigs. Connect a read-only agent to this catalog over MCP and it answers the same question for your GPU, your task and your budget, grounded in these setups and citing them. You keep the final word: it never estimates memory and the site still reserves human review.
Connect an agent to the catalogOnly single-card setups match a memory budget. Unified memory is labelled in each setup. See every capacity guide or the full directory.