START FROM YOUR MACHINE

Can I run
this locally?

Choose your GPU to see setups using that card, or a memory budget to find single-card setups. This is evidence from contributors, not a guarantee that a model fits. Explore multi-GPU setups by GPU model in the directory.

Single-card memory at or below 97 GB+. 56 setups match.

16 GB totalCoding
Qwen 3.8 27B at 100K context on a single RTX 4080 16GB (NInfer 4080)

A purpose-built engine (NInfer 4080) runs ISTA-DASLab's Qwen 3.8 27B GSQ at 100K+ context on one RTX 4080 16GB, measuring up to ~2,720 tok/s prefill and ~262 tok/s decode through DFlash2 speculative decoding.

Qwen 3.8 27B · nInfer
24 GB totalDocuments
Grading 500 student exams with a local model

A teacher runs an entire exam-correction pipeline on a single 3090, from generating answer sheets to scoring hundreds of students automatically.

Qwen 3.8 Flash Next
24 GB totalHybrid orchestration
GPT-6.1 Sol as cloud orchestrator with one RTX 3090 worker: wall clock and cost for three small builds

A single RTX 3090 running Qwen 3.8 27B Q4 via llama.cpp, orchestrated by GPT-6.1 Sol in the cloud, finished three small 3D games in 43.4 minutes and $0.17 total.

Qwen 3.8 27B · llama.cpp
20 GB totalBusiness automation
Running a small e-commerce business on Qwen 3.8 27B

A cloned five-store e-commerce operation — cart, mailings, WhatsApp login, payment gateways — runs day to day on a single 7900XT, turning a real monthly profit.

Qwen 3.8 27B
24 GB totalCoding
Fitting Qwen 27B and a 96K context onto a single 3090

A 3090 owner runs Qwen 3.8 27B daily in Cline at a 96K context by quantising the KV cache to q8_0, reporting about 43 tok/s once the model stops thinking.

Qwen 3.8 27B · Ollama
96 GB totalRegulated work
Three rigs for freelance work an NDA won't let leave the building

Client confidentiality rules out cloud APIs for this freelancer, who spreads different model sizes across three separate machines.

Kimi K3
16 GB totalCoding
Iterative coding with Qwen 27B on a 4070 Ti Super

A developer uses Qwen 27B as a carefully supervised coding partner inside Cline, with explicit planning and changelog-based context management.

Qwen 3.8 27B
24 GB totalBusiness automation
Running a website with Qwen 27B on a 7900XTX

A local Qwen model maintains a website, answers email and runs recurring jobs on a dedicated Radeon workstation.

Qwen 3.8 27B
64 GB totalCoding
Coding a SaaS product on an M1 Max, one well-scoped feature at a time

Running Qwen 3.8 27B on the same Mac they work on, this developer credits per-feature domain glossaries and tight scoping — not the model — for making local coding reliable.

Qwen 3.8 27B
32 GB totalRegulated work
Legal work: a cloud planner driving local anonymisation, OCR and RAG

A legal user has Claude build task plans while a local Qwen 27B executes them, with anonymisation, OCR for difficult documents and a RAG pipeline using nomic embeddings.

Qwen 3.8 27B
32 GB totalCoding
Three concurrent coding sessions at 150 tok/s on one 5090

A developer with 20 years of experience runs three concurrent coding sessions at about 150 tok/s each on a single RTX 5090 and has been cloud-free for three months.

Stack not specified
24 GB totalCoding
Full app development on a single 3090 with OpenCode

A four-bit Qwen 3.8 quant at 200K context handles complete app builds through OpenCode, left running until it hits a decision point.

Qwen 3.8 27B
12 GB totalPersonal assistant
An agent that plans a wedding from your inbox

The same low-key homelab setup also runs a semi-agentic wedding planner that reads email, keeps a running database and document trail, and answers questions about where things stand — plus a separate loop that just deletes marketing mail.

Qwen 3.6 35B-A3B
24 GB totalBusiness automation
PR review and support triage on two budget RTX 3060s

Two 12GB 3060s handle both code review and customer-support lookups, reasoning read-only over live account data and internal docs.

Qwen 3.6 35B-A3B
16 GB totalPersonal assistant
Local is for tinkering; Claude still does the real work

On a VRAM-constrained 5070 Ti, this person keeps local strictly in the 'learning and playing around' lane, and routes actual work through Claude with local models as a validation step at most.

Stack not specified
128 GB totalRegulated work
A HIPAA-style internal LLM platform, built on a cluster of DGX Sparks

Beyond personal daily use, this person stood up a Spark cluster for a client so their employees get an internal, compliance-friendly LLM platform.

Stack not specified
12 GB totalCoding
The first local model this person actually trusts with real work

On a 12GB 5070, a heavily quantized Qwen 3.8 27B through a custom Hermes backend is, in their words, the first local model that's felt confidently productive.

Qwen 3.8 27B
64 GB totalCoding
Using a local model to stress-test the agent a coding agent is building

An unusual layered setup: the local model isn't the coding agent — it's the thing an outer coding agent uses to build and stress-test a separate 'inner' agent.

Qwen 3.8 27B
8 GB totalMedia
Describing security camera events on an 8GB card

A vision model on a modest 3070 turns raw detection events from a home-built surveillance system into plain-language descriptions.

Qwen3-VL-4B
32 GB totalHybrid orchestration
A local model and Codex reviewing each other's work, with a hard stop on disagreement

The same Qwen 3.8 27B instance that drives this person's Hermes agent also sits as an MCP tool for Codex, reviewing its proposals and its final output — and the task simply stops if the two can't agree.

Qwen 3.8 27B
128 GB totalRegulated work
A cardiologist's referral system, fully air-gapped

Referral triage that extracts details, standardizes them, and suggests categories and tests — with the final call always left to a doctor, and nothing ever leaving the network.

Stack not specified
32 GB totalBusiness automation
Daily agent workflows on an RTX 5090

An RTX 5090 runs recurring agents for project maintenance, site contributions and personalized news monitoring.

Qwen 3.8 27B
16 GB totalMedia
Local models as checker and tracker for a text adventure engine

A game developer on an RTX 5080 uses Gemma 4 26B for background passes in a text adventure engine, while larger models handle knowledge-heavy generation.

Gemma 4 26B
640 GB totalCoding
Eight A100s in tensor-parallel, inside a company with a hundred more

A large enterprise deployment: eight A100 80GBs running Qwen 3.8 under SGLang with tensor parallelism, while the company brings newer B200s online for bigger models.

Qwen 3.8 27B · SGLang
24 GB totalRegulated work
Unit tests and reviews for NDA-bound client code on a single 3090

A contractor whose clients forbid commercial providers from reading their source code runs unit tests and code reviews locally on a 3090, with a long-context llama.cpp setup.

Qwen 3.8 27B · llama.cpp
12 GB totalPersonal assistant
A Hermes agent coordinating hobbies, inventories and price tracking

On an RTX 4070 and 128GB of RAM, a Hermes agent is being set up to query a private SQL database of hobby equipment and watch prices on the owner's behalf.

LM Studio
32 GB totalHybrid orchestration
An orchestrator dispatching to five fresh-context subagents

A custom harness layers a written skill and an orchestrator agent that dispatches tasks to five subagents, each starting from a clean context with hard test gates.

llama.cpp
32 GB totalBusiness automation
Company email automation kept off the cloud for legal reasons

This company believes uploading user data to a cloud model would be illegal for their use case, so email and user-text automation runs on a Gemma 4 build on a single Radeon Pro AI R9700.

Gemma 4
8 GB totalMedia
AI tagging for a media library on an 8GB card

Proving out AI tagging and logging for a media archive on one of the smallest cards in the thread, then building toward a self-hosted digital asset manager.

Stack not specified
16 GB totalCoding
Background coding at 10-15 tok/s on an Intel Arc laptop GPU

A NUC12 with a 16GB Intel Arc A770M handles a 262K-context Qwen build entirely in VRAM, coding in the background while the owner does something else.

Qwen 3.6 35B-A3B · llama.cpp
32 GB totalHomelab ops
A daily-refreshed RAG advisor built from your own infrastructure

A Python collector pulls read-only configs, firewall rules, and network health data every day and feeds it into a local RAG pipeline, so the advisor always knows the current state of the homelab without ever being able to touch it.

Qwen 3.8 27B
12 GB totalCoding
Good enough for 80% of coding, honest about the other 20%

Daily local coding on a 4070 is explicitly not as good as Claude for complex refactoring — but the privacy and iteration speed still make it worth keeping.

Qwen 3.8 27B
8 GB totalDocuments
Auto-filling varied lab reports from a multimodal RAG on an RTX 4060

A small research group built a deterministic local pipeline that chunks project data, finds it again with a multimodal RAG and matches it into each project's own report structure.

Qwen 3.8 9B
32 GB totalCoding
One R9700 with card-specific kernels for local coding

A single Radeon Pro AI R9700 runs coding work using Lucebox and a recommended 4-bit quant, relying on custom optimized kernels for the card.

Lucebox
16 GB totalDocuments
Building a personal academic RAG pipeline on a 16GB 5060 Ti

A researcher in applied mathematics is assembling a fully local pipeline to process roughly 20,000 PDFs, comparing PDF extractors, chunking strategies and small summarisation models.

Gemma 4 12B
24 GB totalCoding
A llama.cpp fork reaching 90+ tok/s through a 100K context on a 3090

A custom llama.cpp fork for Ampere cards pushes Qwen 3.8 27B to 90+ tok/s through a 100K context, aimed at caching-heavy agentic work.

Qwen 3.8 27B · llama.cpp (llamAmpere fork)
24 GB totalBatch utility
Generating 60,000 helpdesk tickets overnight instead of burning API credits

A 3080 churned for about 18 hours to produce roughly 60,000 synthetic helpdesk tickets, complete with email threads and time entries, for testing a ticketing system.

Stack not specified
16 GB totalHomelab ops
Going fully local on a home server for under $3,000

Using a gifted 600GB server and a cheap used P100, this owner reports that a capable fully local setup is doable for under $3,000 today.

Qwen 3.8 27B
24 GB totalDocuments
A daily document and writing workflow on a 24GB Mac, wrapped in a desktop app

Rather than chatting with a local model, this user runs local document Q&A, summarisation, translation and grammar correction on an M4 Pro, then built a desktop app to tie the pieces together.

Gemma 4 12B · MLX
96 GB totalCoding
A cloud-free workstation: RTX Pro 6000, vLLM and a DeepSeek harness

A custom Linux workstation with an RTX Pro 6000 and 48GB of RAM runs programming, copy editing and summarisation through vLLM, with no cloud use all year.

Qwen 3.8 Flash Next · vLLM
8 GB totalCoding
Coding on an 8GB 5060, where tuning the GPU/CPU split matters more than tok/s

A daily local coder on an RTX 5060 8GB and 32GB of RAM finds responsiveness under memory pressure matters more than benchmark speed.

Qwen · Ollama
96 GB totalCoding
A workstation GPU as a drop-in subagent for Codex and Claude

Qwen 3.8 Flash Next on an RTX Pro 6000 takes implementation and review tasks delegated from Codex or Claude, replacing Sonnet and Opus for that slice.

Qwen 3.8 Flash Next · DeepSeek Harness
24 GB totalCoding
Qwen3.8 Flash Next on a Radeon 7900XTX and 64GB DDR5 with Strata at about 60 tok/s

A Radeon 7900XTX with 24GB VRAM and 64GB DDR5 runs Qwen 3.8 Flash Next through Strata at about 60 tok/s up to 250k context while leaving VRAM for gaming.

Qwen 3.8 Flash Next · Strata
12 GB totalCoding
Qwen3.8 Flash Next 177B at 11-15 tok/s on a single RTX 5070 12GB with 32GB DDR4-2400

A llama.cpp expert-streaming build runs the 177B Qwen 3.8 Flash Next MoE at roughly 11.5 tok/s on a 12GB RTX 5070 and 32GB DDR4-2400 by streaming experts from SSD.

Qwen 3.8 Flash Next · llama.cpp
16 GB totalResearch & analysis
Running Qwen3.8 Flash Next 176B on a 16GB RTX 3080 Laptop with 32GB RAM and SSD

TensorSharp runs the 176B Qwen 3.8 Flash Next MoE on a 16GB RTX 3080 Laptop with 32GB RAM, matching Strata on decode while finishing the benchmark process far faster.

Qwen 3.8 Flash Next · TensorSharp
32 GB totalCoding
Coding locally with Qwen 3.8 27B Q6 on an RTX 5090: 160k context, ~80 tok/s, Hermes harness

The author runs Qwen 3.8 27B at Q6 with a 160k context and vision on an RTX 5090, using the Hermes agent harness and SearxNG in Docker, and reports around 80 tok/s but about 15 minutes for a one-shot website.

Qwen 3.8 27B
48 GB totalResearch & analysis
Benchmarking Strata n-gram table placement on a modded RTX 4090 48GB with 128GB DDR5

A measured benchmark on a modded 48GB RTX 4090 shows Strata's n-gram table belongs on SSD while conversation parking delivers the large speedup.

Strata
16 GB totalDocuments
Turning math-heavy PDFs into clean Markdown and Word with Qwen vision

Daily OCR of math- and table-heavy PDFs into LaTeX-correct Markdown and Word, using Qwen vision models on llama.cpp instead of a smaller model that misreads numbers.

Qwen 3.8 27B · llama.cpp
8 GB totalPersonal assistant
Creative writing and a Discord bot on an 8GB RTX 4060

A hybrid personal-assistant setup for creative writing, quick searches and Discord bot experiments on an 8GB RTX 4060, where 16GB of system RAM is the real ceiling

Gemma 4 26B A4B QAT · Unsloth Studio
8 GB totalHomelab ops
Making an 8GB laptop useful through DIY agent workflows

A contributor experiments with agents, tools and MCP on an 8GB laptop, using lightweight Gemma models to turn constrained hardware into a practical coding playground.

Gemma 4 · LM Studio
12 GB totalResearch & analysis
Qwen 3.8 Flash Next on 12GB of VRAM via RAM+VRAM offload (Strata engine)

Four quants of a vision model measured on a 12GB RTX 5070 SFF with 64GB DDR5, through a purpose-built inference engine, at 37 to 55 GB combined RAM+VRAM.

Qwen 3.8 Flash Next · Strata
8 GB totalCoding
Qwen 3.8 27B at ~6 tok/s on an 8GB RTX 4060 via RAM offload and MTP

A gaming PC with an RTX 4060 8GB and 32GB DDR5 runs a 17.5GB Qwen 3.8 27B by splitting it between VRAM and system RAM and adding speculative decoding, then drives it as a remote coding agent; the same machine also runs Qwen3-VL for invoice extraction.

Qwen 3.8 27B · llama.cpp
16 GB totalCoding
Qwen 3.8 27B at a true 128K context with vision on a single 16GB RTX 5080 (NInfer)

A purpose-built engine (NInfer v1.3) fits Qwen 3.8 27B at a real 131,072-token context and KV capacity, plus vision, on one RTX 5080 16GB, qualifying ~71.6 tok/s decode and ~1,381 tok/s prefill on a 118,001-token prompt.

Qwen 3.8 27B · nInfer
24 GB totalCoding
Qwen 3.8 27B on a single RTX 4090, measured (llama.cpp)

A documented, measured local coding setup: Qwen 3.8 27B on one RTX 4090 with llama.cpp and OpenCode, holding a 128k context and reporting around 92 tokens/s generation with speculative decoding.

Qwen 3.8 27B · llama.cpp
24 GB totalCoding
Ternary Bonsai 2 27B on an RTX 4090: speed at a quality price

The same project's measured run of a ternary-quantised 27B model: about 92 tokens/s generation with no speculative decoding, from 6.7 GB of weights, at a documented cost in agentic coding ability.

Ternary Bonsai 2 27B · llama.cpp
8 GB totalHybrid orchestration
A three-layer AMD ROCm agent architecture for 24/7 local automation

A Radeon 6650 XT runs quantized Qwen models through vLLM and Ollama, while a custom three-layer harness handles private, deterministic automation around the clock.

Qwen 2.5 7B · vLLM (ROCm 6.x), Ollama fallback
GO FURTHER

Checking your budget is step one. Open the full 97 GB+ guide, or cut the catalog another way.

OR ASK YOUR OWN AGENTCan I run this, for my exact machine?

The lists here are evidence from other people's rigs. Connect a read-only agent to this catalog over MCP and it answers the same question for your GPU, your task and your budget, grounded in these setups and citing them. You keep the final word: it never estimates memory and the site still reserves human review.

Connect an agent to the catalog

Only single-card setups match a memory budget. Unified memory is labelled in each setup. See every capacity guide or the full directory.