A living catalog of local LLM setups, connected to their hardware, stack and results. Benchmarks tell you what fits. This tells you what works.
◉ Born from a r/LocalLLM thread · September 2026
This wiki is only as useful as the details people share. Take your time — every careful contribution makes the map clearer for everyone.
Capacity is the reported total for this setup. Multi-GPU and unified-memory layouts can behave differently.
A small ISP (~7,600 customers) serves its agentic operations work on two identical machines, each with 4× RTX 3090 (250W-capped) and vLLM v0.30.0; llama-benchy measures ~5993 tok/s prefill and ~199 tok/s generation.
A gaming PC with an RTX 4060 8GB and 32GB DDR5 runs a 17.5GB Qwen 3.8 27B by splitting it between VRAM and system RAM and adding speculative decoding, then drives it as a remote coding agent; the same machine also runs Qwen3-VL for invoice extraction.
A Framework 13 with a Ryzen 7 7840U and 64GB of unified memory runs privacy-first coding and homelab work through llama.cpp and hermes, where the larger model is too slow to use interactively.
A 128GB NVIDIA Jetson Thor T5000 runs Qwen 3.8 27B, Qwen-Image 2.1, ComfyUI and the Hermes agent to turn one Telegram message into a comic book and deploy it to a website, entirely locally.
On a 19-task cybersecurity benchmark, an uncensored Qwen 3.8 Flash Next solved 80.6% first try, but the run used someone else's 2x RTX Pro 6000 and only 2 runs per task.
A measured benchmark on a modded 48GB RTX 4090 shows Strata's n-gram table belongs on SSD while conversation parking delivers the large speedup.
Two single-GPU Ubuntu servers, one with a blower-modded Tesla V100 32GB and one with an RTX 3090, run Strata with Qwen 3.8 Flash Next at 262K context for local business management automation.
A purpose-built engine (NInfer 4080) runs ISTA-DASLab's Qwen 3.8 27B GSQ at 100K+ context on one RTX 4080 16GB, measuring up to ~2,720 tok/s prefill and ~262 tok/s decode through DFlash2 speculative decoding.
A Radeon 7900XTX with 24GB VRAM and 64GB DDR5 runs Qwen 3.8 Flash Next through Strata at about 60 tok/s up to 250k context while leaving VRAM for gaming.
A llama.cpp expert-streaming build runs the 177B Qwen 3.8 Flash Next MoE at roughly 11.5 tok/s on a 12GB RTX 5070 and 32GB DDR4-2400 by streaming experts from SSD.
TensorSharp runs the 176B Qwen 3.8 Flash Next MoE on a 16GB RTX 3080 Laptop with 32GB RAM, matching Strata on decode while finishing the benchmark process far faster.
The author runs Qwen 3.8 27B at Q6 with a 160k context and vision on an RTX 5090, using the Hermes agent harness and SearxNG in Docker, and reports around 80 tok/s but about 15 minutes for a one-shot website.
Choose the path that fits you. Both routes go through human review before anything is published.
