Switching from llama.cpp to vLLM on AMD hardware was the unlock for running a complete SaaS build loop through a local model.
Coding1 machineQwen 3.8 27BvLLM
ChangedDescriptionHardware
Compare with revision 1
DescriptionPreviousThis person builds a full-scale SaaS product using Qwen 3.8 27B, describing the key to making it work as 'harness-maxxing' — close attention to context management and feedback loops rather than the model alone. On their AMD cards, they found llama.cpp recomputed the prefill on every subagent return, which killed throughput; switching to vLLM's paged attention kept the GPU doing useful work continuously once tuned, to the point they now consider llama.cpp comparatively toy-like for this workload.This snapshotBuilding a full-scale SaaS product with Qwen 3.8 27B. The author says it works well once you 'harness-maxx' — investing in a product knowledge graph, skills and feedback loops, much like static-analysis rules — rather than relying on the model alone. The rig is two Radeon RX 9070 XT cards (32GB of VRAM total), reporting about 50-60 tok/s at 200K context with MXFP4 quantization.
HardwarePreviousPrimary machine: GPU unspecifiedThis snapshotDual RX 9070 XT build: Radeon 9070 XT, Radeon 9070 XT
This is the currently published snapshot.
Leave a comment