This team's product depends on agentic AI. They started on cloud APIs, then bridged to frontier agents headlessly, before worrying about customer-data privacy and the cost of scaling. They explored two local tracks: a purely local setup using a Strix Halo and a couple of midrange GPUs, and rented inference on Vast.ai.
Crucially, all of the harness and tooling work needed to match frontier agent performance was done locally first. Moving to cloud inference later was only a matter of retargeting the API. In their view they could not have completed the same work within any reasonable budget had they gone straight to cloud, and working around local limitations taught them about failure modes they would have missed on a fast cloud stream. The same local stack can also be deployed on-premise for customers, though the cost is prohibitive.
On the home side, the same person built a heavily quantized Qwen 3.8 27B agent on a couple of old Nvidia cards as a learning buddy for their eight-year-old: vision, tool use, image generation through a small image model, browser-game creation and testing, video and document analysis, and interactive presentations, all from open-source models plus tools and skills. Their honest conclusion: for production, local is either slow or expensive; for home, study and hobbies, it is entirely capable.
Reported anonymously by an r/LocalLLM contributor · score 2