Ran Qwen 3.8 Flash Next on a 2021 M1 Max laptop (32 GPU cores, 64 GB unified memory) through a Splash/ds4 fork (built on antirez's DwarfStar), for real OpenCode agent sessions.
Long-context behaviour (Q2_0, 35 GiB, MTP on)
- One chat grown to its full context: 44 tok/s at 4K, 38.8 at 256K, 35.4 at 398K; prefill 328 to 292 tok/s
- Five prompts on a loop for five minutes: 43.2 tok/s at the start, 43.1 at the end, GPU 73-75 C, no throttling, 0.70 J per token
- Real OpenCode sessions: 162 requests, context up to 371K; median decode 44 tok/s under 64K, 38 at 128-256K
- IQ3_XXS (44 GiB, better answers): 35.2 tok/s at 4K, 31.7 at 259K
Comparisons
- Against the author's Splash port of Qwen 3.8 27B: at 128K+ in OpenCode, 14 tok/s vs 38, and prefill 4-7x faster
- For scale, a DGX Spark with the popular vLLM NVFP4 + MTP recipe does 36.8 tok/s on prose, 45.8 on code
- Bonus: DeepSeek V4 Flash (81 GiB) on the same 64 GB Mac, streamed from the SSD, does 12 tok/s at 4K and 9 at 127K
Limits
The fork is unofficial; Q2_0 loses about 6 points on code versus BF16 in ISTA's evals (81.1 vs 87.4 LiveCodeBench v6, 89.1 vs 93.1 overall), and Flash Next still needs a 64 GB Mac.
Reported anonymously by an r/LocalLLM contributor · 1 upvotes at capture
Leave a comment