Two single-GPU Ubuntu VMs each run Strata at 262K context:
- RTX 3090 (Dell R740, capped at 220W): Qwen 3.8 Flash Next at IQ3_S
- Tesla V100 32GB (blower-modded, EPYC server, capped at 180W): an uncensored Qwen 3.8 Flash Next at IQ4_XS
Reported numbers
- 3090: 60-65 tok/s decode, 1,200-1,300 tok/s prefill
- V100: 50-55 tok/s decode, 850-900 tok/s prefill
- Both drop to roughly 50-55 and 40-45 tok/s respectively on general tasks through Hermes Agent.
Why Flash Next
The workload is local business management: database and memory vaults, orchestrating processes and software, and Computer Use automation. Flash Next is preferred over Qwen 3.8 27B because it completes tasks around 5x faster at xhigh despite a slightly lower raw decode rate. An IQ3_XXS test on the V100 matched 3090 speed but triggered frequent reasoning loops, and the author notes the V100 needs Ubuntu and Strata modifications for Volta.
Reported anonymously by an r/LocalLLM contributor · 1 upvotes at capture
Leave a comment