Most of this person's local work is OCR: turning PDFs full of equations and tables into files they can edit. On a 9950X3D workstation with 96GB of DDR5-6600 and an RTX 5070 Ti, they run Qwen vision models through llama.cpp instead of a small model like Gemma 4 12B, because Gemma regularly misreads small numbers inside equations while the Qwen vision models are more accurate on math and produce LaTeX that survives a real document. They keep two llama-server configurations: a mainline build running Qwen 3.8 27B at a 20K context with an 8-bit KV cache, reporting about 1,000-1,100 prefill and 50-52 generation tokens per second, and an Unsloth fork running a rebuilt Qwen 3.8 Flash Next setup with MTP speculative decoding at a 50K context, reporting 100-120 prefill and 29-32 generation tokens per second. The speed is secondary to the pipeline around it: a VBS script calls a Python script that drives LibreOffice and a prompt tuned so LaTeX resolves correctly in both Markdown and Word, letting most document types, including image-only PDFs, come out as editable Markdown and .docx with formulas intact. For easier source material they use a faster hybrid OCR path and keep the vision model as a safety net.
Reported anonymously by an r/LocalLLM contributor · 1 upvotes at capture
Is this your setup? Claim it to keep the snapshot current