Rather than one big general model, this setup uses several small, fast ones for narrowly scoped, deterministic jobs — starting with OCR and metadata for a self-hosted Paperless document server.
DocumentsHardware unspecifiedModels unspecifiedllama.cpp
ChangedDescriptionHardwareModelsRuntime
Compare with revision 1
DescriptionPreviousThis person's philosophy is that local hardware is best spent on narrowly scoped, largely deterministic automations rather than open-ended chat, saving heavier reasoning for cloud models when it's actually needed. One piece of that is a fully local OCR and metadata-generation pipeline feeding RAG over their self-hosted Paperless document server. Day to day they lean on Gemma 4 12B for anything that needs to be fast, reserving Qwen 3.8 Flash Next for the few tasks — like planning database retrieval — that need more headroom, on a 12GB card they consider entirely sufficient for this.This snapshotThis person's philosophy is that local hardware is best spent on narrowly scoped, largely deterministic automations rather than open-ended chat, saving heavier reasoning for cloud models when it is actually needed. One example is a fully local OCR and metadata-generation pipeline feeding RAG over their self-hosted Paperless document server. They stay on the cheapest Claude plan to help write the workflows and avoid API costs.
HardwarePreviousPrimary machine: RTX 3080 TiThis snapshotNot specified
ModelsPreviousGemma 4 12BThis snapshotNot specified
RuntimePreviousNot specifiedThis snapshotllama.cpp
This is the currently published snapshot.
Leave a comment