This person is not an IT engineer and runs local models on two separate systems, one with 8GB of VRAM and one with 16GB, adopting the small models that feel usable and accurate enough. Their main workflows are extracting text from DJVU and PDF technical books and papers to maintain a knowledge base, including using the same workflow to translate books; organising markdown notes into an indexed, linked structure; and generating detailed notes on old codebases.
The codebase documentation is their favourite, because writing in Python they can now produce notes and blueprints for code written in C, C++, MATLAB or Fortran without first learning the language. Their advice is not to judge local models against frontier ones, but to break daily routine work into small chunks and identify which parts can be offloaded locally.
Reported anonymously by an r/LocalLLM contributor · score 3