RAG and schema classification on data that must not leave the network

This person uses local LLMs at work for data that cannot, or should not, be transmitted over the internet, finding them more than capable for specific bounded tasks. Two examples stand out: a RAG chatbot that answers questions about a document or set of documents, and classification of written material against specific schemas, such as routing tickets, extracting claims from bids, or flagging stale wording.

Frontier models are still used to design and code the systems around these local models, because they do not see the underlying data and produce higher-quality code faster. Once each product is live, only open-source local models do the work.

Correct or update this entry

Is this your setup? Claim it to keep the snapshot current