Building a personal academic RAG pipeline on a 16GB 5060 Ti

This researcher works in applied mathematics and has a Ryzen 7 AM5 machine with an RTX 5060 Ti 16GB. They collected every PDF on their machine into one deduplicated folder, ending with about 20,000 files and 30GB, and estimate a third are academic papers worth processing. They are building the pipeline in Python, on a 50-file test set.

For PDF preprocessing they compared tools: Grobid handles bibliography well but needs retries and drops tables, formulas and images, which matters for their field; Chandra is accurate and even describes images but takes over 30 seconds per page and mangles section boundaries; Marker-PDF is fast in batches but slow per file and also weak on images. Docling is next on the list. For chunking they found recursive structural splitting fast but imperfect, and plan chunks under 128 tokens for embeddings but 1-2K tokens for summarisation, with a bottom-up hierarchy from chunk to subsection to section. Small models such as Gemma 4 12B and Phi 4 summarise at around 50 tok/s, while Qwen 3.8 27B does not fit the 16GB card and drops to roughly 4 tok/s, so aggressive quants hurt quality. Embedding and cross-document relation search are still to come.

Correct or update this entry

Is this your setup? Claim it to keep the snapshot current