Local LLM for PDF and cover letter building for sensitive docs
I am admittedly not super well-versed in AI or tech in general, and would be very grateful for any general guidance. I’ve done some of my own research but find it fairly disorienting.
I am looking to have an air-gapped, local LLM that can look at a number of PDF files, and build either a DOCX cover letter summarizing, or an Excel file summarizing assets as reflected in the PDFs. I would provide it with templates for the cover letter/Excel file. Ideally, I’d like it to rename and number PDFs to correspond with line items on the Excel file.
Each PDF would be roughly 1-6 pages. Each batch would have about 10-30 PDFs.
I don’t really need it to retain any info, just complete the deliverable and wait for the next batch. Speed is not terribly important either.
This is highly repeated work for me, takes a lot of time reading bank statements and entering the data. I would love to automate even a portion, but the high sensitivity of the docs makes me want to keep totally offline at least for now on an air gapped system. I can move files on and off the computer with a USB drive airdrop or something.
Would this be an AnythingLLM type of job? Ollama with LangChain? I really am pretty clueless. Would 32GB VRAM be enough? Again, speed isn’t too important, as it’s usually not time-sensitive for me.