Answering questions about internal documents without sending them anywhere
Start here. What it costs to run retrieval and a language model on hardware you control, measured on a four core laptop, and where I would stop recommending it.
Building a private RAG system
Notes from building and measuring a retrieval augmented generation system that runs entirely on a four core laptop, with nothing leaving the machine. The posts are in reading order, and each one stands on its own, so starting anywhere is fine. Standalone notes on other topics follow after the series.
What a vector actually is, and why my search box is a piece of geometry
The retrieval half of the system: 384 numbers per passage, why cosine rather than dot product, and the first measured results including the ones it got wrong.
My container was healthy two and a half minutes before it could answer
A Docker health check that passed in ten seconds while the model behind it was still loading, and what a check has to prove to be worth having.
My model answered the question, then asked itself four more
A correct answer followed by ninety blank lines, two fixes that changed nothing, and the number in the first response that would have told me why.
I added a sentence to a passage and the vector did not change
A passage and the same passage with an extra sentence appended embedded to exactly the same vector, what that says about chunk size, and what ingesting 1798 pages actually costs.
Azure foundations, the words I had to get straight first
One page, fourteen terms that everything else in Azure is built on, and the one concept with no AWS counterpart.
Azure networking, the parts I keep having to re-check
One page, eleven Azure networking building blocks, and for each one the point that usually catches people. A standalone reference, not part of the RAG series.