We are currently exploring the role of local knowledge bases (KBs) in RAG (retrieval-augmented generation) AI processing. This post is part of a series documenting our “sandbox” knowledge bases (created over a period of about 20 years) and how we’re using them in various GenAI prototyping projects.

What are local knowledge bases (KBs)?

Local knowledge bases play a crucial role in RAG processing by providing a curated source of information that can be efficiently searched and retrieved to augment the outputs of large language models (LLMs).

In RAG systems, the local knowledge base serves as an external repository of up-to-date, domain-specific information that the LLM can draw on. The knowledge base owner is able to keep the information private and local, if desired.

The seven knowledge bases we’re using for RAG prototyping

Although some of our knowledge bases required many files to produce, the published versions that we share with our models are simple and straightforward.

Here is one of our simplest local knowledge bases: a single PDF file for “Cleaning the Garage.”

We try to be sure our KBs have the following characteristics:

• Simple and small, which is best for prototyping purposes
• Contain information generally understood by almost any target audience (or, at the least, the key stakeholders in your organization)
• Contain specific information that is already known to (and likely written by) us, so we can better formulate prompts and queries to the model (both system and real-time) and understand its responses
• Multimodal, if possible, which allows us to challenge the technical capabilities of various models

A multimodal example that we are currently working with  is “Computer History,” shown in the following image.

Although the computer history KB is pretty straightforward, the RAG processing prototype that used it was not running as quickly as we would have liked.

Since we know that only four of the files are required for our next RAG project, we decided to subset the KB for now. When we’re sure it’s doing what we want and expect, we’ll add more files and expand the scope.

This and other experiences with RAG prototyping have taught us that it’s better to start small and increase the load and scope slowly after meaningful experimentation with prompts and queries. Prompt engineering is an inherently iterative process, and having a simple KB makes it easier to evaluate the effectiveness of the prompts and test queries in guiding the model’s responses.

It takes more time than you might think to get on proper “speaking terms” with your model, but the effort pays great dividends in the long run!