Knowledge base (KB), RAG (retrieval-augmented generation) processing

RAG processing: Small language model (SLM) with chat

We are currently exploring the role of local knowledge bases (KBs) in RAG (retrieval-augmented generation) AI processing. This post is part of a series documenting our “sandbox” knowledge bases (created over a period of about 20 years) and how we’re using them in various GenAI prototyping projects.

Overview of the RAG processing pipeline

In case you’re new to RAG processing, here is one of our favorite overview diagrams from Codingscape.

RAG processing pipeline from Codingscape, via Perplexity
RAG processing pipeline from Codingscape, via Perplexity

Our RAG processing prototype projects

Our most recent RAG prototype project is part of a series of three Python programs involving a small language model (SLM) and our “grocery shopping” knowledge base, described in detail in other posts, pages, and widgets on this website.

This post is #2 in the set. Here is a link to #1, which had a single supplied query:

RAG processing: Small language model (SLM) with single query

RAG processing prototype #2: Chat

Our Python program #2, part of which is shown below,  establishes a chat with a user. There is a link to a PDF version of the whole program in the “For more information” section, below.

Prolog section of our RAG chat program
Prolog section of our RAG chat program

The chat loop in program #2

Here is the loop function that handles the chat.

Loop function that handles the chat
Loop function that handles the chat

Our RAG knowledge base: “Shopping for groceries”

Our RAG knowledge base (KB) is “Shopping for groceries”: the smallest and simplest in our DITA/xml KB set.  One of the questions we asked our chatbot was the price of Fuji apples. Here is the topic that provides the answer to that question.

"Fruits and vegetables" topic in "Shopping for Groceries"
“Fruits and vegetables” topic in “Shopping for Groceries”

Sample output from program #2

Here is the output from one of our test runs of the program.

Output from the "chat" version of our Python program
Output from the “chat” version of our Python program

It’s always fun to see how the bot answers questions. It’s often possible to improve the bot’s performance by adding a well-crafted system prompt, which is what we are doing in version #3 of the program!

For more information

Here is a link to a PDF version of our RAG Python program #2.

RAG Small Language Model Chat Python Program (PDF)

What’s next?

We are working on version #3 of our RAG SLM program, and we’ll be posting on that soon. It adds a system prompt that helps direct the attention and refine the response of the bot.