Knowledge base (KB), RAG (retrieval-augmented generation) processing

RAG processing: Generating content derived from multiple sources

We are currently exploring the role of local knowledge bases (KBs) in RAG (retrieval-augmented generation) AI processing. This post is part of a series documenting our “sandbox” knowledge bases (created over a period of about 20 years) and how we’re using them in various GenAI prototyping projects.

Creating a new, temporary knowledge base from multiple sources

In our last experiment (described by a post in this space), we used LLaVA to create descriptions of the images included in the already-existing subset of our “Computer history” knowledge base.

Original and subset of the "Computer history" knowledge base
Original and subset of the “Computer history” knowledge base
LLaVA-generated description of one of the images in the knowledge base
LLaVA-generated description of one of the images in the knowledge base

In our most recent experiment we combined all our “Computer history” subset information into a new, temporary knowledge base containing the elements indicated above.

Objective of our current experiment

We asked our large language model (LLM)  (ChatGPT-3.5 Turbo), running in a Colab notebook, to summarize the new information into a short post, possibly suitable for inclusion on a website. We wanted the summarization to include the following:

  1. A little background about Dick’s career in high-tech
  2. Dick’s professional relationship with a colleague (Dr. Charles Fillerup)
  3. Dr. Fillerup’s experience with the IBM 650 (a vintage computer from the 1950s)
  4. A short description of the IBM 650  (a magnetic drum computer)
Dr. Charles (Chuck) Fillerup in his later years
Dr. Charles (Chuck) Fillerup in his later years

Generating content from the new combined knowledge base

In our system prompt, we asked the model not to make up anything included in the summary: to use only the supplied input plus additional knowledge derived from its general training.

As an experiment we asked the model if it could look up additional information on the internet, but it responded that it was not able to do that. So that’s something to possibly come back to in a future experiment!

Here is the query/request we gave to our LLM during a chat session:

“Please summarize the input into a post-size document that I could put on a website. Include a little background on Dick Johnson and his career history, something about his relationship with Dr. Charles Fillerup, and Dr. Fillerup’s experiences with the IBM 650. Also include a short description of the IBM 650 itself.”

Results and takeaways

Output text generated by the model
Output text generated by the model

We were generally very pleased with the result:

  • The output post was well organized.
  • The language was straightforward and to the point.
  • The model described the human relationships well, and fielded the shift in Dr. Fillerup’s given name and nickname (Charles/Chuck).
  • The model never hallucinated.
  • The system prompt seemed to have been appropriate and effective.
  • The reference to “more information” on our website was a nice touch.

We thought a couple of words were a little off:

  • “the importance of efficient program execution” would have been better as “program design”
  • “program intricacies” would have been better as “programming best practices”

For more information

Python program that generated the image descriptions using LLaVA:

LLaVA_describe_images (PDF)

Python program that generated the computer history post/text:

RAG_Processing_GeneratingNewContentAboutComputerHistory.ipynb – Colab (PDF)

Link to the Colab notebook:

https://colab.research.google.com/drive/1r0I5lb5SiAmETdQ98dQMKWXk2zre05V3?usp=sharing