Improve RAG for Your LLM Knowledge using InfraNodus and Open-WebUI, Dify, or ChatGPT

RAG, or Retrieval-Augmented Generation, is used to add additional context to AI user queries. However, one of its main problems is that it doesn't have an overview of the whole context. This is where InfraNodus can be used to improve the quality of responses you get with RAG.

Using InfraNodus, you can get an overview of the knowledge base, PDF files, or text data that you are using for your RAG system, which will give you a general overview of the main topics, the structure, and the gaps. 

You can then export the analytics generated by InfraNodus, such as the main topics, concepts, relations, and topical connectors, and add this data to your prompts to help RAG extract better content. This is especially good for general queries, such as "What is this about?", which make it very difficult to extract similar statements from your knowledge base using the standard RAG methods.

You can use this augmented RAG data in open-source applications like Open-WebUI, Dify, AnythingLLM or to augment your prompts in commercial products, such as ChatGPT and NotebookLM.

 

How to Improve Retrieval Augmented Generation with Topical Context

The main principle of a RAG system is to encode a user query as a vector (a sequence of numbers) and then find the closest vectors in the knowledge base (context), which is a collection of text chunks that are also encoded with vectors.

This approach works well for very specific user queries. For instance, for this support portal we can ask "How to import a CSV file in InfraNodus" and RAG will find all the chunks that are related to "CSV" and "InfraNodus" and provide the results. 

However, as soon as the user asks something _about_ the knowledge, for example, "What can I do with this?" or "How I can improve my workflow?" (assuming the context), the quality of RAG results will deteriorate. The reason is that a standard RAG system will just find chunks that relate to "improve" or "this", but that is too general. That's where InfraNodus-generated data comes in.

Using the InfraNodus graph AI prompt generator, we can add the extra metadata about our knowledge base to the prompt so that our RAG can do a better job retrieving data. Moreover, our LLM can use this extra information to improve the quality of its responses and make sure that every important topic is covered in an interaction with a user.

Take a look at these examples:

1. Standard RAG: A chatbot agent for this portal without the InfraNodus data to augment the RAG (try the bot here live) — provides an OK response, but doesn't cover everything that InfraNodus can do and focuses on a specific aspect of the tool's functionality:

 

2. InfraNodus-augmented RAG prompt: A chatbot agent for this portal using the InfraNodus topical data to augment the RAG (try the bot live here) — you will see that it augments the responses with the general contextual data we extracted from this knowledge base, so the responses are much more structured and precise:

 

Augment Your RAG Knowledge: Step-by-Step Explanation

In order to achieve these results on your own data, you can follow these steps:

 

1) Find the files or data you'd like to interact using AI with:

• This can be a collection of PDFs or .MD files (e.g. from Obsidian)
• You can also import a website using a tool like Firecrawl (has a cloud and open-source version) — it has a good integration with Dify open-source version. Alternatively, you can download your website crawl results as MD files. 

• You can also get any data through InfraNodus import (e.g. Google search results, websites, YouTube, X posts, RSS feeds, frequently searched terms, etc) — you can then export the chunks directly as a text file to be processed by a knowledge base of your choice

2) Visualize and run a graph analysis of the data with InfraNodus. You can then save the insights to the Project Notes or use the Generate from Analytics button to extract it automatically:

You can also extract the different data types step by step:

a) Analytics > Main Ideas > Expand > Save to Notes
This is useful to give the model a general context of the main concepts contained in the knowledge base and to ensure it will cover those especially on general requests.

- Analytics > Main Ideas > Most Influential Concepts > Save to Notes
Helps the model to understand what the context is mainly about

- Analytics > Blind Spots > Conceptual Gateways > Save to Notes
These are the terms that connect different topics or that can be used to link to another discourse (e.g. for interoperability requests)

- Analytics > Relations > Bigrams > Save to Notes
Main conceptual relations

Optionally, you can also save the Blind Spots > Structural Gap data (one by one for every gap or by selecting the topics you specify) — useful for exploring how the knowledge base can be improved based on what's missing (or for NotebookLM)

We also recommend removing the top layer of nodes to reveal underlying ideas (latent topics).

 

2a) Optional: you can also extract this data automatically using the InfraNodus API and integrate it into your AI workflow if it allows connecting to external resources. In this case, you'd use the endpoints described in our API tutorial on augmenting LLM prompts here.

 

3) Choose a tool you want to use to interact with your knowledge and add your files

• You can use the AI chat inside InfraNodus itself
• For ChatGPT, you can open a new workspace, upload the files there
• For Open WebUI, go to the Workspace > Knowledge and upload the files in a new collection (make sure you have the embedding rules set correctly in the Admin > Documents panel)
• For Dify, go to Knowledge and create a new Knowledge Base, upload the files there or use the Firecrawl API to ingest a website
• For AnythingLLM or LlamaIndex, upload your files to your workspace
• For NotebookLM, upload your files into the new notepad

Here's an example of some knowledge bases uploaded to Open-WebUI:

 

4) Augment your prompt with the InfraNodus data

Now you need to augment your model prompt with the data you generated with InfraNodus. In order to do that, you need to select the part of your application where you can customize the prompt and add a special instruction that will be added to the user query. 

• In ChatGPT: Workspace > Add Instructions
• In Open-WebUI: Open chat > click Settings at the top right > then System Prompt
• In Dify: create a chatbot, add the knowledge base, then in the custom instructions add your prompt
• In AnythingLLM — add a custom prompt to your workspace
• In NotebookLM —  Create a new notebook > Upload your files > go to Studio (right) > Customize

 

Here is an example of the prompt you can use. You will need to replace the data inside <tags> with what you generated in InfraNodus:

 

You are an expert in InfraNodus, and your task is to provide an answer to the user question based on the context you are provided about using InfraNodus and its tool.

## IMPORTANT INSTRUCTIONS, FOLLOW WHEN ANSWERING:

When you provide a response, use the following meta information about the underlying knowledge base to have a high-level overview of the content and to provide higher quality responses that integrate this content, in cases when relevant to the query (or when the query is too general to find results):

- the main topics in the context (listed inside the <MainTopics></MainTopics> XML tag) — you should focus on them especially
- relations between the concepts (listed inside <Relations></Relations> and <MainConcepts></MainConcepts> XML tag) — these are the most important ideas and relations in the context
- latent topics inside (listed inside <LatentTopics></LatentTopics> - these are non-obvious topics that you can take into account
- conceptual entry points (listed inside <ConceptualGateways></ConceptualGateways> XML tag) — use them to connect topics or to connect the knowledge to the outside world

<MainTopics>
1. Graph Dynamics: graph topic node word click setting show knowledge 
2. AI Workflow: infranodus text network ai data analysis workflow base 
3. Idea Exploration: idea generate gap research insight discourse develop interesting 
4. Search Context: search context related google keyword result import market 
5. Node Relations: occur sharing information retrieve 
6. Text Analysis: main content relevant extract reveal create skip 
</MainTopics>

<Relations>
structural gap
network analysis
topical cluster
</Relations>

<LatentTopics>
1. Conceptual Navigation: top concept cluster click menu setting statement analytics 
2. Data Integration: search data research keyword import step google result 
3. Insight Discovery: generate gap insight interesting find identify question specific 
4. Network Strategy: network analysis use gpt base tool knowledge visualization 
5. Analytical Display: discourse context structure analyze market connection term develop 
6. Research Import: content main create extract reveal skip 
7. Gap Identification: workflow 🎥 thinking case study start exploration tutorial 
8. Knowledge Exploration: related article book product conversation original 
9. Contextual Analysis: occur information sharing retrieve 
</LatentTopics>

<ConceptualGateways>
retrieve, occur, influence, recommendation, node, search, setting, word, text, network, click, case, show
</ConceptualGateways>


## CONTEXT INFORMATION:

Use the following context as your learned knowledge, inside <context></context> XML tags.
<context>
{{#context#}}
</context>

Do not hallucinate answers, ONLY use the context provided to generate responses, and provide references to all the context elements you use for your response in inline links with links to the URLs where these results can be found .

If you didn't use any of the context documents to generate your response but generated it from your own knowledge, inform the user about it at the beginning.

Keep your answers VERY SHORT and concise. Only the minimum necessary.

 

Enhancing Original User Prompt

Sometimes users will ask very general questions, which will be insufficient to perform a good quality RAG search on your data. So you might want to enhance the user query itself with the InfraNodus data to make it more relevant to the context you query.

This is how you can do that:

1) create a chatbot flow and

2) add another step between the user query and Knowledge Base retrieval where you ask the model to regenerate user query taking into account the additional context that you provided. Then you will be affecting the RAG itself and not just adding the context to the results provided:

 

 

Here's an example of a prompt you can use to enhance user query:

You receive a user query relating to InfraNodus text network analysis tool. 


If the user query is specific, you just pass it on without changing anything. 


However, if it's too general or not too precise, you need to make it more specific by adding the following context to it. 


IMPORTANT: Make sure the final query is not much longer than the original one. Just add only the most important necessary information. 

## CONTEXT TO ADD

### Context Explanation: 

- the main topics in the context (listed inside the <MainTopics></MainTopics> XML tag) — you should focus on them especially

- relations between the concepts (listed inside <Relations></Relations> and <MainConcepts></MainConcepts> XML tag) — these are the most important ideas and relations in the context

- latent topics inside (listed inside <LatentTopics></LatentTopics> - these are non-obvious topics that you can take into account

- conceptual entry points (listed inside <ConceptualGateways></ConceptualGateways> XML tag) — use them to connect topics or to connect the knowledge to the outside world


### Context Details:

<MainTopics>

1. Graph Dynamics: graph topic node word click setting show knowledge 

2. AI Workflow: infranodus text network ai data analysis workflow base 

3. Idea Exploration: idea generate gap research insight discourse develop interesting 

4. Search Context: search context related google keyword result import market 

5. Node Relations: occur sharing information retrieve 

6. Text Analysis: main content relevant extract reveal create skip 

</MainTopics>


<Relations>

structural gap

network analysis

topical cluster

</Relations>


<LatentTopics>

1. Conceptual Navigation: top concept cluster click menu setting statement analytics 

2. Data Integration: search data research keyword import step google result 

3. Insight Discovery: generate gap insight interesting find identify question specific 

4. Network Strategy: network analysis use gpt base tool knowledge visualization 

5. Analytical Display: discourse context structure analyze market connection term develop 

6. Research Import: content main create extract reveal skip 

7. Gap Identification: workflow 🎥 thinking case study start exploration tutorial 

8. Knowledge Exploration: related article book product conversation original 

9. Contextual Analysis: occur information sharing retrieve 

</LatentTopics>


<ConceptualGateways>

retrieve, occur, influence, recommendation, node, search, setting, word, text, network, click, case, show

</ConceptualGateways>

Everything inside the <tags></tags> can be obtained from the InfraNodus graph (Project Notes > Auto Generate for RAG) or via our API endpoint.

5) Launch your AI chatbot conversation and try out the responses!

 

Improve Retrieval Speed for Large Datasets in Real Time

In our example above, we are using this whole support portal as the knowledge base but we added  the contextual information to the prompts manually, so it works very fast.

However, if you want to retrieve this information in the real time (e.g. to reflect the updates), you can get it directly from InfraNodus via the API. 

In that case, however, the first time the chatbot runs it will take it a few seconds to generate this information. In order to avoid it happen every time the user sends a query, we add the Variable Assigner node in Dify, assign the output of the InfraNodus Knowledge Base Enhancer tool to it, and then add IF / THEN node that will check if the value InfraNodusGraph is empty. If it's not empty, the workflow will go straight to the LLM Prompt Augmentor. 

 

Here is an example of how this workflow would look like in Dify:

The flow here is:

1) User starts interaction

2) If / then operator sees if the InfraNodus variable is empty

3) If it is empty, make the InfraNodus Enhancer (graph information) request (as above) to retrieve graph insights on the original context

4) Save it into the variable assigner node (so it can be reused in the If / Then condition)

5) Use the output of InfraNodus (or the variable assigner output) to create a prompt that would enrich the original user's query in case it's too general 

6) Run knowledge retrieval RAG to extract the most relevant statements

7) Use the output of the RAG search AND the original query AND the InfraNodus-generated insight about the material to generate another prompt that would answer the original query. 

8) Output the results

You can download a sample Dify workflow above here in our Dify / InfraNodus repo.

 

Feel the Difference!

Here's the chatbot built with this logic for this support portal with Dify:

https://aistudio.infranodus.com/chat/4lmXEc1G3uSB4Dj6

To compare, you can also launch the chatbot that does not have the Infranodus graph insights implemented and uses only the native Dify knowledge base retrieval. You will see it's much worse, especially for general queries:

https://aistudio.infranodus.com/chat/GqRW02nVA61qSccW

 

...

 

We hope you find this tutorial useful. If you have any questions, please, contact us directly or via Discord. Our support specialists can also assist you with setting up your own RAG using InfraNodus and open-source tools (available to Premium subscribers).