Prompt Augmentation for LLM RAG

InfraNodus API can be used for prompt augmentation in your LLM RAG workflows. Normally, when a user sends a prompt to an LLM it will go straight to the knowledge base and retrieve the responses based on vector similarity search (what is referred to as retrieval augmented generation or RAG). However, the problem is that if the user's request is too general, LLM won't know the context of the query, so it might deliver irrelevant results.

That's where InfraNodus can be very helpful. You can connect your AI workflow to its graph summarization endpoint, which will analyze the knowledge base, retrieve its underlying graph structure, the main topics and concepts inside, which can then be used to augment the original prompt with contextual data. 

There are two ways to augment your prompts using the InfraNodus API:

1. Using the Reprompt Request (better if you have an existing knowledge graph in InfraNodus that you want to use as a reasoning expert)

Use the `graphAndAdvice` API endpoint with the `requestMode=reprompt` parameter — in this case, InfraNodus will reformulate your original user's prompt based on the underlying graph structure (using GraphRAG to connect different concepts in your existing graph or in the text you send with the request). We recommend to use the graphs you already saved in InfraNodus for that, but you can also send new text (it will just take longer in that case).

After that,, you can feed the InfraNodus' augmented prompt directly into your LLM workflow for further processing.

2. Using the Graph Summary Response

In this scenario, you make a request to the InfraNodus `graphAndStatements` API endpoint which will process your original text (or existing graph) and generate insights about the underlying graph structure (main clusters, gaps, latent concepts, etc). You can then feed this data into your existing LLM workflow, add it into your prompts, and ask your LLM to improve the original prompt using the additional graph context provided. 

Here is how this second scenario works in detail:

For instance, if we use this support portal as the knowledge base and the user asks "What is it good for?", the standard RAG response won't deliver good results. However, if the prompt is augmented with the structural data about the main topics and concepts of the support articles on this portal, then it will can be rephrased as "What can InfraNodus be useful for, particularly in relation to text analysis and finding content gaps?". Then the results retrieved from the knowledge base will be much more relevant and the user will get better responses. 

In the example above, we demonstrate how it can be used with Dify — a popular open-source AI workflow building tool. Practically, it's like adding a portable GraphRAG to your LLM queries.

Below is the description of the main steps:

Step 1: Add a knowledge base to InfraNodus and Dify

First, you need to add a knowledge base to InfraNodus and visualize it as a graph. It will then show the main topics and concepts inside. You can "clean" and optimize the graph to make sure that the responses are focused on the topics that you consider important.

You should also add this same knowledge base to Dify, so you can query it using the knowledge base retrieval node in Dify.

Step 2. Create a Dify AI Chatbot Workflow

Now you can create a new AI chatbot workflow in Dify. You can find a sample workflow here as a YML file, which you can upload to Dify to replicate this workflow: https://github.com/infranodus/dify-infranodus/blob/main/enhance-knowledge-base.yml

The general scheme is:

- receive user query

- retrieve information about the underlying knowledge base from infranodus (graph summary stored in `graphSummary` response string.

- feed this graph summary to the llm for prompt augmentation

- send the augmented prompt to the knowledge base for retrieval

- deliver the results

You can try it in production at https://aistudio.infranodus.com/chat/4lmXEc1G3uSB4Dj6

It will query this very support portal with the augmented prompt, so you can try really general requests to stress test it.