IT Pro: What is Retrieval Augmented Generation (RAG)?

If AI is to become ever more ‘intelligent’ and provide useful answers, it must have greater amounts of contextual information to hand when answering users’ questions.

That’s where new approaches such as retrieval-augmented generation (RAG) are now coming to the fore, integrating into large language models (LLMs) – of which OpenAI’s ChatGPT is now one of the most popular and well-known.

With so many different AI tools currently on the market, Ryan Carr, chief technology officer (CTO) and VP of Engineering at Enveil, believes RAG stands out because of its ability to deliver “real, business-enabling value”.

He explains answers from LLMs alone aren’t always trustworthy, producing so-called “hallucinations”; what he describes as “confident but incorrect responses”. This means it becomes “risky” for businesses to base critical decisions entirely on AI-driven outcomes.

“RAG solves this problem by taking the user’s question or prompt, performing a semantic search for related ground-truth documents, and then feeding these documents to the LLM along with the user’s prompt,” Carr says.

“This allows the LLM to ‘cite its sources’ in the response, allowing users to verify and validate the LLM’s answer.”

Read full article here.