混合RAG：结合了GraphRAG和RAG以利用LLMs进行检索

用LangChain解释带有示例的代码

RAG 在生成式人工智能领域是一项重大的突破，使人们能够连接自己的个人文档，如文本文件、PDF、视频等，并与LLMs 进行交流。最近推出了RAG 的升级版 GraphRAG，它使用知识图来执行使用LLMs 的 RAG 检索。

注意：我已经在之前的帖子和YouTube视频中详细介绍了RAG和GraphRAG。因此，现在暂时跳过它。

RAG和GraphRAG都有它们的优势和局限性。RAG利用向量相似性的能力，而GraphRAG使用图分析和知识图谱来得出更清晰的答案。

如果两者可以一起用于检索呢？

混合RAG

HybridRAG是一个高级框架，融合了RAG和GraphRAG。这种整合旨在提高信息检索的准确性和上下文相关性。简单来说，HybridRAG使用来自检索系统（RAG和GraphRAG）的上下文，最终输出是两个系统的混合。

HybridRAG的优势

提高准确性：HybridRAG同时利用结构化推理和灵活检索，相比单独使用VectorRAG或GraphRAG，能够提供更精确的答案。
增强环境理解：这种组合使得对实体之间关系和它们出现的环境有更深入的理解。
动态推理：知识图可以动态更新，使系统能够根据新信息进行适应。

所以，让我们尝试使用LangChain构建一个HybridRAG系统

我将在这个演示中使用一个名为Moon.txt的文件，这是一个超级英雄故事。请查看以下内容。

In the bustling city of Lunaris, where the streets sparkled with neon lights and the moon hung low in the sky, lived an unassuming young man named Max. By day, he was a mild-mannered astronomer, spending his hours studying the stars and dreaming of adventures beyond Earth. But as the sun dipped below the horizon, Max transformed into something extraordinary—Moon Man, the guardian of the night sky.
Max’s transformation began with a mysterious encounter. One fateful evening, while gazing through his telescope, a brilliant flash of light erupted from the moon. A celestial being, shimmering with silver light, descended and bestowed upon him a magical amulet. “With this, you shall harness the power of the moon,” the being declared. “Use it wisely, for the night sky needs a hero.”
With the amulet around his neck, Max felt energy coursing through him. He could leap great distances, manipulate moonlight, and even communicate with nocturnal creatures. He vowed to protect his city from the shadows that lurked in the night.
As Moon Man, Max donned a sleek, silver suit adorned with celestial patterns that glimmered like the stars. With his newfound abilities, he patrolled the city, rescuing lost pets, helping stranded motorists, and even thwarting petty criminals. The citizens of Lunaris began to whisper tales of their mysterious hero, who appeared under the glow of the moon.
One night, as he soared through the sky, he encountered a gang of thieves attempting to steal a priceless artifact from the Lunaris Museum. With a flick of his wrist, he summoned a beam of moonlight that blinded the thieves, allowing him to swoop in and apprehend them. The city erupted in cheers, and Moon Man became a beloved figure.
However, peace in Lunaris was short-lived. A dark force emerged from the depths of the cosmos—an evil sorceress named Umbra, who sought to extinguish the moon’s light and plunge the world into eternal darkness. With her army of shadow creatures, she began to wreak havoc, stealing the moon’s energy and spreading fear among the citizens.
Moon Man knew he had to confront this new threat. He gathered his courage and sought the wisdom of the celestial being who had granted him his powers. “To defeat Umbra, you must harness the full power of the moon,” the being advised. “Only then can you restore balance to the night sky.”
With determination in his heart, Moon Man prepared for the ultimate battle. He climbed to the highest peak in Lunaris, where the moon shone brightest, and focused on channeling its energy. As Umbra and her shadow creatures descended upon the city, Moon Man unleashed a magnificent wave of moonlight, illuminating the darkness.
The battle raged on, with Umbra conjuring storms of shadows and Moon Man countering with beams of silver light. The clash of powers lit up the night sky, creating a dazzling display that captivated the citizens below. In a final, desperate move, Moon Man summoned all his strength and unleashed a powerful blast of moonlight that enveloped Umbra, banishing her to the farthest reaches of the cosmos.
With Umbra defeated, the moon’s light returned to its full glory, and the city of Lunaris rejoiced. Max, still in his Moon Man guise, stood atop the highest building, watching as the citizens celebrated their hero. They had learned the importance of hope and courage, even in the darkest of times.
From that day forward, Moon Man became a symbol of resilience and bravery. Max continued to protect Lunaris, knowing that as long as the moon shone brightly, he would always be there to guard the night sky. And so, the legend of Moon Man lived on, inspiring generations to look up at the stars and believe in the extraordinary.
As the years passed, stories of Moon Man spread beyond Lunaris, becoming a beacon of hope for those who felt lost in the darkness. Children would gaze at the moon, dreaming of adventures, and Max would smile, knowing that he had made a difference. For in the heart of every dreamer, the spirit of Moon Man lived on, reminding them that even the smallest light can shine brightly against the shadows.

导入包并设置您的LLM端嵌入模型（用于标准RAG）

import os
from langchain_experimental.graph_transformers import LLMGraphTransformer
from langchain_core.documents import Document
from langchain_community.graphs.networkx_graph import NetworkxEntityGraph
from langchain.chains import GraphQAChain
from langchain.text_splitter import CharacterTextSplitter
from langchain.document_loaders import TextLoader
from langchain.chains import RetrievalQA
from langchain.vectorstores import Chroma
from langchain_google_genai import GoogleGenerativeAI,GoogleGenerativeAIEmbeddings

GOOGLE_API_KEY=''

embeddings = GoogleGenerativeAIEmbeddings(model="models/embedding-001",google_api_key=GOOGLE_API_KEY)
llm = GoogleGenerativeAI(model="gemini-pro",google_api_key=GOOGLE_API_KEY)

2. 接下来，我们将为文件“Moon.txt”上的GraphRAG实现（链对象）创建一个函数。

def graphrag():
        with open('Moon.txt', 'r') as file:
            content = file.read()

        documents = [Document(page_content=content)]
        llm_transformer = LLMGraphTransformer(llm=llm)
        graph_documents = llm_transformer.convert_to_graph_documents(documents)

        graph = NetworkxEntityGraph()

        # Add nodes to the graph
        for node in graph_documents[0].nodes:
            graph.add_node(node.id)

        # Add edges to the graph
        for edge in graph_documents[0].relationships:
            graph._graph.add_edge(
                    edge.source.id,
                    edge.target.id,
                    relation=edge.type,
                )

            graph._graph.add_edge(
                    edge.target.id,
                    edge.source.id,
                    relation=edge.type+" by",
                )

        chain = GraphQAChain.from_llm(
            llm=llm, 
            graph=graph, 
            verbose=True
        )
        
        return chain

3. 同样地，我们将创建一个函数来在同一文件上实现标准的RAG。

def rag():
        #Document Loader
        loader = TextLoader('Moon.txt')
        data = loader.load()

        #Document Transformer
        text_splitter = CharacterTextSplitter(chunk_size=1000, chunk_overlap=0)
        texts = text_splitter.split_documents(data)

        #Vector DB
        docsearch = Chroma.from_documents(texts, embeddings)

        #Hyperparameters to know
        retriever = docsearch.as_retriever(search_type='similarity_score_threshold',search_kwargs={"k": 7,"score_threshold":0.3})
        qa = RetrievalQA.from_chain_type(llm=llm, chain_type="stuff", retriever=retriever)
        
        return qa

4. 创建用于两种类型的 RAG 的对象

standard_rag = rag()
graph_rag = graphrag()

5. 是时候实施HybridRAG.

def hybrid_rag(query,standard_rag,graph_rag):
    result1 = standard_rag.run(query)
    
    print("Standard RAG:",result1)
    result2 = graph_rag.run(query)
    
    
    print("Graph RAG:",result2)
    prompt = "Generate a final answer using the two context given : Context 1: {} \n Context 2: {} \n Question: {}".format(result1,result2,query)
    return llm(prompt)

query = "Some characteristics of Moon Man"
hybrid = hybrid_rag(query,standard_rag,graph_rag)
print("Hybrid:",hybrid)

正如你所看到的，

我们分别对所给提示运行了标准的RAG和GraphRAG。
一旦答案被检索出来，最终答案是根据这两个回答的上下文生成的。

简单

谈论输出结果，最终的HybridRAG确实从检索结果中获取了信息，得出了更好的结果。有一些观点被两个RAG系统遗漏了，但最终HybridRAG结合起来给出了完美的答案。

STANDARD RAG: 
 Here are some characteristics of Moon Man, based on the story:

* **Brave:** He confronts danger and fights villains like Umbra.
* **Powerful:** He has superhuman abilities granted by the amulet.
* **Protective:** He safeguards Lunaris and its citizens.
* **Determined:** He doesn't give up, even when facing powerful enemies.
* **Compassionate:** He helps those in need, like rescuing lost pets.
* **Humble:** Despite his powers, he remains grounded and dedicated to his city. 



> Entering new GraphQAChain chain...
Entities Extracted:
Moon Man

Full Context:
Moon Man PROTECTS night sky
Moon Man WEARS silver suit
Moon Man PROTECTED Lunaris
Moon Man CAPTURED thieves
Moon Man DEFEATED Umbra
Moon Man INSPIRES hope
Moon Man INSPIRES courage

> Finished chain.

 @@ 
 GRAPH RAG: 
 Helpful Answer: 
* Protective (protects night sky, protected Lunaris)
* Courageous and Inspiring (inspires hope, inspires courage)
* Strong (captured thieves, defeated Umbra) 


 @@ 
 HYBRID RAG: 
 Moon Man is the **protective** champion of Lunaris, using his **strength** and **courage** to defend its citizens and the night sky. He is **powerful** and **determined**, facing down villains like Umbra without giving up. Yet, despite his abilities, he remains **humble** and **compassionate**, always willing to help those in need. Moon Man is a true inspiration, reminding everyone that even in darkness, hope and heroism can shine through.