结构化分层检索¶
在多个文档上做好 RAG 很难。一个通用的框架是，给定用户查询，首先选择相关文档，然后再选择文档内部的内容。
但选择文档可能很困难 - 我们如何根据不同的属性动态地选择文档，取决于用户查询？
在这个 notebook 中，我们将展示我们的多文档 RAG 架构
- 将每个文档表示为一个包含不同属性的简洁**元数据**字典：提取出的摘要以及结构化元数据。
- 将此元数据字典作为过滤器存储在向量数据库中。
- 给定用户查询，首先执行**自动检索** - 推断相关的语义查询和用于查询此数据的过滤器集合（有效地结合了文本转 SQL 和语义搜索）。
In []
已复制！
%pip install llama-index-readers-github
%pip install llama-index-vector-stores-weaviate
%pip install llama-index-llms-openai
%pip install llama-index-readers-github %pip install llama-index-vector-stores-weaviate %pip install llama-index-llms-openai
In []
已复制！
!pip install llama-index llama-hub
!pip install llama-index llama-hub
设置和下载数据¶
在本节中，我们将加载 LlamaIndex Github issue。
In []
已复制！
import nest_asyncio
nest_asyncio.apply()
import nest_asyncio nest_asyncio.apply()
In []
已复制！
import os
os.environ["GITHUB_TOKEN"] = "ghp_..."
os.environ["OPENAI_API_KEY"] = "sk-..."
import os os.environ["GITHUB_TOKEN"] = "ghp_..." os.environ["OPENAI_API_KEY"] = "sk-..."
In []
已复制！
import os
from llama_index.readers.github import (
GitHubRepositoryIssuesReader,
GitHubIssuesClient,
)
github_client = GitHubIssuesClient()
loader = GitHubRepositoryIssuesReader(
github_client,
owner="run-llama",
repo="llama_index",
verbose=True,
)
orig_docs = loader.load_data()
limit = 100
docs = []
for idx, doc in enumerate(orig_docs):
doc.metadata["index_id"] = int(doc.id_)
if idx >= limit:
break
docs.append(doc)
import os from llama_index.readers.github import ( GitHubRepositoryIssuesReader, GitHubIssuesClient, ) github_client = GitHubIssuesClient() loader = GitHubRepositoryIssuesReader( github_client, owner="run-llama", repo="llama_index", verbose=True, ) orig_docs = loader.load_data() limit = 100 docs = [] for idx, doc in enumerate(orig_docs): doc.metadata["index_id"] = int(doc.id_) if idx >= limit: break docs.append(doc)
Found 100 issues in the repo page 1 Resulted in 100 documents Found 100 issues in the repo page 2 Resulted in 200 documents Found 100 issues in the repo page 3 Resulted in 300 documents Found 64 issues in the repo page 4 Resulted in 364 documents No more issues found, stopping
设置向量存储和索引¶
In []
已复制！
import weaviate
# cloud
auth_config = weaviate.AuthApiKey(
api_key="XRa15cDIkYRT7AkrpqT6jLfE4wropK1c1TGk"
)
client = weaviate.Client(
"https://llama-index-test-v0oggsoz.weaviate.network",
auth_client_secret=auth_config,
)
class_name = "LlamaIndex_docs"
import weaviate # cloud auth_config = weaviate.AuthApiKey( api_key="XRa15cDIkYRT7AkrpqT6jLfE4wropK1c1TGk" ) client = weaviate.Client( "https://llama-index-test-v0oggsoz.weaviate.network", auth_client_secret=auth_config, ) class_name = "LlamaIndex_docs"
In []
已复制！
# optional: delete schema
client.schema.delete_class(class_name)
# optional: delete schema client.schema.delete_class(class_name)
In []
已复制！
from llama_index.vector_stores.weaviate import WeaviateVectorStore
from llama_index.core import VectorStoreIndex, StorageContext
vector_store = WeaviateVectorStore(
weaviate_client=client, index_name=class_name
)
storage_context = StorageContext.from_defaults(vector_store=vector_store)
from llama_index.vector_stores.weaviate import WeaviateVectorStore from llama_index.core import VectorStoreIndex, StorageContext vector_store = WeaviateVectorStore( weaviate_client=client, index_name=class_name ) storage_context = StorageContext.from_defaults(vector_store=vector_store)
In []
已复制！
doc_index = VectorStoreIndex.from_documents(
docs, storage_context=storage_context
)
doc_index = VectorStoreIndex.from_documents( docs, storage_context=storage_context )
创建用于检索和过滤的 IndexNode¶
In []
已复制！
from llama_index.core import SummaryIndex
from llama_index.core.async_utils import run_jobs
from llama_index.llms.openai import OpenAI
from llama_index.core.schema import IndexNode
from llama_index.core.vector_stores import (
FilterOperator,
MetadataFilter,
MetadataFilters,
)
async def aprocess_doc(doc, include_summary: bool = True):
"""Process doc."""
metadata = doc.metadata
date_tokens = metadata["created_at"].split("T")[0].split("-")
year = int(date_tokens[0])
month = int(date_tokens[1])
day = int(date_tokens[2])
assignee = (
"" if "assignee" not in doc.metadata else doc.metadata["assignee"]
)
size = ""
if len(doc.metadata["labels"]) > 0:
size_arr = [l for l in doc.metadata["labels"] if "size:" in l]
size = size_arr[0].split(":")[1] if len(size_arr) > 0 else ""
new_metadata = {
"state": metadata["state"],
"year": year,
"month": month,
"day": day,
"assignee": assignee,
"size": size,
}
# now extract out summary
summary_index = SummaryIndex.from_documents([doc])
query_str = "Give a one-sentence concise summary of this issue."
query_engine = summary_index.as_query_engine(
llm=OpenAI(model="gpt-3.5-turbo")
)
summary_txt = await query_engine.aquery(query_str)
summary_txt = str(summary_txt)
index_id = doc.metadata["index_id"]
# filter for the specific doc id
filters = MetadataFilters(
filters=[
MetadataFilter(
key="index_id", operator=FilterOperator.EQ, value=int(index_id)
),
]
)
# create an index node using the summary text
index_node = IndexNode(
text=summary_txt,
metadata=new_metadata,
obj=doc_index.as_retriever(filters=filters),
index_id=doc.id_,
)
return index_node
async def aprocess_docs(docs):
"""Process metadata on docs."""
index_nodes = []
tasks = []
for doc in docs:
task = aprocess_doc(doc)
tasks.append(task)
index_nodes = await run_jobs(tasks, show_progress=True, workers=3)
return index_nodes
from llama_index.core import SummaryIndex from llama_index.core.async_utils import run_jobs from llama_index.llms.openai import OpenAI from llama_index.core.schema import IndexNode from llama_index.core.vector_stores import ( FilterOperator, MetadataFilter, MetadataFilters, ) async def aprocess_doc(doc, include_summary: bool = True): """处理文档""" metadata = doc.metadata date_tokens = metadata["created_at"].split("T")[0].split("-") year = int(date_tokens[0]) month = int(date_tokens[1]) day = int(date_tokens[2]) assignee = ( "" if "assignee" not in doc.metadata else doc.metadata["assignee"] ) size = "" if len(doc.metadata["labels"]) > 0: size_arr = [l for l in doc.metadata["labels"] if "size:" in l] size = size_arr[0].split(":")[1] if len(size_arr) > 0 else "" new_metadata = { "state": metadata["state"], "year": year, "month": month, "day": day, "assignee": assignee, "size": size, } # 现在提取摘要 summary_index = SummaryIndex.from_documents([doc]) query_str = "请给出此 issue 的一句话简洁摘要。" query_engine = summary_index.as_query_engine( llm=OpenAI(model="gpt-3.5-turbo") ) summary_txt = await query_engine.aquery(query_str) summary_txt = str(summary_txt) index_id = doc.metadata["index_id"] # 过滤特定文档 ID filters = MetadataFilters( filters=[ MetadataFilter( key="index_id", operator=FilterOperator.EQ, value=int(index_id) ), ] ) # 使用摘要文本创建 index node index_node = IndexNode( text=summary_txt, metadata=new_metadata, obj=doc_index.as_retriever(filters=filters), index_id=doc.id_, ) return index_node async def aprocess_docs(docs): """处理文档的元数据""" index_nodes = [] tasks = [] for doc in docs: task = aprocess_doc(doc) tasks.append(task) index_nodes = await run_jobs(tasks, show_progress=True, workers=3) return index_nodes
In []
已复制！
index_nodes = await aprocess_docs(docs)
index_nodes = await aprocess_docs(docs)
1%| | 1/100 [00:00<00:55, 1.78it/s]/home/loganm/llama_index_proper/llama_index/.venv/lib/python3.11/site-packages/openai/_resource.py:38: ResourceWarning: unclosed <socket.socket fd=71, family=2, type=1, proto=6, laddr=('172.25.21.0', 40832), raddr=('104.18.7.192', 443)>
self._delete = client.delete
ResourceWarning: Enable tracemalloc to get the object allocation traceback
/home/loganm/miniconda3/envs/llama_index/lib/python3.11/asyncio/selector_events.py:835: ResourceWarning: unclosed transport <_SelectorSocketTransport fd=73 read=idle write=<idle, bufsize=0>>
_warn(f"unclosed transport {self!r}", ResourceWarning, source=self)
ResourceWarning: Enable tracemalloc to get the object allocation traceback
/home/loganm/miniconda3/envs/llama_index/lib/python3.11/asyncio/selector_events.py:835: ResourceWarning: unclosed transport <_SelectorSocketTransport fd=71 read=idle write=<idle, bufsize=0>>
_warn(f"unclosed transport {self!r}", ResourceWarning, source=self)
ResourceWarning: Enable tracemalloc to get the object allocation traceback
12%|█▏ | 12/100 [00:04<00:31, 2.79it/s]/home/loganm/miniconda3/envs/llama_index/lib/python3.11/asyncio/selector_events.py:835: ResourceWarning: unclosed transport <_SelectorSocketTransport fd=76 read=idle write=<idle, bufsize=0>>
_warn(f"unclosed transport {self!r}", ResourceWarning, source=self)
ResourceWarning: Enable tracemalloc to get the object allocation traceback
/home/loganm/miniconda3/envs/llama_index/lib/python3.11/asyncio/selector_events.py:835: ResourceWarning: unclosed transport <_SelectorSocketTransport fd=77 read=idle write=<idle, bufsize=0>>
_warn(f"unclosed transport {self!r}", ResourceWarning, source=self)
ResourceWarning: Enable tracemalloc to get the object allocation traceback
/home/loganm/miniconda3/envs/llama_index/lib/python3.11/asyncio/selector_events.py:835: ResourceWarning: unclosed transport <_SelectorSocketTransport fd=78 read=idle write=<idle, bufsize=0>>
_warn(f"unclosed transport {self!r}", ResourceWarning, source=self)
ResourceWarning: Enable tracemalloc to get the object allocation traceback
/home/loganm/llama_index_proper/llama_index/.venv/lib/python3.11/site-packages/openai/resources/chat/completions.py:1337: ResourceWarning: unclosed <socket.socket fd=81, family=2, type=1, proto=6, laddr=('172.25.21.0', 40848), raddr=('104.18.7.192', 443)>
completions.create,
ResourceWarning: Enable tracemalloc to get the object allocation traceback
/home/loganm/miniconda3/envs/llama_index/lib/python3.11/asyncio/selector_events.py:835: ResourceWarning: unclosed transport <_SelectorSocketTransport fd=81 read=idle write=<idle, bufsize=0>>
_warn(f"unclosed transport {self!r}", ResourceWarning, source=self)
ResourceWarning: Enable tracemalloc to get the object allocation traceback
/home/loganm/miniconda3/envs/llama_index/lib/python3.11/asyncio/selector_events.py:835: ResourceWarning: unclosed transport <_SelectorSocketTransport fd=82 read=idle write=<idle, bufsize=0>>
_warn(f"unclosed transport {self!r}", ResourceWarning, source=self)
ResourceWarning: Enable tracemalloc to get the object allocation traceback
/home/loganm/miniconda3/envs/llama_index/lib/python3.11/asyncio/selector_events.py:835: ResourceWarning: unclosed transport <_SelectorSocketTransport fd=83 read=idle write=<idle, bufsize=0>>
_warn(f"unclosed transport {self!r}", ResourceWarning, source=self)
ResourceWarning: Enable tracemalloc to get the object allocation traceback
/home/loganm/miniconda3/envs/llama_index/lib/python3.11/asyncio/selector_events.py:835: ResourceWarning: unclosed transport <_SelectorSocketTransport fd=84 read=idle write=<idle, bufsize=0>>
_warn(f"unclosed transport {self!r}", ResourceWarning, source=self)
ResourceWarning: Enable tracemalloc to get the object allocation traceback
21%|██ | 21/100 [00:06<00:22, 3.58it/s]/home/loganm/llama_index_proper/llama_index/.venv/lib/python3.11/site-packages/openai/_resource.py:34: ResourceWarning: unclosed <socket.socket fd=81, family=2, type=1, proto=6, laddr=('172.25.21.0', 40866), raddr=('104.18.7.192', 443)>
self._get = client.get
ResourceWarning: Enable tracemalloc to get the object allocation traceback
/home/loganm/llama_index_proper/llama_index/.venv/lib/python3.11/site-packages/openai/_resource.py:34: ResourceWarning: unclosed <socket.socket fd=82, family=2, type=1, proto=6, laddr=('172.25.21.0', 40868), raddr=('104.18.7.192', 443)>
self._get = client.get
ResourceWarning: Enable tracemalloc to get the object allocation traceback
/home/loganm/miniconda3/envs/llama_index/lib/python3.11/asyncio/selector_events.py:835: ResourceWarning: unclosed transport <_SelectorSocketTransport fd=86 read=idle write=<idle, bufsize=0>>
_warn(f"unclosed transport {self!r}", ResourceWarning, source=self)
ResourceWarning: Enable tracemalloc to get the object allocation traceback
38%|███▊ | 38/100 [00:12<00:24, 2.54it/s]/home/loganm/miniconda3/envs/llama_index/lib/python3.11/asyncio/selector_events.py:835: ResourceWarning: unclosed transport <_SelectorSocketTransport fd=90 read=idle write=<idle, bufsize=0>>
_warn(f"unclosed transport {self!r}", ResourceWarning, source=self)
ResourceWarning: Enable tracemalloc to get the object allocation traceback
/home/loganm/miniconda3/envs/llama_index/lib/python3.11/asyncio/selector_events.py:835: ResourceWarning: unclosed transport <_SelectorSocketTransport fd=92 read=idle write=<idle, bufsize=0>>
_warn(f"unclosed transport {self!r}", ResourceWarning, source=self)
ResourceWarning: Enable tracemalloc to get the object allocation traceback
/home/loganm/llama_index_proper/llama_index/.venv/lib/python3.11/site-packages/openai/_resource.py:34: ResourceWarning: unclosed <socket.socket fd=94, family=2, type=1, proto=6, laddr=('172.25.21.0', 40912), raddr=('104.18.7.192', 443)>
self._get = client.get
ResourceWarning: Enable tracemalloc to get the object allocation traceback
/home/loganm/miniconda3/envs/llama_index/lib/python3.11/asyncio/selector_events.py:835: ResourceWarning: unclosed transport <_SelectorSocketTransport fd=94 read=idle write=<idle, bufsize=0>>
_warn(f"unclosed transport {self!r}", ResourceWarning, source=self)
ResourceWarning: Enable tracemalloc to get the object allocation traceback
50%|█████ | 50/100 [00:17<00:19, 2.51it/s]/home/loganm/miniconda3/envs/llama_index/lib/python3.11/asyncio/selector_events.py:835: ResourceWarning: unclosed transport <_SelectorSocketTransport fd=95 read=idle write=<idle, bufsize=0>>
_warn(f"unclosed transport {self!r}", ResourceWarning, source=self)
ResourceWarning: Enable tracemalloc to get the object allocation traceback
/home/loganm/miniconda3/envs/llama_index/lib/python3.11/asyncio/selector_events.py:835: ResourceWarning: unclosed transport <_SelectorSocketTransport fd=96 read=idle write=<idle, bufsize=0>>
_warn(f"unclosed transport {self!r}", ResourceWarning, source=self)
ResourceWarning: Enable tracemalloc to get the object allocation traceback
/home/loganm/miniconda3/envs/llama_index/lib/python3.11/asyncio/selector_events.py:835: ResourceWarning: unclosed transport <_SelectorSocketTransport fd=97 read=idle write=<idle, bufsize=0>>
_warn(f"unclosed transport {self!r}", ResourceWarning, source=self)
ResourceWarning: Enable tracemalloc to get the object allocation traceback
73%|███████▎ | 73/100 [00:24<00:07, 3.42it/s]/home/loganm/miniconda3/envs/llama_index/lib/python3.11/asyncio/selector_events.py:835: ResourceWarning: unclosed transport <_SelectorSocketTransport fd=101 read=idle write=<idle, bufsize=0>>
_warn(f"unclosed transport {self!r}", ResourceWarning, source=self)
ResourceWarning: Enable tracemalloc to get the object allocation traceback
/home/loganm/miniconda3/envs/llama_index/lib/python3.11/asyncio/selector_events.py:835: ResourceWarning: unclosed transport <_SelectorSocketTransport fd=102 read=idle write=<idle, bufsize=0>>
_warn(f"unclosed transport {self!r}", ResourceWarning, source=self)
ResourceWarning: Enable tracemalloc to get the object allocation traceback
82%|████████▏ | 82/100 [00:27<00:06, 2.94it/s]/home/loganm/miniconda3/envs/llama_index/lib/python3.11/functools.py:76: ResourceWarning: unclosed <socket.socket fd=102, family=2, type=1, proto=6, laddr=('172.25.21.0', 40998), raddr=('104.18.7.192', 443)>
return partial(update_wrapper, wrapped=wrapped,
ResourceWarning: Enable tracemalloc to get the object allocation traceback
92%|█████████▏| 92/100 [00:32<00:03, 2.15it/s]/home/loganm/miniconda3/envs/llama_index/lib/python3.11/asyncio/selector_events.py:835: ResourceWarning: unclosed transport <_SelectorSocketTransport fd=106 read=idle write=<idle, bufsize=0>>
_warn(f"unclosed transport {self!r}", ResourceWarning, source=self)
ResourceWarning: Enable tracemalloc to get the object allocation traceback
/home/loganm/miniconda3/envs/llama_index/lib/python3.11/asyncio/selector_events.py:835: ResourceWarning: unclosed transport <_SelectorSocketTransport fd=111 read=idle write=<idle, bufsize=0>>
_warn(f"unclosed transport {self!r}", ResourceWarning, source=self)
ResourceWarning: Enable tracemalloc to get the object allocation traceback
100%|██████████| 100/100 [00:36<00:00, 2.71it/s]
In []
已复制！
index_nodes[5].metadata
index_nodes[5].metadata
Out[]
{'state': 'open',
'year': 2024,
'month': 1,
'day': 13,
'assignee': '',
'size': 'XL'}
创建顶层自动检索器¶
我们将摘要元数据和原始文档都加载到向量数据库中。
- 摘要元数据：这会进入 `LlamaIndex_auto` 集合。
- 原始文档：这会进入 `LlamaIndex_docs` 集合。
通过同时存储摘要元数据和原始文档，我们可以执行我们的结构化、分层检索策略。
我们加载到一个支持自动检索的向量数据库。
加载摘要元数据¶
这会进入 `LlamaIndex_auto`
In []
已复制！
import weaviate
# cloud
auth_config = weaviate.AuthApiKey(
api_key="XRa15cDIkYRT7AkrpqT6jLfE4wropK1c1TGk"
)
client = weaviate.Client(
"https://llama-index-test-v0oggsoz.weaviate.network",
auth_client_secret=auth_config,
)
class_name = "LlamaIndex_auto"
import weaviate # cloud auth_config = weaviate.AuthApiKey( api_key="XRa15cDIkYRT7AkrpqT6jLfE4wropK1c1TGk" ) client = weaviate.Client( "https://llama-index-test-v0oggsoz.weaviate.network", auth_client_secret=auth_config, ) class_name = "LlamaIndex_auto"
In []
已复制！
# optional: delete schema
client.schema.delete_class(class_name)
# optional: delete schema client.schema.delete_class(class_name)
In []
已复制！
from llama_index.vector_stores.weaviate import WeaviateVectorStore
from llama_index.core import VectorStoreIndex, StorageContext
vector_store_auto = WeaviateVectorStore(
weaviate_client=client, index_name=class_name
)
storage_context_auto = StorageContext.from_defaults(
vector_store=vector_store_auto
)
from llama_index.vector_stores.weaviate import WeaviateVectorStore from llama_index.core import VectorStoreIndex, StorageContext vector_store_auto = WeaviateVectorStore( weaviate_client=client, index_name=class_name ) storage_context_auto = StorageContext.from_defaults( vector_store=vector_store_auto )
In []
已复制！
# Since "index_nodes" are concise summaries, we can directly feed them as objects into VectorStoreIndex
index = VectorStoreIndex(
objects=index_nodes, storage_context=storage_context_auto
)
# 由于 "index_nodes" 是简洁摘要，我们可以直接将它们作为对象输入到 VectorStoreIndex 中 index = VectorStoreIndex( objects=index_nodes, storage_context=storage_context_auto )
设置可组合自动检索器¶
在本节中，我们将设置自动检索器。我们需要执行几个步骤。
- 定义 Schema：定义向量数据库的 schema（例如，元数据字段）。这将在 LLM 决定推断哪些元数据过滤器时作为输入提示的一部分。
- 实例化 VectorIndexAutoRetriever 类：这会在我们的摘要元数据索引之上创建一个检索器，并接受定义的 schema 作为输入。
- 定义一个包装检索器：这允许我们将每个节点后处理成一个
IndexNode，其中包含一个指向源文档的索引 ID。这将允许我们在下一节中进行递归检索（这取决于 IndexNode 对象链接到下游检索器/查询引擎/其他节点）。注意：我们正在努力改进这个抽象。
运行此检索器将根据我们顶层 IndexNode 对象的文本摘要和元数据进行检索。然后，将使用其底层检索器从特定的 Github issue 中检索内容。
1. 定义 Schema¶
In []
已复制！
from llama_index.core.vector_stores import MetadataInfo, VectorStoreInfo
vector_store_info = VectorStoreInfo(
content_info="Github Issues",
metadata_info=[
MetadataInfo(
name="state",
description="Whether the issue is `open` or `closed`",
type="string",
),
MetadataInfo(
name="year",
description="The year issue was created",
type="integer",
),
MetadataInfo(
name="month",
description="The month issue was created",
type="integer",
),
MetadataInfo(
name="day",
description="The day issue was created",
type="integer",
),
MetadataInfo(
name="assignee",
description="The assignee of the ticket",
type="string",
),
MetadataInfo(
name="size",
description="How big the issue is (XS, S, M, L, XL, XXL)",
type="string",
),
],
)
from llama_index.core.vector_stores import MetadataInfo, VectorStoreInfo vector_store_info = VectorStoreInfo( content_info="Github Issues", metadata_info=[ MetadataInfo( name="state", description="Issue 的状态是 `open`（开放）还是 `closed`（关闭）", type="string", ), MetadataInfo( name="year", description="Issue 的创建年份", type="integer", ), MetadataInfo( name="month", description="Issue 的创建月份", type="integer", ), MetadataInfo( name="day", description="Issue 的创建日期", type="integer", ), MetadataInfo( name="assignee", description="Issue 的负责人", type="string", ), MetadataInfo( name="size", description="Issue 的大小 (XS, S, M, L, XL, XXL)", type="string", ), ], )
2. 实例化 VectorIndexAutoRetriever¶
In []
已复制！
from llama_index.core.retrievers import VectorIndexAutoRetriever
retriever = VectorIndexAutoRetriever(
index,
vector_store_info=vector_store_info,
similarity_top_k=2,
empty_query_top_k=10, # if only metadata filters are specified, this is the limit
verbose=True,
)
from llama_index.core.retrievers import VectorIndexAutoRetriever retriever = VectorIndexAutoRetriever( index, vector_store_info=vector_store_info, similarity_top_k=2, empty_query_top_k=10, # 如果只指定了元数据过滤器，这是限制 verbose=True, )
试用¶
现在我们可以开始检索相关的 Github Issue 上下文了！
为了完成 RAG 管道的设置，我们将把递归检索器与我们的 RetrieverQueryEngine 结合起来，除了检索到的节点外，还生成响应。
试用检索¶
In []
已复制！
from llama_index.core import QueryBundle
nodes = retriever.retrieve(QueryBundle("Tell me about some issues on 01/11"))
from llama_index.core import QueryBundle nodes = retriever.retrieve(QueryBundle("Tell me about some issues on 01/11"))
Using query str: issues
Using filters: [('day', '==', '11'), ('month', '==', '01')]
Retrieval entering 9995: VectorIndexRetriever
Retrieving from object VectorIndexRetriever with query issues
Retrieval entering 9985: VectorIndexRetriever
Retrieving from object VectorIndexRetriever with query issues
结果是相关文档中的源分块。
让我们看看附加到源分块的日期（原始元数据中存在）。
In []
已复制！
print(f"Number of source nodes: {len(nodes)}")
nodes[0].node.metadata
print(f"源节点数量：{len(nodes)}") nodes[0].node.metadata
Number of source nodes: 2
Out[]
{'state': 'open',
'created_at': '2024-01-11T20:37:34Z',
'url': 'https://api.github.com/repos/run-llama/llama_index/issues/9995',
'source': 'https://github.com/run-llama/llama_index/pull/9995',
'labels': ['size:XXL'],
'index_id': 9995}
连接到 RetrieverQueryEngine¶
我们连接到 RetrieverQueryEngine 来合成结果。
In []
已复制！
from llama_index.core.query_engine import RetrieverQueryEngine
from llama_index.llms.openai import OpenAI
llm = OpenAI(model="gpt-3.5-turbo")
query_engine = RetrieverQueryEngine.from_args(retriever, llm=llm)
from llama_index.core.query_engine import RetrieverQueryEngine from llama_index.llms.openai import OpenAI llm = OpenAI(model="gpt-3.5-turbo") query_engine = RetrieverQueryEngine.from_args(retriever, llm=llm)
In []
已复制！
response = query_engine.query("Tell me about some issues on 01/11")
response = query_engine.query("告诉我 01/11 的一些 issue")
Using query str: issues
Using filters: [('day', '==', '11'), ('month', '==', '01')]
Retrieval entering 9995: VectorIndexRetriever
Retrieving from object VectorIndexRetriever with query issues
Retrieval entering 9985: VectorIndexRetriever
Retrieving from object VectorIndexRetriever with query issues
In []
已复制！
print(str(response))
print(str(response))
There are two issues that were created on 01/11. The first issue is related to ensuring backwards compatibility with the new Pinecone client version bifurcation. The second issue is a feature request to implement the Language Agent Tree Search (LATS) agent in llama-index.
In []
已复制！
response = query_engine.query(
"Tell me about some open issues related to agents"
)
response = query_engine.query( "告诉我一些与 agent 相关的开放 issue" )
Using query str: agents
Using filters: [('state', '==', 'open')]
Retrieval entering 10058: VectorIndexRetriever
Retrieving from object VectorIndexRetriever with query agents
Retrieval entering 9899: VectorIndexRetriever
Retrieving from object VectorIndexRetriever with query agents
In []
已复制！
print(str(response))
print(str(response))
There are two open issues related to agents. One issue is about adding context for agents, updating a stale link, and adding a notebook to demo a react agent with context. The other issue is a feature request for parallelism when using the top agent from a multi-document agent while comparing multiple documents.
