带有 Pydantic 输出的查询引擎¶

每个查询引擎都支持使用 RetrieverQueryEngine 中的以下 response_mode 进行集成的结构化响应

• refine
• compact
• tree_summarize
• accumulate (beta 版，需要额外解析才能转换为对象)
• compact_accumulate (beta 版，需要额外解析才能转换为对象)

在本 notebook 中，我们将通过一个小例子来演示用法。

在内部，每个 LLM 响应都将是一个 pydantic 对象。如果该响应需要被 refine 或 summarize，它将被转换为 JSON 字符串以供下一个响应使用。然后，最终响应将作为 pydantic 对象返回。

注意： 这理论上可以与任何 LLM 一起工作，但非 OpenAI 的支持仍在开发中，被认为是 beta 版。

设置¶

如果您在 Colab 上打开此 Notebook，您可能需要安装 LlamaIndex 🦙。

In [ ]

%pip install llama-index-llms-anthropic
%pip install llama-index-llms-openai

%pip install llama-index-llms-anthropic %pip install llama-index-llms-openai

In [ ]

!pip install llama-index

!pip install llama-index

In [ ]

import os
import openai

os.environ["OPENAI_API_KEY"] = "sk-..."
openai.api_key = os.environ["OPENAI_API_KEY"]

import os import openai os.environ["OPENAI_API_KEY"] = "sk-..." openai.api_key = os.environ["OPENAI_API_KEY"]

下载数据

In [ ]

!mkdir -p 'data/paul_graham/'
!wget 'https://raw.githubusercontent.com/run-llama/llama_index/main/docs/docs/examples/data/paul_graham/paul_graham_essay.txt' -O 'data/paul_graham/paul_graham_essay.txt'

!mkdir -p 'data/paul_graham/' !wget 'https://raw.githubusercontent.com/run-llama/llama_index/main/docs/docs/examples/data/paul_graham/paul_graham_essay.txt' -O 'data/paul_graham/paul_graham_essay.txt'

In [ ]

from llama_index.core import SimpleDirectoryReader

documents = SimpleDirectoryReader("./data/paul_graham").load_data()

from llama_index.core import SimpleDirectoryReader documents = SimpleDirectoryReader("./data/paul_graham").load_data()

创建我们的 Pydantic 输出对象¶

In [ ]

from typing import List
from pydantic import BaseModel


class Biography(BaseModel):
    """Data model for a biography."""

    name: str
    best_known_for: List[str]
    extra_info: str

from typing import List from pydantic import BaseModel class Biography(BaseModel): """传记的数据模型。""" name: str best_known_for: List[str] extra_info: str

创建索引 + 查询引擎 (OpenAI)¶

使用 OpenAI 时，将利用函数调用 API 实现可靠的结构化输出。

In [ ]

from llama_index.core import VectorStoreIndex
from llama_index.llms.openai import OpenAI

llm = OpenAI(model="gpt-3.5-turbo", temperature=0.1)

index = VectorStoreIndex.from_documents(
    documents,
)

from llama_index.core import VectorStoreIndex from llama_index.llms.openai import OpenAI llm = OpenAI(model="gpt-3.5-turbo", temperature=0.1) index = VectorStoreIndex.from_documents( documents, )

In [ ]

query_engine = index.as_query_engine(
    output_cls=Biography, response_mode="compact", llm=llm
)

query_engine = index.as_query_engine( output_cls=Biography, response_mode="compact", llm=llm )

In [ ]

response = query_engine.query("Who is Paul Graham?")

response = query_engine.query("Who is Paul Graham?")

In [ ]

print(response.name)
print(response.best_known_for)
print(response.extra_info)

print(response.name) print(response.best_known_for) print(response.extra_info)

Paul Graham
['working on Bel', 'co-founding Viaweb', 'creating the programming language Arc']
Paul Graham is a computer scientist, entrepreneur, and writer. He is best known for his work on Bel, a programming language, and for co-founding Viaweb, an early web application company that was later acquired by Yahoo. Graham also created the programming language Arc. He has written numerous essays on topics such as startups, programming, and life.

In [ ]

# get the full pydanitc object
print(type(response.response))

# 获取完整的 pydantic 对象 print(type(response.response))

<class '__main__.Biography'>

创建索引 + 查询引擎 (非 OpenAI，Beta 版)¶

当使用不支持函数调用的 LLM 时，我们依靠 LLM 自己编写 JSON，并将 JSON 解析为适当的 pydantic 对象。

In [ ]

import os

os.environ["ANTHROPIC_API_KEY"] = "sk-..."

import os os.environ["ANTHROPIC_API_KEY"] = "sk-..."

In [ ]

from llama_index.core import VectorStoreIndex
from llama_index.llms.anthropic import Anthropic

llm = Anthropic(model="claude-instant-1.2", temperature=0.1)

index = VectorStoreIndex.from_documents(
    documents,
)

from llama_index.core import VectorStoreIndex from llama_index.llms.anthropic import Anthropic llm = Anthropic(model="claude-instant-1.2", temperature=0.1) index = VectorStoreIndex.from_documents( documents, )

In [ ]

query_engine = index.as_query_engine(
    output_cls=Biography, response_mode="tree_summarize", llm=llm
)

query_engine = index.as_query_engine( output_cls=Biography, response_mode="tree_summarize", llm=llm )

In [ ]

response = query_engine.query("Who is Paul Graham?")

response = query_engine.query("Who is Paul Graham?")

In [ ]

print(response.name)
print(response.best_known_for)
print(response.extra_info)

print(response.name) print(response.best_known_for) print(response.extra_info)

Paul Graham
['Co-founder of Y Combinator', 'Essayist and programmer']
He is known for creating Viaweb, one of the first web application builders, and for founding Y Combinator, one of the world's top startup accelerators. Graham has also written extensively about technology, investing, and philosophy.

In [ ]

# get the full pydanitc object
print(type(response.response))

# 获取完整的 pydantic 对象 print(type(response.response))

<class '__main__.Biography'>

Accumulate 示例 (Beta 版)¶

使用 pydantic 对象的 Accumulate 需要一些额外的解析。这仍然是一个 beta 功能，但仍然可以获得 Accumulate pydantic 对象。

In [ ]

from typing import List
from pydantic import BaseModel


class Company(BaseModel):
    """Data model for a companies mentioned."""

    company_name: str
    context_info: str

from typing import List from pydantic import BaseModel class Company(BaseModel): """提及的公司的数据模型。""" company_name: str context_info: str

In [ ]

from llama_index.core import VectorStoreIndex,
from llama_index.llms.openai import OpenAI

llm = OpenAI(model="gpt-3.5-turbo", temperature=0.1)

index = VectorStoreIndex.from_documents(
    documents,
)

from llama_index.core import VectorStoreIndex, from llama_index.llms.openai import OpenAI llm = OpenAI(model="gpt-3.5-turbo", temperature=0.1) index = VectorStoreIndex.from_documents( documents, )

In [ ]

query_engine = index.as_query_engine(
    output_cls=Company, response_mode="accumulate", llm=llm
)

query_engine = index.as_query_engine( output_cls=Company, response_mode="accumulate", llm=llm )

In [ ]

response = query_engine.query("What companies are mentioned in the text?")

response = query_engine.query("What companies are mentioned in the text?")

在 accumulate 模式下，响应由默认分隔符分隔，并带有前缀。

In [ ]

companies = []

# split by the default separator
for response_str in str(response).split("\n---------------------\n"):
    # remove the prefix --  every response starts like `Response 1: {...}`
    # so, we find the first bracket and remove everything before it
    response_str = response_str[response_str.find("{") :]
    companies.append(Company.parse_raw(response_str))

companies = [] # 按默认分隔符分割 for response_str in str(response).split("\n---------------------\n"): # 移除前缀 -- 每个响应都以 Response 1: {...} 开头 # 因此，我们找到第一个括号并移除其之前的所有内容 response_str = response_str[response_str.find("{") :] companies.append(Company.parse_raw(response_str))

In [ ]

print(companies)

print(companies)

[Company(company_name='Yahoo', context_info='Yahoo bought us'), Company(company_name='Yahoo', context_info="I'd been meaning to since Yahoo bought us")]