使用结构化答案过滤进行精炼¶

当使用我们的 Refine 响应合成器进行响应合时，过滤掉非答案至关重要。常见的问题是单一无用响应（例如“我不知道答案”）的传播，这可能会在整个合成过程中持续存在，导致最终答案也是如此。即使在其他更相关的部分存在实际答案的情况下，也可能发生这种情况。

通过将 structured_answer_filtering 设置为 True，可以过滤掉这些无用响应。默认情况下，它设置为 False，因为目前这只在使用支持函数调用的 OpenAI 模型时效果最佳。

如果您在 Colab 上打开此 Notebook，可能需要安装 LlamaIndex 🦙。

In [ ]

%pip install llama-index-llms-openai

%pip install llama-index-llms-openai

In [ ]

!pip install llama-index

!pip install llama-index

加载数据¶

In [ ]

texts = [
    "The president in the year 2040 is John Cena.",
    "The president in the year 2050 is Florence Pugh.",
    'The president in the year 2060 is Dwayne "The Rock" Johnson.',
]

texts = [ "The president in the year 2040 is John Cena.", "The president in the year 2050 is Florence Pugh.", 'The president in the year 2060 is Dwayne "The Rock" Johnson.', ]

总结¶

In [ ]

import os

os.environ["OPENAI_API_KEY"] = "sk-..."

import os os.environ["OPENAI_API_KEY"] = "sk-..."

In [ ]

from llama_index.llms.openai import OpenAI

llm = OpenAI(model="gpt-3.5-turbo-0613")

from llama_index.llms.openai import OpenAI llm = OpenAI(model="gpt-3.5-turbo-0613")

In [ ]

from llama_index.core import get_response_synthesizer

summarizer = get_response_synthesizer(
    response_mode="refine", llm=llm, verbose=True
)

from llama_index.core import get_response_synthesizer summarizer = get_response_synthesizer( response_mode="refine", llm=llm, verbose=True )

In [ ]

response = summarizer.get_response("who is president in the year 2050?", texts)

response = summarizer.get_response("who is president in the year 2050?", texts)

> Refine context: The president in the year 2050 is Florence Pugh.
> Refine context: The president in the year 2060 is Dwayne "The R...

失败结果¶

如您所见，我们无法从输入的 texts 字符串中获得正确答案，因为最初的“我不知道”答案一直传播到响应合成结束。

In [ ]

print(response)

print(response)

I'm sorry, but I don't have access to information about the future.

现在我们将再次尝试，设置 structured_answer_filtering=True

In [ ]

from llama_index.core import get_response_synthesizer

summarizer = get_response_synthesizer(
    response_mode="refine",
    llm=llm,
    verbose=True,
    structured_answer_filtering=True,
)

from llama_index.core import get_response_synthesizer summarizer = get_response_synthesizer( response_mode="refine", llm=llm, verbose=True, structured_answer_filtering=True, )

In [ ]

response = summarizer.get_response("who is president in the year 2050?", texts)

response = summarizer.get_response("who is president in the year 2050?", texts)

Function call: StructuredRefineResponse with args: {
  "answer": "It is not possible to determine who the president is in the year 2050 based on the given context information.",
  "query_satisfied": false
}
> Refine context: The president in the year 2050 is Florence Pugh.
Function call: StructuredRefineResponse with args: {
  "answer": "Florence Pugh",
  "query_satisfied": true
}
> Refine context: The president in the year 2060 is Dwayne "The R...
Function call: StructuredRefineResponse with args: {
  "answer": "Florence Pugh",
  "query_satisfied": false
}

成功结果¶

如您所见，通过过滤实际上包含问题答案的 texts 字符串，我们能够从给定的上下文中确定正确答案。

In [ ]

print(response)

print(response)

Florence Pugh

非函数调用 LLM¶

您可能希望在不提供函数调用 API 的 LLM 中使用此过滤功能。

在这种情况下，Refine 模块将自动切换到使用不依赖外部函数调用 API 的结构化输出 Program。

In [ ]

# we'll stick with OpenAI but use an older model that does not support function calling
instruct_llm = OpenAI(model="gpt-3.5-turbo-instruct")

# 我们将继续使用 OpenAI，但使用一个不支持函数调用的旧模型 instruct_llm = OpenAI(model="gpt-3.5-turbo-instruct")

In [ ]

from llama_index.core import get_response_synthesizer

summarizer = get_response_synthesizer(
    response_mode="refine",
    llm=instruct_llm,
    verbose=True,
    structured_answer_filtering=True,
)

from llama_index.core import get_response_synthesizer summarizer = get_response_synthesizer( response_mode="refine", llm=instruct_llm, verbose=True, structured_answer_filtering=True, )

In [ ]

response = summarizer.get_response("who is president in the year 2050?", texts)
print(response)

response = summarizer.get_response("who is president in the year 2050?", texts) print(response)

Florence Pugh

CompactAndRefine¶

由于 CompactAndRefine 是构建在 Refine 之上的，此响应模式也支持结构化答案过滤。

In [ ]

from llama_index.core import get_response_synthesizer

summarizer = get_response_synthesizer(
    response_mode="compact",
    llm=instruct_llm,
    verbose=True,
    structured_answer_filtering=True,
)

from llama_index.core import get_response_synthesizer summarizer = get_response_synthesizer( response_mode="compact", llm=instruct_llm, verbose=True, structured_answer_filtering=True, )

In [ ]

response = summarizer.get_response("who is president in the year 2050?", texts)
print(response)

response = summarizer.get_response("who is president in the year 2050?", texts) print(response)

Florence Pugh