正确性评估器¶

本notebook使用CorrectnessEvaluator来评估生成答案相对于参考答案的相关性和正确性。

如果你在colab上打开本notebook，你可能需要安装LlamaIndex 🦙。

In [ ]

%pip install llama-index-llms-openai

%pip install llama-index-llms-openai

In [ ]

!pip install llama-index

!pip install llama-index

In [ ]

from llama_index.core.evaluation import CorrectnessEvaluator
from llama_index.llms.openai import OpenAI

from llama_index.core.evaluation import CorrectnessEvaluator from llama_index.llms.openai import OpenAI

In [ ]

llm = OpenAI("gpt-4")
evaluator = CorrectnessEvaluator(llm=llm)

llm = OpenAI("gpt-4") evaluator = CorrectnessEvaluator(llm=llm)

In [ ]

query = (
    "Can you explain the theory of relativity proposed by Albert Einstein in"
    " detail?"
)

reference = """
Certainly! Albert Einstein's theory of relativity consists of two main components: special relativity and general relativity. Special relativity, published in 1905, introduced the concept that the laws of physics are the same for all non-accelerating observers and that the speed of light in a vacuum is a constant, regardless of the motion of the source or observer. It also gave rise to the famous equation E=mc², which relates energy (E) and mass (m).

General relativity, published in 1915, extended these ideas to include the effects of gravity. According to general relativity, gravity is not a force between masses, as described by Newton's theory of gravity, but rather the result of the warping of space and time by mass and energy. Massive objects, such as planets and stars, cause a curvature in spacetime, and smaller objects follow curved paths in response to this curvature. This concept is often illustrated using the analogy of a heavy ball placed on a rubber sheet, causing it to create a depression that other objects (representing smaller masses) naturally move towards.

In essence, general relativity provided a new understanding of gravity, explaining phenomena like the bending of light by gravity (gravitational lensing) and the precession of the orbit of Mercury. It has been confirmed through numerous experiments and observations and has become a fundamental theory in modern physics.
"""

response = """
Certainly! Albert Einstein's theory of relativity consists of two main components: special relativity and general relativity. Special relativity, published in 1905, introduced the concept that the laws of physics are the same for all non-accelerating observers and that the speed of light in a vacuum is a constant, regardless of the motion of the source or observer. It also gave rise to the famous equation E=mc², which relates energy (E) and mass (m).

However, general relativity, published in 1915, extended these ideas to include the effects of magnetism. According to general relativity, gravity is not a force between masses but rather the result of the warping of space and time by magnetic fields generated by massive objects. Massive objects, such as planets and stars, create magnetic fields that cause a curvature in spacetime, and smaller objects follow curved paths in response to this magnetic curvature. This concept is often illustrated using the analogy of a heavy ball placed on a rubber sheet with magnets underneath, causing it to create a depression that other objects (representing smaller masses) naturally move towards due to magnetic attraction.
"""

query = ( "你能详细解释一下阿尔伯特·爱因斯坦提出的相对论吗？" ) reference = """ 当然！阿尔伯特·爱因斯坦的相对论包含两个主要组成部分：狭义相对论和广义相对论。 狭义相对论发表于1905年，引入了物理定律对所有非加速观察者都相同，以及真空中的光速是常数，与光源或观察者的运动无关的概念。它还催生了著名的方程 E=mc²，该方程关联了能量（E）和质量（m）。 广义相对论发表于1915年，将这些思想扩展到包含引力的影响。根据广义相对论，引力不是牛顿引力理论描述的质量之间的力，而是质量和能量导致的时空弯曲的结果。大质量物体，如行星和恒星，导致时空发生弯曲，较小的物体则沿着弯曲的路径运动以响应这种弯曲。这个概念通常用将一个重球放在橡胶板上，导致橡胶板产生凹陷，其他物体（代表较小的质量）自然会朝着凹陷移动的类比来解释。 本质上，广义相对论提供了对引力的新理解，解释了引力引起的光线弯曲（引力透镜）和水星轨道进动等现象。它已通过大量实验和观测得到证实，并已成为现代物理学中的基本理论。 """ response = """ 当然！阿尔伯特·爱因斯坦的相对论包含两个主要组成部分：狭义相对论和广义相对论。 狭义相对论发表于1905年，引入了物理定律对所有非加速观察者都相同，以及真空中的光速是常数，与光源或观察者的运动无关的概念。它还催生了著名的方程 E=mc²，该方程关联了能量（E）和质量（m）。 然而，广义相对论发表于1915年，将这些思想扩展到包含磁力的影响。根据广义相对论，引力不是质量之间的力，而是大质量物体产生的磁场导致时空弯曲的结果。大质量物体，如行星和恒星，产生磁场，导致时空发生弯曲，较小的物体则沿着弯曲的路径运动以响应这种磁性弯曲。这个概念通常用将一个重球放在橡胶板上并在下方放置磁铁，导致橡胶板产生凹陷，其他物体（代表较小的质量）由于磁引力而自然会朝着凹陷移动的类比来解释。 """

In [ ]

result = evaluator.evaluate(
    query=query,
    response=response,
    reference=reference,
)

result = evaluator.evaluate( query=query, response=response, reference=reference, )

In [ ]

result.score

result.score

Out[ ]

2.5

In [ ]

result.feedback

result.feedback

Out[ ]

'The generated answer is relevant to the user query as it attempts to explain the theory of relativity proposed by Albert Einstein. However, it contains significant mistakes. The explanation of general relativity is incorrect. General relativity is about the warping of space and time by mass and energy, not magnetic fields. The analogy used in the generated answer is also incorrect as it introduces magnets, which are not part of the original analogy or the theory of general relativity. These errors significantly affect the correctness of the information provided.'