IBM watsonx.ai¶

WatsonxRerank 是 IBM watsonx.ai Rerank 的一个包装器。

这些示例旨在展示如何利用 LlamaIndex 后处理器 API 来使用 watsonx.ai 的 Rerank、Embeddings 和 LLMs。

设置¶

安装所需的包

In [ ]

%pip install -qU llama-index
%pip install -qU llama-index-llms-ibm
%pip install -qU llama-index-postprocessor-ibm
%pip install -qU llama-index-embeddings-ibm

%pip install -qU llama-index %pip install -qU llama-index-llms-ibm %pip install -qU llama-index-postprocessor-ibm %pip install -qU llama-index-embeddings-ibm

下面的单元格定义了使用 watsonx Foundation Models, Embeddings 和 Rerank 所需的凭据。

操作：提供 IBM Cloud 用户 API 密钥。详情请参阅管理用户 API 密钥。

In [ ]

import os
from getpass import getpass

watsonx_api_key = getpass()
os.environ["WATSONX_APIKEY"] = watsonx_api_key

import os from getpass import getpass watsonx_api_key = getpass() os.environ["WATSONX_APIKEY"] = watsonx_api_key

此外，你可以将额外的密钥作为环境变量传递。

In [ ]

import os

os.environ["WATSONX_URL"] = "your service instance url"
os.environ["WATSONX_TOKEN"] = "your token for accessing the CPD cluster"
os.environ["WATSONX_PASSWORD"] = "your password for accessing the CPD cluster"
os.environ["WATSONX_USERNAME"] = "your username for accessing the CPD cluster"
os.environ[
    "WATSONX_INSTANCE_ID"
] = "your instance_id for accessing the CPD cluster"

import os os.environ["WATSONX_URL"] = "your service instance url" os.environ["WATSONX_TOKEN"] = "your token for accessing the CPD cluster" os.environ["WATSONX_PASSWORD"] = "your password for accessing the CPD cluster" os.environ["WATSONX_USERNAME"] = "your username for accessing the CPD cluster" os.environ[ "WATSONX_INSTANCE_ID" ] = "your instance_id for accessing the CPD cluster"

注意:

• 为了为 API 调用提供上下文，你必须传递 project_id 或 space_id。要获取你的项目或空间 ID，打开你的项目或空间，前往管理选项卡，然后点击常规。更多信息请参阅：项目文档 或 部署空间文档。
• 根据你预置的服务实例所在的区域，使用watsonx.ai API 认证中列出的 URL 之一。

在此示例中，我们将使用 project_id 和 Dallas URL。

提供将用于初始化每个 watsonx 集成实例的 PROJECT_ID。

In [ ]

PROJECT_ID = "PASTE YOUR PROJECT_ID HERE"
URL = "https://us-south.ml.cloud.ibm.com"

PROJECT_ID = "PASTE YOUR PROJECT_ID HERE" URL = "https://us-south.ml.cloud.ibm.com"

下载数据并加载文档¶

In [ ]

!mkdir -p 'data/paul_graham/'
!wget 'https://raw.githubusercontent.com/run-llama/llama_index/main/docs/docs/examples/data/paul_graham/paul_graham_essay.txt' -O 'data/paul_graham/paul_graham_essay.txt'

!mkdir -p 'data/paul_graham/' !wget 'https://raw.githubusercontent.com/run-llama/llama_index/main/docs/docs/examples/data/paul_graham/paul_graham_essay.txt' -O 'data/paul_graham/paul_graham_essay.txt'

--2025-02-24 10:46:16--  https://raw.githubusercontent.com/run-llama/llama_index/main/docs/docs/examples/data/paul_graham/paul_graham_essay.txt
Resolving raw.githubusercontent.com (raw.githubusercontent.com)... 2606:50c0:8000::154, 2606:50c0:8001::154, 2606:50c0:8002::154, ...
Connecting to raw.githubusercontent.com (raw.githubusercontent.com)|2606:50c0:8000::154|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 75042 (73K) [text/plain]
Saving to: ‘data/paul_graham/paul_graham_essay.txt’

data/paul_graham/pa 100%[===================>]  73,28K  --.-KB/s    in 0,06s   

2025-02-24 10:46:17 (1,30 MB/s) - ‘data/paul_graham/paul_graham_essay.txt’ saved [75042/75042]

In [ ]

from llama_index.core import SimpleDirectoryReader

documents = SimpleDirectoryReader("./data/paul_graham/").load_data()

from llama_index.core import SimpleDirectoryReader documents = SimpleDirectoryReader("./data/paul_graham/").load_data()

加载 Rerank¶

你可能需要针对不同的任务调整 rerank 参数。

In [ ]

truncate_input_tokens = 512

truncate_input_tokens = 512

初始化 WatsonxRerank 实例。¶

你需要指定将用于 rerank 的 model_id。你可以在支持的 rerank 模型中找到所有可用模型的列表。

In [ ]

from llama_index.postprocessor.ibm import WatsonxRerank

watsonx_rerank = WatsonxRerank(
    model_id="cross-encoder/ms-marco-minilm-l-12-v2",
    top_n=2,
    url=URL,
    project_id=PROJECT_ID,
    truncate_input_tokens=truncate_input_tokens,
)

from llama_index.postprocessor.ibm import WatsonxRerank watsonx_rerank = WatsonxRerank( model_id="cross-encoder/ms-marco-minilm-l-12-v2", top_n=2, url=URL, project_id=PROJECT_ID, truncate_input_tokens=truncate_input_tokens, )

或者，你可以使用 Cloud Pak for Data 凭据。详情请参阅watsonx.ai 软件设置。

In [ ]

from llama_index.postprocessor.ibm import WatsonxRerank

watsonx_rerank = WatsonxRerank(
    model_id="cross-encoder/ms-marco-minilm-l-12-v2",
    url=URL,
    username="PASTE YOUR USERNAME HERE",
    password="PASTE YOUR PASSWORD HERE",
    instance_id="openshift",
    version="5.1",
    project_id=PROJECT_ID,
    truncate_input_tokens=truncate_input_tokens,
)

from llama_index.postprocessor.ibm import WatsonxRerank watsonx_rerank = WatsonxRerank( model_id="cross-encoder/ms-marco-minilm-l-12-v2", url=URL, username="PASTE YOUR USERNAME HERE", password="PASTE YOUR PASSWORD HERE", instance_id="openshift", version="5.1", project_id=PROJECT_ID, truncate_input_tokens=truncate_input_tokens, )

加载嵌入模型¶

初始化 WatsonxEmbeddings 实例。¶

关于 WatsonxEmbeddings 的更多信息，请参阅示例笔记本：

你可能需要针对不同的任务调整嵌入参数。

In [ ]

truncate_input_tokens = 512

truncate_input_tokens = 512

你需要指定将用于嵌入的 model_id。你可以在支持的嵌入模型中找到所有可用模型的列表。

In [ ]

from llama_index.embeddings.ibm import WatsonxEmbeddings

watsonx_embedding = WatsonxEmbeddings(
    model_id="ibm/slate-30m-english-rtrvr",
    url=URL,
    project_id=PROJECT_ID,
    truncate_input_tokens=truncate_input_tokens,
)

from llama_index.embeddings.ibm import WatsonxEmbeddings watsonx_embedding = WatsonxEmbeddings( model_id="ibm/slate-30m-english-rtrvr", url=URL, project_id=PROJECT_ID, truncate_input_tokens=truncate_input_tokens, )

更改默认设置

In [ ]

from llama_index.core import Settings

Settings.chunk_size = 512

from llama_index.core import Settings Settings.chunk_size = 512

构建索引¶

In [ ]

from llama_index.core import VectorStoreIndex

index = VectorStoreIndex.from_documents(
    documents=documents, embed_model=watsonx_embedding
)

from llama_index.core import VectorStoreIndex index = VectorStoreIndex.from_documents( documents=documents, embed_model=watsonx_embedding )

加载 LLM¶

初始化 WatsonxLLM 实例。¶

关于 WatsonxLLM 的更多信息，请参阅示例笔记本：

你需要指定将用于推理的 model_id。你可以在支持的基础模型中找到所有可用模型的列表。

你可能需要针对不同的模型或任务调整模型 parameters。详情请参阅可用的 MetaNames。

In [ ]

max_new_tokens = 128

max_new_tokens = 128

In [ ]

from llama_index.llms.ibm import WatsonxLLM

watsonx_llm = WatsonxLLM(
    model_id="meta-llama/llama-3-3-70b-instruct",
    url=URL,
    project_id=PROJECT_ID,
    max_new_tokens=max_new_tokens,
)

from llama_index.llms.ibm import WatsonxLLM watsonx_llm = WatsonxLLM( model_id="meta-llama/llama-3-3-70b-instruct", url=URL, project_id=PROJECT_ID, max_new_tokens=max_new_tokens, )

发送查询¶

检索前 10 个最相关的节点，然后使用 WatsonxRerank 进行过滤¶

In [ ]

query_engine = index.as_query_engine(
    llm=watsonx_llm,
    similarity_top_k=10,
    node_postprocessors=[watsonx_rerank],
)
response = query_engine.query(
    "What did Sam Altman do in this essay?",
)

query_engine = index.as_query_engine( llm=watsonx_llm, similarity_top_k=10, node_postprocessors=[watsonx_rerank], ) response = query_engine.query( "What did Sam Altman do in this essay?", )

In [ ]

from llama_index.core.response.pprint_utils import pprint_response

pprint_response(response, show_source=True)

from llama_index.core.response.pprint_utils import pprint_response pprint_response(response, show_source=True)

Final Response: In this essay, Sam Altman was recruited to be the
president of Y Combinator (YC), and he agreed to take over the role
starting with the winter 2014 batch. He initially declined the offer,
wanting to start a startup to make nuclear reactors, but eventually
agreed after being persuaded. He began learning the job and taking
over responsibilities from the author in the latter part of 2013, and
officially took over as president in 2014.
______________________________________________________________________
Source Node 1/2
Node ID: 2ed5d8e7-2681-49b0-a112-ea35cc9a8b9e
Similarity: 3.2075154781341553
Text: "You know," he said, "you should make sure Y Combinator isn't
the last cool thing you do."  At the time I didn't understand what he
meant, but gradually it dawned on me that he was saying I should quit.
This seemed strange advice, because YC was doing great. But if there
was one thing rarer than Rtm offering advice, it was Rtm being wrong.
So th...
______________________________________________________________________
Source Node 2/2
Node ID: 6ae17865-aaa7-46a5-bc49-f38abf4a825e
Similarity: -1.3127477169036865
Text: I asked Jessica if she wanted to be president, but she didn't,
so we decided we'd try to recruit Sam Altman. We talked to Robert and
Trevor and we agreed to make it a complete changing of the guard. Up
till that point YC had been controlled by the original LLC we four had
started. But we wanted YC to last for a long time, and to do that it
could...

直接检索前 2 个最相似的节点¶

In [ ]

query_engine = index.as_query_engine(
    llm=watsonx_llm,
    similarity_top_k=2,
)
response = query_engine.query(
    "What did Sam Altman do in this essay?",
)

query_engine = index.as_query_engine( llm=watsonx_llm, similarity_top_k=2, ) response = query_engine.query( "What did Sam Altman do in this essay?", )

检索到的上下文不相关，并且响应是幻觉产生的。

In [ ]

pprint_response(response, show_source=True)

pprint_response(response, show_source=True)

Final Response: Sam Altman was one of the founders of the first batch
of startups funded by the Summer Founders Program, and he later became
the second president of YC.
______________________________________________________________________
Source Node 1/2
Node ID: ba52769a-7342-4e6c-af02-4159216a79a8
Similarity: 0.6396056863136902
Text: We knew undergrads were deciding then about summer jobs, so in a
matter of days we cooked up something we called the Summer Founders
Program, and I posted an announcement on my site, inviting undergrads
to apply. I had never imagined that writing essays would be a way to
get "deal flow," as investors call it, but it turned out to be the
perfect ...
______________________________________________________________________
Source Node 2/2
Node ID: 43a6cf9f-8284-45db-bbbd-44109fcb9373
Similarity: 0.6334836031239921
Text: I wrote this new Lisp, called Bel, in itself in Arc. That may
sound like a contradiction, but it's an indication of the sort of
trickery I had to engage in to make this work. By means of an
egregious collection of hacks I managed to make something close enough
to an interpreter written in itself that could actually run. Not fast,
but fast enough...