Predibase¶

本 Notebook 展示了如何在 Llamaindex 中使用 Predibase 托管的 LLM。您可以将 Predibase 添加到现有的 Llamaindex 工作流中，以：

1. 轻松部署和查询预训练或自定义开源 LLM
2. 实现端到端检索增强生成 (RAG) 系统的运行
3. 只需几行代码即可微调您自己的 LLM

入门¶

1. 在此处注册免费的 Predibase 账户 here
2. 创建账户
3. 转到 Settings > My profile 并 Generate a new API Token。

In [ ]

%pip install llama-index-llms-predibase

%pip install llama-index-llms-predibase

In [ ]

!pip install llama-index --quiet
!pip install predibase --quiet
!pip install sentence-transformers --quiet

!pip install llama-index --quiet !pip install predibase --quiet !pip install sentence-transformers --quiet

In [ ]

import os

os.environ["PREDIBASE_API_TOKEN"] = "{PREDIBASE_API_TOKEN}"
from llama_index.llms.predibase import PredibaseLLM

import os os.environ["PREDIBASE_API_TOKEN"] = "{PREDIBASE_API_TOKEN}" from llama_index.llms.predibase import PredibaseLLM

流程 1：直接查询 Predibase LLM¶

In [ ]

# Predibase-hosted fine-tuned adapter example
llm = PredibaseLLM(
    model_name="mistral-7b",
    predibase_sdk_version=None,  # optional parameter (defaults to the latest Predibase SDK version if omitted)
    adapter_id="e2e_nlg",  # adapter_id is optional
    adapter_version=1,  # optional parameter (applies to Predibase only)
    api_token=None,  # optional parameter for accessing services hosting adapters (e.g., HuggingFace)
    max_new_tokens=512,
    temperature=0.3,
)
# The `model_name` parameter is the Predibase "serverless" base_model ID
# (see https://docs.predibase.com/user-guide/inference/models for the catalog).
# You can also optionally specify a fine-tuned adapter that's hosted on Predibase or HuggingFace
# In the case of Predibase-hosted adapters, you must also specify the adapter_version

# Predibase 托管的微调 Adapter 示例 llm = PredibaseLLM( model_name="mistral-7b", predibase_sdk_version=None, # 可选参数 (如果省略，则默认为最新的 Predibase SDK 版本) adapter_id="e2e_nlg", # adapter_id 是可选的 adapter_version=1, # 可选参数 (仅适用于 Predibase) api_token=None, # 用于访问托管 Adapter 的服务（如 HuggingFace）的可选参数 max_new_tokens=512, temperature=0.3, ) # `model_name` 参数是 Predibase“无服务器”基础模型 ID # (目录请参阅 https://docs.predibase.com/user-guide/inference/models)。 # 您也可以选择指定托管在 Predibase 或 HuggingFace 上的微调 Adapter # 对于 Predibase 托管的 Adapter，您必须同时指定 `adapter_version`

In [ ]

# HuggingFace-hosted fine-tuned adapter example
llm = PredibaseLLM(
    model_name="mistral-7b",
    predibase_sdk_version=None,  # optional parameter (defaults to the latest Predibase SDK version if omitted)
    adapter_id="predibase/e2e_nlg",  # adapter_id is optional
    api_token=os.environ.get(
        "HUGGING_FACE_HUB_TOKEN"
    ),  # optional parameter for accessing services hosting adapters (e.g., HuggingFace)
    max_new_tokens=512,
    temperature=0.3,
)
# The `model_name` parameter is the Predibase "serverless" base_model ID
# (see https://docs.predibase.com/user-guide/inference/models for the catalog).
# You can also optionally specify a fine-tuned adapter that's hosted on Predibase or HuggingFace
# In the case of Predibase-hosted adapters, you can also specify the adapter_version (assumed latest if omitted)

# HuggingFace 托管的微调 Adapter 示例 llm = PredibaseLLM( model_name="mistral-7b", predibase_sdk_version=None, # 可选参数 (如果省略，则默认为最新的 Predibase SDK 版本) adapter_id="predibase/e2e_nlg", # adapter_id 是可选的 api_token=os.environ.get( "HUGGING_FACE_HUB_TOKEN" ), # 用于访问托管 Adapter 的服务（如 HuggingFace）的可选参数 max_new_tokens=512, temperature=0.3, ) # `model_name` 参数是 Predibase“无服务器”基础模型 ID # (目录请参阅 https://docs.predibase.com/user-guide/inference/models)。 # 您也可以选择指定托管在 Predibase 或 HuggingFace 上的微调 Adapter # 对于 Predibase 托管的 Adapter，您也可以指定 `adapter_version`（如果省略则假定为最新版本）

In [ ]

result = llm.complete("Can you recommend me a nice dry white wine?")
print(result)

result = llm.complete("Can you recommend me a nice dry white wine?") print(result)

流程 2：使用 Predibase LLM 进行检索增强生成 (RAG)¶

In [ ]

from llama_index.core import VectorStoreIndex, SimpleDirectoryReader
from llama_index.core.embeddings import resolve_embed_model
from llama_index.core.node_parser import SentenceSplitter

from llama_index.core import VectorStoreIndex, SimpleDirectoryReader from llama_index.core.embeddings import resolve_embed_model from llama_index.core.node_parser import SentenceSplitter

下载数据¶

In [ ]

!mkdir -p 'data/paul_graham/'
!wget 'https://raw.githubusercontent.com/run-llama/llama_index/main/docs/docs/examples/data/paul_graham/paul_graham_essay.txt' -O 'data/paul_graham/paul_graham_essay.txt'

!mkdir -p 'data/paul_graham/' !wget 'https://raw.githubusercontent.com/run-llama/llama_index/main/docs/docs/examples/data/paul_graham/paul_graham_essay.txt' -O 'data/paul_graham/paul_graham_essay.txt'

加载文档¶

In [ ]

documents = SimpleDirectoryReader("./data/paul_graham/").load_data()

documents = SimpleDirectoryReader("./data/paul_graham/").load_data()

配置 Predibase LLM¶

In [ ]

# Predibase-hosted fine-tuned adapter
llm = PredibaseLLM(
    model_name="mistral-7b",
    predibase_sdk_version=None,  # optional parameter (defaults to the latest Predibase SDK version if omitted)
    adapter_id="e2e_nlg",  # adapter_id is optional
    api_token=None,  # optional parameter for accessing services hosting adapters (e.g., HuggingFace)
    temperature=0.3,
    context_window=1024,
)

# Predibase 托管的微调 Adapter llm = PredibaseLLM( model_name="mistral-7b", predibase_sdk_version=None, # 可选参数 (如果省略，则默认为最新的 Predibase SDK 版本) adapter_id="e2e_nlg", # adapter_id 是可选的 api_token=None, # 用于访问托管 Adapter 的服务（如 HuggingFace）的可选参数 temperature=0.3, context_window=1024, )

In [ ]

# HuggingFace-hosted fine-tuned adapter
llm = PredibaseLLM(
    model_name="mistral-7b",
    predibase_sdk_version=None,  # optional parameter (defaults to the latest Predibase SDK version if omitted)
    adapter_id="predibase/e2e_nlg",  # adapter_id is optional
    api_token=os.environ.get(
        "HUGGING_FACE_HUB_TOKEN"
    ),  # optional parameter for accessing services hosting adapters (e.g., HuggingFace)
    temperature=0.3,
    context_window=1024,
)

# HuggingFace 托管的微调 Adapter llm = PredibaseLLM( model_name="mistral-7b", predibase_sdk_version=None, # 可选参数 (如果省略，则默认为最新的 Predibase SDK 版本) adapter_id="predibase/e2e_nlg", # adapter_id 是可选的 api_token=os.environ.get( "HUGGING_FACE_HUB_TOKEN" ), # 用于访问托管 Adapter 的服务（如 HuggingFace）的可选参数 temperature=0.3, context_window=1024, )

In [ ]

embed_model = resolve_embed_model("local:BAAI/bge-small-en-v1.5")
splitter = SentenceSplitter(chunk_size=1024)

embed_model = resolve_embed_model("local:BAAI/bge-small-en-v1.5") splitter = SentenceSplitter(chunk_size=1024)

设置并查询索引¶

In [ ]

index = VectorStoreIndex.from_documents(
    documents, transformations=[splitter], embed_model=embed_model
)
query_engine = index.as_query_engine(llm=llm)
response = query_engine.query("What did the author do growing up?")

index = VectorStoreIndex.from_documents( documents, transformations=[splitter], embed_model=embed_model ) query_engine = index.as_query_engine(llm=llm) response = query_engine.query("What did the author do growing up?")

In [ ]

print(response)

print(response)