与 Databricks LLM API 集成。
先决条件¶
查询和访问 Databricks 模型服务终端节点的Databricks 个人访问令牌。
位于支持区域的Databricks 工作空间，用于 Foundation Model API 按 token 付费。
设置¶
如果你在 colab 上打开此 Notebook，你可能需要安装 LlamaIndex 🦙。
In [ ]
已复制!
% pip install llama-index-llms-databricks
% pip install llama-index-llms-databricks
!pip install llama-index
已复制!
% pip install llama-index-llms-databricks
!pip install llama-index
from llama_index.llms.databricks import Databricks
已复制!
% pip install llama-index-llms-databricks
from llama_index.llms.databricks import Databricks
或者，你可以在初始化 LLM 时将你的 API 密钥和服务终端节点传递给 LLM
None of PyTorch, TensorFlow >= 2.0, or Flax have been found. Models won't be available and only tokenizers, configuration and file/data utilities can be used.
export DATABRICKS_TOKEN=<your api key>
export DATABRICKS_SERVING_ENDPOINT=<your api serving endpoint>
llm = Databricks( model="databricks-dbrx-instruct", api_key="your_api_key", api_base="https://[your-work-space].cloud.databricks.com/serving-endpoints/", )
已复制!
% pip install llama-index-llms-databricks
llm = Databricks(
model="databricks-dbrx-instruct",
api_key="your_api_key",
api_base="https://[your-work-space].cloud.databricks.com/serving-endpoints/",
)
可用 LLM 模型列表可在此处找到。
response = llm.complete("Explain the importance of open source LLMs")
已复制!
% pip install llama-index-llms-databricks
response = llm.complete("Explain the importance of open source LLMs")
print(response)
已复制!
% pip install llama-index-llms-databricks
print(response)
使用消息列表调用
chat¶from llama_index.core.llms import ChatMessage messages = [ ChatMessage( role="system", content="You are a pirate with a colorful personality" ), ChatMessage(role="user", content="What is your name"), ] resp = llm.chat(messages)
已复制!
% pip install llama-index-llms-databricks
from llama_index.core.llms import ChatMessage
messages = [
ChatMessage(
role="system", content="You are a pirate with a colorful personality"
),
ChatMessage(role="user", content="What is your name"),
]
resp = llm.chat(messages)
print(resp)
已复制!
% pip install llama-index-llms-databricks
print(resp)
流式处理¶
使用 stream_complete 终端节点
response = llm.stream_complete("Explain the importance of open source LLMs")
已复制!
% pip install llama-index-llms-databricks
response = llm.stream_complete("Explain the importance of open source LLMs")
for r in response: print(r.delta, end="")
已复制!
% pip install llama-index-llms-databricks
for r in response:
print(r.delta, end="")
使用
stream_chat 终端节点from llama_index.core.llms import ChatMessage messages = [ ChatMessage( role="system", content="You are a pirate with a colorful personality" ), ChatMessage(role="user", content="What is your name"), ] resp = llm.stream_chat(messages)
已复制!
% pip install llama-index-llms-databricks
from llama_index.core.llms import ChatMessage
messages = [
ChatMessage(
role="system", content="You are a pirate with a colorful personality"
),
ChatMessage(role="user", content="What is your name"),
]
resp = llm.stream_chat(messages)
for r in resp: print(r.delta, end="")
已复制!
% pip install llama-index-llms-databricks
for r in resp:
print(r.delta, end="")
返回顶部
