OctoAI¶

如果您在 colab 上打开此 Notebook，您可能需要安装 LlamaIndex 🦙。

In [ ]

%pip install llama-index-llms-octoai
%pip install llama-index
%pip install octoai-sdk

%pip install llama-index-llms-octoai %pip install llama-index %pip install octoai-sdk

在下方包含您的 OctoAI API 密钥。您可以在 OctoAI 获取。

如果您需要更多指导，这里有一些说明。

In [ ]

OCTOAI_API_KEY = ""

OCTOAI_API_KEY = ""

使用默认模型初始化集成¶¶

In [ ]

from llama_index.llms.octoai import OctoAI

octoai = OctoAI(token=OCTOAI_API_KEY)

from llama_index.llms.octoai import OctoAI octoai = OctoAI(token=OCTOAI_API_KEY)

使用提示词调用 complete¶¶

In [ ]

response = octoai.complete("Paul Graham is ")
print(response)

response = octoai.complete("Paul Graham is ") print(response)

使用消息列表调用 chat¶¶

In [ ]

from llama_index.core.llms import ChatMessage

messages = [
    ChatMessage(
        role="system",
        content="Below is an instruction that describes a task. Write a response that appropriately completes the request.",
    ),
    ChatMessage(role="user", content="Write a blog about Seattle"),
]
response = octoai.chat(messages)
print(response)

from llama_index.core.llms import ChatMessage messages = [ ChatMessage( role="system", content="下面是一个描述任务的说明。请撰写一个恰当完成请求的响应。", ), ChatMessage(role="user", content="写一篇关于西雅图的博客"), ] response = octoai.chat(messages) print(response)

流式传输¶¶

使用 stream_complete 端点

In [ ]

response = octoai.stream_complete("Paul Graham is ")
for r in response:
    print(r.delta, end="")

response = octoai.stream_complete("Paul Graham is ") for r in response: print(r.delta, end="")

使用消息列表调用 stream_chat

In [ ]

from llama_index.core.llms import ChatMessage

messages = [
    ChatMessage(
        role="system",
        content="Below is an instruction that describes a task. Write a response that appropriately completes the request.",
    ),
    ChatMessage(role="user", content="Write a blog about Seattle"),
]
response = octoai.stream_chat(messages)
for r in response:
    print(r.delta, end="")

from llama_index.core.llms import ChatMessage messages = [ ChatMessage( role="system", content="下面是一个描述任务的说明。请撰写一个恰当完成请求的响应。", ), ChatMessage(role="user", content="写一篇关于西雅图的博客"), ] response = octoai.stream_chat(messages) for r in response: print(r.delta, end="")

配置模型¶¶

In [ ]

# To customize your API token, do this
# otherwise it will lookup OCTOAI_TOKEN from your env variable
octoai = OctoAI(
    model="mistral-7b-instruct", max_tokens=128, token=OCTOAI_API_KEY
)

response = octoai.complete("Paul Graham is ")
print(response)

# To customize your API token, do this # otherwise it will lookup OCTOAI_TOKEN from your env variable octoai = OctoAI( model="mistral-7b-instruct", max_tokens=128, token=OCTOAI_API_KEY ) response = octoai.complete("Paul Graham is ") print(response)