Nebius AI LLM¶

本 notebook 演示了如何将来自 Nebius AI Studio 的 LLMs 与 LlamaIndex 一起使用。Nebius AI Studio 实现了所有可用于商业用途的最新 LLMs。

首先，让我们安装 LlamaIndex 和 Nebius AI Studio 的依赖项。

In [ ]

%pip install llama-index-llms-nebius llama-index

%pip install llama-index-llms-nebius llama-index

从下面的系统变量上传您的 Nebius AI Studio 密钥或直接插入。您可以免费注册 Nebius AI Studio 并在API Keys 部分获取密钥。"

In [ ]

import os

NEBIUS_API_KEY = os.getenv("NEBIUS_API_KEY")  # NEBIUS_API_KEY = ""

import os NEBIUS_API_KEY = os.getenv("NEBIUS_API_KEY") # NEBIUS_API_KEY = ""

In [ ]

from llama_index.llms.nebius import NebiusLLM

llm = NebiusLLM(
    api_key=NEBIUS_API_KEY, model="meta-llama/Llama-3.3-70B-Instruct-fast"
)

from llama_index.llms.nebius import NebiusLLM llm = NebiusLLM( api_key=NEBIUS_API_KEY, model="meta-llama/Llama-3.3-70B-Instruct-fast" )

None of PyTorch, TensorFlow >= 2.0, or Flax have been found. Models won't be available and only tokenizers, configuration and file/data utilities can be used.

使用 prompt 调用 complete¶

In [ ]

response = llm.complete("Amsterdam is the capital of ")
print(response)

response = llm.complete("Amsterdam is the capital of ") print(response)

The Netherlands! Amsterdam is indeed the capital and largest city of the Netherlands.

使用消息列表调用 chat¶

In [ ]

from llama_index.core.llms import ChatMessage

messages = [
    ChatMessage(role="system", content="You are a helpful AI assistant."),
    ChatMessage(
        role="user",
        content="Answer briefly: who is Wall-e?",
    ),
]
response = llm.chat(messages)
print(response)

from llama_index.core.llms import ChatMessage messages = [ ChatMessage(role="system", content="你是一个有用的AI助手。"), ChatMessage( role="user", content="简短回答：谁是Wall-e?", ), ] response = llm.chat(messages) print(response)

assistant: WALL-E is a small waste-collecting robot and the main character in the 2008 Pixar animated film of the same name.

流式处理¶

使用 stream_complete 端点¶

In [ ]

response = llm.stream_complete("Amsterdam is the capital of ")
for r in response:
    print(r.delta, end="")

response = llm.stream_complete("Amsterdam is the capital of ") for r in response: print(r.delta, end="")

The Netherlands! Amsterdam is indeed the capital and largest city of the Netherlands.

使用消息列表调用 stream_chat¶

In [ ]

from llama_index.core.llms import ChatMessage

messages = [
    ChatMessage(role="system", content="You are a helpful AI assistant."),
    ChatMessage(
        role="user",
        content="Answer briefly: who is Wall-e?",
    ),
]
response = llm.stream_chat(messages)
for r in response:
    print(r.delta, end="")

from llama_index.core.llms import ChatMessage messages = [ ChatMessage(role="system", content="你是一个有用的AI助手。"), ChatMessage( role="user", content="简短回答：谁是Wall-e?", ), ] response = llm.stream_chat(messages) for r in response: print(r.delta, end="")

WALL-E is a small waste-collecting robot and the main character in the 2008 Pixar animated film of the same name.