OpenRouter¶

OpenRouter 提供一个标准化的 API，以最优价格访问多种大型语言模型。您可以在其主页了解更多信息。

如果您在 Colab 上打开此 Notebook，可能需要安装 LlamaIndex 🦙。

In [ ]

%pip install llama-index-llms-openrouter

%pip install llama-index-llms-openrouter

In [ ]

!pip install llama-index

!pip install llama-index

In [ ]

from llama_index.llms.openrouter import OpenRouter
from llama_index.core.llms import ChatMessage

from llama_index.llms.openrouter import OpenRouter from llama_index.core.llms import ChatMessage

使用 ChatMessage 列表调用 chat¶

您需要设置环境变量 OPENROUTER_API_KEY 或在类构造函数中设置 api_key

In [ ]

# import os
# os.environ['OPENROUTER_API_KEY'] = '<your-api-key>'

llm = OpenRouter(
    api_key="<your-api-key>",
    max_tokens=256,
    context_window=4096,
    model="gryphe/mythomax-l2-13b",
)

# import os # os.environ['OPENROUTER_API_KEY'] = '' llm = OpenRouter( api_key="", max_tokens=256, context_window=4096, model="gryphe/mythomax-l2-13b", )

In [ ]

message = ChatMessage(role="user", content="Tell me a joke")
resp = llm.chat([message])
print(resp)

message = ChatMessage(role="user", content="Tell me a joke") resp = llm.chat([message]) print(resp)

assistant: Why did the tomato turn red? Because it saw the salad dressing!

流式处理¶

In [ ]

message = ChatMessage(role="user", content="Tell me a story in 250 words")
resp = llm.stream_chat([message])
for r in resp:
    print(r.delta, end="")

message = ChatMessage(role="user", content="Tell me a story in 250 words") resp = llm.stream_chat([message]) for r in resp: print(r.delta, end="")

Once upon a time, there was a young girl named Maria who lived in a small village surrounded by lush green forests. Maria was a kind and gentle soul, loved by everyone in the village. She spent most of her days exploring the forests, discovering new species of plants and animals, and helping the villagers with their daily chores.

One day, while Maria was out on a walk, she stumbled upon a hidden path she had never seen before. The path was overgrown with weeds and vines, but something about it called to her. She decided to follow it, and it led her deeper and deeper into the forest.

As she walked, the trees grew taller and the air grew colder. Maria began to feel a sense of unease, but she was determined to see where the path led. Finally, she came to a clearing, and in the center of it stood an enormous tree, its trunk as wide as a house.

Maria approached the tree and saw that it was covered in strange symbols. She reached out to touch one of the symbols, and suddenly, the tree began to glow. The glow grew brighter and brighter, until Maria

使用 Prompt 调用 complete¶

In [ ]

resp = llm.complete("Tell me a joke")
print(resp)

resp = llm.complete("Tell me a joke") print(resp)

Sure, here's a joke for you:

Why couldn't the bicycle stand up by itself?

Because it was two-tired!

I hope that brought a smile to your face!

In [ ]

resp = llm.stream_complete("Tell me a story in 250 words")
for r in resp:
    print(r.delta, end="")

resp = llm.stream_complete("Tell me a story in 250 words") for r in resp: print(r.delta, end="")

Once upon a time, there was a young girl named Maria. She lived in a small village surrounded by lush green forests and sparkling rivers. Maria was a kind and gentle soul, loved by everyone in the village. She spent her days helping her parents with their farm work and exploring the surrounding nature.

One day, while wandering in the forest, Maria stumbled upon a hidden path she had never seen before. She decided to follow it, and it led her to a beautiful meadow filled with wildflowers. In the center of the meadow, she found a small pond, where she saw her own reflection in the water.

As she gazed into the pond, Maria saw a figure approaching her. It was a wise old woman, who introduced herself as the guardian of the meadow. The old woman told Maria that she had been chosen to receive a special gift, one that would bring her great joy and happiness.

The old woman then presented Maria with a small, delicate flower. She told her that this flower had the power to heal any wound, both physical and emotional. Maria was amazed and grateful, and she promised to use the flower wisely.

模型配置¶

In [ ]

# View options at https://openrouter.ai/models
# This example uses Mistral's MoE, Mixtral:
llm = OpenRouter(model="mistralai/mixtral-8x7b-instruct")

# View options at https://openrouter.ai/models # This example uses Mistral's MoE, Mixtral: llm = OpenRouter(model="mistralai/mixtral-8x7b-instruct")

In [ ]

resp = llm.complete("Write a story about a dragon who can code in Rust")
print(resp)

resp = llm.complete("Write a story about a dragon who can code in Rust") print(resp)