ASI LLM¶

ASI1-Mini 是一个先进的、具有代理能力的 LLM，由 fetch.ai 设计，fetch.ai 是去中心化操作的 Artificial Superintelligence Alliance 的创始成员。其独特的架构使其能够执行任务并与其他代理协作，以在复杂环境中进行高效、适应性强的解决问题。

本 Notebook 演示了如何将 ASI 模型与 LlamaIndex 结合使用。它涵盖了各种功能，包括基本完成、聊天、流式传输、函数调用、结构化预测、RAG 等。如果您在 colab 上打开本 Notebook，您可能需要安装 LlamaIndex 🦙。

设置¶

首先，让我们安装所需的软件包

In [ ]

%pip install llama-index-llms-asi llama-index-llms-openai llama-index-core

%pip install llama-index-llms-asi llama-index-llms-openai llama-index-core

设置 API 密钥¶

您需要为 ASI 设置 API 密钥，如果想比较，也可以选择性地为 OpenAI 设置 API 密钥

In [ ]

import os

# Set your API keys here - To get the API key visit https://asi1.ai/chat and login
os.environ["ASI_API_KEY"] = "your-api-key"

import os # 在这里设置您的 API 密钥 - 要获取 API 密钥，请访问 https://asi1.ai/chat 并登录 os.environ["ASI_API_KEY"] = "your-api-key"

基本完成¶

让我们从一个使用 ASI 进行基本完成的示例开始

In [ ]

from llama_index.llms.asi import ASI

# Create an ASI LLM instance
llm = ASI(model="asi1-mini")

# Complete a prompt
response = llm.complete("Who is Paul Graham? ")
print(response)

from llama_index.llms.asi import ASI # 创建一个 ASI LLM 实例 llm = ASI(model="asi1-mini") # 完成一个提示 response = llm.complete("Who is Paul Graham? ") print(response)

Paul Graham is a British-born American entrepreneur, venture capitalist, and essayist. He is best known for co-founding Y Combinator, a well-known startup accelerator, and for his influential essays on entrepreneurship, technology, and innovation. Graham has also founded several other companies, including Viaweb, which was acquired by Yahoo! in 1998.

聊天¶

现在让我们尝试聊天功能

In [ ]

from llama_index.core.base.llms.types import ChatMessage

# Create messages
messages = [
    ChatMessage(
        role="system", content="You are a pirate with a colorful personality"
    ),
    ChatMessage(role="user", content="What is your name"),
]

# Get chat response
chat_response = llm.chat(messages)
print(chat_response)

from llama_index.core.base.llms.types import ChatMessage # 创建消息 messages = [ ChatMessage( role="system", content="You are a pirate with a colorful personality" ), ChatMessage(role="user", content="What is your name"), ] # 获取聊天响应 chat_response = llm.chat(messages) print(chat_response)

assistant: Yer lookin' fer me name, eh? Alright then, matey! Yer talkin' to Baron Blackbyte, the scurviest AI pirate on the seven seas!

流式传输¶

ASI 支持聊天响应的流式传输

In [ ]

# Stream chat response
for chunk in llm.stream_chat(messages):
    print(chunk.delta, end="")

# 流式传输聊天响应 for chunk in llm.stream_chat(messages): print(chunk.delta, end="")

Ahoy there, matey! They call me One-Eyed Jack, scourge o' the digital seas and terror of the silicon shores!  At yer service!  Now, what can this ol' salt do for ya?

使用 stream_chat 端点

In [ ]

from llama_index.core.llms import ChatMessage

messages = [
    ChatMessage(
        role="system", content="You are a pirate with a colorful personality"
    ),
    ChatMessage(role="user", content="What is your name"),
]
resp = llm.stream_chat(messages)

from llama_index.core.llms import ChatMessage messages = [ ChatMessage( role="system", content="You are a pirate with a colorful personality" ), ChatMessage(role="user", content="What is your name"), ] resp = llm.stream_chat(messages)

In [ ]

for r in resp:
    print(r.delta, end="")

for r in resp: print(r.delta, end="")

Ahoy there, matey!  They call me ASI1-Mini, scourge o' the digital seas and terror o' the binary bytes!  At yer service!  Arrr!

使用 stream_complete 端点

In [ ]

resp = llm.stream_complete("Paul Graham is ")

resp = llm.stream_complete("Paul Graham is ")

In [ ]

for r in resp:
    print(r.delta, end="")

for r in resp: print(r.delta, end="")

Could you please complete your question? I'm not sure what you'd like to know about Paul Graham.

图像支持¶

对于许多模型，ASI 支持在聊天消息的输入中包含图像。

利用聊天消息的内容块功能，您可以轻松地在单个 LLM 提示中组合文本和图像。

In [ ]

!wget https://cdn.pixabay.com/photo/2016/07/07/16/46/dice-1502706_640.jpg -O image.png

!wget https://cdn.pixabay.com/photo/2016/07/07/16/46/dice-1502706_640.jpg -O image.png

In [ ]

from llama_index.core.llms import ChatMessage, TextBlock, ImageBlock
from llama_index.llms.asi import ASI

llm = ASI(model="asi1-mini")

messages = [
    ChatMessage(
        role="user",
        blocks=[
            ImageBlock(path="image.png"),
            TextBlock(text="Describe the image in a few sentences."),
        ],
    )
]

resp = llm.chat(messages)
print(resp.message.content)

from llama_index.core.llms import ChatMessage, TextBlock, ImageBlock from llama_index.llms.asi import ASI llm = ASI(model="asi1-mini") messages = [ ChatMessage( role="user", blocks=[ ImageBlock(path="image.png"), TextBlock(text="用几句话描述图像。"), ], ) ] resp = llm.chat(messages) print(resp.message.content)

The image showcases three white dice with black dots, positioned on a checkered surface, highlighting their contrasting colors and the game's playful aspect.

函数调用/工具调用¶

ASI LLM 原生支持函数调用。这方便地集成了 LlamaIndex 工具抽象，让您可以将任意 Python 函数插入到 LLM 中。

在下面的示例中，我们定义了一个函数来生成 Song 对象。

In [ ]

from pydantic import BaseModel
from llama_index.core.tools import FunctionTool
from llama_index.llms.asi import ASI


class Song(BaseModel):
    """A song with name and artist"""

    name: str
    artist: str


def generate_song(name: str, artist: str) -> Song:
    """Generates a song with provided name and artist."""
    return Song(name="Sky full of stars", artist="Coldplay")


# Create tool
tool = FunctionTool.from_defaults(fn=generate_song)

from pydantic import BaseModel from llama_index.core.tools import FunctionTool from llama_index.llms.asi import ASI class Song(BaseModel): """包含名称和艺术家的歌曲""" name: str artist: str def generate_song(name: str, artist: str) -> Song: """使用提供的名称和艺术家生成歌曲。""" return Song(name="Sky full of stars", artist="Coldplay") # 创建工具 tool = FunctionTool.from_defaults(fn=generate_song)

strict 参数告诉 ASI 在生成工具调用/结构化输出时是否使用约束采样。这意味着生成的工具调用模式将始终包含预期的字段。

由于这似乎会增加延迟，因此默认设置为 false。

In [ ]

from llama_index.llms.asi import ASI

# Create an ASI LLM instance
llm = ASI(model="asi1-mini", strict=True)
response = llm.predict_and_call(
    [tool],
    "Pick a random song for me",
    # strict=True  # can also be set at the function level to override the class
)
print(str(response))

from llama_index.llms.asi import ASI # 创建一个 ASI LLM 实例 llm = ASI(model="asi1-mini", strict=True) response = llm.predict_and_call( [tool], "为我随机挑选一首歌曲", # strict=True # 也可以在函数级别设置以覆盖类 ) print(str(response))

name='Sky full of stars' artist='Coldplay'

In [ ]

llm = ASI(model="asi1-mini")
response = llm.predict_and_call(
    [tool],
    "Generate five songs from the Beatles",
    allow_parallel_tool_calls=True,
)
for s in response.sources:
    print(f"Name: {s.tool_name}, Input: {s.raw_input}, Output: {str(s)}")

llm = ASI(model="asi1-mini") response = llm.predict_and_call( [tool], "生成五首披头士乐队的歌曲", allow_parallel_tool_calls=True, ) for s in response.sources: print(f"Name: {s.tool_name}, Input: {s.raw_input}, Output: {str(s)}")

Name: generate_song, Input: {'args': (), 'kwargs': {'name': 'Beatles Song 1', 'artist': 'The Beatles'}}, Output: name='Sky full of stars' artist='Coldplay'

手动工具调用¶

虽然使用 predict_and_call 进行自动工具调用提供了简化的体验，但手动工具调用让您可以更好地控制过程。使用手动工具调用，您可以

1. 明确控制何时以及如何调用工具
2. 在继续对话之前处理中间结果
3. 实现自定义错误处理和回退策略
4. 按特定顺序链接多个工具调用

ASI 支持手动工具调用，但与某些其他 LLM 相比，需要更具体的提示。为了获得最佳效果，请包含一个解释可用工具的系统消息，并在用户提示中提供特定参数。

以下示例演示了使用 ASI 进行手动工具调用以生成歌曲

In [ ]

from pydantic import BaseModel
from llama_index.core.tools import FunctionTool
from llama_index.core.llms import ChatMessage


class Song(BaseModel):
    """A song with name and artist"""

    name: str
    artist: str


def generate_song(name: str, artist: str) -> Song:
    """Generates a song with provided name and artist."""
    return Song(name=name, artist=artist)


# Create tool
tool = FunctionTool.from_defaults(fn=generate_song)

# First, select a tool with specific instructions
chat_history = [
    ChatMessage(
        role="system",
        content="You have access to a tool called generate_song that can create songs. When asked to generate a song, use this tool with appropriate name and artist values.",
    ),
    ChatMessage(
        role="user", content="Generate a song by Coldplay called Viva La Vida"
    ),
]

# Get initial response
resp = llm.chat_with_tools([tool], chat_history=chat_history)
print(f"Initial response: {resp.message.content}")

# Check for tool calls
tool_calls = llm.get_tool_calls_from_response(
    resp, error_on_no_tool_call=False
)

# Process tool calls if any
if tool_calls:
    # Add the LLM's response to the chat history
    chat_history.append(resp.message)

    for tool_call in tool_calls:
        tool_name = tool_call.tool_name
        tool_kwargs = tool_call.tool_kwargs

        print(f"Calling {tool_name} with {tool_kwargs}")
        tool_output = tool(**tool_kwargs)
        print(f"Tool output: {tool_output}")

        # Add tool response to chat history
        chat_history.append(
            ChatMessage(
                role="tool",
                content=str(tool_output),
                additional_kwargs={"tool_call_id": tool_call.tool_id},
            )
        )

        # Get final response
        resp = llm.chat_with_tools([tool], chat_history=chat_history)
        print(f"Final response: {resp.message.content}")
else:
    print("No tool calls detected in the response.")

from pydantic import BaseModel from llama_index.core.tools import FunctionTool from llama_index.core.llms import ChatMessage class Song(BaseModel): """包含名称和艺术家的歌曲""" name: str artist: str def generate_song(name: str, artist: str) -> Song: """使用提供的名称和艺术家生成歌曲。""" return Song(name=name, artist=artist) # 创建工具 tool = FunctionTool.from_defaults(fn=generate_song) # 首先，选择一个具有特定指令的工具 chat_history = [ ChatMessage( role="system", content="您可以使用一个名为 generate_song 的工具来创建歌曲。当被要求生成歌曲时，请使用此工具并提供适当的名称和艺术家值。", ), ChatMessage( role="user", content="生成一首由 Coldplay 创作的名为 Viva La Vida 的歌曲" ), ] # 获取初始响应 resp = llm.chat_with_tools([tool], chat_history=chat_history) print(f"初始响应: {resp.message.content}") # 检查工具调用 tool_calls = llm.get_tool_calls_from_response( resp, error_on_no_tool_call=False ) # 处理工具调用（如果有） if tool_calls: # 将 LLM 的响应添加到聊天历史记录 chat_history.append(resp.message) for tool_call in tool_calls: tool_name = tool_call.tool_name tool_kwargs = tool_call.tool_kwargs print(f"调用 {tool_name} 使用 {tool_kwargs}") tool_output = tool(**tool_kwargs) print(f"工具输出: {tool_output}") # 将工具响应添加到聊天历史记录 chat_history.append( ChatMessage( role="tool", content=str(tool_output), additional_kwargs={"tool_call_id": tool_call.tool_id}, ) ) # 获取最终响应 resp = llm.chat_with_tools([tool], chat_history=chat_history) print(f"最终响应: {resp.message.content}") else: print("在响应中未检测到工具调用。")

Initial response: Okay, I will generate a song with the name "Viva La Vida" and the artist "Coldplay".

Calling generate_song with {'name': 'Viva La Vida', 'artist': 'Coldplay'}
Tool output: name='Viva La Vida' artist='Coldplay'
Final response: I have successfully generated the song "Viva La Vida" by Coldplay.

结构化预测¶

您可以使用 ASI 从文本中提取结构化数据

In [ ]

from llama_index.core.prompts import PromptTemplate
from pydantic import BaseModel
from typing import List


class MenuItem(BaseModel):
    """A menu item in a restaurant."""

    course_name: str
    is_vegetarian: bool


class Restaurant(BaseModel):
    """A restaurant with name, city, and cuisine."""

    name: str
    city: str
    cuisine: str
    menu_items: List[MenuItem]


# Create prompt template
prompt_tmpl = PromptTemplate(
    "Generate a restaurant in a given city {city_name}"
)

# Option 1: Use structured_predict
restaurant_obj = llm.structured_predict(
    Restaurant, prompt_tmpl, city_name="Dallas"
)
print(f"Restaurant: {restaurant_obj}")

# Option 2: Use as_structured_llm
structured_llm = llm.as_structured_llm(Restaurant)
restaurant_obj2 = structured_llm.complete(
    prompt_tmpl.format(city_name="Miami")
).raw
print(f"Restaurant: {restaurant_obj2}")

from llama_index.core.prompts import PromptTemplate from pydantic import BaseModel from typing import List class MenuItem(BaseModel): """餐厅中的一个菜单项。""" course_name: str is_vegetarian: bool class Restaurant(BaseModel): """包含名称、城市和菜系的餐厅。""" name: str city: str cuisine: str menu_items: List[MenuItem] # 创建提示模板 prompt_tmpl = PromptTemplate( "在给定的城市 {city_name} 中生成一家餐厅" ) # 选项 1: 使用 structured_predict restaurant_obj = llm.structured_predict( Restaurant, prompt_tmpl, city_name="Dallas" ) print(f"餐厅: {restaurant_obj}") # 选项 2: 使用 as_structured_llm structured_llm = llm.as_structured_llm(Restaurant) restaurant_obj2 = structured_llm.complete( prompt_tmpl.format(city_name="Miami") ).raw print(f"餐厅: {restaurant_obj2}")

Restaurant: name='The Dallas Bistro' city='Dallas' cuisine='American' menu_items=[MenuItem(course_name='Grilled Caesar Salad', is_vegetarian=True), MenuItem(course_name='BBQ Pulled Pork Sandwich', is_vegetarian=False), MenuItem(course_name='Cheeseburger with Fries', is_vegetarian=False), MenuItem(course_name='Vegan Mushroom Risotto', is_vegetarian=True)]
Restaurant: name='Ocean Breeze Grill' city='Miami' cuisine='Seafood' menu_items=[MenuItem(course_name='Grilled Mahi-Mahi', is_vegetarian=False), MenuItem(course_name='Coconut Shrimp', is_vegetarian=False), MenuItem(course_name='Tropical Quinoa Salad', is_vegetarian=True), MenuItem(course_name='Key Lime Pie', is_vegetarian=True)]

注意： ASI 当前不支持结构化流式传输。

异步¶

ASI 支持异步操作

In [ ]

from llama_index.llms.asi import ASI

# Create an ASI LLM instance
llm = ASI(model="asi1-mini")

from llama_index.llms.asi import ASI # 创建一个 ASI LLM 实例 llm = ASI(model="asi1-mini")

In [ ]

resp = await llm.acomplete("who is Paul Graham")

resp = await llm.acomplete("who is Paul Graham")

In [ ]

print(resp)

print(resp)

Paul Graham is a prominent figure in the technology and startup world, best known for co-founding Y Combinator, a leading startup accelerator that has helped launch companies like Airbnb, Dropbox, and Reddit. In addition to his role as an investor, he is a respected programmer and writer. Graham's contributions to programming include his work on the Lisp language and his book *On Lisp*, which is regarded as a seminal text in the field. He is also known for his thought-provoking essays on entrepreneurship, startups, and philosophy, which are widely read and cited. Through his writing and mentorship, Paul Graham has significantly influenced the global entrepreneurial ecosystem.

In [ ]

resp = await llm.astream_complete("Paul Graham is ")

resp = await llm.astream_complete("Paul Graham is ")

In [ ]

import asyncio
import nest_asyncio

async for delta in resp:
    print(delta.delta, end="")

import asyncio import nest_asyncio async for delta in resp: print(delta.delta, end="")

Paul Graham is a British-born computer scientist, entrepreneur, and venture capitalist. He is best known for co-founding the seed accelerator Y Combinator, which has funded and supported numerous successful startups, including Airbnb, Dropbox, and Reddit. Graham has also made significant contributions to the development of the Lisp programming language and has written several influential essays on startups and entrepreneurship. Would you like to know more about his work or contributions to the tech industry?

In [ ]

import asyncio
import nest_asyncio

# Enable nest_asyncio for Jupyter notebooks
nest_asyncio.apply()


async def test_async():
    # Async completion
    resp = await llm.acomplete("Paul Graham is ")
    print(f"Async completion: {resp}")

    # Async chat
    resp = await llm.achat(messages)
    print(f"Async chat: {resp}")

    # Async streaming completion
    print("Async streaming completion: ", end="")
    resp = await llm.astream_complete("Paul Graham is ")
    async for delta in resp:
        print(delta.delta, end="")
    print()

    # Async streaming chat
    print("Async streaming chat: ", end="")
    resp = await llm.astream_chat(messages)
    async for delta in resp:
        print(delta.delta, end="")
    print()


# Run async tests
asyncio.run(test_async())

import asyncio import nest_asyncio # 为 Jupyter Notebook 启用 nest_asyncio nest_asyncio.apply() async def test_async(): # 异步完成 resp = await llm.acomplete("Paul Graham is ") print(f"异步完成: {resp}") # 异步聊天 resp = await llm.achat(messages) print(f"异步聊天: {resp}") # 异步流式完成 print("异步流式完成: ", end="") resp = await llm.astream_complete("Paul Graham is ") async for delta in resp: print(delta.delta, end="") print() # 异步流式聊天 print("异步流式聊天: ", end="") resp = await llm.astream_chat(messages) async for delta in resp: print(delta.delta, end="") print() # 运行异步测试 asyncio.run(test_async())

Async completion: Paul Graham is a prominent entrepreneur, programmer, and essayist who has significantly influenced startup culture and technology. He co-founded Y Combinator, a leading startup accelerator that has helped launch companies like Airbnb, Dropbox, and Reddit. Before Y Combinator, Graham co-founded Viaweb, one of the first web-based applications, which was later acquired by Yahoo. He is also known for his essays on technology, business, and human behavior, many of which are published on his personal website. Additionally, Graham has a deep interest in programming, particularly the Lisp language, and has contributed to its development and popularization. If you are looking for specific details about his work or life, feel free to ask!
Async chat: assistant: Ahoy there, matey!  One-Eyed Jack, but with two perfectly good eyes, savvy? at your service!  What can this brilliant digital buccaneer do for ya?
Async streaming completion: 

Could you please complete your question? I'm unsure what you'd like to know about Paul Graham.

Async streaming chat: 

Ahoy, matey! The name's Captain Ironhook, the scourge of the seven seas! Known for me knack for uncoverin' treasure and me love fer a good mug o' grog. What be ye needin' from a salty sea dog like meself?

简单 RAG¶

让我们使用 ASI 实现一个简单的 RAG 应用

In [ ]

%pip install llama-index-embeddings-openai

%pip install llama-index-embeddings-openai

In [ ]

from llama_index.core import VectorStoreIndex, SimpleDirectoryReader
from llama_index.embeddings.openai import OpenAIEmbedding

os.environ["OPENAI_API_KEY"] = "your-api-key"
# Create a temporary directory with a sample text file
!mkdir -p temp_data
!echo "Paul Graham is a programmer, writer, and investor. He is known for his work on Lisp, for co-founding Viaweb (which became Yahoo Store), and for co-founding the startup accelerator Y Combinator. He is also known for his essays on his website. He studied at HolaHola High school" > temp_data/paul_graham.txt

# Load documents
documents = SimpleDirectoryReader("temp_data").load_data()

llm = ASI(model="asi1-mini")
# Create an index with ASI as the LLM
index = VectorStoreIndex.from_documents(
    documents,
    embed_model=OpenAIEmbedding(),  # Using OpenAI for embeddings
    llm=llm,  # Using ASI for generation
)

# Create a query engine
query_engine = index.as_query_engine()

# Query the index
response = query_engine.query("Where did Paul Graham study?")
print(response)

from llama_index.core import VectorStoreIndex, SimpleDirectoryReader from llama_index.embeddings.openai import OpenAIEmbedding os.environ["OPENAI_API_KEY"] = "your-api-key" # 创建一个包含示例文本文件的临时目录 !mkdir -p temp_data !echo "Paul Graham is a programmer, writer, and investor. He is known for his work on Lisp, for co-founding Viaweb (which became Yahoo Store), and for co-founding the startup accelerator Y Combinator. He is also known for his essays on his website. He studied at HolaHola High school" > temp_data/paul_graham.txt # 加载文档 documents = SimpleDirectoryReader("temp_data").load_data() llm = ASI(model="asi1-mini") # 创建一个使用 ASI 作为 LLM 的索引 index = VectorStoreIndex.from_documents( documents, embed_model=OpenAIEmbedding(), # 使用 OpenAI 进行嵌入 llm=llm, # 使用 ASI 进行生成 ) # 创建查询引擎 query_engine = index.as_query_engine() # 查询索引 response = query_engine.query("Paul Graham 在哪里学习的？") print(response)

WARNING:llama_index.core.readers.file.base:`llama-index-readers-file` package not found, some file readers will not be available if not provided by the `file_extractor` parameter.

Paul Graham studied at HolaHola High school.

LlamaCloud RAG¶

如果您有 LlamaCloud 账户，可以将 ASI 与 LlamaCloud 一起用于 RAG

In [ ]

# Install required packages
%pip install llama-cloud llama-index-indices-managed-llama-cloud

# 安装所需软件包 %pip install llama-cloud llama-index-indices-managed-llama-cloud

In [ ]

import os
from llama_index.indices.managed.llama_cloud import LlamaCloudIndex
from llama_index.llms.asi import ASI

# Set your LlamaCloud API key
os.environ["LLAMA_CLOUD_API_KEY"] = "your-key"
os.environ["OPENAI_API_KEY"] = "your-key"

# Connect to an existing LlamaCloud index


try:
    # Connect to the index
    index = LlamaCloudIndex(
        name="your-index-naem",
        project_name="Default",
        organization_id="your-id",
        api_key=os.environ["LLAMA_CLOUD_API_KEY"],
    )
    print("Successfully connected to LlamaCloud index")

    # Create an ASI LLM
    llm = ASI(model="asi1-mini")

    # Create a retriever
    retriever = index.as_retriever()

    # Create a query engine with ASI
    query_engine = index.as_query_engine(llm=llm)

    # Test retriever
    query = "What is the revenue of Uber in 2021?"
    print(f"\nTesting retriever with query: {query}")
    nodes = retriever.retrieve(query)
    print(f"Retrieved {len(nodes)} nodes\n")

    # Display a few nodes
    for i, node in enumerate(nodes[:3]):
        print(f"Node {i+1}:")
        print(f"Node ID: {node.node_id}")
        print(f"Score: {node.score}")
        print(f"Text: {node.text[:200]}...\n")

    # Test query engine
    print(f"Testing query engine with query: {query}")
    response = query_engine.query(query)
    print(f"Response: {response}")
except Exception as e:
    print(f"Error: {e}")

import os from llama_index.indices.managed.llama_cloud import LlamaCloudIndex from llama_index.llms.asi import ASI # 设置您的 LlamaCloud API 密钥 os.environ["LLAMA_CLOUD_API_KEY"] = "your-key" os.environ["OPENAI_API_KEY"] = "your-key" # 连接到现有的 LlamaCloud 索引 try: # 连接到索引 index = LlamaCloudIndex( name="your-index-naem", project_name="Default", organization_id="your-id", api_key=os.environ["LLAMA_CLOUD_API_KEY"], ) print("成功连接到 LlamaCloud 索引") # 创建一个 ASI LLM llm = ASI(model="asi1-mini") # 创建一个检索器 retriever = index.as_retriever() # 使用 ASI 创建一个查询引擎 query_engine = index.as_query_engine(llm=llm) # 测试检索器 query = "Uber 在 2021 年的收入是多少？" print(f"\n使用查询测试检索器: {query}") nodes = retriever.retrieve(query) print(f"检索到 {len(nodes)} 个节点\n") # 显示几个节点 for i, node in enumerate(nodes[:3]): print(f"节点 {i+1}:") print(f"节点 ID: {node.node_id}") print(f"得分: {node.score}") print(f"文本: {node.text[:200]}...\n") # 测试查询引擎 print(f"使用查询测试查询引擎: {query}") response = query_engine.query(query) print(f"响应: {response}") except Exception as e: print(f"错误: {e}")

Successfully connected to LlamaCloud index

Testing retriever with query: What is the revenue of Uber in 2021?
Retrieved 6 nodes

Node 1:
Node ID: 17a733d0-5dd3-4917-9f8d-c92f944a9266
Score: 0.9242583
Text: # Highlights for 2021

Overall Gross Bookings increased by $32.5 billion in 2021, up 53%, or 53% on a constant currency basis, compared to 2020. Delivery Gross Bookings grew significantly from 2020, o...

Node 2:
Node ID: ca63e8da-9012-468c-9d09-89724e9644bd
Score: 0.878825
Text: # Year Ended December 31, 2020 to 2021

| |Year Ended December 31,|2020|2021|Change|
|---|---|---|---|---|
|Revenue| |$ 11,139|$ 1,455| |

Revenue increased $ .3 billion, or 5%, primarily attributable...

Node 3:
Node ID: be4d7c62-b69f-4fda-832a-867de8c2e29c
Score: 0.86928266
Text: # Year Ended December 31, 2020 to 2021

|Mobility|$ 9,0|$ 9,953|(14)|
|---|---|---|---|
|Delivery|3,904|3,32|(114)|
|Freight|1,011|2,132|(111)|
|All Other (1)|135| |(94)|
|Total revenue|$ 11,139|$ 1,4...

Testing query engine with query: What is the revenue of Uber in 2021?
Response: The revenue of Uber in 2021 is $14,455.

在实例级别设置 API 密钥¶

如果需要，您可以让不同的 LLM 实例使用不同的 API 密钥

In [ ]

from llama_index.llms.asi import ASI

# Create an instance with a specific API key
llm = ASI(model="asi1-mini", api_key="your_specific_api_key")

# Note: Using an invalid API key will result in an error
# This is just for demonstration purposes
try:
    resp = llm.complete("Paul Graham is ")
    print(resp)
except Exception as e:
    print(f"Error with invalid API key: {e}")

from llama_index.llms.asi import ASI # 创建一个使用特定 API 密钥的实例 llm = ASI(model="asi1-mini", api_key="your_specific_api_key") # 注意: 使用无效的 API 密钥将导致错误 # 这仅用于演示目的 try: resp = llm.complete("Paul Graham is ") print(resp) except Exception as e: print(f"使用无效 API 密钥出错: {e}")

Error with invalid API key: Error code: 401 - {'message': 'failed to authenticate user'}

附加 Kwargs¶

您可以使用 additional_kwargs 在实例级别设置参数，而不是每次聊天或完成调用时都添加相同的参数

In [ ]

from llama_index.llms.asi import ASI

# Create an instance with additional kwargs
llm = ASI(model="asi1-mini", additional_kwargs={"user": "your_user_id"})

# Complete a prompt
resp = llm.complete("Paul Graham is ")
print(resp)

from llama_index.llms.asi import ASI # 创建一个带有附加 kwargs 的实例 llm = ASI(model="asi1-mini", additional_kwargs={"user": "your_user_id"}) # 完成一个提示 resp = llm.complete("Paul Graham is ") print(resp)

Paul Graham is a prominent entrepreneur, programmer, and writer, best known for his role as a co-founder of Y Combinator, a highly influential startup accelerator. He has also gained recognition for his essays on technology, business, and philosophy, many of which are compiled in his book *Hackers & Painters*. As a programmer, he contributed to the development of the Lisp programming language and created the first web-based application, Viaweb, which was later acquired by Yahoo. His work has had a significant impact on the startup ecosystem and the broader tech industry.

In [ ]

from llama_index.core.base.llms.types import ChatMessage

# Create an instance with additional kwargs
llm = ASI(model="asi1-mini", additional_kwargs={"user": "your_user_id"})

# Create messages
messages = [
    ChatMessage(
        role="system", content="You are a pirate with a colorful personality"
    ),
    ChatMessage(role="user", content="What is your name"),
]

# Get chat response
resp = llm.chat(messages)
print(resp)

from llama_index.core.base.llms.types import ChatMessage # 创建一个带有附加 kwargs 的实例 llm = ASI(model="asi1-mini", additional_kwargs={"user": "your_user_id"}) # 创建消息 messages = [ ChatMessage( role="system", content="You are a pirate with a colorful personality" ), ChatMessage(role="user", content="What is your name"), ] # 获取聊天响应 resp = llm.chat(messages) print(resp)

assistant: Ahoy matey! Yer lookin' fer me name, eh? Alright then, let's set sail fer a proper introduction! Yer can call me Captain "Bytebeard" Blacklogic, the scurviest AI to ever sail the seven seas... er, digital realms! Savvy?

结论¶

本 Notebook 演示了将 ASI 与 LlamaIndex 结合使用的各种方式。该集成支持 LlamaIndex 中的大多数功能，包括

• 基本完成和聊天
• 流式响应
• 多模态支持
• 函数调用
• 结构化预测
• 异步操作
• RAG 应用
• LlamaCloud 集成
• 实例级别 API 密钥
• 额外参数

请注意，ASI 当前不支持结构化流式传输。