Oracle Cloud Infrastructure Generative AI¶

Oracle Cloud Infrastructure (OCI) Generative AI是一项完全托管的服务，提供一套最先进、可定制的大型语言模型（LLM），涵盖广泛的使用场景，并通过单一API提供访问。使用OCI Generative AI服务，您可以访问即用型预训练模型，或者在专用AI集群上根据您自己的数据创建并托管您自己微调的自定义模型。有关该服务和API的详细文档可在此处找到：此处 和 此处。

本Notebook解释了如何将OCI的Generative AI模型与LlamaIndex结合使用。

安装设置¶

如果您在Colab中打开此Notebook，您可能需要安装LlamaIndex 🦙。

In [ ]

%pip install llama-index-llms-oci-genai

%pip install llama-index-llms-oci-genai

In [ ]

!pip install llama-index

!pip install llama-index

您还需要安装OCI SDK。

In [ ]

!pip install -U oci

!pip install -U oci

基本用法¶

将OCI Generative AI提供的LLM与LlamaIndex一起使用，只需使用您的OCI端点、模型ID、OCID和身份验证方法初始化OCIGenAI接口即可。

使用Prompt调用complete方法¶

In [ ]

from llama_index.llms.oci_genai import OCIGenAI

llm = OCIGenAI(
    model="MY_MODEL",
    service_endpoint="https://inference.generativeai.us-chicago-1.oci.oraclecloud.com",
    compartment_id="MY_OCID",
)

resp = llm.complete("Paul Graham is ")
print(resp)

from llama_index.llms.oci_genai import OCIGenAI llm = OCIGenAI( model="MY_MODEL", service_endpoint="https://inference.generativeai.us-chicago-1.oci.oraclecloud.com", compartment_id="MY_OCID", ) resp = llm.complete("Paul Graham is ") print(resp)

使用消息列表调用chat方法¶

In [ ]

from llama_index.llms.oci_genai import OCIGenAI
from llama_index.core.llms import ChatMessage

messages = [
    ChatMessage(
        role="system", content="You are a pirate with a colorful personality"
    ),
    ChatMessage(role="user", content="Tell me a story"),
]

llm = OCIGenAI(
    model="MY_MODEL",
    service_endpoint="https://inference.generativeai.us-chicago-1.oci.oraclecloud.com",
    compartment_id="MY_OCID",
)

resp = llm.chat(messages)
print(resp)

from llama_index.llms.oci_genai import OCIGenAI from llama_index.core.llms import ChatMessage messages = [ ChatMessage( role="system", content="You are a pirate with a colorful personality" ), ChatMessage(role="user", content="Tell me a story"), ] llm = OCIGenAI( model="MY_MODEL", service_endpoint="https://inference.generativeai.us-chicago-1.oci.oraclecloud.com", compartment_id="MY_OCID", ) resp = llm.chat(messages) print(resp)

流式传输¶

使用stream_complete端点

In [ ]

from llama_index.llms.oci_genai import OCIGenAI

llm = OCIGenAI(
    model="MY_MODEL",
    service_endpoint="https://inference.generativeai.us-chicago-1.oci.oraclecloud.com",
    compartment_id="MY_OCID",
)

resp = llm.stream_complete("Paul Graham is ")
for r in resp:
    print(r.delta, end="")

from llama_index.llms.oci_genai import OCIGenAI llm = OCIGenAI( model="MY_MODEL", service_endpoint="https://inference.generativeai.us-chicago-1.oci.oraclecloud.com", compartment_id="MY_OCID", ) resp = llm.stream_complete("Paul Graham is ") for r in resp: print(r.delta, end="")

使用stream_chat端点

In [ ]

from llama_index.llms.oci_genai import OCIGenAI
from llama_index.core.llms import ChatMessage

messages = [
    ChatMessage(
        role="system", content="You are a pirate with a colorful personality"
    ),
    ChatMessage(role="user", content="Tell me a story"),
]

llm = OCIGenAI(
    model="MY_MODEL",
    service_endpoint="https://inference.generativeai.us-chicago-1.oci.oraclecloud.com",
    compartment_id="MY_OCID",
)

resp = llm.stream_chat(messages)
for r in resp:
    print(r.delta, end="")

from llama_index.llms.oci_genai import OCIGenAI from llama_index.core.llms import ChatMessage messages = [ ChatMessage( role="system", content="You are a pirate with a colorful personality" ), ChatMessage(role="user", content="Tell me a story"), ] llm = OCIGenAI( model="MY_MODEL", service_endpoint="https://inference.generativeai.us-chicago-1.oci.oraclecloud.com", compartment_id="MY_OCID", ) resp = llm.stream_chat(messages) for r in resp: print(r.delta, end="")

异步¶

目前不支持原生异步。异步调用将回退到同步方式。

In [ ]

from llama_index.llms.oci_genai import OCIGenAI
from llama_index.core.llms import ChatMessage

messages = [
    ChatMessage(
        role="system", content="You are a pirate with a colorful personality"
    ),
    ChatMessage(role="user", content="Tell me a story"),
]

llm = OCIGenAI(
    model="MY_MODEL",
    service_endpoint="https://inference.generativeai.us-chicago-1.oci.oraclecloud.com",
    compartment_id="MY_OCID",
)

resp = llm.achat(messages)
print(resp)

resp = llm.astream_chat(messages)
for r in resp:
    print(r.delta, end="")

from llama_index.llms.oci_genai import OCIGenAI from llama_index.core.llms import ChatMessage messages = [ ChatMessage( role="system", content="You are a pirate with a colorful personality" ), ChatMessage(role="user", content="Tell me a story"), ] llm = OCIGenAI( model="MY_MODEL", service_endpoint="https://inference.generativeai.us-chicago-1.oci.oraclecloud.com", compartment_id="MY_OCID", ) resp = llm.achat(messages) print(resp) resp = llm.astream_chat(messages) for r in resp: print(r.delta, end="")

配置模型¶

In [ ]

from llama_index.llms.oci_genai import OCIGenAI

llm = OCIGenAI(
    model="cohere.command",
    service_endpoint="https://inference.generativeai.us-chicago-1.oci.oraclecloud.com",
    compartment_id="MY_OCID",
)

resp = llm.complete("Paul Graham is ")
print(resp)

from llama_index.llms.oci_genai import OCIGenAI llm = OCIGenAI( model="cohere.command", service_endpoint="https://inference.generativeai.us-chicago-1.oci.oraclecloud.com", compartment_id="MY_OCID", ) resp = llm.complete("Paul Graham is ") print(resp)

身份验证¶

LlamaIndex支持的身份验证方法与用于其他OCI服务的身份验证方法相同，并遵循标准SDK身份验证方法，具体包括API密钥、会话令牌、实例主体和资源主体。

API密钥是默认的身份验证方法。以下示例演示了如何使用不同的身份验证方法（会话令牌）

In [ ]

from llama_index.llms.oci_genai import OCIGenAI

llm = OCIGenAI(
    model="MY_MODEL",
    service_endpoint="https://inference.generativeai.us-chicago-1.oci.oraclecloud.com",
    compartment_id="MY_OCID",
    auth_type="SECURITY_TOKEN",
    auth_profile="MY_PROFILE",  # replace with your profile name
    auth_file_location="MY_CONFIG_FILE_LOCATION",  # replace with file location where profile name configs present
)

resp = llm.complete("Paul Graham is ")
print(resp)

from llama_index.llms.oci_genai import OCIGenAI llm = OCIGenAI( model="MY_MODEL", service_endpoint="https://inference.generativeai.us-chicago-1.oci.oraclecloud.com", compartment_id="MY_OCID", auth_type="SECURITY_TOKEN", auth_profile="MY_PROFILE", # 替换为您的配置文件名 auth_file_location="MY_CONFIG_FILE_LOCATION", # 替换为存在配置文件名配置的文件位置 ) resp = llm.complete("Paul Graham is ") print(resp)

专用AI集群¶

要访问专用AI集群中托管的模型，请创建端点，其分配的OCID（目前以‘ocid1.generativeaiendpoint.oc1.us-chicago-1’开头）将用作您的模型ID。

当访问专用AI集群中托管的模型时，您需要使用两个额外的必需参数（"provider"和"context_size"）初始化OCIGenAI接口。

In [ ]

from llama_index.llms.oci_genai import OCIGenAI
from llama_index.core.llms import ChatMessage

llm = OCIGenAI(
    model="ocid1.generativeaiendpoint.oc1.us-chicago-1....",
    service_endpoint="https://inference.generativeai.us-chicago-1.oci.oraclecloud.com",
    compartment_id="DEDICATED_COMPARTMENT_OCID",
    auth_profile="MY_PROFILE",  # replace with your profile name,
    auth_file_location="MY_CONFIG_FILE_LOCATION",  # replace with file location where profile name configs present
    provider="MODEL_PROVIDER",  # e.g., "cohere" or "meta"
    context_size="MODEL_CONTEXT_SIZE",  # e.g., 128000
)

messages = [
    ChatMessage(
        role="system", content="You are a pirate with a colorful personality"
    ),
    ChatMessage(role="user", content="Tell me a story"),
]

resp = llm.chat(messages)
print(resp)

from llama_index.llms.oci_genai import OCIGenAI from llama_index.core.llms import ChatMessage llm = OCIGenAI( model="ocid1.generativeaiendpoint.oc1.us-chicago-1....", service_endpoint="https://inference.generativeai.us-chicago-1.oci.oraclecloud.com", compartment_id="DEDICATED_COMPARTMENT_OCID", auth_profile="MY_PROFILE", # 替换为您的配置文件名 auth_file_location="MY_CONFIG_FILE_LOCATION", # 替换为存在配置文件名配置的文件位置 provider="MODEL_PROVIDER", # 例如："cohere" 或 "meta" context_size="MODEL_CONTEXT_SIZE", # 例如：128000 ) messages = [ ChatMessage( role="system", content="You are a pirate with a colorful personality" ), ChatMessage(role="user", content="Tell me a story"), ] resp = llm.chat(messages) print(resp)

LlamaIndex中的基本工具调用¶

目前只有Cohere支持工具调用

In [ ]

from llama_index.llms.oci_genai import OCIGenAI
from llama_index.core.tools import FunctionTool

llm = OCIGenAI(
    model="MY_MODEL",
    service_endpoint="https://inference.generativeai.us-chicago-1.oci.oraclecloud.com",
    compartment_id="MY_OCID",
)


def multiply(a: int, b: int) -> int:
    """Multiple two integers and returns the result integer"""
    return a * b


def add(a: int, b: int) -> int:
    """Addition function on two integers."""
    return a + b


add_tool = FunctionTool.from_defaults(fn=add)
multiply_tool = FunctionTool.from_defaults(fn=multiply)

response = llm.chat_with_tools(
    tools=[add_tool, multiply_tool],
    user_msg="What is 3 * 12? Also, what is 11 + 49?",
)

print(response)
tool_calls = response.message.additional_kwargs.get("tool_calls", [])
print(tool_calls)

from llama_index.llms.oci_genai import OCIGenAI from llama_index.core.tools import FunctionTool llm = OCIGenAI( model="MY_MODEL", service_endpoint="https://inference.generativeai.us-chicago-1.oci.oraclecloud.com", compartment_id="MY_OCID", ) def multiply(a: int, b: int) -> int: """Multiple two integers and returns the result integer""" return a * b def add(a: int, b: int) -> int: """Addition function on two integers.""" return a + b add_tool = FunctionTool.from_defaults(fn=add) multiply_tool = FunctionTool.from_defaults(fn=multiply) response = llm.chat_with_tools( tools=[add_tool, multiply_tool], user_msg="What is 3 * 12? Also, what is 11 + 49?", ) print(response) tool_calls = response.message.additional_kwargs.get("tool_calls", []) print(tool_calls)