Oracle Cloud Infrastructure Data Science¶

Oracle Cloud Infrastructure (OCI) Data Science 是一个完全托管、无服务器的平台，供数据科学团队在 Oracle Cloud Infrastructure 中构建、训练和管理机器学习模型。

它提供了 AI Quick Actions，可用于在 OCI Data Science 中部署、评估和微调基础 LLM 模型。AI Quick Actions 面向希望快速利用 AI 能力的用户。它们旨在通过提供一个精简、无代码且高效的环境来扩展基础模型的使用范围，让更广泛的用户能够使用基础模型。AI Quick Actions 可从 Data Science Notebook 访问。

关于如何使用 AI Quick Actions 在 OCI Data Science 中部署 LLM 模型的详细文档可在此处 here 和此处 here 查看。

本 Notebook 解释了如何在 LlamaIndex 中使用 OCI 的 Data Science 模型。

设置¶

如果您在 colab 上打开本 Notebook，您可能需要安装 LlamaIndex 🦙。

In [ ]

已复制!

%pip install llama-index-llms-oci-data-science
%pip install llama-index-llms-oci-data-science

In [ ]

已复制!

!pip install llama-index
!pip install llama-index

您还需要安装 oracle-ads SDK。

In [ ]

已复制!

!pip install -U oracle-ads
!pip install -U oracle-ads

身份验证¶

LlamaIndex 支持的身份验证方法与用于其他 OCI 服务的方法相同，并遵循标准 SDK 身份验证方法，特别是 API 密钥、会话令牌、实例主体和资源主体。更多详细信息可在 here 找到。请确保拥有所需的 policies 以访问 OCI Data Science 模型部署端点。oracle-ads 有助于简化 OCI Data Science 中的身份验证。

基本用法¶

使用 OCI Data Science AI 提供的 LLM 与 LlamaIndex 只需要您使用您的 Data Science 模型部署端点和模型 ID 初始化 OCIDataScience 接口。默认情况下，AI Quick Actions 中的所有已部署模型都获取 odsc-model ID。但是，此 ID 可以在部署期间更改。

使用 Prompt 调用 `complete`¶

In [ ]

已复制!

import ads
from llama_index.llms.oci_data_science import OCIDataScience

ads.set_auth(auth="security_token", profile="<replace-with-your-profile>")

llm = OCIDataScience(
    model="odsc-llm",
    endpoint="https://<MD_OCID>/predict",
)
response = llm.complete("Tell me a joke")

print(response)
import ads from llama_index.llms.oci_data_science import OCIDataScience ads.set_auth(auth="security_token", profile="/predict", ) llm = OCIDataScience( model="odsc-llm", endpoint="https:///predict", ) response = llm.complete("给我讲个笑话") print(response)

使用消息列表调用 `chat`¶

In [ ]

已复制!





import ads
from llama_index.llms.oci_data_science import OCIDataScience
from llama_index.core.base.llms.types import ChatMessage

ads.set_auth(auth="security_token", profile="<replace-with-your-profile>")

llm = OCIDataScience(
    model="odsc-llm",
    endpoint="https://<MD_OCID>/predict",
)
response = llm.chat(
    [
        ChatMessage(role="user", content="Tell me a joke"),
        ChatMessage(
            role="assistant", content="Why did the chicken cross the road?"
        ),
        ChatMessage(role="user", content="I don't know, why?"),
    ]
)

print(response)
import ads from llama_index.llms.oci_data_science import OCIDataScience from llama_index.core.base.llms.types import ChatMessage ads.set_auth(auth="security_token", profile="/predict", ) llm = OCIDataScience( model="odsc-llm", endpoint="https:///predict", ) response = llm.chat( [ ChatMessage(role="user", content="给我讲个笑话"), ChatMessage( role="assistant", content="鸡为什么要过马路？" ), ChatMessage(role="user", content="我不知道，为什么？"), ] ) print(response)

流式传输¶

使用 `stream_complete` 端点¶

In [ ]

已复制!

import ads
from llama_index.llms.oci_data_science import OCIDataScience

ads.set_auth(auth="security_token", profile="<replace-with-your-profile>")

llm = OCIDataScience(
    model="odsc-llm",
    endpoint="https://<MD_OCID>/predict",
)

for chunk in llm.stream_complete("Tell me a joke"):
    print(chunk.delta, end="")
import ads from llama_index.llms.oci_data_science import OCIDataScience ads.set_auth(auth="security_token", profile="/predict", ) llm = OCIDataScience( model="odsc-llm", endpoint="https:///predict", ) for chunk in llm.stream_complete("给我讲个笑话"): print(chunk.delta, end="")

使用 `stream_chat` 端点¶

In [ ]

已复制!





import ads
from llama_index.llms.oci_data_science import OCIDataScience
from llama_index.core.base.llms.types import ChatMessage

ads.set_auth(auth="security_token", profile="<replace-with-your-profile>")

llm = OCIDataScience(
    model="odsc-llm",
    endpoint="https://<MD_OCID>/predict",
)
response = llm.stream_chat(
    [
        ChatMessage(role="user", content="Tell me a joke"),
        ChatMessage(
            role="assistant", content="Why did the chicken cross the road?"
        ),
        ChatMessage(role="user", content="I don't know, why?"),
    ]
)

for chunk in response:
    print(chunk.delta, end="")
import ads from llama_index.llms.oci_data_science import OCIDataScience from llama_index.core.base.llms.types import ChatMessage ads.set_auth(auth="security_token", profile="/predict", ) llm = OCIDataScience( model="odsc-llm", endpoint="https:///predict", ) response = llm.stream_chat( [ ChatMessage(role="user", content="给我讲个笑话"), ChatMessage( role="assistant", content="鸡为什么要过马路？" ), ChatMessage(role="user", content="我不知道，为什么？"), ] ) for chunk in response: print(chunk.delta, end="")

异步¶

使用 Prompt 调用 `acomplete`¶

In [ ]

已复制!

import ads
from llama_index.llms.oci_data_science import OCIDataScience

ads.set_auth(auth="security_token", profile="<replace-with-your-profile>")

llm = OCIDataScience(
    model="odsc-llm",
    endpoint="https://<MD_OCID>/predict",
)
response = await llm.acomplete("Tell me a joke")

print(response)
import ads from llama_index.llms.oci_data_science import OCIDataScience ads.set_auth(auth="security_token", profile="/predict", ) llm = OCIDataScience( model="odsc-llm", endpoint="https:///predict", ) response = await llm.acomplete("给我讲个笑话") print(response)

使用消息列表调用 `achat`¶

In [ ]

已复制!





import ads
from llama_index.llms.oci_data_science import OCIDataScience
from llama_index.core.base.llms.types import ChatMessage

ads.set_auth(auth="security_token", profile="<replace-with-your-profile>")

llm = OCIDataScience(
    model="odsc-llm",
    endpoint="https://<MD_OCID>/predict",
)
response = await llm.achat(
    [
        ChatMessage(role="user", content="Tell me a joke"),
        ChatMessage(
            role="assistant", content="Why did the chicken cross the road?"
        ),
        ChatMessage(role="user", content="I don't know, why?"),
    ]
)

print(response)
import ads from llama_index.llms.oci_data_science import OCIDataScience from llama_index.core.base.llms.types import ChatMessage ads.set_auth(auth="security_token", profile="/predict", ) llm = OCIDataScience( model="odsc-llm", endpoint="https:///predict", ) response = await llm.achat( [ ChatMessage(role="user", content="给我讲个笑话"), ChatMessage( role="assistant", content="鸡为什么要过马路？" ), ChatMessage(role="user", content="我不知道，为什么？"), ] ) print(response)

使用 `astream_complete` 端点¶

In [ ]

已复制!

import ads
from llama_index.llms.oci_data_science import OCIDataScience

ads.set_auth(auth="security_token", profile="<replace-with-your-profile>")

llm = OCIDataScience(
    model="odsc-llm",
    endpoint="https://<MD_OCID>/predict",
)

async for chunk in await llm.astream_complete("Tell me a joke"):
    print(chunk.delta, end="")
import ads from llama_index.llms.oci_data_science import OCIDataScience ads.set_auth(auth="security_token", profile="/predict", ) llm = OCIDataScience( model="odsc-llm", endpoint="https:///predict", ) async for chunk in await llm.astream_complete("给我讲个笑话"): print(chunk.delta, end="")

使用 `astream_chat` 端点¶

In [ ]

已复制!





import ads
from llama_index.llms.oci_data_science import OCIDataScience
from llama_index.core.base.llms.types import ChatMessage

ads.set_auth(auth="security_token", profile="<replace-with-your-profile>")

llm = OCIDataScience(
    model="odsc-llm",
    endpoint="https://<MD_OCID>/predict",
)
response = await llm.stream_chat(
    [
        ChatMessage(role="user", content="Tell me a joke"),
        ChatMessage(
            role="assistant", content="Why did the chicken cross the road?"
        ),
        ChatMessage(role="user", content="I don't know, why?"),
    ]
)

async for chunk in response:
    print(chunk.delta, end="")
import ads from llama_index.llms.oci_data_science import OCIDataScience from llama_index.core.base.llms.types import ChatMessage ads.set_auth(auth="security_token", profile="/predict", ) llm = OCIDataScience( model="odsc-llm", endpoint="https:///predict", ) response = await llm.stream_chat( [ ChatMessage(role="user", content="给我讲个笑话"), ChatMessage( role="assistant", content="鸡为什么要过马路？" ), ChatMessage(role="user", content="我不知道，为什么？"), ] ) async for chunk in response: print(chunk.delta, end="")

配置模型¶

In [ ]

已复制!





import ads
from llama_index.llms.oci_data_science import OCIDataScience

ads.set_auth(auth="security_token", profile="<replace-with-your-profile>")

llm = OCIDataScience(
    model="odsc-llm",
    endpoint="https://<MD_OCID>/predict",
    temperature=0.2,
    max_tokens=500,
    timeout=120,
    context_window=2500,
    additional_kwargs={
        "top_p": 0.75,
        "logprobs": True,
        "top_logprobs": 3,
    },
)
response = llm.chat(
    [
        ChatMessage(role="user", content="Tell me a joke"),
    ]
)
print(response)
import ads from llama_index.llms.oci_data_science import OCIDataScience ads.set_auth(auth="security_token", profile="/predict", ) llm = OCIDataScience( model="odsc-llm", endpoint="https:///predict", temperature=0.2, max_tokens=500, timeout=120, context_window=2500, additional_kwargs={ "top_p": 0.75, "logprobs": True, "top_logprobs": 3, }, ) response = llm.chat( [ ChatMessage(role="user", content="给我讲个笑话"), ] ) print(response)

函数调用¶

AI Quick Actions 提供预构建的服务容器，使部署和提供大型语言模型变得非常容易。服务容器中使用 vLLM（一个用于 LLM 的高吞吐量和内存高效的推理和服务引擎）或 TGI（一个用于流行开源 LLM 的高性能文本生成服务器）来托管模型，创建的端点支持 OpenAI API 协议。这使得模型部署可以作为使用 OpenAI API 的应用程序的直接替代品。如果部署的模型支持函数调用，那么通过 llm 上的 predict_and_call 函数与 LlamaIndex 工具集成，允许附加任何工具并让 LLM 决定调用哪些工具（如果需要）。

In [ ]

已复制!





import ads
from llama_index.llms.oci_data_science import OCIDataScience
from llama_index.core.tools import FunctionTool

ads.set_auth(auth="security_token", profile="<replace-with-your-profile>")

llm = OCIDataScience(
    model="odsc-llm",
    endpoint="https://<MD_OCID>/predict",
    temperature=0.2,
    max_tokens=500,
    timeout=120,
    context_window=2500,
    additional_kwargs={
        "top_p": 0.75,
        "logprobs": True,
        "top_logprobs": 3,
    },
)


def multiply(a: float, b: float) -> float:
    print(f"---> {a} * {b}")
    return a * b


def add(a: float, b: float) -> float:
    print(f"---> {a} + {b}")
    return a + b


def subtract(a: float, b: float) -> float:
    print(f"---> {a} - {b}")
    return a - b


def divide(a: float, b: float) -> float:
    print(f"---> {a} / {b}")
    return a / b


multiply_tool = FunctionTool.from_defaults(fn=multiply)
add_tool = FunctionTool.from_defaults(fn=add)
sub_tool = FunctionTool.from_defaults(fn=subtract)
divide_tool = FunctionTool.from_defaults(fn=divide)

response = llm.predict_and_call(
    [multiply_tool, add_tool, sub_tool, divide_tool],
    user_msg="Calculate the result of `8 + 2 - 6`.",
    verbose=True,
)

print(response)
import ads from llama_index.llms.oci_data_science import OCIDataScience from llama_index.core.tools import FunctionTool ads.set_auth(auth="security_token", profile="/predict", ) llm = OCIDataScience( model="odsc-llm", endpoint="https:///predict", temperature=0.2, max_tokens=500, timeout=120, context_window=2500, additional_kwargs={ "top_p": 0.75, "logprobs": True, "top_logprobs": 3, }, ) def multiply(a: float, b: float) -> float: print(f"---> {a} * {b}") return a * b def add(a: float, b: float) -> float: print(f"---> {a} + {b}") return a + b def subtract(a: float, b: float) -> float: print(f"---> {a} - {b}") return a - b def divide(a: float, b: float) -> float: print(f"---> {a} / {b}") return a / b multiply_tool = FunctionTool.from_defaults(fn=multiply) add_tool = FunctionTool.from_defaults(fn=add) sub_tool = FunctionTool.from_defaults(fn=subtract) divide_tool = FunctionTool.from_defaults(fn=divide) response = llm.predict_and_call( [multiply_tool, add_tool, sub_tool, divide_tool], user_msg="计算 `8 + 2 - 6` 的结果。", verbose=True, ) print(response)

使用 `FunctionCallingAgent`¶

In [ ]

已复制!





import ads
from llama_index.llms.oci_data_science import OCIDataScience
from llama_index.core.tools import FunctionTool
from llama_index.core.agent import FunctionCallingAgent

ads.set_auth(auth="security_token", profile="<replace-with-your-profile>")

llm = OCIDataScience(
    model="odsc-llm",
    endpoint="https://<MD_OCID>/predict",
    temperature=0.2,
    max_tokens=500,
    timeout=120,
    context_window=2500,
    additional_kwargs={
        "top_p": 0.75,
        "logprobs": True,
        "top_logprobs": 3,
    },
)


def multiply(a: float, b: float) -> float:
    print(f"---> {a} * {b}")
    return a * b


def add(a: float, b: float) -> float:
    print(f"---> {a} + {b}")
    return a + b


def subtract(a: float, b: float) -> float:
    print(f"---> {a} - {b}")
    return a - b


def divide(a: float, b: float) -> float:
    print(f"---> {a} / {b}")
    return a / b


multiply_tool = FunctionTool.from_defaults(fn=multiply)
add_tool = FunctionTool.from_defaults(fn=add)
sub_tool = FunctionTool.from_defaults(fn=subtract)
divide_tool = FunctionTool.from_defaults(fn=divide)

agent = FunctionCallingAgent.from_tools(
    tools=[multiply_tool, add_tool, sub_tool, divide_tool],
    llm=llm,
    verbose=True,
)
response = agent.chat(
    "Calculate the result of `8 + 2 - 6`. Use tools. Return the calculated result."
)

print(response)
import ads from llama_index.llms.oci_data_science import OCIDataScience from llama_index.core.tools import FunctionTool from llama_index.core.agent import FunctionCallingAgent ads.set_auth(auth="security_token", profile="/predict", ) llm = OCIDataScience( model="odsc-llm", endpoint="https:///predict", temperature=0.2, max_tokens=500, timeout=120, context_window=2500, additional_kwargs={ "top_p": 0.75, "logprobs": True, "top_logprobs": 3, }, ) def multiply(a: float, b: float) -> float: print(f"---> {a} * {b}") return a * b def add(a: float, b: float) -> float: print(f"---> {a} + {b}") return a + b def subtract(a: float, b: float) -> float: print(f"---> {a} - {b}") return a - b def divide(a: float, b: float) -> float: print(f"---> {a} / {b}") return a / b multiply_tool = FunctionTool.from_defaults(fn=multiply) add_tool = FunctionTool.from_defaults(fn=add) sub_tool = FunctionTool.from_defaults(fn=subtract) divide_tool = FunctionTool.from_defaults(fn=divide) agent = FunctionCallingAgent.from_tools( tools=[multiply_tool, add_tool, sub_tool, divide_tool], llm=llm, verbose=True, ) response = agent.chat( "计算 `8 + 2 - 6` 的结果。使用工具。返回计算结果。" ) print(response)

Oracle Cloud Infrastructure Data Science¶

设置¶

身份验证¶

基本用法¶

使用 Prompt 调用 complete¶

使用消息列表调用 chat¶

流式传输¶

使用 stream_complete 端点¶

使用 stream_chat 端点¶

异步¶

使用 Prompt 调用 acomplete¶

使用消息列表调用 achat¶

使用 astream_complete 端点¶

使用 astream_chat 端点¶

配置模型¶

函数调用¶

使用 FunctionCallingAgent¶

使用 Prompt 调用 `complete`¶

使用消息列表调用 `chat`¶

使用 `stream_complete` 端点¶

使用 `stream_chat` 端点¶

使用 Prompt 调用 `acomplete`¶

使用消息列表调用 `achat`¶

使用 `astream_complete` 端点¶

使用 `astream_chat` 端点¶

使用 `FunctionCallingAgent`¶