MyMagic AI LLM¶

介绍¶

本 Notebook 演示了如何使用 MyMagicAI 对存储在云存储桶中的海量数据进行批量推理。唯一实现的端点是 complete 和 acomplete，它们适用于许多用例，包括补全、摘要和提取。要使用本 Notebook，您需要 MyMagicAI 的 API 密钥 (个人访问令牌) 以及存储在云存储桶中的数据。请点击 MyMagicAI 网站上的“开始使用”注册以获取您的 API 密钥。

设置¶

要设置您的存储桶并授予 MyMagic API 对您的云存储的安全访问权限，请参阅 MyMagic 文档。如果您在 Colab 上打开此 Notebook，您可能需要安装 LlamaIndex 🦙。

In [ ]

%pip install llama-index-llms-mymagic

%pip install llama-index-llms-mymagic

In [ ]

!pip install llama-index

!pip install llama-index

In [ ]

from llama_index.llms.mymagic import MyMagicAI

from llama_index.llms.mymagic import MyMagicAI

In [ ]

llm = MyMagicAI(
    api_key="your-api-key",
    storage_provider="s3",  # s3, gcs
    bucket_name="your-bucket-name",
    session="your-session-name",  # files should be located in this folder on which batch inference will be run
    role_arn="your-role-arn",
    system_prompt="your-system-prompt",
    region="your-bucket-region",
    return_output=False,  # Whether you want MyMagic API to return the output json
    input_json_file=None,  # name of the input file (stored on the bucket)
    list_inputs=None,  # Option to provide inputs as a list in case of small batch
    structured_output=None,  # json schema of the output
)

llm = MyMagicAI( api_key="your-api-key", storage_provider="s3", # s3, gcs bucket_name="your-bucket-name", session="your-session-name", # files should be located in this folder on which batch inference will be run role_arn="your-role-arn", system_prompt="your-system-prompt", region="your-bucket-region", return_output=False, # Whether you want MyMagic API to return the output json input_json_file=None, # name of the input file (stored on the bucket) list_inputs=None, # Option to provide inputs as a list in case of small batch structured_output=None, # json schema of the output )

注意：如果上面将 return_output 设置为 True，max_tokens 应该至少设置为 100

In [ ]

resp = llm.complete(
    question="your-question",
    model="chhoose-model",  # currently we support mistral7b, llama7b, mixtral8x7b, codellama70b, llama70b, more to come...
    max_tokens=5,  # number of tokens to generate, default is 10
)

resp = llm.complete( question="your-question", model="chhoose-model", # currently we support mistral7b, llama7b, mixtral8x7b, codellama70b, llama70b, more to come... max_tokens=5, # number of tokens to generate, default is 10 )

In [ ]

# The response indicated that the final output is stored in your bucket or raises an exception if the job failed
print(resp)

# 响应表明最终输出存储在您的存储桶中，如果作业失败则会引发异常 print(resp)

使用 acomplete 端点进行异步请求¶

对于异步操作，请使用以下方法。

In [ ]

import asyncio

import asyncio

In [ ]

async def main():
    response = await llm.acomplete(
        question="your-question",
        model="choose-model",  # supported models constantly updated and are listed at docs.mymagic.ai
        max_tokens=5,  # number of tokens to generate, default is 10
    )

    print("Async completion response:", response)

async def main(): response = await llm.acomplete( question="your-question", model="choose-model", # supported models constantly updated and are listed at docs.mymagic.ai max_tokens=5, # number of tokens to generate, default is 10 ) print("异步补全响应:", response)

In [ ]

await main()

await main()