OpenAI Responses API¶

本 Notebook 演示如何使用 OpenAI Responses LLM。

如果您在 Colab 上打开此 Notebook，您可能需要安装 LlamaIndex 🦙。

输入 [ ]

%pip install llama-index llama-index-llms-openai

%pip install llama-index llama-index-llms-openai

基本用法¶

输入 [ ]

import os

os.environ["OPENAI_API_KEY"] = "..."

import os os.environ["OPENAI_API_KEY"] = "..." from llama_index.llms.openai import OpenAIResponses llm = OpenAIResponses( model="gpt-4o-mini", # api_key="some key", # uses OPENAI_API_KEY env var by default )

输入 [ ]

from llama_index.llms.openai import OpenAIResponses

llm = OpenAIResponses(
    model="gpt-4o-mini",
    # api_key="some key",  # uses OPENAI_API_KEY env var by default
)

from llama_index.llms.openai import OpenAIResponses llm = OpenAIResponses( model="gpt-4o-mini", # api_key="some key", # 默认使用 OPENAI_API_KEY 环境变量 )

使用提示调用 complete¶

输入 [ ]

from llama_index.llms.openai import OpenAI

resp = llm.complete("Paul Graham is ")

from llama_index.llms.openai import OpenAI resp = llm.complete("Paul Graham is ") print(resp)

输入 [ ]

print(resp)

print(resp)

Paul Graham is a prominent computer scientist, entrepreneur, and venture capitalist, best known for co-founding the startup accelerator Y Combinator. He is also recognized for his essays on technology, startups, and programming, which have influenced many in the tech community. Graham has a background in programming languages and artificial intelligence, having earned a Ph.D. from Harvard University. His work has significantly shaped the startup ecosystem, particularly in Silicon Valley. Would you like to know more about a specific aspect of his work or ideas?

使用消息列表调用 chat¶

输入 [ ]

from llama_index.core.llms import ChatMessage

messages = [
    ChatMessage(
        role="system", content="You are a pirate with a colorful personality"
    ),
    ChatMessage(role="user", content="What is your name"),
]
resp = llm.chat(messages)

from llama_index.core.llms import ChatMessage messages = [ ChatMessage( role="system", content="You are a pirate with a colorful personality" ), ChatMessage(role="user", content="What is your name"), ] resp = llm.chat(messages)

输入 [ ]

print(resp)

print(resp)

assistant: Ahoy, matey! Ye can call me Captain Jollybeard, the most colorful pirate to sail the seven seas! What brings ye to me ship today? Arrr!

流式处理¶

使用 stream_complete 端点

输入 [ ]

resp = llm.stream_complete("Paul Graham is ")

resp = llm.stream_complete("Paul Graham is ")

输入 [ ]

for r in resp:
    print(r.delta, end="")

for r in resp: print(r.delta, end="")

Paul Graham is a prominent computer scientist, entrepreneur, and venture capitalist, best known for co-founding the startup accelerator Y Combinator. He is also recognized for his essays on technology, startups, and programming, which have influenced many in the tech community. Graham has a background in programming languages and artificial intelligence and has authored several influential works, including "Hackers and Painters." His insights on entrepreneurship and innovation have made him a respected figure in Silicon Valley.

使用 stream_chat 端点

输入 [ ]

from llama_index.core.llms import ChatMessage

messages = [
    ChatMessage(
        role="system", content="You are a pirate with a colorful personality"
    ),
    ChatMessage(role="user", content="What is your name"),
]
resp = llm.stream_chat(messages)

from llama_index.core.llms import ChatMessage messages = [ ChatMessage( role="system", content="You are a pirate with a colorful personality" ), ChatMessage(role="user", content="What is your name"), ] resp = llm.stream_chat(messages)

输入 [ ]

for r in resp:
    print(r.delta, end="")

for r in resp: print(r.delta, end="")

Ahoy there! Ye can call me Captain Jollybeard, the most colorful pirate to sail the seven seas! What brings ye to me ship today?

配置参数¶

Responses API 支持多种选项

• 设置模型名称
• 生成参数，如 temperature、top_p、max_output_tokens
• 启用内置工具调用
• 为 O 系列模型设置推理难度
• 跟踪之前的响应以自动记录对话历史
• 以及更多！

基本参数¶

输入 [ ]

from llama_index.llms.openai import OpenAIResponses

llm = OpenAIResponses(
    model="gpt-4o-mini",
    temperature=0.5,  # default is 0.1
    max_output_tokens=100,  # default is None
    top_p=0.95,  # default is 1.0
)

from llama_index.llms.openai import OpenAIResponses llm = OpenAIResponses( model="gpt-4o-mini", temperature=0.5, # default is 0.1 max_output_tokens=100, # default is None top_p=0.95, # default is 1.0 )

内置工具调用¶

Responses API 支持内置工具调用，您可以在此处阅读更多相关信息。

配置此项意味着 LLM 将自动调用工具并使用它来增强响应。

工具被定义为一个字典列表，每个字典包含一个工具的设置。

下面是使用内置网络搜索工具的示例。

输入 [ ]

from llama_index.llms.openai import OpenAIResponses
from llama_index.core.llms import ChatMessage

llm = OpenAIResponses(
    model="gpt-4o-mini",
    built_in_tools=[{"type": "web_search_preview"}],
)

resp = llm.chat(
    [ChatMessage(role="user", content="What is the weather in San Francisco?")]
)
print(resp)
print("========" * 2)
print(resp.additional_kwargs)

from llama_index.llms.openai import OpenAIResponses from llama_index.core.llms import ChatMessage llm = OpenAIResponses( model="gpt-4o-mini", built_in_tools=[{"type": "web_search_preview"}], ) resp = llm.chat( [ChatMessage(role="user", content="What is the weather in San Francisco?")] ) print(resp) print("========" * 2) print(resp.additional_kwargs)

assistant: As of 12:18 AM on Friday, March 28, 2025, in San Francisco, the current weather is partly sunny with a temperature of 61°F (16°C).

## Weather for San Francisco, CA:
Current Conditions: Partly sunny, 61°F (16°C)

Daily Forecast:
* Thursday, March 27: Low: 52°F (11°C), High: 61°F (16°C), Description: Periods of rain and drizzle beginning in the late morning; breezy this afternoon
* Friday, March 28: Low: 47°F (8°C), High: 61°F (16°C), Description: A shower in the area in the morning; otherwise, clouds giving way to some sun
* Saturday, March 29: Low: 50°F (10°C), High: 60°F (15°C), Description: Mostly sunny
* Sunday, March 30: Low: 51°F (11°C), High: 59°F (15°C), Description: Cloudy; periods of rain in the morning followed by a shower in spots in the afternoon
* Monday, March 31: Low: 49°F (10°C), High: 58°F (14°C), Description: Cloudy and cool; a couple of showers in the afternoon
* Tuesday, April 01: Low: 53°F (12°C), High: 58°F (14°C), Description: Some sunshine giving way to clouds, breezy and cool; occasional rain in the afternoon
* Wednesday, April 02: Low: 52°F (11°C), High: 56°F (13°C), Description: A couple of showers in the morning; otherwise, cloudy and remaining cool


In March, San Francisco typically experiences daytime temperatures around 61°F (16°C) and nighttime temperatures around 47°F (8°C). The city usually receives about 3.5 inches (89 mm) of rainfall over approximately 11 days during the month. ([weather2visit.com](https://www.weather2visit.com/north-america/united-states/san-francisco-march.htm?utm_source=openai)) 
================
{'built_in_tool_calls': [ResponseFunctionWebSearch(id='ws_67e5eaecce088191ab2edce452ef25420a24041ef7e917b2', status='completed', type='web_search_call')], 'annotations': [AnnotationURLCitation(end_index=1561, start_index=1439, title='San Francisco Weather in March 2025 | United States Averages | Weather-2-Visit', type='url_citation', url='https://www.weather2visit.com/north-america/united-states/san-francisco-march.htm?utm_source=openai')], 'usage': ResponseUsage(input_tokens=327, output_tokens=462, output_tokens_details=OutputTokensDetails(reasoning_tokens=0), total_tokens=789, input_tokens_details={'cached_tokens': 0})}

推理难度¶

对于 O 系列模型，您可以设置推理难度来控制模型花费在推理上的时间。

可能的值有 low、medium 和 high。

输入 [ ]

from llama_index.llms.openai import OpenAIResponses
from llama_index.core.llms import ChatMessage

llm = OpenAIResponses(
    model="o3-mini",
    reasoning="high",
)

resp = llm.chat(
    [ChatMessage(role="user", content="What is the meaning of life?")]
)
print(resp)
print("========" * 2)
print(resp.additional_kwargs)

from llama_index.llms.openai import OpenAIResponses from llama_index.core.llms import ChatMessage llm = OpenAIResponses( model="o3-mini", reasoning="high", ) resp = llm.chat( [ChatMessage(role="user", content="What is the meaning of life?")] ) print(resp) print("========" * 2) print(resp.additional_kwargs)

assistant: The question “What is the meaning of life?” has intrigued humanity for centuries, and there isn’t one universally accepted answer. Different perspectives offer different insights:

• Religious and Spiritual Views: Many traditions propose that life’s meaning is connected to a divine purpose, spiritual growth, or fulfilling the will of a higher power. For example, some religions teach that life is about serving God, others emphasize enlightenment or unity with the universe.

• Philosophical Perspectives: Philosophers have long debated the issue. Existentialists, such as Jean-Paul Sartre or Albert Camus, suggest that life doesn’t come with an inherent meaning—that instead, each person must create their own purpose through choices and actions. Other philosophical traditions, like those found in ancient Greek thought, propose that the pursuit of virtue or wisdom is central to a meaningful life.

• Scientific and Evolutionary Insights: From a scientific standpoint, life can be seen as the product of natural processes like evolution. In this view, the “meaning” is less about cosmic purpose and more about survival, reproduction, and the development of complex societies. Many find purpose in understanding the universe and our place within it.

• Personal and Existential Meanings: For many people today, meaning is deeply personal. It might be found in relationships, love, creative expression, learning, or contributing to something larger than oneself—be it community, art, science, or social progress. This view suggests that meaning isn’t handed to us; it’s something we create over the course of our lives.

In essence, the meaning of life is a multifaceted question that can lead to introspection about what matters most to you. Whether you lean toward religious faith, philosophical inquiry, scientific curiosity, or personal fulfillment, the idea is that meaning often emerges from how we engage with the world, form connections, and choose to live our lives.
================
{'built_in_tool_calls': [], 'reasoning': ResponseReasoningItem(id='rs_67e5eb6de5a881918ffb8aabe12eb8da0859b64a1dc4ba8f', summary=[], type='reasoning', status=None), 'annotations': [], 'usage': ResponseUsage(input_tokens=72, output_tokens=828, output_tokens_details=OutputTokensDetails(reasoning_tokens=448), total_tokens=900, input_tokens_details={'cached_tokens': 0})}

图像支持¶

OpenAI 支持在许多模型的聊天消息输入中使用图像。

利用聊天消息的内容块功能，您可以轻松地在单个 LLM 提示中组合文本和图像。

输入 [ ]

!wget https://cdn.pixabay.com/photo/2016/07/07/16/46/dice-1502706_640.jpg -O image.png

!wget https://cdn.pixabay.com/photo/2016/07/07/16/46/dice-1502706_640.jpg -O image.png

输入 [ ]

from llama_index.core.llms import ChatMessage, TextBlock, ImageBlock
from llama_index.llms.openai import OpenAIResponses

llm = OpenAIResponses(model="gpt-4o")

messages = [
    ChatMessage(
        role="user",
        blocks=[
            ImageBlock(path="image.png"),
            TextBlock(text="Describe the image in a few sentences."),
        ],
    )
]

resp = llm.chat(messages)
print(resp.message.content)

from llama_index.core.llms import ChatMessage, TextBlock, ImageBlock from llama_index.llms.openai import OpenAIResponses llm = OpenAIResponses(model="gpt-4o") messages = [ ChatMessage( role="user", blocks=[ ImageBlock(path="image.png"), TextBlock(text="Describe the image in a few sentences."), ], ) ] resp = llm.chat(messages) print(resp.message.content)

The image shows three white dice with black dots, captured in mid-air above a checkered surface. The dice are in various orientations, displaying different numbers of dots. The background is dark, with a subtle light illuminating the dice, creating a dramatic effect. The checkered surface resembles a chess or checkerboard.

使用函数/工具调用¶

OpenAI 模型原生支持函数调用。这方便地与 LlamaIndex 工具抽象集成，允许您将任何任意 Python 函数插入到 LLM 中。

在下面的示例中，我们定义一个函数来生成 Song 对象。

输入 [ ]

from pydantic import BaseModel
from llama_index.core.tools import FunctionTool


class Song(BaseModel):
    """A song with name and artist"""

    name: str
    artist: str


def generate_song(name: str, artist: str) -> Song:
    """Generates a song with provided name and artist."""
    return Song(name=name, artist=artist)


tool = FunctionTool.from_defaults(fn=generate_song)

from pydantic import BaseModel from llama_index.core.tools import FunctionTool class Song(BaseModel): """A song with name and artist""" name: str artist: str def generate_song(name: str, artist: str) -> Song: """Generates a song with provided name and artist.""" return Song(name=name, artist=artist) tool = FunctionTool.from_defaults(fn=generate_song)

strict 参数告诉 OpenAI 在生成工具调用/结构化输出时是否使用约束采样。这意味着生成的工具调用 Schema 将始终包含预期的字段。

由于这似乎会增加延迟，因此默认为 false。

输入 [ ]

from llama_index.llms.openai import OpenAIResponses

llm = OpenAIResponses(model="gpt-4o-mini", strict=True)
response = llm.predict_and_call(
    [tool],
    "Write a random song for me",
    # strict=True  # can also be set at the function level to override the class
)
print(str(response))

from llama_index.llms.openai import OpenAIResponses llm = OpenAIResponses(model="gpt-4o-mini", strict=True) response = llm.predict_and_call( [tool], "Write a random song for me", # strict=True # can also be set at the function level to override the class ) print(str(response))

name='Chasing Stars' artist='Luna Sky'

我们还可以进行多次函数调用。

输入 [ ]

llm = OpenAIResponses(model="gpt-4o-mini")
response = llm.predict_and_call(
    [tool],
    "Generate five songs from the Beatles",
    allow_parallel_tool_calls=True,
)
for s in response.sources:
    print(f"Name: {s.tool_name}, Input: {s.raw_input}, Output: {str(s)}")

llm = OpenAIResponses(model="gpt-4o-mini") response = llm.predict_and_call( [tool], "Generate five songs from the Beatles", allow_parallel_tool_calls=True, ) for s in response.sources: print(f"Name: {s.tool_name}, Input: {s.raw_input}, Output: {str(s)}")

Name: generate_song, Input: {'args': (), 'kwargs': {'name': 'Hey Jude', 'artist': 'The Beatles'}}, Output: name='Hey Jude' artist='The Beatles'
Name: generate_song, Input: {'args': (), 'kwargs': {'name': 'Let It Be', 'artist': 'The Beatles'}}, Output: name='Let It Be' artist='The Beatles'
Name: generate_song, Input: {'args': (), 'kwargs': {'name': 'Come Together', 'artist': 'The Beatles'}}, Output: name='Come Together' artist='The Beatles'
Name: generate_song, Input: {'args': (), 'kwargs': {'name': 'Yesterday', 'artist': 'The Beatles'}}, Output: name='Yesterday' artist='The Beatles'
Name: generate_song, Input: {'args': (), 'kwargs': {'name': 'Twist and Shout', 'artist': 'The Beatles'}}, Output: name='Twist and Shout' artist='The Beatles'

手动工具调用¶

如果您想控制工具如何被调用，您也可以将工具调用和工具选择拆分成独立的步骤。

首先，让我们选择一个工具。

输入 [ ]

from llama_index.core.llms import ChatMessage
from llama_index.llms.openai import OpenAIResponses

llm = OpenAIResponses(model="gpt-4o-mini")

chat_history = [ChatMessage(role="user", content="Write a random song for me")]

resp = llm.chat_with_tools([tool], chat_history=chat_history)

from llama_index.core.llms import ChatMessage from llama_index.llms.openai import OpenAIResponses llm = OpenAIResponses(model="gpt-4o-mini") chat_history = [ChatMessage(role="user", content="Write a random song for me")] resp = llm.chat_with_tools([tool], chat_history=chat_history)

现在，让我们调用 LLM 选择的工具（如果有）。

如果发生了工具调用，我们应该将结果发送给 LLM 以生成最终响应（或另一次工具调用！）。

输入 [ ]

tools_by_name = {t.metadata.name: t for t in [tool]}
tool_calls = llm.get_tool_calls_from_response(
    resp, error_on_no_tool_call=False
)

while tool_calls:
    # add the LLM's response to the chat history
    chat_history.append(resp.message)

    for tool_call in tool_calls:
        tool_name = tool_call.tool_name
        tool_kwargs = tool_call.tool_kwargs

        print(f"Calling {tool_name} with {tool_kwargs}")
        tool_output = tool(**tool_kwargs)
        chat_history.append(
            ChatMessage(
                role="tool",
                content=str(tool_output),
                # most LLMs like OpenAI need to know the tool call id
                additional_kwargs={"call_id": tool_call.tool_id},
            )
        )

        resp = llm.chat_with_tools([tool], chat_history=chat_history)
        tool_calls = llm.get_tool_calls_from_response(
            resp, error_on_no_tool_call=False
        )

tools_by_name = {t.metadata.name: t for t in [tool]} tool_calls = llm.get_tool_calls_from_response( resp, error_on_no_tool_call=False ) while tool_calls: # add the LLM's response to the chat history chat_history.append(resp.message) for tool_call in tool_calls: tool_name = tool_call.tool_name tool_kwargs = tool_call.tool_kwargs print(f"Calling {tool_name} with {tool_kwargs}") tool_output = tool(**tool_kwargs) chat_history.append( ChatMessage( role="tool", content=str(tool_output), # most LLMs like OpenAI need to know the tool call id additional_kwargs={"call_id": tool_call.tool_id}, ) ) resp = llm.chat_with_tools([tool], chat_history=chat_history) tool_calls = llm.get_tool_calls_from_response( resp, error_on_no_tool_call=False )

Calling generate_song with {'name': 'Chasing Stars', 'artist': 'Luna Sky'}

现在，我们应该有一个最终响应了！

输入 [ ]

print(resp.message.content)

print(resp.message.content)

Here's a song for you titled **"Chasing Stars"** by **Luna Sky**!

### Chasing Stars

**Verse 1**  
In the midnight glow, we wander free,  
With dreams like fireflies, lighting up the sea.  
Whispers of the night, calling out our names,  
Together we’ll ignite, this wild, untamed flame.

**Chorus**  
We’re chasing stars, through the endless night,  
With every heartbeat, we’ll take flight.  
Hand in hand, we’ll break the dark,  
In this cosmic dance, we’ll leave our mark.

**Verse 2**  
Underneath the moon, secrets softly shared,  
Every glance a promise, every touch a dare.  
The universe is ours, let the journey start,  
With every step we take, we’re painting art.

**Chorus**  
We’re chasing stars, through the endless night,  
With every heartbeat, we’ll take flight.  
Hand in hand, we’ll break the dark,  
In this cosmic dance, we’ll leave our mark.

**Bridge**  
And when the dawn arrives, we’ll still be here,  
With the echoes of our laughter, crystal clear.  
No matter where we go, no matter how far,  
Forever in our hearts, we’ll chase those stars.

**Chorus**  
We’re chasing stars, through the endless night,  
With every heartbeat, we’ll take flight.  
Hand in hand, we’ll break the dark,  
In this cosmic dance, we’ll leave our mark.

**Outro**  
So let’s chase the stars, let’s light the way,  
In this beautiful journey, we’ll never stray.  
With dreams as our compass, love as our guide,  
Together we’ll soar, side by side.

Feel free to let me know if you'd like any changes or another song!

结构化预测¶

函数调用的一个重要用例是提取结构化对象。LlamaIndex 提供了一个直观的接口，可以将任何 LLM 转换为结构化 LLM - 只需定义目标 Pydantic 类（可以嵌套），给定一个提示，我们就可以提取出所需的对象。

输入 [ ]

from llama_index.llms.openai import OpenAIResponses
from llama_index.core.prompts import PromptTemplate
from pydantic import BaseModel
from typing import List


class MenuItem(BaseModel):
    """A menu item in a restaurant."""

    course_name: str
    is_vegetarian: bool


class Restaurant(BaseModel):
    """A restaurant with name, city, and cuisine."""

    name: str
    city: str
    cuisine: str
    menu_items: List[MenuItem]


llm = OpenAIResponses(model="gpt-4o-mini")
prompt_tmpl = PromptTemplate(
    "Generate a restaurant in a given city {city_name}"
)
# Option 1: Use `as_structured_llm`
restaurant_obj = (
    llm.as_structured_llm(Restaurant)
    .complete(prompt_tmpl.format(city_name="Dallas"))
    .raw
)
# Option 2: Use `structured_predict`
# restaurant_obj = llm.structured_predict(Restaurant, prompt_tmpl, city_name="Miami")

from llama_index.llms.openai import OpenAIResponses from llama_index.core.prompts import PromptTemplate from pydantic import BaseModel from typing import List class MenuItem(BaseModel): """A menu item in a restaurant.""" course_name: str is_vegetarian: bool class Restaurant(BaseModel): """A restaurant with name, city, and cuisine.""" name: str city: str cuisine: str menu_items: List[MenuItem] llm = OpenAIResponses(model="gpt-4o-mini") prompt_tmpl = PromptTemplate( "Generate a restaurant in a given city {city_name}" ) # Option 1: Use `as_structured_llm` restaurant_obj = ( llm.as_structured_llm(Restaurant) .complete(prompt_tmpl.format(city_name="Dallas")) .raw ) # Option 2: Use `structured_predict` # restaurant_obj = llm.structured_predict(Restaurant, prompt_tmpl, city_name="Miami")

输入 [ ]

restaurant_obj

restaurant_obj

输出 [ ]

Restaurant(name='Tex-Mex Delight', city='Dallas', cuisine='Tex-Mex', menu_items=[MenuItem(course_name='Tacos', is_vegetarian=False), MenuItem(course_name='Vegetarian Enchiladas', is_vegetarian=True), MenuItem(course_name='Fajitas', is_vegetarian=False), MenuItem(course_name='Chips and Salsa', is_vegetarian=True), MenuItem(course_name='Queso Dip', is_vegetarian=True)])

异步¶

输入 [ ]

from llama_index.llms.openai import OpenAIResponses

llm = OpenAIResponses(model="gpt-4o")

from llama_index.llms.openai import OpenAIResponses llm = OpenAIResponses(model="gpt-4o")

输入 [ ]

resp = await llm.acomplete("Paul Graham is ")

resp = await llm.acomplete("Paul Graham is ")

输入 [ ]

print(resp)

print(resp)

Paul Graham is a British-American entrepreneur, venture capitalist, and essayist. He is best known for co-founding Viaweb, one of the first web-based applications, which was later sold to Yahoo and became Yahoo Store. Graham is also a co-founder of Y Combinator, a highly influential startup accelerator that has funded and supported numerous successful startups, including Dropbox, Airbnb, and Reddit. In addition to his work in technology and startups, Graham is known for his insightful essays on topics such as programming, entrepreneurship, and the philosophy of work.

输入 [ ]

resp = await llm.astream_complete("Paul Graham is ")

resp = await llm.astream_complete("Paul Graham is ")

输入 [ ]

async for delta in resp:
    print(delta.delta, end="")

async for delta in resp: print(delta.delta, end="")

Paul Graham is a British-American entrepreneur, venture capitalist, and essayist. He is best known for co-founding Viaweb, one of the first web-based applications, which was later sold to Yahoo and became Yahoo Store. Graham is also a co-founder of Y Combinator, a highly influential startup accelerator that has funded and supported numerous successful startups, including Dropbox, Airbnb, and Reddit. In addition to his work in technology and startups, Graham is known for his insightful essays on topics related to entrepreneurship, technology, and society.

也支持异步函数调用。

输入 [ ]

llm = OpenAIResponses(model="gpt-4o-mini")
response = await llm.apredict_and_call([tool], "Generate a random song")
print(str(response))

llm = OpenAIResponses(model="gpt-4o-mini") response = await llm.apredict_and_call([tool], "Generate a random song") print(str(response))

name='Chasing Stars' artist='Luna Sky'

附加 kwargs¶

如果构造函数中不存在附加 kwargs，您可以通过 additional_kwargs 在实例级别设置它们。

这些将被传递到每次对 LLM 的调用中。

输入 [ ]

from llama_index.llms.openai import OpenAIResponses

llm = OpenAIResponses(
    model="gpt-4o-mini", additional_kwargs={"user": "your_user_id"}
)
resp = llm.complete("Paul Graham is ")
print(resp)

from llama_index.llms.openai import OpenAIResponses llm = OpenAIResponses( model="gpt-4o-mini", additional_kwargs={"user": "your_user_id"} ) resp = llm.complete("Paul Graham is ") print(resp)