使用 RichPromptTemplate 构建¶

RichPromptTemplate 在 llama-index-core==0.12.27 中引入，它是一个新的提示模板，允许你使用 Jinja 语法构建具有丰富格式的提示。

使用它，你可以构建

• 带有变量的基本提示
• 单个字符串中的聊天提示模板
• 接受文本、图像和音频的提示
• 循环或解析对象的高级提示
• 等等！

让我们看一些示例。

In [ ]

%pip install llama-index

%pip install llama-index

带有变量的基本提示¶

在 RichPromptTemplate 中，你可以使用 {{ }} 语法将变量插入到你的提示中。

In [ ]

from llama_index.core.prompts import RichPromptTemplate

prompt = RichPromptTemplate("Hello, {{ name }}!")

from llama_index.core.prompts import RichPromptTemplate prompt = RichPromptTemplate("Hello, {{ name }}!")

你可以将提示格式化为字符串或聊天消息列表。

In [ ]

print(prompt.format(name="John"))

print(prompt.format(name="John"))

Hello, John!

In [ ]

print(prompt.format_messages(name="John"))

print(prompt.format_messages(name="John"))

[ChatMessage(role=<MessageRole.USER: 'user'>, additional_kwargs={}, blocks=[TextBlock(block_type='text', text='Hello, John!')])]

聊天提示模板¶

你还可以直接在提示模板中定义聊天消息块。

In [ ]

prompt = RichPromptTemplate(
    """
{% chat role="system" %}
You are now chatting with {{ user }}
{% endchat %}

{% chat role="user" %}
{{ user_msg }}
{% endchat %}
"""
)

prompt = RichPromptTemplate( """ {% chat role="system" %} You are now chatting with {{ user }} {% endchat %} {% chat role="user" %} {{ user_msg }} {% endchat %} """ )

In [ ]

print(prompt.format_messages(user="John", user_msg="Hello!"))

print(prompt.format_messages(user="John", user_msg="Hello!"))

[ChatMessage(role=<MessageRole.SYSTEM: 'system'>, additional_kwargs={}, blocks=[TextBlock(block_type='text', text='You are now chatting with John')]), ChatMessage(role=<MessageRole.USER: 'user'>, additional_kwargs={}, blocks=[TextBlock(block_type='text', text='Hello!')])]

带有图像和音频的提示¶

假设你使用的 LLM 支持，你也可以在提示中包含图像和音频！

图像¶

In [ ]

!wget https://cdn.pixabay.com/photo/2016/07/07/16/46/dice-1502706_640.jpg -O image.png

!wget https://cdn.pixabay.com/photo/2016/07/07/16/46/dice-1502706_640.jpg -O image.png

In [ ]

from llama_index.llms.openai import OpenAI

llm = OpenAI(model="gpt-4o-mini", api_key="sk-...")

prompt = RichPromptTemplate(
    """
Describe the following image:
{{ image_path | image}}
"""
)

from llama_index.llms.openai import OpenAI llm = OpenAI(model="gpt-4o-mini", api_key="sk-...") prompt = RichPromptTemplate( """ Describe the following image: {{ image_path | image}} """ )

In [ ]

messages = prompt.format_messages(image_path="./image.png")
response = llm.chat(messages)
print(response.message.content)

messages = prompt.format_messages(image_path="./image.png") response = llm.chat(messages) print(response.message.content)

The image features three white dice with black dots, captured in a monochrome setting. The dice are positioned on a checkered surface, which appears to be a wooden board. The background is blurred, creating a sense of depth, while the focus remains on the dice. The overall composition emphasizes the randomness and chance associated with rolling dice.

音频¶

In [ ]

!wget AUDIO_URL = "https://science.nasa.gov/wp-content/uploads/2024/04/sounds-of-mars-one-small-step-earth.wav" -O audio.wav

!wget AUDIO_URL = "https://science.nasa.gov/wp-content/uploads/2024/04/sounds-of-mars-one-small-step-earth.wav" -O audio.wav

In [ ]

prompt = RichPromptTemplate(
    """
Describe the following audio:
{{ audio_path | audio }}
"""
)
messages = prompt.format_messages(audio_path="./audio.wav")

prompt = RichPromptTemplate( """ Describe the following audio: {{ audio_path | audio }} """ ) messages = prompt.format_messages(audio_path="./audio.wav")

In [ ]

llm = OpenAI(model="gpt-4o-audio-preview", api_key="sk-...")
response = llm.chat(messages)
print(response.message.content)

llm = OpenAI(model="gpt-4o-audio-preview", api_key="sk-...") response = llm.chat(messages) print(response.message.content)

The audio features a famous quote, "That's one small step for man, one giant leap for mankind." This statement was made during a significant historical event, symbolizing a monumental achievement for humanity.

[高级] 循环和对象¶

现在，我们可以更进一步。假设我们有一个图像和文本列表，我们想将其包含在提示中。

我们可以使用 {% for x in y %} 循环语法遍历列表，并将图像和文本包含在提示中。

In [ ]

text_and_images = [
    ("This is a test", "./image.png"),
    ("This is another test", "./image.png"),
]

prompt = RichPromptTemplate(
    """
{% for text, image_path in text_and_images %}
Here is some text:
{{ text }}
Here is an image:
{{ image_path | image }}
{% endfor %}
"""
)

messages = prompt.format_messages(text_and_images=text_and_images)

text_and_images = [ ("This is a test", "./image.png"), ("This is another test", "./image.png"), ] prompt = RichPromptTemplate( """ {% for text, image_path in text_and_images %} Here is some text: {{ text }} Here is an image: {{ image_path | image }} {% endfor %} """ ) messages = prompt.format_messages(text_and_images=text_and_images)

让我们检查消息看看我们有什么。

In [ ]

for message in messages:
    print(message.role.value)
    for block in message.blocks:
        print(str(block)[:100])
    print("\n")

for message in messages: print(message.role.value) for block in message.blocks: print(str(block)[:100]) print("\n")

user
block_type='text' text='Here is some text:'
block_type='text' text='This is a test'
block_type='text' text='Here is an image:'
block_type='image' image=None path=None url=AnyUrl('data:image/jpeg;base64,/9j/4AAQSkZJRgABAQAAAQABA
block_type='text' text='Here is some text:'
block_type='text' text='This is another test'
block_type='text' text='Here is an image:'
block_type='image' image=None path=None url=AnyUrl('data:image/jpeg;base64,/9j/4AAQSkZJRgABAQAAAQABA

如你所见，我们有一条包含块列表的消息，每个块代表一个新的内容块（文本或图像）。

（注意：图像在渲染提示时被解析为 base64 编码的字符串）