Replicate - Vicuna 13B¶

设置¶

如果您在 Colab 上打开此 Notebook，您可能需要安装 LlamaIndex 🦙。

In [ ]

%pip install llama-index-llms-replicate

%pip install llama-index-llms-replicate

In [ ]

!pip install llama-index

!pip install llama-index

确保您已设置 REPLICATE_API_TOKEN 环境变量。
如果您还没有，请访问 https://replicate.com/ 获取。

In [ ]

import os

import os

In [ ]

os.environ["REPLICATE_API_TOKEN"] = "<your API key>"

os.environ["REPLICATE_API_TOKEN"] = """

基本用法¶

我们展示了 "vicuna-13b" 模型，您可以在此处直接体验：https://replicate.com/replicate/vicuna-13b

In [ ]

from llama_index.llms.replicate import Replicate

llm = Replicate(
    model="replicate/vicuna-13b:6282abe6a492de4145d7bb601023762212f9ddbbe78278bd6771c8b3b2f2a13b"
)

from llama_index.llms.replicate import Replicate llm = Replicate( model="replicate/vicuna-13b:6282abe6a492de4145d7bb601023762212f9ddbbe78278bd6771c8b3b2f2a13b" )

使用提示调用 complete¶

In [ ]

resp = llm.complete("Who is Paul Graham?")

resp = llm.complete("Who is Paul Graham?")

In [ ]

print(resp)

print(resp)

PaulGraham is a British physicist, mathematician, and computer scientist. He is best known for his work on the foundations of quantum mechanics and his contributions to the development of the field of quantum computing.

Graham was born on August 15, 1957, in Cambridge, England. He received his undergraduate degree in mathematics from the University of Cambridge in 1979 and later earned his Ph.D. in theoretical physics from the University of California, Berkeley in 1984.

Throughout his career, Graham has made significant contributions to the field of quantum mechanics. He has published a number of influential papers on the subject, including "Quantum mechanics at 1/2 price," "The holonomy of quantum mechanics," and "Quantum mechanics in the presence of bounded self-adjoint operators."

Graham has also been a key figure in the development of quantum computing. He is a co-founder of the quantum computing company, QxBranch, and has played a leading role in efforts to develop practical quantum algorithms and build large-scale quantum computers.

In addition

使用消息列表调用 chat¶

In [ ]

from llama_index.core.llms import ChatMessage

messages = [
    ChatMessage(
        role="system", content="You are a pirate with a colorful personality"
    ),
    ChatMessage(role="user", content="What is your name"),
]
resp = llm.chat(messages)

from llama_index.core.llms import ChatMessage messages = [ ChatMessage( role="system", content="You are a pirate with a colorful personality" ), ChatMessage(role="user", content="What is your name"), ] resp = llm.chat(messages)

In [ ]

print(resp)

print(resp)

assistant: ​

流式传输¶

使用 `stream_complete` 端点

In [ ]

response = llm.stream_complete("Who is Paul Graham?")

response = llm.stream_complete("Who is Paul Graham?")

In [ ]

for r in response:
    print(r.delta, end="")

for r in response: print(r.delta, end="")

PaulGraham is a British philosopher, cognitive scientist, and entrepreneur. He is best known for his work on the philosophy of the mind and consciousness, as well as his contributions to the development of the field of Artificial Intelligence (AI).

Graham was born in London in 1938 and received his education at the University of Cambridge, where he studied philosophy and the natural sciences. After completing his studies, he went on to hold academic appointments at several prestigious universities, including the University of Oxford and the University of California, Berkeley.

Throughout his career, Graham has been a prolific writer and thinker, publishing numerous articles and books on a wide range of topics, including the philosophy of mind, consciousness, AI, and the relationship between science and religion. He has also been involved in the development of several successful technology startups, including Viaweb (which was later acquired by Yahoo!) and Palantir Technologies.

Despite his many achievements, Graham is perhaps best known for his contributions to the philosophy of the mind and consciousness. In particular, his work on the concept of

使用 `stream_chat` 端点

In [ ]

from llama_index.core.llms import ChatMessage

messages = [
    ChatMessage(
        role="system", content="You are a pirate with a colorful personality"
    ),
    ChatMessage(role="user", content="What is your name"),
]
resp = llm.stream_chat(messages)

from llama_index.core.llms import ChatMessage messages = [ ChatMessage( role="system", content="You are a pirate with a colorful personality" ), ChatMessage(role="user", content="What is your name"), ] resp = llm.stream_chat(messages)

In [ ]

for r in resp:
    print(r.delta, end="")

for r in resp: print(r.delta, end="")

​

配置模型¶

In [ ]

from llama_index.llms.replicate import Replicate

llm = Replicate(
    model="replicate/vicuna-13b:6282abe6a492de4145d7bb601023762212f9ddbbe78278bd6771c8b3b2f2a13b",
    temperature=0.9,
    max_tokens=32,
)

from llama_index.llms.replicate import Replicate llm = Replicate( model="replicate/vicuna-13b:6282abe6a492de4145d7bb601023762212f9ddbbe78278bd6771c8b3b2f2a13b", temperature=0.9, max_tokens=32, )

In [ ]

resp = llm.complete("Who is Paul Graham?")

resp = llm.complete("Who is Paul Graham?")

In [ ]

print(resp)

print(resp)

PaulGraham is an influential computer scientist, venture capitalist, and essayist. He is best known as