Friendli¶

基本用法¶

如果您正在 Colab 上打开此 Notebook，您可能需要安装 LlamaIndex 🦙。

In [ ]

%pip install llama-index-llms-friendli

%pip install llama-index-llms-friendli

In [ ]

!pip install llama-index

!pip install llama-index

In [ ]

%env FRIENDLI_TOKEN=...

%env FRIENDLI_TOKEN=...

env: FRIENDLI_TOKEN=...

In [ ]

from llama_index.llms.friendli import Friendli

# To customize your friendli token, do this
# otherwise it will lookup FRIENDLI_TOKEN from your env variable
# llm = Friendli(friendli_token="Your personal access token")

llm = Friendli()

from llama_index.llms.friendli import Friendli # 要自定义您的 friendli token，请执行此操作 # 否则它将从您的环境变量中查找 FRIENDLI_TOKEN # llm = Friendli(friendli_token="您的个人访问 token") llm = Friendli()

使用消息列表调用 `chat`¶

In [ ]

from llama_index.core.llms import ChatMessage, MessageRole

message = ChatMessage(role=MessageRole.USER, content="Tell me a joke.")
resp = llm.chat([message])

print(resp)

from llama_index.core.llms import ChatMessage, MessageRole message = ChatMessage(role=MessageRole.USER, content="讲个笑话。") resp = llm.chat([message]) print(resp)

assistant: Of course, I'd be happy to share a joke with you! Here it is:

Why don't scientists trust atoms?

Because they make up everything!

I hope that brought a smile to your face. Would you like to hear another joke, or is there something else you'd like to talk about?

流式¶

In [ ]

resp = llm.stream_chat([message])
for r in resp:
    print(r.delta, end="")

resp = llm.stream_chat([message]) for r in resp: print(r.delta, end="")

Of course, I'd be happy to share a joke with you! Here it is:

Why don't scientists trust atoms?

Because they make up everything!

I hope that brought a smile to your face. Would you like to hear another joke, or is there something else you'd like to talk about?

异步¶

In [ ]

resp = await llm.achat([message])

print(resp)

resp = await llm.achat([message]) print(resp)

assistant: Sure, here's one:

Why don't scientists trust atoms?

Because they make up everything!

异步流式¶

In [ ]

resp = await llm.astream_chat([message])
async for r in resp:
    print(r.delta, end="")

resp = await llm.astream_chat([message]) async for r in resp: print(r.delta, end="")

Sure, here's one:

Why don't scientists trust atoms?

Because they make up everything!

使用提示调用 `complete`¶

In [ ]

prompt = "Draft a cover letter for a role in software engineering."
resp = llm.complete(prompt)

print(resp)

prompt = "起草一份软件工程师职位的求职信。" resp = llm.complete(prompt) print(resp)

Dear Hiring Manager,

I am writing to express my interest in the Software Engineer position at XYZ Company. As a highly skilled and motivated software engineer with over five years of experience in the field, I am confident that I have the skills and expertise necessary to make a valuable contribution to your team.

Throughout my career, I have gained extensive experience in designing, developing, and maintaining complex software systems. I have a strong background in programming languages such as Java, Python, and C++, and I am proficient in using various software development tools and frameworks. I am also experienced in working with agile methodologies and have a proven track record of delivering high-quality software on time and within budget.

In my current role at ABC Company, I have been responsible for leading a team of software engineers in the development of a mission-critical application used by over 10,000 users. I have successfully managed the entire software development lifecycle, from requirements gathering to deployment, and have implemented various performance optimization techniques to ensure the application runs smoothly even during peak usage times.

I am particularly drawn to XYZ Company because of its reputation as a leader in the software engineering industry.

流式¶

In [ ]

resp = llm.stream_complete(prompt)
for r in resp:
    print(r.delta, end="")

resp = llm.stream_complete(prompt) for r in resp: print(r.delta, end="")

Dear Hiring Manager,

I am writing to express my interest in the Software Engineer position at XYZ Company. With a Bachelor's degree in Computer Science and over five years of experience in software development, I am confident in my ability to make a valuable contribution to your team.

Throughout my career, I have gained experience in various programming languages such as Java, Python, and C++. I have also worked on full-stack development projects, where I was responsible for both front-end and back-end development. My experience includes designing and implementing software solutions, collaborating with cross-functional teams, and conducting code reviews.

I am particularly interested in XYZ Company's focus on innovation and cutting-edge technology. I am excited about the opportunity to work with a team that values creativity and continuous learning. I am confident that my skills and experience make me a strong candidate for this role.

Thank you for considering my application. I look forward to the opportunity to discuss my qualifications further.

Sincerely,

[Your Name]

异步¶

In [ ]

resp = await llm.acomplete(prompt)

print(resp)

resp = await llm.acomplete(prompt) print(resp)

Dear Hiring Manager,

I am writing to express my interest in the Software Engineer position at XYZ Company. With a Bachelor's degree in Computer Science and over five years of experience in software development, I am confident in my ability to make a valuable contribution to your team.

Throughout my career, I have gained experience in various programming languages such as Java, Python, and C++. I have also worked on full-stack development projects, where I was responsible for both front-end and back-end development. My experience includes working on cloud-based applications, developing APIs, and integrating third-party services.

I am passionate about writing clean, efficient, and maintainable code. I am also a strong believer in Agile methodologies and have experience working in Agile teams. I am a team player and enjoy collaborating with others to find solutions to complex problems.

In my current role at ABC Company, I have been responsible for leading a team of software engineers in the development of a cloud-based application. I have also been involved in the design and implementation of the company's DevOps practices, which has helped to improve the team's productivity and efficiency.

异步流式¶

In [ ]

resp = await llm.astream_complete(prompt)
async for r in resp:
    print(r.delta, end="")

resp = await llm.astream_complete(prompt) async for r in resp: print(r.delta, end="")

Dear Hiring Manager,

I am writing to express my interest in the Software Engineer position at XYZ Corporation. With a Bachelor's degree in Computer Science and over five years of experience in software development, I am confident in my ability to make a valuable contribution to your team.

Throughout my career, I have gained extensive experience in various programming languages such as Java, Python, and C++. I have also worked on developing web applications using frameworks such as React and Angular. My experience includes working on both front-end and back-end development, as well as leading teams to deliver high-quality software products.

At my current role, I have been responsible for designing and implementing software solutions for clients in various industries. I have worked closely with cross-functional teams, including product managers, designers, and quality assurance engineers, to ensure that the final product meets the client's requirements. I have also been involved in the entire software development lifecycle, from requirements gathering to deployment.

I am particularly interested in the Software Engineer role at XYZ Corporation because of the company's reputation for innovation and excellence. I am excited about the opportunity to work with a talented team

配置模型¶

In [ ]

from llama_index.llms.friendli import Friendli

llm = Friendli(model="llama-2-70b-chat")

from llama_index.llms.friendli import Friendli llm = Friendli(model="llama-2-70b-chat")

In [ ]

resp = llm.chat([message])

print(resp)

resp = llm.chat([message]) print(resp)

assistant:  Sure, here's a joke for you:

Why couldn't the bicycle stand up by itself?

Because it was two-tired!

I hope that brought a smile to your face! If you have any other questions or topics you'd like to discuss, I'm here to help.