Astra DB¶

DataStax Astra DB 是一个无服务器且支持向量的数据库，构建在 Apache Cassandra 之上，并通过易于使用的 JSON API 访问。

要运行此 Notebook，您需要一个运行在云端的 DataStax Astra DB 实例（您可以在 datastax.com 免费获取）。

您应确保已安装 llama-index 和 astrapy

In [ ]

%pip install llama-index-vector-stores-astra-db
%pip install llama-index-embeddings-openai

%pip install llama-index-vector-stores-astra-db %pip install llama-index-embeddings-openai

In [ ]

!pip install llama-index
!pip install "astrapy>=1.0"

!pip install llama-index !pip install "astrapy>=1.0"

请提供数据库连接参数和秘密:¶

In [ ]

import os
import getpass

api_endpoint = input(
    "\nPlease enter your Database Endpoint URL (e.g. 'https://4bc...datastax.com'):"
)

token = getpass.getpass(
    "\nPlease enter your 'Database Administrator' Token (e.g. 'AstraCS:...'):"
)

os.environ["OPENAI_API_KEY"] = getpass.getpass(
    "\nPlease enter your OpenAI API Key (e.g. 'sk-...'):"
)

import os import getpass api_endpoint = input( "\nPlease enter your Database Endpoint URL (e.g. 'https://4bc...datastax.com'):" ) token = getpass.getpass( "\nPlease enter your 'Database Administrator' Token (e.g. 'AstraCS:...'):" ) os.environ["OPENAI_API_KEY"] = getpass.getpass( "\nPlease enter your OpenAI API Key (e.g. 'sk-...'):" )

导入所需的包依赖项:¶

In [ ]

from llama_index.core import (
    VectorStoreIndex,
    SimpleDirectoryReader,
    StorageContext,
)
from llama_index.embeddings.openai import OpenAIEmbedding
from llama_index.vector_stores.astra_db import AstraDBVectorStore

from llama_index.core import ( VectorStoreIndex, SimpleDirectoryReader, StorageContext, ) from llama_index.embeddings.openai import OpenAIEmbedding from llama_index.vector_stores.astra_db import AstraDBVectorStore

加载一些示例数据:¶

In [ ]

!mkdir -p 'data/paul_graham/'
!wget 'https://raw.githubusercontent.com/run-llama/llama_index/main/docs/docs/examples/data/paul_graham/paul_graham_essay.txt' -O 'data/paul_graham/paul_graham_essay.txt'

!mkdir -p 'data/paul_graham/' !wget 'https://raw.githubusercontent.com/run-llama/llama_index/main/docs/docs/examples/data/paul_graham/paul_graham_essay.txt' -O 'data/paul_graham/paul_graham_essay.txt'

读取数据:¶

In [ ]

# load documents
documents = SimpleDirectoryReader("./data/paul_graham/").load_data()
print(f"Total documents: {len(documents)}")
print(f"First document, id: {documents[0].doc_id}")
print(f"First document, hash: {documents[0].hash}")
print(
    "First document, text"
    f" ({len(documents[0].text)} characters):\n{'='*20}\n{documents[0].text[:360]} ..."
)

# load documents documents = SimpleDirectoryReader("./data/paul_graham/").load_data() print(f"总文档数：{len(documents)}") print(f"第一个文档 ID：{documents[0].doc_id}") print(f"第一个文档哈希：{documents[0].hash}") print( "第一个文档文本" f" （{len(documents[0].text)} 字符）：\n{'='*20}\n{documents[0].text[:360]} ..." )

创建 Astra DB 向量存储对象:¶

In [ ]

astra_db_store = AstraDBVectorStore(
    token=token,
    api_endpoint=api_endpoint,
    collection_name="astra_v_table",
    embedding_dimension=1536,
)

astra_db_store = AstraDBVectorStore( token=token, api_endpoint=api_endpoint, collection_name="astra_v_table", embedding_dimension=1536, )

从文档构建索引:¶

In [ ]

embed_model = OpenAIEmbedding(model_name="text-embedding-3-small")

storage_context = StorageContext.from_defaults(vector_store=astra_db_store)

index = VectorStoreIndex.from_documents(
    documents, storage_context=storage_context, embed_model=embed_model
)

embed_model = OpenAIEmbedding(model_name="text-embedding-3-small") storage_context = StorageContext.from_defaults(vector_store=astra_db_store) index = VectorStoreIndex.from_documents( documents, storage_context=storage_context, embed_model=embed_model )

使用索引进行查询:¶

In [ ]

query_engine = index.as_query_engine()
response = query_engine.query("Why did the author choose to work on AI?")

print(response.response)

query_engine = index.as_query_engine() response = query_engine.query("作者为什么选择研究人工智能？") print(response.response)