Tair Vector Store¶

在本 Notebook 中，我们将快速演示如何使用 TairVectorStore。

如果您是在 Colab 上打开此 Notebook，您可能需要安装 LlamaIndex 🦙。

In [ ]

%pip install llama-index-vector-stores-tair

%pip install llama-index-vector-stores-tair

In [ ]

!pip install llama-index

!pip install llama-index

In [ ]

import os
import sys
import logging
import textwrap

import warnings

warnings.filterwarnings("ignore")

# stop huggingface warnings
os.environ["TOKENIZERS_PARALLELISM"] = "false"

# Uncomment to see debug logs
# logging.basicConfig(stream=sys.stdout, level=logging.INFO)
# logging.getLogger().addHandler(logging.StreamHandler(stream=sys.stdout))

from llama_index.core import (
    GPTVectorStoreIndex,
    SimpleDirectoryReader,
    Document,
)
from llama_index.vector_stores.tair import TairVectorStore
from IPython.display import Markdown, display

import os import sys import logging import textwrap import warnings warnings.filterwarnings("ignore") # 停止 huggingface 警告 os.environ["TOKENIZERS_PARALLELISM"] = "false" # 取消注释以查看调试日志 # logging.basicConfig(stream=sys.stdout, level=logging.INFO) # logging.getLogger().addHandler(logging.StreamHandler(stream=sys.stdout)) from llama_index.core import ( GPTVectorStoreIndex, SimpleDirectoryReader, Document, ) from llama_index.vector_stores.tair import TairVectorStore from IPython.display import Markdown, display

Setup OpenAI¶

首先，让我们添加 OpenAI API 密钥。这将允许我们访问 OpenAI 进行 Embedding 并使用 ChatGPT。

In [ ]

import os

os.environ["OPENAI_API_KEY"] = "sk-<your key here>"

import os os.environ["OPENAI_API_KEY"] = "sk-"

下载数据¶

In [ ]

!mkdir -p 'data/paul_graham/'
!wget 'https://raw.githubusercontent.com/run-llama/llama_index/main/docs/docs/examples/data/paul_graham/paul_graham_essay.txt' -O 'data/paul_graham/paul_graham_essay.txt'

!mkdir -p 'data/paul_graham/' !wget 'https://raw.githubusercontent.com/run-llama/llama_index/main/docs/docs/examples/data/paul_graham/paul_graham_essay.txt' -O 'data/paul_graham/paul_graham_essay.txt'

读取数据集¶

In [ ]

# load documents
documents = SimpleDirectoryReader("./data/paul_graham").load_data()
print(
    "Document ID:",
    documents[0].doc_id,
    "Document Hash:",
    documents[0].doc_hash,
)

# 加载文档 documents = SimpleDirectoryReader("./data/paul_graham").load_data() print( "文档 ID:", documents[0].doc_id, "文档哈希:", documents[0].doc_hash, )

从文档构建索引¶

让我们使用 GPTVectorStoreIndex 构建一个向量索引，并使用 TairVectorStore 作为其后端。将 tair_url 替换为您 Tair 实例的实际 URL。

In [ ]

from llama_index.core import StorageContext

tair_url = "redis://{username}:{password}@r-bp****************.redis.rds.aliyuncs.com:{port}"

vector_store = TairVectorStore(
    tair_url=tair_url, index_name="pg_essays", overwrite=True
)
storage_context = StorageContext.from_defaults(vector_store=vector_store)
index = GPTVectorStoreIndex.from_documents(
    documents, storage_context=storage_context
)

from llama_index.core import StorageContext tair_url = "redis://{username}:{password}@r-bp****************.redis.rds.aliyuncs.com:{port}" vector_store = TairVectorStore( tair_url=tair_url, index_name="pg_essays", overwrite=True ) storage_context = StorageContext.from_defaults(vector_store=vector_store) index = GPTVectorStoreIndex.from_documents( documents, storage_context=storage_context )

查询数据¶

现在我们可以使用该索引作为知识库并向其提问。

In [ ]

query_engine = index.as_query_engine()
response = query_engine.query("What did the author learn?")
print(textwrap.fill(str(response), 100))

query_engine = index.as_query_engine() response = query_engine.query("What did the author learn?") print(textwrap.fill(str(response), 100))

In [ ]

response = query_engine.query("What was a hard moment for the author?")
print(textwrap.fill(str(response), 100))

response = query_engine.query("What was a hard moment for the author?") print(textwrap.fill(str(response), 100))

删除文档¶

要从索引中删除文档，请使用 delete 方法。

In [ ]

document_id = documents[0].doc_id
document_id

document_id = documents[0].doc_id document_id

In [ ]

info = vector_store.client.tvs_get_index("pg_essays")
print("Number of documents", int(info["data_count"]))

info = vector_store.client.tvs_get_index("pg_essays") print("文档数量", int(info["data_count"]))

In [ ]

vector_store.delete(document_id)

vector_store.delete(document_id)

In [ ]

info = vector_store.client.tvs_get_index("pg_essays")
print("Number of documents", int(info["data_count"]))

info = vector_store.client.tvs_get_index("pg_essays") print("文档数量", int(info["data_count"]))

删除索引¶

使用 delete_index 方法删除整个索引。

In [ ]

vector_store.delete_index()

vector_store.delete_index()

In [ ]

print("Check index existence:", vector_store.client._index_exists())

print("检查索引是否存在:", vector_store.client._index_exists())