Opensearch

OpensearchReader

基类: BaseReader

从 Opensearch 索引读取文档。

这些文档随后可在下游的 Llama Index 数据结构中使用。

参数

名称	类型	描述	默认值
endpoint	str	集群的 URL (http/https)，不含端口	必需
index	str	索引名称 (必需)	必需
basic_auth	set	基本认证用户名和密码	无

源代码位于 llama-index-integrations/readers/llama-index-readers-opensearch/llama_index/readers/opensearch/base.py

class OpensearchReader(BaseReader):
    """
    Read documents from an Opensearch index.

    These documents can then be used in a downstream Llama Index data structure.

    Args:
        endpoint (str): URL (http/https) of cluster without port
        index (str): Name of the index (required)
        basic_auth (set): basic authentication username password

    """

    def __init__(
        self, host: str, port: int, index: str, basic_auth: Optional[set] = None
    ):
        """Initialize with parameters."""
        from opensearchpy import OpenSearch

        self._opster_client = OpenSearch(
            hosts=[{"host": host, "port": port}],
            http_compress=True,  # enables gzip compression for request bodies
            http_auth=basic_auth,
            use_ssl=True,
            verify_certs=False,
            ssl_assert_hostname=False,
            ssl_show_warn=False,
        )
        self._index = index

    def load_data(
        self,
        field: str,
        query: Optional[dict] = None,
        embedding_field: Optional[str] = None,
    ) -> List[Document]:
        """
        Read data from the Opensearch index.

        Args:
            field (str): Field in the document to retrieve text from
            query (Optional[dict]): Opensearch JSON query DSL object.
                For example:
                { "query" : {"match": {"message": {"query": "this is a test"}}}}
            embedding_field (Optional[str]): If there are embeddings stored in
                this index, this field can be used
                to set the embedding field on the returned Document list.


        Returns:
            List[Document]: A list of documents.

        """
        res = self._opster_client.search(body=query, index=self._index)
        documents = []
        for hit in res["hits"]["hits"]:
            value = hit["_source"][field]
            _ = hit["_source"].pop(field)
            embedding = hit["_source"].get(embedding_field or "", None)
            documents.append(
                Document(text=value, extra_info=hit["_source"], embedding=embedding)
            )
        return documents

load_data

load_data(field: str, query: Optional[dict] = None, embedding_field: Optional[str] = None) -> List[Document]

从 Opensearch 索引读取数据。

参数

名称	类型	描述	默认值
field	str	文档中用于检索文本的字段	必需
query	Optional[dict]	Opensearch JSON 查询 DSL 对象。例如：{ "query" : {"match": {"message": {"query": "这是一个测试"}}}}	无
embedding_field	Optional[str]	如果此索引中存储了嵌入向量，则此字段可用于设置返回的 Document 列表中的嵌入字段。	无

返回

类型	描述
List[Document]	List[Document]：文档列表。

源代码位于 llama-index-integrations/readers/llama-index-readers-opensearch/llama_index/readers/opensearch/base.py

def load_data(
    self,
    field: str,
    query: Optional[dict] = None,
    embedding_field: Optional[str] = None,
) -> List[Document]:
    """
    Read data from the Opensearch index.

    Args:
        field (str): Field in the document to retrieve text from
        query (Optional[dict]): Opensearch JSON query DSL object.
            For example:
            { "query" : {"match": {"message": {"query": "this is a test"}}}}
        embedding_field (Optional[str]): If there are embeddings stored in
            this index, this field can be used
            to set the embedding field on the returned Document list.


    Returns:
        List[Document]: A list of documents.

    """
    res = self._opster_client.search(body=query, index=self._index)
    documents = []
    for hit in res["hits"]["hits"]:
        value = hit["_source"][field]
        _ = hit["_source"].pop(field)
        embedding = hit["_source"].get(embedding_field or "", None)
        documents.append(
            Document(text=value, extra_info=hit["_source"], embedding=embedding)
        )
    return documents