Faiss

FaissReader

基类: BaseReader

Faiss 阅读器。

通过现有的内存 Faiss 索引检索文档。这些文档可以在下游的 LlamaIndex 数据结构中使用。如果您希望使用 Faiss 本身作为索引来组织文档、插入文档并对其执行查询，请使用带有 FaissVectorStore 的 VectorStoreIndex。

参数

名称	类型	描述	默认值
faiss_index	Index	一个 Faiss Index 对象 (必需)	必需

源代码位于 llama-index-integrations/readers/llama-index-readers-faiss/llama_index/readers/faiss/base.py

class FaissReader(BaseReader):
    """
    Faiss reader.

    Retrieves documents through an existing in-memory Faiss index.
    These documents can then be used in a downstream LlamaIndex data structure.
    If you wish use Faiss itself as an index to to organize documents,
    insert documents, and perform queries on them, please use VectorStoreIndex
    with FaissVectorStore.

    Args:
        faiss_index (faiss.Index): A Faiss Index object (required)

    """

    def __init__(self, index: Any):
        """Initialize with parameters."""
        import_err_msg = """
            `faiss` package not found. For instructions on
            how to install `faiss` please visit
            https://github.com/facebookresearch/faiss/wiki/Installing-Faiss
        """
        try:
            import faiss  # noqa
        except ImportError:
            raise ImportError(import_err_msg)

        self._index = index

    def load_data(
        self,
        query: np.ndarray,
        id_to_text_map: Dict[str, str],
        k: int = 4,
        separate_documents: bool = True,
    ) -> List[Document]:
        """
        Load data from Faiss.

        Args:
            query (np.ndarray): A 2D numpy array of query vectors.
            id_to_text_map (Dict[str, str]): A map from ID's to text.
            k (int): Number of nearest neighbors to retrieve. Defaults to 4.
            separate_documents (Optional[bool]): Whether to return separate
                documents. Defaults to True.

        Returns:
            List[Document]: A list of documents.

        """
        dists, indices = self._index.search(query, k)
        documents = []
        for qidx in range(indices.shape[0]):
            for didx in range(indices.shape[1]):
                doc_id = indices[qidx, didx]
                if doc_id not in id_to_text_map:
                    raise ValueError(
                        f"Document ID {doc_id} not found in id_to_text_map."
                    )
                text = id_to_text_map[doc_id]
                documents.append(Document(text=text))

        if not separate_documents:
            # join all documents into one
            text_list = [doc.get_content() for doc in documents]
            text = "\n\n".join(text_list)
            documents = [Document(text=text)]

        return documents

load_data

load_data(query: ndarray, id_to_text_map: Dict[str, str], k: int = 4, separate_documents: bool = True) -> List[Document]

从 Faiss 加载数据。

参数

名称	类型	描述	默认值
query	ndarray	一个 2D numpy 数组形式的查询向量。	必需
id_to_text_map	Dict[str, str]	一个从 ID 映射到文本的字典。	必需
k	int	要检索的最近邻居数量。默认为 4。	4
separate_documents	Optional[bool]	是否返回单独的文档。默认为 True。	True

返回

类型	描述
List[Document]	List[Document]: 文档列表。

源代码位于 llama-index-integrations/readers/llama-index-readers-faiss/llama_index/readers/faiss/base.py

def load_data(
    self,
    query: np.ndarray,
    id_to_text_map: Dict[str, str],
    k: int = 4,
    separate_documents: bool = True,
) -> List[Document]:
    """
    Load data from Faiss.

    Args:
        query (np.ndarray): A 2D numpy array of query vectors.
        id_to_text_map (Dict[str, str]): A map from ID's to text.
        k (int): Number of nearest neighbors to retrieve. Defaults to 4.
        separate_documents (Optional[bool]): Whether to return separate
            documents. Defaults to True.

    Returns:
        List[Document]: A list of documents.

    """
    dists, indices = self._index.search(query, k)
    documents = []
    for qidx in range(indices.shape[0]):
        for didx in range(indices.shape[1]):
            doc_id = indices[qidx, didx]
            if doc_id not in id_to_text_map:
                raise ValueError(
                    f"Document ID {doc_id} not found in id_to_text_map."
                )
            text = id_to_text_map[doc_id]
            documents.append(Document(text=text))

    if not separate_documents:
        # join all documents into one
        text_list = [doc.get_content() for doc in documents]
        text = "\n\n".join(text_list)
        documents = [Document(text=text)]

    return documents