智能 PDF 加载器

SmartPDFLoader

基类: BaseReader

SmartPDFLoader 利用嵌套布局信息（如章节、段落、列表和表格）智能地分块 PDF，以优化 LLM 上下文窗口的使用。

参数

名称	类型	描述	默认值
llmsherpa_api_url	str	托管 llmsherpa PDF 解析器的服务地址	无

源代码位于 llama-index-integrations/readers/llama-index-readers-smart-pdf-loader/llama_index/readers/smart_pdf_loader/base.py

class SmartPDFLoader(BaseReader):
    """
    SmartPDFLoader uses nested layout information such as sections, paragraphs, lists and tables to smartly chunk PDFs for optimal usage of LLM context window.

    Args:
        llmsherpa_api_url (str): Address of the service hosting llmsherpa PDF parser

    """

    def __init__(
        self, *args: Any, llmsherpa_api_url: str = None, **kwargs: Any
    ) -> None:
        super().__init__(*args, **kwargs)
        from llmsherpa.readers import LayoutPDFReader

        self.pdf_reader = LayoutPDFReader(llmsherpa_api_url)

    def load_data(
        self, pdf_path_or_url: str, extra_info: Optional[Dict] = None
    ) -> List[Document]:
        """
        Load data and extract table from PDF file.

        Args:
            pdf_path_or_url (str): A url or file path pointing to the PDF

        Returns:
            List[Document]: List of documents.

        """
        results = []
        doc = self.pdf_reader.read_pdf(str(pdf_path_or_url))
        for chunk in doc.chunks():
            document = Document(
                text=chunk.to_context_text(),
                extra_info={**extra_info, "chunk_type": chunk.tag}
                if extra_info
                else {"chunk_type": chunk.tag},
            )
            results.append(document)
        return results

load_data

load_data(pdf_path_or_url: str, extra_info: Optional[Dict] = None) -> List[Document]

加载数据并从 PDF 文件中提取表格。

参数

名称	类型	描述	默认值
pdf_path_or_url	str	指向 PDF 的 URL 或文件路径	必需

返回值

类型	描述
List[Document]	List[Document]: 文档列表。

源代码位于 llama-index-integrations/readers/llama-index-readers-smart-pdf-loader/llama_index/readers/smart_pdf_loader/base.py

def load_data(
    self, pdf_path_or_url: str, extra_info: Optional[Dict] = None
) -> List[Document]:
    """
    Load data and extract table from PDF file.

    Args:
        pdf_path_or_url (str): A url or file path pointing to the PDF

    Returns:
        List[Document]: List of documents.

    """
    results = []
    doc = self.pdf_reader.read_pdf(str(pdf_path_or_url))
    for chunk in doc.chunks():
        document = Document(
            text=chunk.to_context_text(),
            extra_info={**extra_info, "chunk_type": chunk.tag}
            if extra_info
            else {"chunk_type": chunk.tag},
        )
        results.append(document)
    return results