--- name: local-document-research-with-traceable-citations version: 1.0.0 description: Enable users to import local documents, asynchronously process them into an indexed knowledge base, and obtain AI-generated answers that cite specific page locations and OCR evidence. inputs: - Local document files (PDF, DOCX, PPTX, XLSX, images, TXT) - Configured LLM API endpoint and keys (via .env or settings) - Optional web search service config if enabled - Local OCR model cache (downloaded on first use) steps: - Import documents into a project; files are queued for asynchronous processing. - Convert non-PDF formats (DOCX, PPTX, XLSX) to PDF using native Office or LibreOffice fallback. - Run OCR (PaddleOCR) on PDF pages/images to extract text, page numbers, polygons, and confidence scores. - Chunk text and generate embeddings; publish to LanceDB hybrid index (dense vector + full-text) only when fully processed. - User starts a research session or branch; Leader agent analyzes query. - Hybrid RAG retrieves candidate chunks from ready documents; dynamic material scope ensures no half-indexed docs. - Leader delegates tasks to sub-agents (researcher, reviewer, writer) via constrained task capability; each delegation logs start, completion, duration, and evidence. - Generate answer that references only actually used evidence; citations include document ID, page, coordinates. - User clicks citation to open original document and view highlighted OCR location. outputs: - Project with indexed document library (SQLite metadata + LanceDB vectors) - AI answers with verifiable citations to source pages - Evidence preview with page image and OCR highlight polygons - Persistent session history, branches, and long-term memory tags: [] metadata: source_repo: https://github.com/0verL1nk/PaperSage.git extracted_at: '' confidence: 0.85 --- # local-document-research-with-traceable-citations Enable users to import local documents, asynchronously process them into an indexed knowledge base, and obtain AI-generated answers that cite specific page locations and OCR evidence. ## Steps 1. Import documents into a project; files are queued for asynchronous processing. 2. Convert non-PDF formats (DOCX, PPTX, XLSX) to PDF using native Office or LibreOffice fallback. 3. Run OCR (PaddleOCR) on PDF pages/images to extract text, page numbers, polygons, and confidence scores. 4. Chunk text and generate embeddings; publish to LanceDB hybrid index (dense vector + full-text) only when fully processed. 5. User starts a research session or branch; Leader agent analyzes query. 6. Hybrid RAG retrieves candidate chunks from ready documents; dynamic material scope ensures no half-indexed docs. 7. Leader delegates tasks to sub-agents (researcher, reviewer, writer) via constrained task capability; each delegation logs start, completion, duration, and evidence. 8. Generate answer that references only actually used evidence; citations include document ID, page, coordinates. 9. User clicks citation to open original document and view highlighted OCR location. ## Inputs - Local document files (PDF, DOCX, PPTX, XLSX, images, TXT) - Configured LLM API endpoint and keys (via .env or settings) - Optional web search service config if enabled - Local OCR model cache (downloaded on first use) ## Outputs - Project with indexed document library (SQLite metadata + LanceDB vectors) - AI answers with verifiable citations to source pages - Evidence preview with page image and OCR highlight polygons - Persistent session history, branches, and long-term memory ## Failure Modes - OCR quality low for scanned images leading to poor extraction - Missing Office/LibreOffice causes conversion failure for Office docs - Interrupted indexing leaves documents unpublished and excluded from retrieval - LLM API outage or misconfiguration yields no answer - Citation coordinates mismatch due to chunk drift - Sub-agent recursion if constraints not enforced ## Source Extracted from: [https://github.com/0verL1nk/PaperSage.git](https://github.com/0verL1nk/PaperSage.git) Confidence: 0.85