Add Skill: local-document-research-with-traceable-citations #52

Open
tonyjbala wants to merge 1 commits from skill/local-document-research-with-traceable-citations-20260810-170746 into main
5 changed files with 147 additions and 0 deletions
@@ -0,0 +1,82 @@
---
name: local-document-research-with-traceable-citations
version: 1.0.0
description: Enable users to import local documents, asynchronously process them into
an indexed knowledge base, and obtain AI-generated answers that cite specific page
locations and OCR evidence.
inputs:
- Local document files (PDF, DOCX, PPTX, XLSX, images, TXT)
- Configured LLM API endpoint and keys (via .env or settings)
- Optional web search service config if enabled
- Local OCR model cache (downloaded on first use)
steps:
- Import documents into a project; files are queued for asynchronous processing.
- Convert non-PDF formats (DOCX, PPTX, XLSX) to PDF using native Office or LibreOffice
fallback.
- Run OCR (PaddleOCR) on PDF pages/images to extract text, page numbers, polygons,
and confidence scores.
- Chunk text and generate embeddings; publish to LanceDB hybrid index (dense vector
+ full-text) only when fully processed.
- User starts a research session or branch; Leader agent analyzes query.
- Hybrid RAG retrieves candidate chunks from ready documents; dynamic material scope
ensures no half-indexed docs.
- Leader delegates tasks to sub-agents (researcher, reviewer, writer) via constrained
task capability; each delegation logs start, completion, duration, and evidence.
- Generate answer that references only actually used evidence; citations include document
ID, page, coordinates.
- User clicks citation to open original document and view highlighted OCR location.
outputs:
- Project with indexed document library (SQLite metadata + LanceDB vectors)
- AI answers with verifiable citations to source pages
- Evidence preview with page image and OCR highlight polygons
- Persistent session history, branches, and long-term memory
tags: []
metadata:
source_repo: https://github.com/0verL1nk/PaperSage.git
extracted_at: ''
confidence: 0.85
---
# local-document-research-with-traceable-citations
Enable users to import local documents, asynchronously process them into an indexed knowledge base, and obtain AI-generated answers that cite specific page locations and OCR evidence.
## Steps
1. Import documents into a project; files are queued for asynchronous processing.
2. Convert non-PDF formats (DOCX, PPTX, XLSX) to PDF using native Office or LibreOffice fallback.
3. Run OCR (PaddleOCR) on PDF pages/images to extract text, page numbers, polygons, and confidence scores.
4. Chunk text and generate embeddings; publish to LanceDB hybrid index (dense vector + full-text) only when fully processed.
5. User starts a research session or branch; Leader agent analyzes query.
6. Hybrid RAG retrieves candidate chunks from ready documents; dynamic material scope ensures no half-indexed docs.
7. Leader delegates tasks to sub-agents (researcher, reviewer, writer) via constrained task capability; each delegation logs start, completion, duration, and evidence.
8. Generate answer that references only actually used evidence; citations include document ID, page, coordinates.
9. User clicks citation to open original document and view highlighted OCR location.
## Inputs
- Local document files (PDF, DOCX, PPTX, XLSX, images, TXT)
- Configured LLM API endpoint and keys (via .env or settings)
- Optional web search service config if enabled
- Local OCR model cache (downloaded on first use)
## Outputs
- Project with indexed document library (SQLite metadata + LanceDB vectors)
- AI answers with verifiable citations to source pages
- Evidence preview with page image and OCR highlight polygons
- Persistent session history, branches, and long-term memory
## Failure Modes
- OCR quality low for scanned images leading to poor extraction
- Missing Office/LibreOffice causes conversion failure for Office docs
- Interrupted indexing leaves documents unpublished and excluded from retrieval
- LLM API outage or misconfiguration yields no answer
- Citation coordinates mismatch due to chunk drift
- Sub-agent recursion if constraints not enforced
## Source
Extracted from: [https://github.com/0verL1nk/PaperSage.git](https://github.com/0verL1nk/PaperSage.git)
Confidence: 0.85
@@ -0,0 +1,6 @@
# Commands: local-document-research-with-traceable-citations
## Available Commands
- `/skill local-document-research-with-traceable-citations` — Load this skill
- `/run local-document-research-with-traceable-citations` — Execute workflow
@@ -0,0 +1,10 @@
# Examples: local-document-research-with-traceable-citations
## Usage Example
```python
# How to use this skill
# Inputs: Local document files (PDF, DOCX, PPTX, XLSX, images, TXT), Configured LLM API endpoint and keys (via .env or settings), Optional web search service config if enabled, Local OCR model cache (downloaded on first use)
# Process: Import documents into a project; files are queued for asynchronous processing. → Convert non-PDF formats (DOCX, PPTX, XLSX) to PDF using native Office or LibreOffice fallback. → Run OCR (PaddleOCR) on PDF pages/images to extract text, page numbers, polygons, and confidence scores.
# Outputs: Project with indexed document library (SQLite metadata + LanceDB vectors), AI answers with verifiable citations to source pages, Evidence preview with page image and OCR highlight polygons, Persistent session history, branches, and long-term memory
```
@@ -0,0 +1,40 @@
{
"name": "local-document-research-with-traceable-citations",
"version": "1.0.0",
"goal": "Enable users to import local documents, asynchronously process them into an indexed knowledge base, and obtain AI-generated answers that cite specific page locations and OCR evidence.",
"inputs": [
"Local document files (PDF, DOCX, PPTX, XLSX, images, TXT)",
"Configured LLM API endpoint and keys (via .env or settings)",
"Optional web search service config if enabled",
"Local OCR model cache (downloaded on first use)"
],
"steps": [
"Import documents into a project; files are queued for asynchronous processing.",
"Convert non-PDF formats (DOCX, PPTX, XLSX) to PDF using native Office or LibreOffice fallback.",
"Run OCR (PaddleOCR) on PDF pages/images to extract text, page numbers, polygons, and confidence scores.",
"Chunk text and generate embeddings; publish to LanceDB hybrid index (dense vector + full-text) only when fully processed.",
"User starts a research session or branch; Leader agent analyzes query.",
"Hybrid RAG retrieves candidate chunks from ready documents; dynamic material scope ensures no half-indexed docs.",
"Leader delegates tasks to sub-agents (researcher, reviewer, writer) via constrained task capability; each delegation logs start, completion, duration, and evidence.",
"Generate answer that references only actually used evidence; citations include document ID, page, coordinates.",
"User clicks citation to open original document and view highlighted OCR location."
],
"outputs": [
"Project with indexed document library (SQLite metadata + LanceDB vectors)",
"AI answers with verifiable citations to source pages",
"Evidence preview with page image and OCR highlight polygons",
"Persistent session history, branches, and long-term memory"
],
"failure_modes": [
"OCR quality low for scanned images leading to poor extraction",
"Missing Office/LibreOffice causes conversion failure for Office docs",
"Interrupted indexing leaves documents unpublished and excluded from retrieval",
"LLM API outage or misconfiguration yields no answer",
"Citation coordinates mismatch due to chunk drift",
"Sub-agent recursion if constraints not enforced"
],
"confidence": 0.85,
"explanation": "The README outlines a clear pipeline from document import to cited answer with evidence location, which is a reusable pattern for local-first RAG applications requiring traceability.",
"source_repo": "https://github.com/0verL1nk/PaperSage.git",
"score": 1.0
}
@@ -0,0 +1,9 @@
# Tests: local-document-research-with-traceable-citations
## Test Checklist
- [ ] Workflow has at least 3 steps
- [ ] All inputs are defined
- [ ] All outputs are defined
- [ ] Failure modes are documented
- [ ] Skill can be loaded without errors