Compare commits
1 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
| 7273cbe4ba |
@@ -0,0 +1,108 @@
|
||||
---
|
||||
name: graph-based-node-orchestration
|
||||
version: 1.0.0
|
||||
description: Build a typed node graph that processes data through a sequence of operations
|
||||
and produces a final result
|
||||
inputs:
|
||||
- NIM_MODEL constant (e.g., 'nim:meta/llama-3.1-8b-instruct') for the LLM model
|
||||
- blacknode package with Graph, Node, and cook functionality
|
||||
- Python script defining node types with inputs/outputs and connecting them via edges
|
||||
steps:
|
||||
- Import required modules (blacknode as bn, NIM_MODEL) and ensure API keys are available
|
||||
via require_nim_api_key()
|
||||
- Create a bn.Graph() instance to hold all nodes and their connections
|
||||
- Define individual nodes with specific input/output parameters (e.g., Text node with
|
||||
'value' input, LLMAgent node with model parameter, Output node with 'value' output)
|
||||
- Connect nodes together using edge definitions (from_port -> to_port) to establish
|
||||
data flow
|
||||
- Execute the graph using g.cook() to run the pipeline and process data through the
|
||||
node chain
|
||||
- Extract the final result from the output node to complete the workflow
|
||||
outputs:
|
||||
- A fully constructed graph with typed nodes and defined connections
|
||||
- Executed result (e.g., processed text, summary, or other output) from the final
|
||||
node
|
||||
- A reusable pattern that can be adapted to different models, hardware, or task types
|
||||
tags: []
|
||||
metadata:
|
||||
source_repo: https://github.com/temiroff/Blacknode.git
|
||||
extracted_at: ''
|
||||
confidence: 0.95
|
||||
---
|
||||
|
||||
# graph-based-node-orchestration
|
||||
|
||||
Build a typed node graph that processes data through a sequence of operations and produces a final result
|
||||
|
||||
## Setup
|
||||
|
||||
**Dependencies:**
|
||||
|
||||
```text
|
||||
pip install blacknode >= 0.3.0 Python >= 3.11 NVIDIA NIM API key (optional but recommended) Anthropic, OpenAI, or other LLM models
|
||||
```
|
||||
|
||||
**Setup steps:**
|
||||
|
||||
1. Clone the repository and install dependencies: pip install -e .
|
||||
1. Set NVIDIA_API_KEY or other required API keys in .env
|
||||
1. Run the example script: python examples/hello_agent.py
|
||||
1. For production, configure hardware pairing and deploy via the blacknode CLI
|
||||
|
||||
## Key Files
|
||||
|
||||
- `examples/converted_nvidia_nim.py - Full graph with Model, Text, LLMAgent, Output nodes`
|
||||
- `examples/hello_agent.py - Minimal agent example connecting Literal → LLMAgent → Print`
|
||||
- `blacknode/core - Graph and node implementation (internal)`
|
||||
|
||||
## Steps
|
||||
|
||||
1. Import required modules (blacknode as bn, NIM_MODEL) and ensure API keys are available via require_nim_api_key()
|
||||
2. Create a bn.Graph() instance to hold all nodes and their connections
|
||||
3. Define individual nodes with specific input/output parameters (e.g., Text node with 'value' input, LLMAgent node with model parameter, Output node with 'value' output)
|
||||
4. Connect nodes together using edge definitions (from_port -> to_port) to establish data flow
|
||||
5. Execute the graph using g.cook() to run the pipeline and process data through the node chain
|
||||
6. Extract the final result from the output node to complete the workflow
|
||||
|
||||
## Implementation Details
|
||||
|
||||
```python
|
||||
g = bn.Graph()
|
||||
```
|
||||
|
||||
```python
|
||||
model = g.node('Model', **{'value': 'nim:meta/llama-3.1-8b-instruct'})
|
||||
```
|
||||
|
||||
```python
|
||||
agent = g.node('LLMAgent', **{model})
|
||||
```
|
||||
|
||||
```python
|
||||
result = g.cook(output, 'value')
|
||||
```
|
||||
|
||||
## Inputs
|
||||
|
||||
- NIM_MODEL constant (e.g., 'nim:meta/llama-3.1-8b-instruct') for the LLM model
|
||||
- blacknode package with Graph, Node, and cook functionality
|
||||
- Python script defining node types with inputs/outputs and connecting them via edges
|
||||
|
||||
## Outputs
|
||||
|
||||
- A fully constructed graph with typed nodes and defined connections
|
||||
- Executed result (e.g., processed text, summary, or other output) from the final node
|
||||
- A reusable pattern that can be adapted to different models, hardware, or task types
|
||||
|
||||
## Failure Modes
|
||||
|
||||
- Missing or invalid API key (NIM_API_KEY, OPENAI_API_KEY) causing require_nim_api_key() to fail
|
||||
- Type mismatch in node connections (e.g., expecting 'value' but receiving 'text') breaking the data flow
|
||||
- Model not available or unreachable (NIM_MODEL not found) causing the LLMAgent node to fail
|
||||
- Graph execution error due to incorrect edge configuration or circular dependencies
|
||||
- Resource exhaustion (e.g., GPU memory) when processing large inputs in the pipeline
|
||||
|
||||
## Source
|
||||
|
||||
Extracted from: [https://github.com/temiroff/Blacknode.git](https://github.com/temiroff/Blacknode.git)
|
||||
Confidence: 0.95
|
||||
@@ -0,0 +1,6 @@
|
||||
# Commands: graph-based-node-orchestration
|
||||
|
||||
## Available Commands
|
||||
|
||||
- `/skill graph-based-node-orchestration` — Load this skill
|
||||
- `/run graph-based-node-orchestration` — Execute workflow
|
||||
@@ -0,0 +1,10 @@
|
||||
# Examples: graph-based-node-orchestration
|
||||
|
||||
## Usage Example
|
||||
|
||||
```python
|
||||
# How to use this skill
|
||||
# Inputs: NIM_MODEL constant (e.g., 'nim:meta/llama-3.1-8b-instruct') for the LLM model, blacknode package with Graph, Node, and cook functionality, Python script defining node types with inputs/outputs and connecting them via edges
|
||||
# Process: Import required modules (blacknode as bn, NIM_MODEL) and ensure API keys are available via require_nim_api_key() → Create a bn.Graph() instance to hold all nodes and their connections → Define individual nodes with specific input/output parameters (e.g., Text node with 'value' input, LLMAgent node with model parameter, Output node with 'value' output)
|
||||
# Outputs: A fully constructed graph with typed nodes and defined connections, Executed result (e.g., processed text, summary, or other output) from the final node, A reusable pattern that can be adapted to different models, hardware, or task types
|
||||
```
|
||||
@@ -0,0 +1,34 @@
|
||||
{
|
||||
"name": "graph-based-node-orchestration",
|
||||
"version": "1.0.0",
|
||||
"goal": "Build a typed node graph that processes data through a sequence of operations and produces a final result",
|
||||
"inputs": [
|
||||
"NIM_MODEL constant (e.g., 'nim:meta/llama-3.1-8b-instruct') for the LLM model",
|
||||
"blacknode package with Graph, Node, and cook functionality",
|
||||
"Python script defining node types with inputs/outputs and connecting them via edges"
|
||||
],
|
||||
"steps": [
|
||||
"Import required modules (blacknode as bn, NIM_MODEL) and ensure API keys are available via require_nim_api_key()",
|
||||
"Create a bn.Graph() instance to hold all nodes and their connections",
|
||||
"Define individual nodes with specific input/output parameters (e.g., Text node with 'value' input, LLMAgent node with model parameter, Output node with 'value' output)",
|
||||
"Connect nodes together using edge definitions (from_port -> to_port) to establish data flow",
|
||||
"Execute the graph using g.cook() to run the pipeline and process data through the node chain",
|
||||
"Extract the final result from the output node to complete the workflow"
|
||||
],
|
||||
"outputs": [
|
||||
"A fully constructed graph with typed nodes and defined connections",
|
||||
"Executed result (e.g., processed text, summary, or other output) from the final node",
|
||||
"A reusable pattern that can be adapted to different models, hardware, or task types"
|
||||
],
|
||||
"failure_modes": [
|
||||
"Missing or invalid API key (NIM_API_KEY, OPENAI_API_KEY) causing require_nim_api_key() to fail",
|
||||
"Type mismatch in node connections (e.g., expecting 'value' but receiving 'text') breaking the data flow",
|
||||
"Model not available or unreachable (NIM_MODEL not found) causing the LLMAgent node to fail",
|
||||
"Graph execution error due to incorrect edge configuration or circular dependencies",
|
||||
"Resource exhaustion (e.g., GPU memory) when processing large inputs in the pipeline"
|
||||
],
|
||||
"confidence": 0.95,
|
||||
"explanation": "This workflow demonstrates a declarative graph-based orchestration pattern where nodes are connected via explicit ports and data flows through the graph. The pattern is highly reusable across different domains (robotics, research, data processing) because it separates graph structure from execution logic. The same graph construction and cook pattern can be adapted to different models (NIM, Anthropic, OpenAI), hardware targets (CPU, GPU, Jetson), and task types (LLM reasoning, file I/O, sensor processing).",
|
||||
"source_repo": "https://github.com/temiroff/Blacknode.git",
|
||||
"score": 1.0
|
||||
}
|
||||
+1
-1
@@ -1,4 +1,4 @@
|
||||
# Tests: local-document-research-with-traceable-citations
|
||||
# Tests: graph-based-node-orchestration
|
||||
|
||||
## Test Checklist
|
||||
|
||||
@@ -1,82 +0,0 @@
|
||||
---
|
||||
name: local-document-research-with-traceable-citations
|
||||
version: 1.0.0
|
||||
description: Enable users to import local documents, asynchronously process them into
|
||||
an indexed knowledge base, and obtain AI-generated answers that cite specific page
|
||||
locations and OCR evidence.
|
||||
inputs:
|
||||
- Local document files (PDF, DOCX, PPTX, XLSX, images, TXT)
|
||||
- Configured LLM API endpoint and keys (via .env or settings)
|
||||
- Optional web search service config if enabled
|
||||
- Local OCR model cache (downloaded on first use)
|
||||
steps:
|
||||
- Import documents into a project; files are queued for asynchronous processing.
|
||||
- Convert non-PDF formats (DOCX, PPTX, XLSX) to PDF using native Office or LibreOffice
|
||||
fallback.
|
||||
- Run OCR (PaddleOCR) on PDF pages/images to extract text, page numbers, polygons,
|
||||
and confidence scores.
|
||||
- Chunk text and generate embeddings; publish to LanceDB hybrid index (dense vector
|
||||
+ full-text) only when fully processed.
|
||||
- User starts a research session or branch; Leader agent analyzes query.
|
||||
- Hybrid RAG retrieves candidate chunks from ready documents; dynamic material scope
|
||||
ensures no half-indexed docs.
|
||||
- Leader delegates tasks to sub-agents (researcher, reviewer, writer) via constrained
|
||||
task capability; each delegation logs start, completion, duration, and evidence.
|
||||
- Generate answer that references only actually used evidence; citations include document
|
||||
ID, page, coordinates.
|
||||
- User clicks citation to open original document and view highlighted OCR location.
|
||||
outputs:
|
||||
- Project with indexed document library (SQLite metadata + LanceDB vectors)
|
||||
- AI answers with verifiable citations to source pages
|
||||
- Evidence preview with page image and OCR highlight polygons
|
||||
- Persistent session history, branches, and long-term memory
|
||||
tags: []
|
||||
metadata:
|
||||
source_repo: https://github.com/0verL1nk/PaperSage.git
|
||||
extracted_at: ''
|
||||
confidence: 0.85
|
||||
---
|
||||
|
||||
# local-document-research-with-traceable-citations
|
||||
|
||||
Enable users to import local documents, asynchronously process them into an indexed knowledge base, and obtain AI-generated answers that cite specific page locations and OCR evidence.
|
||||
|
||||
## Steps
|
||||
|
||||
1. Import documents into a project; files are queued for asynchronous processing.
|
||||
2. Convert non-PDF formats (DOCX, PPTX, XLSX) to PDF using native Office or LibreOffice fallback.
|
||||
3. Run OCR (PaddleOCR) on PDF pages/images to extract text, page numbers, polygons, and confidence scores.
|
||||
4. Chunk text and generate embeddings; publish to LanceDB hybrid index (dense vector + full-text) only when fully processed.
|
||||
5. User starts a research session or branch; Leader agent analyzes query.
|
||||
6. Hybrid RAG retrieves candidate chunks from ready documents; dynamic material scope ensures no half-indexed docs.
|
||||
7. Leader delegates tasks to sub-agents (researcher, reviewer, writer) via constrained task capability; each delegation logs start, completion, duration, and evidence.
|
||||
8. Generate answer that references only actually used evidence; citations include document ID, page, coordinates.
|
||||
9. User clicks citation to open original document and view highlighted OCR location.
|
||||
|
||||
## Inputs
|
||||
|
||||
- Local document files (PDF, DOCX, PPTX, XLSX, images, TXT)
|
||||
- Configured LLM API endpoint and keys (via .env or settings)
|
||||
- Optional web search service config if enabled
|
||||
- Local OCR model cache (downloaded on first use)
|
||||
|
||||
## Outputs
|
||||
|
||||
- Project with indexed document library (SQLite metadata + LanceDB vectors)
|
||||
- AI answers with verifiable citations to source pages
|
||||
- Evidence preview with page image and OCR highlight polygons
|
||||
- Persistent session history, branches, and long-term memory
|
||||
|
||||
## Failure Modes
|
||||
|
||||
- OCR quality low for scanned images leading to poor extraction
|
||||
- Missing Office/LibreOffice causes conversion failure for Office docs
|
||||
- Interrupted indexing leaves documents unpublished and excluded from retrieval
|
||||
- LLM API outage or misconfiguration yields no answer
|
||||
- Citation coordinates mismatch due to chunk drift
|
||||
- Sub-agent recursion if constraints not enforced
|
||||
|
||||
## Source
|
||||
|
||||
Extracted from: [https://github.com/0verL1nk/PaperSage.git](https://github.com/0verL1nk/PaperSage.git)
|
||||
Confidence: 0.85
|
||||
@@ -1,6 +0,0 @@
|
||||
# Commands: local-document-research-with-traceable-citations
|
||||
|
||||
## Available Commands
|
||||
|
||||
- `/skill local-document-research-with-traceable-citations` — Load this skill
|
||||
- `/run local-document-research-with-traceable-citations` — Execute workflow
|
||||
@@ -1,10 +0,0 @@
|
||||
# Examples: local-document-research-with-traceable-citations
|
||||
|
||||
## Usage Example
|
||||
|
||||
```python
|
||||
# How to use this skill
|
||||
# Inputs: Local document files (PDF, DOCX, PPTX, XLSX, images, TXT), Configured LLM API endpoint and keys (via .env or settings), Optional web search service config if enabled, Local OCR model cache (downloaded on first use)
|
||||
# Process: Import documents into a project; files are queued for asynchronous processing. → Convert non-PDF formats (DOCX, PPTX, XLSX) to PDF using native Office or LibreOffice fallback. → Run OCR (PaddleOCR) on PDF pages/images to extract text, page numbers, polygons, and confidence scores.
|
||||
# Outputs: Project with indexed document library (SQLite metadata + LanceDB vectors), AI answers with verifiable citations to source pages, Evidence preview with page image and OCR highlight polygons, Persistent session history, branches, and long-term memory
|
||||
```
|
||||
@@ -1,40 +0,0 @@
|
||||
{
|
||||
"name": "local-document-research-with-traceable-citations",
|
||||
"version": "1.0.0",
|
||||
"goal": "Enable users to import local documents, asynchronously process them into an indexed knowledge base, and obtain AI-generated answers that cite specific page locations and OCR evidence.",
|
||||
"inputs": [
|
||||
"Local document files (PDF, DOCX, PPTX, XLSX, images, TXT)",
|
||||
"Configured LLM API endpoint and keys (via .env or settings)",
|
||||
"Optional web search service config if enabled",
|
||||
"Local OCR model cache (downloaded on first use)"
|
||||
],
|
||||
"steps": [
|
||||
"Import documents into a project; files are queued for asynchronous processing.",
|
||||
"Convert non-PDF formats (DOCX, PPTX, XLSX) to PDF using native Office or LibreOffice fallback.",
|
||||
"Run OCR (PaddleOCR) on PDF pages/images to extract text, page numbers, polygons, and confidence scores.",
|
||||
"Chunk text and generate embeddings; publish to LanceDB hybrid index (dense vector + full-text) only when fully processed.",
|
||||
"User starts a research session or branch; Leader agent analyzes query.",
|
||||
"Hybrid RAG retrieves candidate chunks from ready documents; dynamic material scope ensures no half-indexed docs.",
|
||||
"Leader delegates tasks to sub-agents (researcher, reviewer, writer) via constrained task capability; each delegation logs start, completion, duration, and evidence.",
|
||||
"Generate answer that references only actually used evidence; citations include document ID, page, coordinates.",
|
||||
"User clicks citation to open original document and view highlighted OCR location."
|
||||
],
|
||||
"outputs": [
|
||||
"Project with indexed document library (SQLite metadata + LanceDB vectors)",
|
||||
"AI answers with verifiable citations to source pages",
|
||||
"Evidence preview with page image and OCR highlight polygons",
|
||||
"Persistent session history, branches, and long-term memory"
|
||||
],
|
||||
"failure_modes": [
|
||||
"OCR quality low for scanned images leading to poor extraction",
|
||||
"Missing Office/LibreOffice causes conversion failure for Office docs",
|
||||
"Interrupted indexing leaves documents unpublished and excluded from retrieval",
|
||||
"LLM API outage or misconfiguration yields no answer",
|
||||
"Citation coordinates mismatch due to chunk drift",
|
||||
"Sub-agent recursion if constraints not enforced"
|
||||
],
|
||||
"confidence": 0.85,
|
||||
"explanation": "The README outlines a clear pipeline from document import to cited answer with evidence location, which is a reusable pattern for local-first RAG applications requiring traceability.",
|
||||
"source_repo": "https://github.com/0verL1nk/PaperSage.git",
|
||||
"score": 1.0
|
||||
}
|
||||
Reference in New Issue
Block a user