Compare commits

..

5 Commits

Author SHA1 Message Date
Hermes Pipeline e20b5370e1 Add Skill: literature-review-with-traceable-ai-evidence
Extracted from: https://github.com/0verL1nk/PaperSage.git
Score: 1.0
2026-08-08 23:04:46 +00:00
Epictetus 271f79610d Add 5 skills from LFM + 12 skills total
New skills:
- blacknode-graph-workflow
- multi-agent-workflow-execution
- langgraph-agent-workflow
- langgraph-multi-agent-router
- three-tier-evaluation-pipeline

Config: LLM pipeline uses LFM on llama.cpp (8080)
2026-08-05 17:05:21 +00:00
Epictetus d81ddeda88 Scout window: 6 months (Feb-Aug 2026)
- pushed_after: 2026-02-01 (was 2024-06-01)
- Keeps scope tight on recent, relevant repos
2026-08-05 16:44:22 +00:00
Epictetus a14f09bec2 Add 2 new skills + dedup fix
New skills:
- langgraph-workflow-creation (from LangGraphProjects)
- agent-supervisor (from multi_agent_workflow_demo_in_langgraph)

Fixes:
- Publisher dedup: skip existing skills
- Older repos: pushed_after 2024-06-01 (was 2026-05-01)
- Lower stars: 10 (was 15)
2026-08-05 15:55:36 +00:00
Epictetus 7f496feb90 Publisher dedup check + run.py skip display
- Skip skills already in skills/ directory (no duplicate PRs)
- Run.py shows SKIP status with reason
- Fixed: was re-publishing same 5 skills every run
2026-08-05 15:49:57 +00:00
46 changed files with 1295 additions and 19 deletions
+5 -5
View File
@@ -16,10 +16,10 @@ llm:
api_key: "" api_key: ""
max_tokens: 8000 max_tokens: 8000
# Secondary LLM for pipeline tasks — uses Ollama on 3060 (non-reasoning model) # Secondary LLM for pipeline tasks — uses LFM on 3060 (llama.cpp)
llm_pipeline: llm_pipeline:
base_url: http://100.64.0.4:11434 base_url: http://100.64.0.4:8080
model: qwen2.5:7b model: C:\models\LFM2.5-2.6B-Q4_K_M.gguf
api_key: "" api_key: ""
max_tokens: 6000 max_tokens: 6000
@@ -34,8 +34,8 @@ scout:
- 'rag agent workflow' - 'rag agent workflow'
- 'tool calling workflow' - 'tool calling workflow'
filters: filters:
stars_min: 15 stars_min: 10
pushed_after: 2026-05-01 pushed_after: 2026-02-01
language: Python language: Python
archived: false archived: false
size_max_kb: 10000 size_max_kb: 10000
+12
View File
@@ -52,6 +52,18 @@ def publish_skill(review_result, config):
subprocess.run(["git", "config", "user.email", "hermes@agent.local"], cwd=repo_dir) subprocess.run(["git", "config", "user.email", "hermes@agent.local"], cwd=repo_dir)
subprocess.run(["git", "config", "user.name", "Hermes Pipeline"], cwd=repo_dir) subprocess.run(["git", "config", "user.name", "Hermes Pipeline"], cwd=repo_dir)
# Check for duplicates in skills/ directory
skills_dir = os.path.join(repo_dir, "skills")
existing_skills = []
if os.path.isdir(skills_dir):
existing_skills = [d for d in os.listdir(skills_dir) if os.path.isdir(os.path.join(skills_dir, d))]
if skill_name in existing_skills:
return {
"status": "SKIP",
"reason": f"Skill '{skill_name}' already exists in skills/ directory",
}
# Create skill directory # Create skill directory
skill_dir = os.path.join(repo_dir, "skills", skill_name) skill_dir = os.path.join(repo_dir, "skills", skill_name)
os.makedirs(skill_dir, exist_ok=True) os.makedirs(skill_dir, exist_ok=True)
+2
View File
@@ -137,6 +137,8 @@ def main():
if publish_output.get("status") == "PUBLISHED": if publish_output.get("status") == "PUBLISHED":
print(f" ✓ Published! PR: {publish_output.get('pr_url', '')}") print(f" ✓ Published! PR: {publish_output.get('pr_url', '')}")
results["published"] += 1 results["published"] += 1
elif publish_output.get("status") == "SKIP":
print(f" ⏸ Skipped: {publish_output.get('reason', '')}")
else: else:
print(f" ! {publish_output.get('status', '?')}: {publish_output.get('message', publish_output.get('error', ''))[:100]}") print(f" ! {publish_output.get('status', '?')}: {publish_output.get('message', publish_output.get('error', ''))[:100]}")
+84
View File
@@ -0,0 +1,84 @@
---
name: agent-supervisor
version: 1.0.0
description: Demonstrate a supervisor-worker architecture for intelligent task delegation
and real-time decision-making.
inputs:
- name: OPENAI_API_KEY
description: OpenAI API key for language models.
- name: TAVILY_API_KEY
description: Tavily API key for search functionality.
steps:
- step: 1
action: Load environment variables.
details: Set the OPENAI_API_KEY and TAVILY_API_KEY environment variables.
- step: 2
action: Configure LangChain tools.
details: Initialize TavilySearchResults and PythonREPLTool.
- step: 3
action: Define agent nodes.
details: Create functions for the Researcher and Coder agents that process state
through their respective tasks.
- step: 4
action: Set up supervisor agent.
details: Create a supervisor agent function that decides which worker should act
next based on user input.
- step: 5
action: Build state graph.
details: Construct the state graph with nodes for each agent and edges connecting
them to the supervisor node.
- step: 6
action: Add conditional edges.
details: Define conditions for transitioning between agents based on their responses.
- step: 7
action: Compile graph.
details: Compile the state graph into a runnable workflow.
- step: 8
action: Run example queries.
details: Stream through the workflow with example inputs to demonstrate its functionality.
outputs:
- name: 'Example 1: Code Hello World'
description: A demonstration of coding a simple hello world program.
- name: 'Example 2: Research Report'
description: A demonstration of researching and writing a brief report on pikas.
tags: []
metadata:
source_repo: https://github.com/extrawest/multi_agent_workflow_demo_in_langgraph.git
extracted_at: ''
confidence: 0.9
---
# agent-supervisor
Demonstrate a supervisor-worker architecture for intelligent task delegation and real-time decision-making.
## Steps
1. {'step': 1, 'action': 'Load environment variables.', 'details': 'Set the OPENAI_API_KEY and TAVILY_API_KEY environment variables.'}
2. {'step': 2, 'action': 'Configure LangChain tools.', 'details': 'Initialize TavilySearchResults and PythonREPLTool.'}
3. {'step': 3, 'action': 'Define agent nodes.', 'details': 'Create functions for the Researcher and Coder agents that process state through their respective tasks.'}
4. {'step': 4, 'action': 'Set up supervisor agent.', 'details': 'Create a supervisor agent function that decides which worker should act next based on user input.'}
5. {'step': 5, 'action': 'Build state graph.', 'details': 'Construct the state graph with nodes for each agent and edges connecting them to the supervisor node.'}
6. {'step': 6, 'action': 'Add conditional edges.', 'details': 'Define conditions for transitioning between agents based on their responses.'}
7. {'step': 7, 'action': 'Compile graph.', 'details': 'Compile the state graph into a runnable workflow.'}
8. {'step': 8, 'action': 'Run example queries.', 'details': 'Stream through the workflow with example inputs to demonstrate its functionality.'}
## Inputs
- {'name': 'OPENAI_API_KEY', 'description': 'OpenAI API key for language models.'}
- {'name': 'TAVILY_API_KEY', 'description': 'Tavily API key for search functionality.'}
## Outputs
- {'name': 'Example 1: Code Hello World', 'description': 'A demonstration of coding a simple hello world program.'}
- {'name': 'Example 2: Research Report', 'description': 'A demonstration of researching and writing a brief report on pikas.'}
## Failure Modes
- {'mode': 'Invalid API keys', 'description': 'The workflow may fail if the provided API keys are invalid or expired.'}
- {'mode': 'Insufficient permissions', 'description': 'The workflow may fail if the user does not have sufficient permissions to use the Tavily search functionality.'}
## Source
Extracted from: [https://github.com/extrawest/multi_agent_workflow_demo_in_langgraph.git](https://github.com/extrawest/multi_agent_workflow_demo_in_langgraph.git)
Confidence: 0.9
+6
View File
@@ -0,0 +1,6 @@
# Commands: agent-supervisor
## Available Commands
- `/skill agent-supervisor` — Load this skill
- `/run agent-supervisor` — Execute workflow
+10
View File
@@ -0,0 +1,10 @@
# Examples: agent-supervisor
## Usage Example
```python
# How to use this skill
# Inputs: {'name': 'OPENAI_API_KEY', 'description': 'OpenAI API key for language models.'}, {'name': 'TAVILY_API_KEY', 'description': 'Tavily API key for search functionality.'}
# Process: {'step': 1, 'action': 'Load environment variables.', 'details': 'Set the OPENAI_API_KEY and TAVILY_API_KEY environment variables.'} → {'step': 2, 'action': 'Configure LangChain tools.', 'details': 'Initialize TavilySearchResults and PythonREPLTool.'} → {'step': 3, 'action': 'Define agent nodes.', 'details': 'Create functions for the Researcher and Coder agents that process state through their respective tasks.'}
# Outputs: {'name': 'Example 1: Code Hello World', 'description': 'A demonstration of coding a simple hello world program.'}, {'name': 'Example 2: Research Report', 'description': 'A demonstration of researching and writing a brief report on pikas.'}
```
+81
View File
@@ -0,0 +1,81 @@
{
"name": "agent-supervisor",
"version": "1.0.0",
"goal": "Demonstrate a supervisor-worker architecture for intelligent task delegation and real-time decision-making.",
"inputs": [
{
"name": "OPENAI_API_KEY",
"description": "OpenAI API key for language models."
},
{
"name": "TAVILY_API_KEY",
"description": "Tavily API key for search functionality."
}
],
"steps": [
{
"step": 1,
"action": "Load environment variables.",
"details": "Set the OPENAI_API_KEY and TAVILY_API_KEY environment variables."
},
{
"step": 2,
"action": "Configure LangChain tools.",
"details": "Initialize TavilySearchResults and PythonREPLTool."
},
{
"step": 3,
"action": "Define agent nodes.",
"details": "Create functions for the Researcher and Coder agents that process state through their respective tasks."
},
{
"step": 4,
"action": "Set up supervisor agent.",
"details": "Create a supervisor agent function that decides which worker should act next based on user input."
},
{
"step": 5,
"action": "Build state graph.",
"details": "Construct the state graph with nodes for each agent and edges connecting them to the supervisor node."
},
{
"step": 6,
"action": "Add conditional edges.",
"details": "Define conditions for transitioning between agents based on their responses."
},
{
"step": 7,
"action": "Compile graph.",
"details": "Compile the state graph into a runnable workflow."
},
{
"step": 8,
"action": "Run example queries.",
"details": "Stream through the workflow with example inputs to demonstrate its functionality."
}
],
"outputs": [
{
"name": "Example 1: Code Hello World",
"description": "A demonstration of coding a simple hello world program."
},
{
"name": "Example 2: Research Report",
"description": "A demonstration of researching and writing a brief report on pikas."
}
],
"failure_modes": [
{
"mode": "Invalid API keys",
"description": "The workflow may fail if the provided API keys are invalid or expired."
},
{
"mode": "Insufficient permissions",
"description": "The workflow may fail if the user does not have sufficient permissions to use the Tavily search functionality."
}
],
"confidence": 0.9,
"explanation": "This workflow demonstrates a hierarchical multi-agent system where a supervisor agent makes routing decisions based on user input, delegating tasks to specialized worker agents (Researcher and Coder). It is designed to be reusable for similar task delegation scenarios.",
"source_repo": "https://github.com/extrawest/multi_agent_workflow_demo_in_langgraph.git",
"score": 1.0
}
+9
View File
@@ -0,0 +1,9 @@
# Tests: agent-supervisor
## Test Checklist
- [ ] Workflow has at least 3 steps
- [ ] All inputs are defined
- [ ] All outputs are defined
- [ ] Failure modes are documented
- [ ] Skill can be loaded without errors
+96
View File
@@ -0,0 +1,96 @@
---
name: blacknode-graph-workflow
version: 1.0.0
description: Build and execute node-based AI workflows with LLM agents and processing
nodes
inputs:
- Model configuration (NIM_MODEL, API keys for NVIDIA NIM, OpenAI, Anthropic)
- Node definitions with specified inputs and outputs (Literal, LLMAgent, FileWrite,
etc.)
- Data sources (URLs, text content, or other inputs for the workflow)
steps:
- Initialize a blacknode.Graph instance to create the workflow structure
- Add nodes to the graph with defined inputs and outputs (e.g., Literal, LLMAgent,
FileWrite)
- Define edges connecting nodes to establish data flow between them
- Execute the graph using cook() to run the workflow and generate outputs
outputs:
- Processed results from the final node (e.g., printed text, written files, or generated
data)
- Graph execution status and any errors encountered during execution
tags: []
metadata:
source_repo: https://github.com/temiroff/Blacknode.git
extracted_at: ''
confidence: 0.95
---
# blacknode-graph-workflow
Build and execute node-based AI workflows with LLM agents and processing nodes
## Setup
**Dependencies:**
```text
pip install blacknode (core package) anthropic>=0.25 openai>=1.0 petgraph (for graph operations)
```
**Setup steps:**
1. Install blacknode package: pip install blacknode
1. Configure model API keys (NIM_API_KEY, OPENAI_API_KEY, etc.) in .env or editor
1. Create a Graph instance and add nodes with inputs/outputs
1. Define node connections in g._edges list
1. Execute with g.cook() to run the workflow and capture results
## Key Files
- `blacknode/blacknode.py (Graph class implementation)`
- `examples/hello_agent.py (simple LLM agent workflow)`
- `examples/converted_nvidia_nim.py (NIM model workflow)`
## Steps
1. Initialize a blacknode.Graph instance to create the workflow structure
2. Add nodes to the graph with defined inputs and outputs (e.g., Literal, LLMAgent, FileWrite)
3. Define edges connecting nodes to establish data flow between them
4. Execute the graph using cook() to run the workflow and generate outputs
## Implementation Details
```python
g = bn.Graph()
```
```python
g._edges = [{'from': 'model', 'from_port': 'value', 'to': 'agent', 'to_port': 'model'}]
```
```python
result = g.cook(output_node, 'value')
```
## Inputs
- Model configuration (NIM_MODEL, API keys for NVIDIA NIM, OpenAI, Anthropic)
- Node definitions with specified inputs and outputs (Literal, LLMAgent, FileWrite, etc.)
- Data sources (URLs, text content, or other inputs for the workflow)
## Outputs
- Processed results from the final node (e.g., printed text, written files, or generated data)
- Graph execution status and any errors encountered during execution
## Failure Modes
- Missing or invalid model API key causing graph initialization failure
- Incorrect node connections or missing edge definitions leading to runtime errors
- Model not found or unavailable in the specified environment causing execution failure
- Graph edges not properly defined or mismatched causing cook() to fail
## Source
Extracted from: [https://github.com/temiroff/Blacknode.git](https://github.com/temiroff/Blacknode.git)
Confidence: 0.95
@@ -0,0 +1,6 @@
# Commands: blacknode-graph-workflow
## Available Commands
- `/skill blacknode-graph-workflow` — Load this skill
- `/run blacknode-graph-workflow` — Execute workflow
@@ -0,0 +1,10 @@
# Examples: blacknode-graph-workflow
## Usage Example
```python
# How to use this skill
# Inputs: Model configuration (NIM_MODEL, API keys for NVIDIA NIM, OpenAI, Anthropic), Node definitions with specified inputs and outputs (Literal, LLMAgent, FileWrite, etc.), Data sources (URLs, text content, or other inputs for the workflow)
# Process: Initialize a blacknode.Graph instance to create the workflow structure → Add nodes to the graph with defined inputs and outputs (e.g., Literal, LLMAgent, FileWrite) → Define edges connecting nodes to establish data flow between them
# Outputs: Processed results from the final node (e.g., printed text, written files, or generated data), Graph execution status and any errors encountered during execution
```
@@ -0,0 +1,30 @@
{
"name": "blacknode-graph-workflow",
"version": "1.0.0",
"goal": "Build and execute node-based AI workflows with LLM agents and processing nodes",
"inputs": [
"Model configuration (NIM_MODEL, API keys for NVIDIA NIM, OpenAI, Anthropic)",
"Node definitions with specified inputs and outputs (Literal, LLMAgent, FileWrite, etc.)",
"Data sources (URLs, text content, or other inputs for the workflow)"
],
"steps": [
"Initialize a blacknode.Graph instance to create the workflow structure",
"Add nodes to the graph with defined inputs and outputs (e.g., Literal, LLMAgent, FileWrite)",
"Define edges connecting nodes to establish data flow between them",
"Execute the graph using cook() to run the workflow and generate outputs"
],
"outputs": [
"Processed results from the final node (e.g., printed text, written files, or generated data)",
"Graph execution status and any errors encountered during execution"
],
"failure_modes": [
"Missing or invalid model API key causing graph initialization failure",
"Incorrect node connections or missing edge definitions leading to runtime errors",
"Model not found or unavailable in the specified environment causing execution failure",
"Graph edges not properly defined or mismatched causing cook() to fail"
],
"confidence": 0.95,
"explanation": "Blacknode provides a standardized Graph-based workflow pattern where users create node graphs using the blacknode.Graph class. This pattern is reusable across projects as it follows a consistent structure: initialize a graph, add nodes with defined inputs/outputs, connect them with edges, and execute with cook(). The examples demonstrate this pattern with LLM agents and text processing pipelines, making it adaptable to various robotics and AI workflows.",
"source_repo": "https://github.com/temiroff/Blacknode.git",
"score": 1.0
}
+9
View File
@@ -0,0 +1,9 @@
# Tests: blacknode-graph-workflow
## Test Checklist
- [ ] Workflow has at least 3 steps
- [ ] All inputs are defined
- [ ] All outputs are defined
- [ ] Failure modes are documented
- [ ] Skill can be loaded without errors
+115
View File
@@ -0,0 +1,115 @@
---
name: langgraph-agent-workflow
version: 1.0.0
description: Orchestrate multi-step AI agents using LangGraph with SerperDevTool for
RAG, code execution, and citation generation
inputs:
- LangGraph chain configuration files defining agent workflows
- SerperDevTool integration for LLM tool access
- React agent creation scripts via create_react_agent
- Knowledge graph retrieval and citation generation pipelines
steps:
- 'Step 1: Define LangGraph chain architecture with SerperDevTool integration - Create
a LangGraph chain that combines retrieval, reasoning, and response generation using
SerperDevTool for tool access'
- 'Step 2: Implement React agent wrapper - Use create_react_agent to build a frontend
agent that can interact with the LangGraph chain'
- 'Step 3: Configure RAG pipeline - Set up vector search (Qdrant/OpenSearch) and knowledge
graph retrieval (Neo4j/ArangoDB) with citation generation'
- 'Step 4: Add code execution sandbox - Integrate artifact generation capabilities
for code-related tasks'
- "Step 5: Orchestrate multi-step research workflow - Chain search \u2192 deep research\
\ \u2192 agent response in a single LangGraph workflow"
outputs:
- Reusable LangGraph chain definition (pyfile) with configurable steps
- React agent frontend component that can be deployed independently
- RAG pipeline that generates block citations and grounded answers
- Code execution sandbox for artifact generation
- Documentation for parameterizing workflows for different tasks
tags: []
metadata:
source_repo: https://github.com/pipeshub-ai/pipeshub-ai.git
extracted_at: ''
confidence: 0.95
---
# langgraph-agent-workflow
Orchestrate multi-step AI agents using LangGraph with SerperDevTool for RAG, code execution, and citation generation
## Setup
**Dependencies:**
```text
pip install langgraph>=0.7.0 serper-dev-tool>=0.1.0 qdrant-client or opensearch-dsl neo4j-driver or arango-database-driver react, next.js
```
**Setup steps:**
1. Install LangGraph and SerperDevTool dependencies
1. Configure vector store (Qdrant/OpenSearch) and knowledge graph (Neo4j/ArangoDB)
1. Define chain topology with retrieval, reasoning, and response steps
1. Build React agent frontend using create_react_agent
1. Test multi-step agent workflows end-to-end
## Key Files
- `pipeshub-ai/workflows/agent_chain.py - Main LangGraph chain definition`
- `pipeshub-ai/workflows/agent_react.py - React agent wrapper`
- `pipeshub-ai/workflows/rag_pipeline.py - RAG with citation generation`
- `pipeshub-ai/workflows/code_sandbox.py - Code execution sandbox`
## Steps
1. Step 1: Define LangGraph chain architecture with SerperDevTool integration - Create a LangGraph chain that combines retrieval, reasoning, and response generation using SerperDevTool for tool access
2. Step 2: Implement React agent wrapper - Use create_react_agent to build a frontend agent that can interact with the LangGraph chain
3. Step 3: Configure RAG pipeline - Set up vector search (Qdrant/OpenSearch) and knowledge graph retrieval (Neo4j/ArangoDB) with citation generation
4. Step 4: Add code execution sandbox - Integrate artifact generation capabilities for code-related tasks
5. Step 5: Orchestrate multi-step research workflow - Chain search → deep research → agent response in a single LangGraph workflow
## Implementation Details
```python
chain = LangGraph()
```
```python
chain.add_step(SerperDevToolAgent())
```
```python
agent = create_react_agent(chain, SerperDevToolAgent())
```
```python
workflow = chain.start()
```
## Inputs
- LangGraph chain configuration files defining agent workflows
- SerperDevTool integration for LLM tool access
- React agent creation scripts via create_react_agent
- Knowledge graph retrieval and citation generation pipelines
## Outputs
- Reusable LangGraph chain definition (pyfile) with configurable steps
- React agent frontend component that can be deployed independently
- RAG pipeline that generates block citations and grounded answers
- Code execution sandbox for artifact generation
- Documentation for parameterizing workflows for different tasks
## Failure Modes
- GraphDB connection failures if Neo4j/ArangoDB is not properly configured
- Vector store unavailability (Qdrant/OpenSearch) causing RAG pipeline to fail
- LLM tool access errors if SerperDevTool is not properly initialized
- Agent timeout if complex multi-step reasoning exceeds time limits
- Sandbox execution failures if code has security vulnerabilities or infinite loops
## Source
Extracted from: [https://github.com/pipeshub-ai/pipeshub-ai.git](https://github.com/pipeshub-ai/pipeshub-ai.git)
Confidence: 0.95
@@ -0,0 +1,6 @@
# Commands: langgraph-agent-workflow
## Available Commands
- `/skill langgraph-agent-workflow` — Load this skill
- `/run langgraph-agent-workflow` — Execute workflow
@@ -0,0 +1,10 @@
# Examples: langgraph-agent-workflow
## Usage Example
```python
# How to use this skill
# Inputs: LangGraph chain configuration files defining agent workflows, SerperDevTool integration for LLM tool access, React agent creation scripts via create_react_agent, Knowledge graph retrieval and citation generation pipelines
# Process: Step 1: Define LangGraph chain architecture with SerperDevTool integration - Create a LangGraph chain that combines retrieval, reasoning, and response generation using SerperDevTool for tool access → Step 2: Implement React agent wrapper - Use create_react_agent to build a frontend agent that can interact with the LangGraph chain → Step 3: Configure RAG pipeline - Set up vector search (Qdrant/OpenSearch) and knowledge graph retrieval (Neo4j/ArangoDB) with citation generation
# Outputs: Reusable LangGraph chain definition (pyfile) with configurable steps, React agent frontend component that can be deployed independently, RAG pipeline that generates block citations and grounded answers, Code execution sandbox for artifact generation, Documentation for parameterizing workflows for different tasks
```
@@ -0,0 +1,36 @@
{
"name": "langgraph-agent-workflow",
"version": "1.0.0",
"goal": "Orchestrate multi-step AI agents using LangGraph with SerperDevTool for RAG, code execution, and citation generation",
"inputs": [
"LangGraph chain configuration files defining agent workflows",
"SerperDevTool integration for LLM tool access",
"React agent creation scripts via create_react_agent",
"Knowledge graph retrieval and citation generation pipelines"
],
"steps": [
"Step 1: Define LangGraph chain architecture with SerperDevTool integration - Create a LangGraph chain that combines retrieval, reasoning, and response generation using SerperDevTool for tool access",
"Step 2: Implement React agent wrapper - Use create_react_agent to build a frontend agent that can interact with the LangGraph chain",
"Step 3: Configure RAG pipeline - Set up vector search (Qdrant/OpenSearch) and knowledge graph retrieval (Neo4j/ArangoDB) with citation generation",
"Step 4: Add code execution sandbox - Integrate artifact generation capabilities for code-related tasks",
"Step 5: Orchestrate multi-step research workflow - Chain search \u2192 deep research \u2192 agent response in a single LangGraph workflow"
],
"outputs": [
"Reusable LangGraph chain definition (pyfile) with configurable steps",
"React agent frontend component that can be deployed independently",
"RAG pipeline that generates block citations and grounded answers",
"Code execution sandbox for artifact generation",
"Documentation for parameterizing workflows for different tasks"
],
"failure_modes": [
"GraphDB connection failures if Neo4j/ArangoDB is not properly configured",
"Vector store unavailability (Qdrant/OpenSearch) causing RAG pipeline to fail",
"LLM tool access errors if SerperDevTool is not properly initialized",
"Agent timeout if complex multi-step reasoning exceeds time limits",
"Sandbox execution failures if code has security vulnerabilities or infinite loops"
],
"confidence": 0.95,
"explanation": "PipesHub provides a reusable LangGraph-based agent workflow framework that can be parameterized for different tasks. The core pattern involves defining a LangGraph chain with SerperDevTool integration for tool access, creating a React agent wrapper, and configuring RAG pipelines with citation generation. This framework can be reused across RAG, code execution, and research workflows by adjusting the chain definition and agent configuration.",
"source_repo": "https://github.com/pipeshub-ai/pipeshub-ai.git",
"score": 1.0
}
+9
View File
@@ -0,0 +1,9 @@
# Tests: langgraph-agent-workflow
## Test Checklist
- [ ] Workflow has at least 3 steps
- [ ] All inputs are defined
- [ ] All outputs are defined
- [ ] Failure modes are documented
- [ ] Skill can be loaded without errors
@@ -0,0 +1,96 @@
---
name: langgraph-multi-agent-router
version: 1.0.0
description: Orchestrate a multi-agent workflow where specialized agents collaborate
sequentially to gather information, structure it, and generate a final response
inputs:
- User query string (e.g., destination location)
- BedrockModel configuration (model_id, temperature, top_p)
- Pre-configured agents with specific system prompts and tool sets
steps:
- Researcher agent executes with system prompt to gather raw destination facts (places,
history, accommodations, food, web pages) using BedrockModel and available tools
(calculator, current_time)
- Travel guide agent receives raw research output and structures it into labeled sections
(Must-See Attractions, Historical Highlights, Accommodation Areas, Culinary Delights,
Suggested Web Pages)
- Writer agent receives the structured guide and synthesizes it into a professional
client-facing response with clear formatting and emphasis on the suggested web pages
outputs:
- Raw research data (JSON string containing gathered facts)
- Structured guide content (markdown-formatted travel guide with labeled sections)
- Final client response (professional formatted response ready for delivery)
tags: []
metadata:
source_repo: https://github.com/omerbsezer/Fast-LLM-Agent-MCP.git
extracted_at: ''
confidence: 0.95
---
# langgraph-multi-agent-router
Orchestrate a multi-agent workflow where specialized agents collaborate sequentially to gather information, structure it, and generate a final response
## Setup
**Dependencies:**
```text
pip install langchain langgraph bedrock-model pydantic
```
**Setup steps:**
1. Install langchain and langgraph packages
1. Configure BedrockModel with desired parameters (model_id, temperature, top_p)
1. Create three Agent instances with specific system prompts and tool sets
1. Initialize LangGraph with the agent chain and run the workflow
## Key Files
- `agents/langchain_langgraph/00-basic-agent/agent.py`
- `agents/langchain_langgraph/02-agent-with-tools-structured-output/agent.py`
- `agents/langchain_langgraph/08-langgraph-multi-agents-sequential-pattern/agent.py`
## Steps
1. Researcher agent executes with system prompt to gather raw destination facts (places, history, accommodations, food, web pages) using BedrockModel and available tools (calculator, current_time)
2. Travel guide agent receives raw research output and structures it into labeled sections (Must-See Attractions, Historical Highlights, Accommodation Areas, Culinary Delights, Suggested Web Pages)
3. Writer agent receives the structured guide and synthesizes it into a professional client-facing response with clear formatting and emphasis on the suggested web pages
## Implementation Details
```python
Researcher agent uses BedrockModel with temperature=0.7, top_p=0.9 to gather destination facts
```
```python
Travel guide agent receives raw output and formats into 5 labeled sections
```
```python
Writer agent takes structured guide and writes professional client response
```
## Inputs
- User query string (e.g., destination location)
- BedrockModel configuration (model_id, temperature, top_p)
- Pre-configured agents with specific system prompts and tool sets
## Outputs
- Raw research data (JSON string containing gathered facts)
- Structured guide content (markdown-formatted travel guide with labeled sections)
- Final client response (professional formatted response ready for delivery)
## Failure Modes
- Researcher agent fails to gather sufficient data or returns incomplete results
- Travel guide agent fails to structure information correctly or produces unreadable output
- Writer agent fails to format the final response properly or loses key information from the guide
## Source
Extracted from: [https://github.com/omerbsezer/Fast-LLM-Agent-MCP.git](https://github.com/omerbsezer/Fast-LLM-Agent-MCP.git)
Confidence: 0.95
@@ -0,0 +1,6 @@
# Commands: langgraph-multi-agent-router
## Available Commands
- `/skill langgraph-multi-agent-router` — Load this skill
- `/run langgraph-multi-agent-router` — Execute workflow
@@ -0,0 +1,10 @@
# Examples: langgraph-multi-agent-router
## Usage Example
```python
# How to use this skill
# Inputs: User query string (e.g., destination location), BedrockModel configuration (model_id, temperature, top_p), Pre-configured agents with specific system prompts and tool sets
# Process: Researcher agent executes with system prompt to gather raw destination facts (places, history, accommodations, food, web pages) using BedrockModel and available tools (calculator, current_time) → Travel guide agent receives raw research output and structures it into labeled sections (Must-See Attractions, Historical Highlights, Accommodation Areas, Culinary Delights, Suggested Web Pages) → Writer agent receives the structured guide and synthesizes it into a professional client-facing response with clear formatting and emphasis on the suggested web pages
# Outputs: Raw research data (JSON string containing gathered facts), Structured guide content (markdown-formatted travel guide with labeled sections), Final client response (professional formatted response ready for delivery)
```
@@ -0,0 +1,29 @@
{
"name": "langgraph-multi-agent-router",
"version": "1.0.0",
"goal": "Orchestrate a multi-agent workflow where specialized agents collaborate sequentially to gather information, structure it, and generate a final response",
"inputs": [
"User query string (e.g., destination location)",
"BedrockModel configuration (model_id, temperature, top_p)",
"Pre-configured agents with specific system prompts and tool sets"
],
"steps": [
"Researcher agent executes with system prompt to gather raw destination facts (places, history, accommodations, food, web pages) using BedrockModel and available tools (calculator, current_time)",
"Travel guide agent receives raw research output and structures it into labeled sections (Must-See Attractions, Historical Highlights, Accommodation Areas, Culinary Delights, Suggested Web Pages)",
"Writer agent receives the structured guide and synthesizes it into a professional client-facing response with clear formatting and emphasis on the suggested web pages"
],
"outputs": [
"Raw research data (JSON string containing gathered facts)",
"Structured guide content (markdown-formatted travel guide with labeled sections)",
"Final client response (professional formatted response ready for delivery)"
],
"failure_modes": [
"Researcher agent fails to gather sufficient data or returns incomplete results",
"Travel guide agent fails to structure information correctly or produces unreadable output",
"Writer agent fails to format the final response properly or loses key information from the guide"
],
"confidence": 0.95,
"explanation": "This workflow demonstrates a reusable multi-stage agent pattern where specialized agents collaborate in sequence. The Researcher agent gathers raw information using a domain-specific model, the Travel Guide agent structures that information into a consistent format, and the Writer agent synthesizes the final output. This pattern can be adapted to other domains (e.g., code generation, data analysis, research workflows) by swapping the agent types and system prompts while maintaining the same three-step structure.",
"source_repo": "https://github.com/omerbsezer/Fast-LLM-Agent-MCP.git",
"score": 1.0
}
@@ -0,0 +1,9 @@
# Tests: langgraph-multi-agent-router
## Test Checklist
- [ ] Workflow has at least 3 steps
- [ ] All inputs are defined
- [ ] All outputs are defined
- [ ] Failure modes are documented
- [ ] Skill can be loaded without errors
@@ -0,0 +1,83 @@
---
name: langgraph-workflow-creation
version: 1.0.0
description: Create a LangGraph workflow to gather facts using SerperDevTool and process
them with an AI agent.
inputs:
- API Key for SerperDevTool
- Search Query
steps:
- 'Step 1: Import necessary modules from langgraph and langchain libraries'
- 'Step 2: Create a LangGraph agent using `langgraph.create_react_agent` function
with SerperDevTool as the tool node'
- 'Step 3: Define the search query and pass it to the agent for fact gathering'
- 'Step 4: Process the gathered facts within the AI agent'
outputs:
- Processed Facts
tags: []
metadata:
source_repo: https://github.com/jkmaina/LangGraphProjects.git
extracted_at: ''
confidence: 0.95
---
# langgraph-workflow-creation
Create a LangGraph workflow to gather facts using SerperDevTool and process them with an AI agent.
## Setup
**Dependencies:**
```text
pip install langchain serperdev
```
**Setup steps:**
1. Install required libraries: pip install langchain serperdev
1. Add API key to .env file: OPENAPI_API_KEY=your_api_key
## Key Files
- `agent.py - Contains the LangGraph agent creation logic`
- `tool_node.py - Defines the SerperDevTool node`
## Steps
1. Step 1: Import necessary modules from langgraph and langchain libraries
2. Step 2: Create a LangGraph agent using `langgraph.create_react_agent` function with SerperDevTool as the tool node
3. Step 3: Define the search query and pass it to the agent for fact gathering
4. Step 4: Process the gathered facts within the AI agent
## Implementation Details
```python
import langgraph
from serperdev import SerperDevTool
def create_agent(api_key, query):
tool = SerperDevTool(api_key)
agent = langgraph.create_react_agent(tool=tool)
facts = agent.run(query)
return process_facts(facts)
```
## Inputs
- API Key for SerperDevTool
- Search Query
## Outputs
- Processed Facts
## Failure Modes
- API Key not provided
- Invalid Search Query
## Source
Extracted from: [https://github.com/jkmaina/LangGraphProjects.git](https://github.com/jkmaina/LangGraphProjects.git)
Confidence: 0.95
@@ -0,0 +1,6 @@
# Commands: langgraph-workflow-creation
## Available Commands
- `/skill langgraph-workflow-creation` — Load this skill
- `/run langgraph-workflow-creation` — Execute workflow
@@ -0,0 +1,10 @@
# Examples: langgraph-workflow-creation
## Usage Example
```python
# How to use this skill
# Inputs: API Key for SerperDevTool, Search Query
# Process: Step 1: Import necessary modules from langgraph and langchain libraries → Step 2: Create a LangGraph agent using `langgraph.create_react_agent` function with SerperDevTool as the tool node → Step 3: Define the search query and pass it to the agent for fact gathering
# Outputs: Processed Facts
```
@@ -0,0 +1,26 @@
{
"name": "langgraph-workflow-creation",
"version": "1.0.0",
"goal": "Create a LangGraph workflow to gather facts using SerperDevTool and process them with an AI agent.",
"inputs": [
"API Key for SerperDevTool",
"Search Query"
],
"steps": [
"Step 1: Import necessary modules from langgraph and langchain libraries",
"Step 2: Create a LangGraph agent using `langgraph.create_react_agent` function with SerperDevTool as the tool node",
"Step 3: Define the search query and pass it to the agent for fact gathering",
"Step 4: Process the gathered facts within the AI agent"
],
"outputs": [
"Processed Facts"
],
"failure_modes": [
"API Key not provided",
"Invalid Search Query"
],
"confidence": 0.95,
"explanation": "This workflow is specific to fact gathering and can be adapted for different search queries or tools.",
"source_repo": "https://github.com/jkmaina/LangGraphProjects.git",
"score": 1.0
}
@@ -0,0 +1,9 @@
# Tests: langgraph-workflow-creation
## Test Checklist
- [ ] Workflow has at least 3 steps
- [ ] All inputs are defined
- [ ] All outputs are defined
- [ ] Failure modes are documented
- [ ] Skill can be loaded without errors
@@ -0,0 +1,77 @@
---
name: literature-review-with-traceable-ai-evidence
version: 1.0.0
description: Enable researchers to ingest documents, asynchronously index them, and
interact with multi-agent AI to answer questions with verifiable citations to original
text.
inputs:
- Documents in PDF, Office, image, or text formats
- Research questions or topics of interest
- 'Optional: user model configuration via .env or settings'
steps:
- Upload documents to a project (via desktop app or web UI)
- System asynchronously converts Office docs to PDF if needed, runs OCR to extract
text with coordinates, chunks and embeds into vector store
- User starts a main research session or creates exploration branches without waiting
for indexing to finish
- Leader agent receives query and delegates subtasks to researcher, reviewer, writer
subagents
- Subagents perform hybrid retrieval and rerank to find relevant chunks with source
coordinates
- Agents synthesize answers and return evidence with clickable citations that highlight
original pages
- User verifies conclusions by navigating to cited source locations and can save notes
to research memory or mind map
outputs:
- AI-generated answers with traceable evidence (coordinates, page highlights)
- Research session history with branches
- Indexed document library for future queries
- Mind maps or structured notes
- Persistent run events for resuming sessions
tags: []
metadata:
source_repo: https://github.com/0verL1nk/PaperSage.git
extracted_at: ''
confidence: 0.85
---
# literature-review-with-traceable-ai-evidence
Enable researchers to ingest documents, asynchronously index them, and interact with multi-agent AI to answer questions with verifiable citations to original text.
## Steps
1. Upload documents to a project (via desktop app or web UI)
2. System asynchronously converts Office docs to PDF if needed, runs OCR to extract text with coordinates, chunks and embeds into vector store
3. User starts a main research session or creates exploration branches without waiting for indexing to finish
4. Leader agent receives query and delegates subtasks to researcher, reviewer, writer subagents
5. Subagents perform hybrid retrieval and rerank to find relevant chunks with source coordinates
6. Agents synthesize answers and return evidence with clickable citations that highlight original pages
7. User verifies conclusions by navigating to cited source locations and can save notes to research memory or mind map
## Inputs
- Documents in PDF, Office, image, or text formats
- Research questions or topics of interest
- Optional: user model configuration via .env or settings
## Outputs
- AI-generated answers with traceable evidence (coordinates, page highlights)
- Research session history with branches
- Indexed document library for future queries
- Mind maps or structured notes
- Persistent run events for resuming sessions
## Failure Modes
- Missing local Office/LibreOffice converter causes document conversion failure
- First-time model download may be slow or require network
- OCR may have low confidence on poor quality scans
- Retrieval might miss context if chunking splits semantics
- Multi-agent coordination could produce conflicting intermediate results
## Source
Extracted from: [https://github.com/0verL1nk/PaperSage.git](https://github.com/0verL1nk/PaperSage.git)
Confidence: 0.85
@@ -0,0 +1,6 @@
# Commands: literature-review-with-traceable-ai-evidence
## Available Commands
- `/skill literature-review-with-traceable-ai-evidence` — Load this skill
- `/run literature-review-with-traceable-ai-evidence` — Execute workflow
@@ -0,0 +1,10 @@
# Examples: literature-review-with-traceable-ai-evidence
## Usage Example
```python
# How to use this skill
# Inputs: Documents in PDF, Office, image, or text formats, Research questions or topics of interest, Optional: user model configuration via .env or settings
# Process: Upload documents to a project (via desktop app or web UI) → System asynchronously converts Office docs to PDF if needed, runs OCR to extract text with coordinates, chunks and embeds into vector store → User starts a main research session or creates exploration branches without waiting for indexing to finish
# Outputs: AI-generated answers with traceable evidence (coordinates, page highlights), Research session history with branches, Indexed document library for future queries, Mind maps or structured notes, Persistent run events for resuming sessions
```
@@ -0,0 +1,37 @@
{
"name": "literature-review-with-traceable-ai-evidence",
"version": "1.0.0",
"goal": "Enable researchers to ingest documents, asynchronously index them, and interact with multi-agent AI to answer questions with verifiable citations to original text.",
"inputs": [
"Documents in PDF, Office, image, or text formats",
"Research questions or topics of interest",
"Optional: user model configuration via .env or settings"
],
"steps": [
"Upload documents to a project (via desktop app or web UI)",
"System asynchronously converts Office docs to PDF if needed, runs OCR to extract text with coordinates, chunks and embeds into vector store",
"User starts a main research session or creates exploration branches without waiting for indexing to finish",
"Leader agent receives query and delegates subtasks to researcher, reviewer, writer subagents",
"Subagents perform hybrid retrieval and rerank to find relevant chunks with source coordinates",
"Agents synthesize answers and return evidence with clickable citations that highlight original pages",
"User verifies conclusions by navigating to cited source locations and can save notes to research memory or mind map"
],
"outputs": [
"AI-generated answers with traceable evidence (coordinates, page highlights)",
"Research session history with branches",
"Indexed document library for future queries",
"Mind maps or structured notes",
"Persistent run events for resuming sessions"
],
"failure_modes": [
"Missing local Office/LibreOffice converter causes document conversion failure",
"First-time model download may be slow or require network",
"OCR may have low confidence on poor quality scans",
"Retrieval might miss context if chunking splits semantics",
"Multi-agent coordination could produce conflicting intermediate results"
],
"confidence": 0.85,
"explanation": "The README describes PaperSage's core workflow: asynchronous document ingestion with OCR/indexing, followed by multi-agent question answering with cited evidence. This process is not tied to the specific codebase and can be reused as a general literature review methodology for any document-centric research using RAG and agent collaboration.",
"source_repo": "https://github.com/0verL1nk/PaperSage.git",
"score": 1.0
}
@@ -0,0 +1,9 @@
# Tests: literature-review-with-traceable-ai-evidence
## Test Checklist
- [ ] Workflow has at least 3 steps
- [ ] All inputs are defined
- [ ] All outputs are defined
- [ ] Failure modes are documented
- [ ] Skill can be loaded without errors
@@ -9,10 +9,10 @@ steps:
- 'Step 1: Researcher agent (agent.py) uses LangGraph create_react_agent with BedrockModel - 'Step 1: Researcher agent (agent.py) uses LangGraph create_react_agent with BedrockModel
to gather raw facts about the destination.' to gather raw facts about the destination.'
- 'Step 2: Travel Guide Generator agent (agent.py) synthesizes the gathered data into - 'Step 2: Travel Guide Generator agent (agent.py) synthesizes the gathered data into
a structured travel guide based on the user''s request and the raw information provided a structured travel guide based on the user''s request and raw information provided
by the researcher.' by the researcher.'
- 'Step 3: Writer agent (agent.py) formats the final response, including the structured - 'Step 3: Writer agent (agent.py) formats the final response, including the structured
travel guide content and prominently featuring the ''Suggested Web Pages'' section.' guide content and prominently features the ''Suggested Web Pages'' section.'
outputs: outputs:
- Structured travel guide with key sections - Structured travel guide with key sections
- Final client response - Final client response
@@ -32,23 +32,24 @@ Gather and process information from multiple agents to generate a comprehensive
**Dependencies:** **Dependencies:**
```text ```text
pip install python langchain strands pip install python3 fastapi uvicorn strands bedrock-model
``` ```
**Setup steps:** **Setup steps:**
1. Install required dependencies using pip and ensure the BedrockModel is properly configured. 1. Install required dependencies using pip
1. Set up environment variables for API keys and model IDs
## Key Files ## Key Files
- `agents/aws_strands/05-agent-strands-multiagent-workflow-sequential/agent.py - Contains the sequential workflow logic for gathering and processing information.` - `agents/aws_strands/05-agent-strands-multiagent-workflow-sequential/agent.py - Contains the multi-agent workflow logic.`
- `agents/aws_strands/05-agent-strands-multiagent-workflow-sequential/app.py - Provides a FastAPI endpoint to interact with the multi-agent system.` - `agents/aws_strands/05-agent-strands-multiagent-workflow-sequential/app.py - FastAPI app to handle user queries.`
## Steps ## Steps
1. Step 1: Researcher agent (agent.py) uses LangGraph create_react_agent with BedrockModel to gather raw facts about the destination. 1. Step 1: Researcher agent (agent.py) uses LangGraph create_react_agent with BedrockModel to gather raw facts about the destination.
2. Step 2: Travel Guide Generator agent (agent.py) synthesizes the gathered data into a structured travel guide based on the user's request and the raw information provided by the researcher. 2. Step 2: Travel Guide Generator agent (agent.py) synthesizes the gathered data into a structured travel guide based on the user's request and raw information provided by the researcher.
3. Step 3: Writer agent (agent.py) formats the final response, including the structured travel guide content and prominently featuring the 'Suggested Web Pages' section. 3. Step 3: Writer agent (agent.py) formats the final response, including the structured guide content and prominently features the 'Suggested Web Pages' section.
## Implementation Details ## Implementation Details
@@ -75,7 +76,7 @@ final_response = writer_agent(writer_prompt, stream=False)
## Failure Modes ## Failure Modes
- Specific failure scenario with mitigation: If any of the agents fail to process their tasks (e.g., network issues, model errors), the workflow will fail. Mitigation involves robust error handling and fallback mechanisms. - Network issues during API calls could lead to incomplete data collection or processing failures
## Source ## Source
@@ -5,6 +5,6 @@
```python ```python
# How to use this skill # How to use this skill
# Inputs: User query with location # Inputs: User query with location
# Process: Step 1: Researcher agent (agent.py) uses LangGraph create_react_agent with BedrockModel to gather raw facts about the destination. → Step 2: Travel Guide Generator agent (agent.py) synthesizes the gathered data into a structured travel guide based on the user's request and the raw information provided by the researcher. → Step 3: Writer agent (agent.py) formats the final response, including the structured travel guide content and prominently featuring the 'Suggested Web Pages' section. # Process: Step 1: Researcher agent (agent.py) uses LangGraph create_react_agent with BedrockModel to gather raw facts about the destination. → Step 2: Travel Guide Generator agent (agent.py) synthesizes the gathered data into a structured travel guide based on the user's request and raw information provided by the researcher. → Step 3: Writer agent (agent.py) formats the final response, including the structured guide content and prominently features the 'Suggested Web Pages' section.
# Outputs: Structured travel guide with key sections, Final client response # Outputs: Structured travel guide with key sections, Final client response
``` ```
@@ -7,18 +7,18 @@
], ],
"steps": [ "steps": [
"Step 1: Researcher agent (agent.py) uses LangGraph create_react_agent with BedrockModel to gather raw facts about the destination.", "Step 1: Researcher agent (agent.py) uses LangGraph create_react_agent with BedrockModel to gather raw facts about the destination.",
"Step 2: Travel Guide Generator agent (agent.py) synthesizes the gathered data into a structured travel guide based on the user's request and the raw information provided by the researcher.", "Step 2: Travel Guide Generator agent (agent.py) synthesizes the gathered data into a structured travel guide based on the user's request and raw information provided by the researcher.",
"Step 3: Writer agent (agent.py) formats the final response, including the structured travel guide content and prominently featuring the 'Suggested Web Pages' section." "Step 3: Writer agent (agent.py) formats the final response, including the structured guide content and prominently features the 'Suggested Web Pages' section."
], ],
"outputs": [ "outputs": [
"Structured travel guide with key sections", "Structured travel guide with key sections",
"Final client response" "Final client response"
], ],
"failure_modes": [ "failure_modes": [
"Specific failure scenario with mitigation: If any of the agents fail to process their tasks (e.g., network issues, model errors), the workflow will fail. Mitigation involves robust error handling and fallback mechanisms." "Network issues during API calls could lead to incomplete data collection or processing failures"
], ],
"confidence": 0.95, "confidence": 0.95,
"explanation": "This workflow is specific to generating travel guides but can be adapted for other types of structured content creation.", "explanation": "This workflow is specific but can be adapted for other types of guides or information gathering tasks.",
"source_repo": "https://github.com/omerbsezer/Fast-LLM-Agent-MCP.git", "source_repo": "https://github.com/omerbsezer/Fast-LLM-Agent-MCP.git",
"score": 1.0 "score": 1.0
} }
@@ -0,0 +1,113 @@
---
name: multi-agent-workflow-execution
version: 1.0.0
description: Execute multi-agent AI workflows defined in YAML blueprints by creating
sessions, submitting user prompts, and polling for completion until final answers
are returned.
inputs:
- Blueprint ID or name (to identify the workflow to execute)
- User shortcut (authentication identifier for the user)
- User question or prompt (input to the workflow)
- Base URL of the UnifAI API (endpoint for session management)
- Polling interval (seconds between status checks during execution)
steps:
- Resolve the blueprint ID from either direct ID or name lookup via the API, handling
cases where the blueprint is not found or not unique
- Create a new session from the resolved blueprint using the session creation endpoint
- Submit the session with the user's prompt to start the multi-agent workflow execution
- Poll the session status at regular intervals until the session completes, fails,
or is cancelled
- Retrieve and return the final answer from the completed workflow
outputs:
- Final workflow result or answer (text or structured data)
- Session status (completed, failed, or cancelled)
- Error details if the workflow execution fails or times out
tags: []
metadata:
source_repo: https://github.com/redhat-community-ai-tools/UnifAI.git
extracted_at: ''
confidence: 0.95
---
# multi-agent-workflow-execution
Execute multi-agent AI workflows defined in YAML blueprints by creating sessions, submitting user prompts, and polling for completion until final answers are returned.
## Setup
**Dependencies:**
```text
pip install requests urllib3 python-langgraph temporalio
```
**Setup steps:**
1. Install Python 3.11+ and required packages (requests, langgraph, temporalio)
1. Configure API base URL and user credentials in environment variables or config
1. Define or select a blueprint from the available workflows in the system
1. Run the execution_workflow.py script with blueprint ID/name and user prompt
## Key Files
- `scripts/execution_workflow.py - Main workflow execution script`
- `multi-agent/lib/mas/engine/ - LangGraph-based orchestration modules`
- `multi-agent/lib/mas/elements/ - Node definitions (custom_agent_node, merger_node, etc.)`
- `multi-agent/lib/mas/blueprints/ - Blueprint resolution and validation logic`
## Steps
1. Resolve the blueprint ID from either direct ID or name lookup via the API, handling cases where the blueprint is not found or not unique
2. Create a new session from the resolved blueprint using the session creation endpoint
3. Submit the session with the user's prompt to start the multi-agent workflow execution
4. Poll the session status at regular intervals until the session completes, fails, or is cancelled
5. Retrieve and return the final answer from the completed workflow
## Implementation Details
```python
resolve_blueprint_id() - Resolves blueprint by ID or name lookup with error handling
```
```python
create_session() - Creates a new session from a blueprint via POST /user.session.create
```
```python
submit_session() - Submits user prompt to start workflow via POST /user.session.submit
```
```python
poll_session_status() - Polls session.stream.status at configurable intervals
```
```python
get_final_answer() - Retrieves final output via GET /session.chat.get
```
## Inputs
- Blueprint ID or name (to identify the workflow to execute)
- User shortcut (authentication identifier for the user)
- User question or prompt (input to the workflow)
- Base URL of the UnifAI API (endpoint for session management)
- Polling interval (seconds between status checks during execution)
## Outputs
- Final workflow result or answer (text or structured data)
- Session status (completed, failed, or cancelled)
- Error details if the workflow execution fails or times out
## Failure Modes
- Blueprint not found or not unique - script exits with an error listing available blueprints
- Session creation fails - may be due to invalid blueprint ID, authentication issues, or rate limiting
- Session submission fails - could be due to network issues, invalid parameters, or API rate limits
- Polling loop times out - session may be stuck in a long-running state without progress
- Final answer retrieval fails - could be due to session cleanup or network issues after completion
## Source
Extracted from: [https://github.com/redhat-community-ai-tools/UnifAI.git](https://github.com/redhat-community-ai-tools/UnifAI.git)
Confidence: 0.95
@@ -0,0 +1,6 @@
# Commands: multi-agent-workflow-execution
## Available Commands
- `/skill multi-agent-workflow-execution` — Load this skill
- `/run multi-agent-workflow-execution` — Execute workflow
@@ -0,0 +1,10 @@
# Examples: multi-agent-workflow-execution
## Usage Example
```python
# How to use this skill
# Inputs: Blueprint ID or name (to identify the workflow to execute), User shortcut (authentication identifier for the user), User question or prompt (input to the workflow), Base URL of the UnifAI API (endpoint for session management), Polling interval (seconds between status checks during execution)
# Process: Resolve the blueprint ID from either direct ID or name lookup via the API, handling cases where the blueprint is not found or not unique → Create a new session from the resolved blueprint using the session creation endpoint → Submit the session with the user's prompt to start the multi-agent workflow execution
# Outputs: Final workflow result or answer (text or structured data), Session status (completed, failed, or cancelled), Error details if the workflow execution fails or times out
```
@@ -0,0 +1,35 @@
{
"name": "multi-agent-workflow-execution",
"version": "1.0.0",
"goal": "Execute multi-agent AI workflows defined in YAML blueprints by creating sessions, submitting user prompts, and polling for completion until final answers are returned.",
"inputs": [
"Blueprint ID or name (to identify the workflow to execute)",
"User shortcut (authentication identifier for the user)",
"User question or prompt (input to the workflow)",
"Base URL of the UnifAI API (endpoint for session management)",
"Polling interval (seconds between status checks during execution)"
],
"steps": [
"Resolve the blueprint ID from either direct ID or name lookup via the API, handling cases where the blueprint is not found or not unique",
"Create a new session from the resolved blueprint using the session creation endpoint",
"Submit the session with the user's prompt to start the multi-agent workflow execution",
"Poll the session status at regular intervals until the session completes, fails, or is cancelled",
"Retrieve and return the final answer from the completed workflow"
],
"outputs": [
"Final workflow result or answer (text or structured data)",
"Session status (completed, failed, or cancelled)",
"Error details if the workflow execution fails or times out"
],
"failure_modes": [
"Blueprint not found or not unique - script exits with an error listing available blueprints",
"Session creation fails - may be due to invalid blueprint ID, authentication issues, or rate limiting",
"Session submission fails - could be due to network issues, invalid parameters, or API rate limits",
"Polling loop times out - session may be stuck in a long-running state without progress",
"Final answer retrieval fails - could be due to session cleanup or network issues after completion"
],
"confidence": 0.95,
"explanation": "The UnifAI repository contains a concrete, reusable workflow pattern for executing multi-agent AI workflows. The scripts/execution_workflow.py script demonstrates a complete pipeline: resolving blueprints by ID or name, creating sessions from blueprints, submitting user prompts to start workflows, polling session status until completion, and retrieving final answers. This pattern can be adapted to any multi-agent workflow defined in the YAML blueprint system, making it reusable across different use cases and teams.",
"source_repo": "https://github.com/redhat-community-ai-tools/UnifAI.git",
"score": 1.0
}
@@ -0,0 +1,9 @@
# Tests: multi-agent-workflow-execution
## Test Checklist
- [ ] Workflow has at least 3 steps
- [ ] All inputs are defined
- [ ] All outputs are defined
- [ ] Failure modes are documented
- [ ] Skill can be loaded without errors
@@ -0,0 +1,94 @@
---
name: three-tier-evaluation-pipeline
version: 1.0.0
description: Run tasks through three evaluation tiers (Run, Trace, Thread) to produce
comprehensive reports with human-in-the-loop validation
inputs:
- query/input text for the task
- search results (for trace tier evaluation)
- evaluation criteria and thresholds
steps:
- 'Step 1: Execute the main task using the Run tier of the evaluation pipeline (agentkit/runtime/LangGraph
engine) to generate initial outputs and results'
- 'Step 2: Run the Trace tier where an LLM-as-Judge evaluates the output against defined
criteria, generating detailed analysis and scoring'
- 'Step 3: Execute the Thread tier which facilitates human-in-the-loop discussion,
approval, and iterative refinement of the output'
outputs:
- Final consolidated report combining results from all three tiers
- Detailed scores and metrics per tier
- Threaded discussion logs for human review and approval
tags: []
metadata:
source_repo: https://github.com/itszhaoziyan-n/AgentKit.git
extracted_at: ''
confidence: 0.95
---
# three-tier-evaluation-pipeline
Run tasks through three evaluation tiers (Run, Trace, Thread) to produce comprehensive reports with human-in-the-loop validation
## Setup
**Dependencies:**
```text
pip install langgraph>=0.3 langchain-core>=0.3 langchain-anthropic>=0.3 langfuse>=2.0 mcp[server]>=1.24 tenacity>=9.0 fastapi>=0.115 psycopg[binary]>=3.1
```
**Setup steps:**
1. Install dependencies with pip install -e .[dev]
1. Start infrastructure: docker compose up -d (PostgreSQL, Langfuse, MCP server)
1. Configure environment variables (DATABASE_URL, MCP_API_KEY, etc.)
1. Run the pipeline: python -m eval.runner --tiers run,thread,trace
## Key Files
- `eval/ - contains the three-tier evaluation logic`
- `scripts/ci_gate.py - threshold update and benchmark validation`
- `agentkit/runtime/ - LangGraph engine for state management and graph execution`
## Steps
1. Step 1: Execute the main task using the Run tier of the evaluation pipeline (agentkit/runtime/LangGraph engine) to generate initial outputs and results
2. Step 2: Run the Trace tier where an LLM-as-Judge evaluates the output against defined criteria, generating detailed analysis and scoring
3. Step 3: Execute the Thread tier which facilitates human-in-the-loop discussion, approval, and iterative refinement of the output
## Implementation Details
```python
The eval/ directory implements Run, Trace, and Thread stages with configurable tiers
```
```python
Benchmark suite (40 test cases) validates the pipeline's reliability
```
```python
CI/CD workflows (ci.yml, eval-fast.yml, eval-trace.yml) orchestrate the evaluation pipeline
```
## Inputs
- query/input text for the task
- search results (for trace tier evaluation)
- evaluation criteria and thresholds
## Outputs
- Final consolidated report combining results from all three tiers
- Detailed scores and metrics per tier
- Threaded discussion logs for human review and approval
## Failure Modes
- If the Run tier fails (e.g., code execution error), the pipeline can retry but may produce incomplete outputs
- If the Trace tier LLM-as-Judge produces low-quality evaluations, the Thread tier may need additional human intervention
- Threshold mismatches between tiers could cause the pipeline to exit early or require manual adjustment
## Source
Extracted from: [https://github.com/itszhaoziyan-n/AgentKit.git](https://github.com/itszhaoziyan-n/AgentKit.git)
Confidence: 0.95
@@ -0,0 +1,6 @@
# Commands: three-tier-evaluation-pipeline
## Available Commands
- `/skill three-tier-evaluation-pipeline` — Load this skill
- `/run three-tier-evaluation-pipeline` — Execute workflow
@@ -0,0 +1,10 @@
# Examples: three-tier-evaluation-pipeline
## Usage Example
```python
# How to use this skill
# Inputs: query/input text for the task, search results (for trace tier evaluation), evaluation criteria and thresholds
# Process: Step 1: Execute the main task using the Run tier of the evaluation pipeline (agentkit/runtime/LangGraph engine) to generate initial outputs and results → Step 2: Run the Trace tier where an LLM-as-Judge evaluates the output against defined criteria, generating detailed analysis and scoring → Step 3: Execute the Thread tier which facilitates human-in-the-loop discussion, approval, and iterative refinement of the output
# Outputs: Final consolidated report combining results from all three tiers, Detailed scores and metrics per tier, Threaded discussion logs for human review and approval
```
@@ -0,0 +1,29 @@
{
"name": "three-tier-evaluation-pipeline",
"version": "1.0.0",
"goal": "Run tasks through three evaluation tiers (Run, Trace, Thread) to produce comprehensive reports with human-in-the-loop validation",
"inputs": [
"query/input text for the task",
"search results (for trace tier evaluation)",
"evaluation criteria and thresholds"
],
"steps": [
"Step 1: Execute the main task using the Run tier of the evaluation pipeline (agentkit/runtime/LangGraph engine) to generate initial outputs and results",
"Step 2: Run the Trace tier where an LLM-as-Judge evaluates the output against defined criteria, generating detailed analysis and scoring",
"Step 3: Execute the Thread tier which facilitates human-in-the-loop discussion, approval, and iterative refinement of the output"
],
"outputs": [
"Final consolidated report combining results from all three tiers",
"Detailed scores and metrics per tier",
"Threaded discussion logs for human review and approval"
],
"failure_modes": [
"If the Run tier fails (e.g., code execution error), the pipeline can retry but may produce incomplete outputs",
"If the Trace tier LLM-as-Judge produces low-quality evaluations, the Thread tier may need additional human intervention",
"Threshold mismatches between tiers could cause the pipeline to exit early or require manual adjustment"
],
"confidence": 0.95,
"explanation": "The AgentKit repository contains a production-ready three-tier evaluation pipeline (Run \u2192 Trace \u2192 Thread) that can be adapted to any task requiring multi-stage validation. This workflow uses LangGraph for orchestration and LangChain for tool integration, making it portable across different agent engineering scenarios. The pattern is reusable because it separates concerns into distinct stages with clear inputs/outputs, allowing teams to plug in different evaluation criteria or human reviewers as needed.",
"source_repo": "https://github.com/itszhaoziyan-n/AgentKit.git",
"score": 1.0
}
@@ -0,0 +1,9 @@
# Tests: three-tier-evaluation-pipeline
## Test Checklist
- [ ] Workflow has at least 3 steps
- [ ] All inputs are defined
- [ ] All outputs are defined
- [ ] Failure modes are documented
- [ ] Skill can be loaded without errors