Self-Healing AI Pipeline — Claude Code vs. Claude Code + Neo MCP

A side-by-side benchmark of two ways to build a production-grade self-healing AI agent: one written with Claude Code alone, and one written with Claude Code + Neo MCP. Both solve the same compound task — search for customer data, calculate the average lifetime value from the database, and write an output file — against a deliberately flaky tool environment. The interesting result isn't the code; it's how differently the two systems approach failure recovery.

📖 Read the write-up: BLOG.md

Get Neo

| | | |---|---| | 🌐 Website | heyneo.com | | 📚 Neo MCP docs | docs.heyneo.com/neo-mcp | | 🧩 VS Code extension | marketplace.visualstudio.com → NeoResearchInc.heyneo | | ▶️ Cursor extension | marketplace.cursorapi.com → NeoResearchInc.heyneo |

Repository layout

selfhealingPipeline/
├── BLOG.md            # The write-up comparing both approaches
├── assets/            # SVG infographics used in the blog (+ rendered PNG previews)
├── claudecode/        # Implementation A — Claude Code only (ReflectiveAgent)
└── neo-mcp/           # Implementation B — Claude Code + Neo MCP (SelfHealingAgent)

| Folder | What it is | Entry point | |--------|-----------|-------------| | claudecode/ | Pure-Python reflective agent: bounded retry loop, stack-trace reflection, retry-vs-repair classification, exponential backoff. | python3 run.py | | neo-mcp/ | Modular, interface-based platform: pluggable planning engines, failure classifier, argument repair, backoff + jitter, security-guarded tools, metrics, and a pytest suite. | python3 main.py | | assets/ | Hand-authored SVG infographics for the blog. | see assets/README.md |

Each implementation has its own README with full design notes: claudecode/README.md · neo-mcp/README.md.

Quick start

Both implementations run offline with zero dependencies on the deterministic path (Python 3.10+).

# Implementation A — Claude Code only
cd claudecode
python3 run.py

# Implementation B — Claude Code + Neo MCP
cd ../neo-mcp
python3 main.py

Run the Neo MCP test suite

cd neo-mcp
pip install -r requirements-dev.txt
python3 -m pytest tests/ -v

Drive planning with a real model (optional)

Both agents transparently switch from their offline deterministic planner to a real Claude (claude-opus-4-8) planner when an API key is present:

export ANTHROPIC_API_KEY=sk-...
python3 main.py        # (or run.py) — no code changes needed

The key is read from the environment only; never commit it. .env is gitignored.

What the benchmark shows

| Capability | Claude Code | Claude Code + Neo MCP | |------------|-------------|-----------------------| | Functional agent | ✅ | ✅ | | Retry handling | ✅ | ✅ | | Failure classification | Limited | Comprehensive | | Argument repair | ❌ | ✅ | | Structured observability | Partial | Built-in | | Pluggable planning engines | ❌ | ✅ | | Testing maturity | Basic | Extensive |

Full discussion and infographics in BLOG.md.

Requirements

• Python 3.10+
• No runtime dependencies for the offline path
• Optional: anthropic SDK + ANTHROPIC_API_KEY for the real-LLM planner
• Dev/testing: pip install -r neo-mcp/requirements-dev.txt

License

MIT — see LICENSE.