MicroHEROX / Mult-Hands-Eyes-MCP

A platform-agnostic MCP server that lets your online LLM (opencode, Claude Desktop, Cursor, Cline, Windsurf, Cherry Studio, …) hand off repetitive, token-wasting grunt work to OpenAI-compatible local inference services running on this machine — text work and vision work (image understanding / OCR / image comparison).

The main model stays exactly where your deployment puts it. When a task is cheaper to do locally, the model calls:

• local_run — run one prompt on a local text model: batch rewrites, name translations, string munging, deduplication, short summaries, structured extraction.
• local_vision — send images to a local multimodal model: OCR, image analysis, multi-image comparison, using structured report templates. Text-only online models get their eyes this way: pass an image path or URL, get text back.
• local_status — list configured backends and probe their health.

The server is a pure client: it never starts, owns, or kills any process, and never writes any file. It only talks HTTP to the services you run.

What it does

• Two model-facing tools (local_run, local_vision) plus a diagnostics tool (local_status), registered with the official MCP Python SDK (≥ 2.0).
• Any OpenAI-compatible endpoint: KoboldCpp, Unsloth Desktop, llama.cpp server, LM Studio, Ollama /v1, text-generation-webui — declared per backend in one JSON config, with text / vision capability routing.
• Structured vision reports: analyze (8-section report), ocr (character-exact extraction), compare (2–4 images, joint reasoning, 5-section report), plus a fidelity rule (relay verbatim, never invent, preserve uncertainty).
• Three image sources for vision: local file paths, data: URLs, http(s):// URLs (downloaded server-side and re-encoded — the local model never needs internet).
• Three transports: stdio (default, zero-arg launch), SSE, and Streamable HTTP — protocol versions 2024-11-05 / 2025-03-26 / 2025-06-18 / 2025-11-25 verified against the official MCP client.
• Hot-reloaded config: the JSON file is re-read on every call; edit it without restarting. A temporarily broken file falls back to the last good one.
• Spec-compliant errors: every failure surfaces as protocol-level CallToolResult.isError=true with a stable [CODE] and an actionable hint the model can self-heal from.
• Installable two ways: uv tool install . (Python) or npx mult-hands-eyes-mcp (npm wrapper that bootstraps its own Python venv).

What it does NOT do

• Does not replace your model provider — the online model is always the brain; local models are reachable only through the two tools.
• Does not start, configure, or stop any local service, and never scans ports or auto-detects servers. Endpoints are explicitly configured by you.
• Does not bundle or host model files (GGUF / mmproj). Bring your own.
• Does not stream — one tool call, one complete answer (simpler and sufficient).
• Does not modify any client config file — you add the MCP entry yourself.

Requirements

Installation

git clone https://github.com/MicroHEROX/Mult-Hands-Eyes-MCP.git
cd Mult-Hands-Eyes-MCP
uv tool install .            # isolated install, no system pollution
multhands --help

or from npm (bundles the same Python server; first run creates a private venv and installs two small deps):

npx mult-hands-eyes-mcp

Configuration

Create multhands.json anywhere and point MULTHANDS_CONFIG at it (fallback: multhands.json in the working directory):

{
  "defaultBackend": "koboldcpp",
  "backends": {
    "koboldcpp": {
      "baseURL": "http://127.0.0.1:5001",
      "model": "koboldcpp",
      "capabilities": ["text", "vision"],
      "timeoutMs": 120000,
      "maxTokens": 8192
    },
    "unsloth": {
      "baseURL": "http://127.0.0.1:8888",
      "model": "unsloth",
      "apiKey": "sk-unsloth-xxxxxxxx",
      "capabilities": ["text", "vision"]
    }
  }
}

Quick single-backend alternative: MULTHANDS_BASE_URL=http://127.0.0.1:5001 (+ optional MULTHANDS_MODEL, MULTHANDS_API_KEY).

Usage

local_run — text

Returns { text, reasoning?, model, backend, usage, elapsed_ms }.

local_vision — OCR / analysis / comparison

Returns { text, reasoning?, model, backend, images, usage, elapsed_ms }. compare sends 2–4 images in one request for joint reasoning.

local_status — backends & health

No args. Returns each configured backend with reachable (live GET /v1/models probe) and a note (an AUTH note means the server is up but the key was rejected).

Connect a client

All clients share one premise: multhands on PATH (uv tool install .), config passed via environment.

{
  "mcp": {
    "multhands": {
      "type": "local",
      "command": ["multhands"],
      "enabled": true,
      "environment": { "MULTHANDS_CONFIG": "/path/to/multhands.json" }
    }
  }
}

{
  "mcpServers": {
    "multhands": {
      "command": "multhands",
      "env": { "MULTHANDS_CONFIG": "/path/to/multhands.json" }
    }
  }
}

{
  "mcpServers": {
    "multhands": {
      "command": "multhands",
      "env": { "MULTHANDS_CONFIG": "/path/to/multhands.json" }
    }
  }
}

{
  "mcpServers": {
    "multhands": {
      "command": "multhands",
      "env": { "MULTHANDS_CONFIG": "/path/to/multhands.json" }
    }
  }
}

{
  "mcpServers": {
    "multhands": {
      "command": "multhands",
      "env": { "MULTHANDS_CONFIG": "/path/to/multhands.json" }
    }
  }
}

multhands --transport streamable-http --host 0.0.0.0 --port 8020   # endpoint: http://<host>:8020/mcp
multhands --transport sse --host 0.0.0.0 --port 8021               # endpoint: http://<host>:8021/sse

Uninstall

Clean, zero residue:

1. Remove the multhands entry from your client config (other MCP entries are unaffected).
2. uv tool uninstall multhands-mcp (or npm uninstall -g mult-hands-eyes-mcp).
3. Delete the cloned folder.

The server writes nothing anywhere at runtime, so there is nothing else to clean up.

Documentation

The docs ship with the repository but are intentionally excluded from the pip/npm packages.

Roadmap

Directions we can go:

• More vision modes & templates (document layout, table extraction).
• Batch jobs: one agent turn driving many local calls.
• Optional auto-detection of common ports (currently deliberate: explicit config only).
• Publish the Python package to PyPI (uvx mult-hands-eyes-mcp).

Directions we will not go:

• Becoming an LLM provider adapter — the online model stays the main brain.
• Managing processes — the server remains a pure client; your services are yours.
• Streaming responses — one round trip per tool call is simpler and sufficient.
• Bundling model files (GGUF / mmproj) or modifying any local service.

Acknowledgments

• DeepSeek AI — the DeepSeek Harness platform and its plugin patterns shaped this project, via the reference plugins dsh-koboldcpp-hands and dsh-unsloth-hands (MIT).
• LostRuins / KoboldCpp and Unsloth — the local inference servers that make this possible.
• llama.cpp and the GGUF quantization ecosystem.
• Model Context Protocol — the protocol and its Python SDK.
• The open models running on your machine.

No affiliation with DeepSeek AI, LostRuins, or Unsloth AI; all trademarks belong to their owners.

License

MIT