Skip to main content
Overmind ingests OpenTelemetry traces, scores them as they arrive and turns them into datasets. Those datasets drive evals, optimisation runs and training runs, and every result is benchmarked against the agent you ship today. Use the hosted Console or self-host the same code with Docker Compose. The platform is AGPL-3.0 and the SDK is MIT; see Licensing. Everything speaks standard protocols: OTLP/HTTP for ingest, REST JSON for the API, MCP for coding agents and OpenAI-compatible serving for models. The Python SDK instruments OpenAI, Anthropic, Gemini, Agno and LangChain, and any OpenTelemetry exporter can send to the same endpoint. What Overmind does:
  • Capability scan: your coding agent maps the repository into capabilities
  • Live scoring: every trace is carved into task executions and judged against the capability’s eval set
  • Datasets: traces and files become eval and training data, shaped by the dataset’s own agent
  • Eval: variants measured against a baseline with typed scores
  • Optimiser: candidate code changes replayed in your repository and scored; the winning diff is yours to apply
  • Training: fine-tuned models you own, trained on Modal from your data
  • Inference: an OpenAI-compatible API for your trained models and frontier models
  • MCP and CLI: the same workflows from Cursor, Claude Code, OpenCode or Codex

​
Guides

Quickstart

One pasted prompt: SDK, MCP, capability scan, first trace.

Self-hosting

Docker Compose, the environment reference, backends and billing modes.

Observability

OTLP ingest, span model, attribute mapping, the executions view.

Python SDK

init(), the run boundary, decorators, tasks, runtime expectations.

Agent & Capabilities

Local discovery, overmind sync, tasks, telemetry attribution.

Datasets

Source, cells, versions, contracts, the workshop agent.

Eval

Evaluator kinds, claims, task bindings, eval runs.

Trace scoring

Units, tasks, verdicts, execution and session scores.

Optimisers

The client-driven loop, harness and backtesting modes, the winning diff.

MCP

Setup per coding agent, the tool catalog, prompts, resources.

​
Models & API

Models

Runs, the model catalog, loss curves, the baseline benchmark.

Inference

Serving lifecycle, the capability alias, /api/v1/chat/completions.

REST API

Auth, endpoint map, conventions, Swagger.

CLI

overmind init, sync, dataset, optimise, model.