Armature Adds Analytics for MCP Sessions

Armature is a YC P26 project from Theodore and Louis that adds product analytics and evals for agent sessions on your MCP server. It deals with a new debugging problem: MCP tool calls arrive without the full user-agent story that caused them. The useful takeaway is simple: session analytics can make agent behavior reviewable, but the first integration should stay narrow, read-only, and easy to turn off. Product analytics for MCP agent sessions is instrumentation that reconstructs what an agent was trying to do around the tool calls your server receives.

That matters to Cursor, Anysphere's AI code editor, because more coding work now flows through agents, rules, repo context, and external tools. If you want to implement code review habits for ai-generated code, traces like these can show whether the agent used the right context before it opened a pull request. For readers following agentic coding governance or ai coding training for teams, the interesting part is not the dashboard. It is the move from reviewing only the final diff to reviewing the path that produced it.

See the session behind the MCP call

Armature's pitch is that an MCP server should not have to guess why a tool was called. As of August 2026, the Show HN description says you wrap an MCP server with a small SDK in TypeScript, Python, or Go, then see reconstructed sessions, clustered use cases, and frequent issues in a dashboard.

MCP, the Model Context Protocol, is a standard way for AI apps and agents to connect to external tools, data, and services. A coding agent might use MCP to read a GitHub issue, query docs, search a design system, or call an internal deployment tool.

The daily pain is familiar. Your MCP server sees search_docs(query='auth middleware timeout'), but not always the larger story: the user asked Claude, Anthropic's assistant, or ChatGPT, OpenAI's assistant, to fix a flaky login test; the agent tried three searches; then it called your tool with a vague query. Armature is trying to put that story back together.

The trap is to treat reconstruction as mind reading. A Hacker News objection asked how the product gets the model's thoughts. That is the right question. If a host does not expose private reasoning, an analytics layer cannot magically recover it; it can only reconstruct from available messages, tool calls, metadata, and integration points. Read any claim about agent thoughts as a product claim to verify, not as a law of nature.

Use analytics to find broken tool edges

The first practical win is not surveillance. It is finding the jagged edges where agents repeatedly fail.

Imagine a repo with a Cursor rule that tells the agent to use an internal docs MCP before editing authentication code. In the final diff, you only see a changed middleware file and a new test. In an MCP session view, you may see that the agent searched the wrong namespace five times, skipped the runbook, and finally guessed.

That is a product signal. Your docs search tool may need better examples, a safer default collection, or a clearer error when the query is too broad. Armature's session clustering idea is useful here because individual traces are noisy, but repeated clusters show where agents actually spend time.

The trap is drowning in logs. Raw request logs already exist in many systems, and they are cheaper. The reason to try a session product is when tool calls are no longer enough: you need the user request, the agent's route through tools, and the failure pattern in one reviewable object.

Keep the first MCP integration boring

Wire analytics around the least dangerous MCP server first. A docs search server, issue lookup server, or read-only repo metadata server is a better starting point than a deployment, billing, database write, or secrets tool.

For Cursor users, pair the integration with a tiny repo boundary. Put the agent's rules near the code it affects, then make the expected review behavior explicit.

---
description: Review AI-generated auth changes
---
When an agent edits auth, session, or permission code:
- Use the internal docs MCP before changing behavior.
- Include the MCP session link or trace ID in the PR notes when available.
- Keep generated changes small enough for human review.
- Do not approve changes that alter permission checks without a test and a human explanation.

That little rule is not a policy empire. It is a handrail. It helps you connect Cursor's reviewable IDE workflow to the session data Armature wants to expose.

The permission risk is the part to take seriously. Before production traffic leaves the server, ask whether redaction runs before upload, whether it is configurable, whether collection can be disabled, and what fields leave your environment. The Show HN thread raised those questions directly, and they are not nitpicks.

Compare Armature with logs and eval traces

Armature sits between raw observability and test-style evals. That is the useful comparison, because most teams already have one of those two habits.

Verdict: Armature wins when real sessions are the missing artifact. Raw logs win when the problem is operational and narrow. CI eval traces win when you know the behavior you want and need a repeatable gate before merging.

There is a nice parallel with tiny local agents too. In MicroCodex Reimplements Codex in C++, the interesting lesson is how much you can learn by shrinking the surface area. MCP analytics asks the same kind of question from the other side: once agents get bigger, what is the smallest trace that still explains the work?

Try it safely with one review boundary

Use the integration as a review aid before you use it as a scorecard. The goal is to make one risky path easier to inspect, not to rank every developer, agent, or prompt.

A small experiment can look like this:

Permission-boundary note: keep the first MCP server read-only, redact before export, and make collection opt-out or off-by-config for sensitive repos. If you cannot explain what data leaves the machine in one paragraph, the integration is not ready for production use.

A Cursor review checklist can stay short:

• Does the PR say which agent produced the change and which MCP tools it used?
• Did the agent consult the expected docs or issue context before editing?
• Is the generated diff small enough to review without trusting the agent?
• Are permission, auth, billing, or migration changes covered by tests?
• Is there a trace ID, session link, or clear note saying no trace was collected?

This is the practical place to implement code review habits for ai-generated code. You are not asking reviewers to become prompt archaeologists. You are giving them one extra artifact when the final diff is not enough.

Common questions

• What are the best ways to implement code review habits for ai-generated code?

  Start with small diffs, explicit repo rules, and one traceable artifact for risky changes. For MCP-backed agents, that artifact can be a session ID or link showing which tools the agent used before editing code. Keep the habit narrow at first: auth, permissions, payments, migrations, and generated tests deserve the extra review.

• Does Armature really take three lines of code to add?

  Maybe for the wrapper, not for the whole safe integration. The Show HN pitch says the SDK wraps an MCP server in a few lines across TypeScript, Python, and Go. Real production use still needs redaction decisions, environment config, access control, and a documented disable path.

• Can it actually see what the model thought?

  Only if the surrounding system exposes something that can be captured. An MCP analytics product can reconstruct messages, tool calls, errors, and available reasoning-like artifacts, but it cannot recover hidden chain-of-thought that a host never sends. Treat the trace as evidence of behavior, not a perfect transcript of cognition.

• Is this part of engineering team ai adoption or just observability?

  It is observability first, with adoption lessons as a side effect. The strongest use is seeing where agents fail against your tools and docs. Once those patterns are visible, you can improve Cursor rules, MCP descriptions, eval cases, and review checklists without turning the dashboard into a management scoreboard.

Best ways to use this research

• Best for: engineers running MCP servers that coding agents call often enough for raw logs to feel incomplete.
• Best first artifact: a read-only MCP integration plus a Cursor .mdc rule that asks for a trace ID on sensitive PRs.
• Best comparison angle: compare Armature against raw logs and CI eval traces; they answer different questions and work better together than as replacements.
• Best caution: verify redaction, collection controls, and hosted data boundaries before sending private repo or customer context.

Further reading

• Product analytics (and evals) for agent sessions on your MCP — source
• Model Context Protocol — specification
• Cursor — Agent
• OpenTelemetry documentation

Next step

Pick one read-only MCP server and add a review boundary before you add a dashboard habit. If the first trace helps a reviewer understand a real AI-generated diff faster, the integration has earned its next experiment.

One methodology lens

One useful way to read this through our methodology is the Plan step: delegate first-pass decomposition and dependency mapping, review the sequencing and assumptions, and keep ownership of scope and priorities. If that split is still fuzzy, the workflow usually is too.