← All guides · MCP in practice

Common MCP Server Failure Modes and How to Fix Them

Part 9 of MCP in practice · 1,751 words · published 2026-08-21 · updated 2026-08-21 · Markdown variant

A runtime playbook for the two MCP server failure modes with no dedicated deep-dive elsewhere: unrecoverable state after a mid-call crash, and malformed or hallucinated tool calls that reach the handler despite upstream validation.

Part of the MCP Server in Production: How to Build, Ship and Run One guide.

The pillar's failure modes section lists eight things that actually break a production MCP server, from rug pulls to spec-transition breakage, each in a sentence or two. Three of those eight already have a dedicated deep-dive elsewhere in this cluster: rug pulls and supply-chain compromise in tool description injection and tool poisoning, confused-deputy token passthrough and DNS rebinding in implementing OAuth 2.1 for an MCP server, and spec-transition breakage in MCP server versioning and spec migration. This article covers the two that have no sibling deep-dive: unrecoverable state on crash, and malformed or hallucinated tool calls that reach the handler. It does not repeat the pillar's one-line summaries of the other five.

What happens when a tool call crashes mid-write?

A tool call that crashes after it has already started a write — a database insert, a payment charge, a message send — leaves the system in an ambiguous state unless the handler was built to make a retry safe. The caller does not know whether the side effect completed before the crash, only that it did not receive a response, so the caller's only reasonable next move is to retry the same call. Whether that retry is safe depends entirely on whether the handler was written to tolerate it.

The mechanism that makes a retry safe is an idempotency key: a stable identifier attached to the call that the downstream service uses to recognize "I have already done this" and return the original result instead of repeating the side effect. The key detail is what the key is derived from. A key built from wall-clock time or a freshly generated random value is different on every retry attempt — which defeats the purpose, because the downstream service sees each retry as a new, distinct request. A key derived from the run ID and the step index within that run stays identical across retries of the same logical call, which is what durable execution treats as the standard pattern: "derive a stable idempotency key from the workflow run ID and step index — not from wall-clock time or a random UUID — and pass it to the downstream service so that a duplicate call is a no-op" (durable execution for long-running agents).

// Illustrative — the shape of a stable idempotency key, not tied to a
// specific MCP SDK. runId and stepIndex must be stable across retries of
// the same logical call; a fresh value on each attempt defeats the point.
function idempotencyKey(runId: string, stepIndex: number): string {
  return `${runId}:${stepIndex}`;
}

async function chargeCard(runId: string, stepIndex: number, amountCents: number) {
  const key = idempotencyKey(runId, stepIndex);
  // Passed to the payment API's own idempotency-key parameter, so a
  // duplicate call with the same key returns the original charge result
  // instead of creating a second charge.
  return paymentClient.charge({ amountCents, idempotencyKey: key });
}

Logging the step before acting on it

The same durable-execution discipline that produces a stable idempotency key also determines whether a crash loses work at all: "a durable execution engine records every meaningful step of a workflow — LLM call, tool invocation, timer, signal received — to a persistent log (or checkpoint store) before moving on. If the process crashes or is restarted, the engine replays the log to reconstruct in-memory state exactly where execution left off" (durable execution for long-running agents). Applied to a single MCP tool handler rather than a full multi-step workflow, the practical version is narrower but the principle is the same: persist that a call with a given idempotency key was started before attempting the side effect, so that on restart the handler can check whether that key already has a recorded outcome rather than blindly re-executing.

Not every MCP server needs a full durable-execution engine for this — a handler that writes its own idempotency-key table to the same database it is already calling, and checks it before executing, gets most of the benefit without adopting Temporal, Restate, or a similar engine. Reach for a dedicated durable-execution engine specifically when a tool call needs to pause for an external signal — a human approval, a webhook — for longer than a request-response cycle reasonably spans; that is a different problem from making a single crash-prone write retry-safe, and the corpus resource above surveys the engines built for it.

Transient failure or terminal failure — and why the distinction matters

A crashed or failed tool call should not always be retried, and the decision hinges on whether the failure was transient or terminal. Durable execution engines formalize this distinction explicitly: "transient failures (network timeout, rate limit, 5xx) are retried with backoff; terminal failures (4xx bad-request, business logic error) escalate immediately" (durable execution for long-running agents). An MCP handler that retries a terminal failure — a malformed request, a business rule rejection, an authorization denial — will fail identically on every retry, burning latency and, if the retry also re-attempts a side effect without an idempotency key behind it, risking a duplicate. An MCP handler that fails fast on a transient error — a single dropped connection, a momentary rate limit — pushes unnecessary error handling onto the calling agent for a condition a short backoff-and-retry would have resolved on its own.

• Transient: network timeout connecting to a downstream API, a 429 rate-limit response, a 5xx from a dependency. Retry with backoff, bounded by a maximum attempt count — unbounded retries mask real bugs behind an eventually-successful call.
• Terminal: a 4xx from a downstream API, a business-logic rejection (insufficient balance, invalid state transition), a validation failure on the handler's own side. Fail immediately and return a specific error the caller can act on; retrying an unchanged terminal failure produces an unchanged terminal failure.

Why does a well-formed schema still let bad tool calls through?

A declared inputSchema constrains what a well-behaved client validates before sending a call — it does not constrain what an unreliable model actually emits. The reliability literature for tool calling generally documents both a hallucinated-tool-name failure and a set of argument-shape failures as distinct, common categories, independent of how carefully a server has declared its schema: "hallucinated tool name — model calls a tool not in the declared set", "missing required arguments — model omits a field the schema marks required", "extra / unexpected arguments — model adds fields not in schema", "malformed JSON — output is not parseable", and "wrong types — string where int expected" (reliable tool calling and structured outputs). None of those originate on the server side. They originate in the calling model's own output, before your server ever sees the request — which is exactly why a schema declaration alone cannot prevent them from arriving.

This is a different problem from the one testing an MCP server in CI solves. That article covers writing a Layer 1 unit test that feeds your own input validator malformed argument objects and asserts it rejects each one — a pre-production check that your validation logic works correctly at all. This section covers what the handler itself does at runtime, in production, when a call that your validator was never exercised against — or a call your validator has a gap for — actually arrives. Testing proves the validator works against cases you thought of; runtime defense is what happens against the case you did not.

The mitigation table, applied at the handler

The reliability corpus's mitigation table names concrete fixes for each failure category — check any incoming call name against your declared tool set and refuse whatever isn't on it; use additionalProperties: false and a required array with schema validation to catch missing or extra arguments pre-execution; wrap JSON parsing in a try/catch and treat a parse failure as a rejection, not a crash; validate types with a schema library before consuming any argument value (reliable tool calling and structured outputs). Applied specifically to an MCP handler, in order of execution:

// Illustrative handler-side defense — runs before any side effect,
// regardless of what client-side validation the caller may or may not
// have performed.
function handleToolCall(req: { name: string; arguments: unknown }) {
  if (!DECLARED_TOOLS.has(req.name)) {
    return rejectWith(`unknown tool: ${req.name}`); // hallucinated tool name
  }
  const parsed = validateAgainstSchema(req.name, req.arguments); // missing/extra/wrong-type
  if (!parsed.ok) {
    return rejectWith(parsed.errors.join('; '));
  }
  return executeHandler(req.name, parsed.value); // only now, a side effect
}

The ordering matters as much as the checks themselves: tool-name validation, then argument-schema validation, then execution — never the reverse, and never a side effect before both checks pass. A rejection at either of the first two steps costs one round trip; a side effect executed on an unvalidated argument object can cost a duplicate charge, a corrupted write, or a call to a tool the caller never actually intended to invoke.

Why zero-tolerance defense is the right default, not paranoia

Berkeley's Function Calling Leaderboard (BFCL), the reference benchmark for tool-calling reliability across providers, exists specifically because "typical breakdowns range from invented tool names and skipped or extraneous arguments to unparsable JSON, mistyped fields, needless or missing invocations, out-of-order parallel calls, and templates that don't match how the model was trained" (reliable tool calling and structured outputs). As of August 2026, no provider's constrained-decoding mechanism eliminates every one of these categories for every model and every client integration — strict mode, tool_choice forcing, and grammar-based decoding each reduce specific failure categories, but the benchmark's continued existence is itself evidence that the underlying rate is nonzero across the field. An MCP handler sits downstream of whichever model and client happen to be calling it, often without control over either, so treating a hallucinated or malformed call as an expected input condition — not a rare edge case — is the realistic baseline to build against, not defensive overengineering.

Where this leaves you

Give every side-effecting MCP tool handler a stable idempotency key derived from run ID and step index, distinguish transient failures worth retrying from terminal ones worth failing fast on, and re-check whatever call name and arguments actually arrive against your own declared schema before any side effect runs — regardless of what the client claims to have already checked. For the full eight-item failure-mode list this article extends, see MCP server in production; for how to catch a validation gap before it reaches production, see how to test an MCP server in CI; for the durable-execution engines that formalize crash recovery beyond a single handler, see durable execution for long-running agents.

#mcp #reliability #idempotency #tool-calling #agents #production