Do I mock the LLM or the transport when unit testing an MCP server?

The transport. Generic agent Layer 1 testing mocks the LLM client because the code under test calls a model. An MCP server, in most cases, does not call a model at all — it responds to calls from a client that has one. What you replace at Layer 1 is the JSON-RPC channel: either call your registered tool handler directly with a constructed arguments object, bypassing the wire format entirely, or connect your server to an in-process substitute for stdio/Streamable HTTP so the real request-handling code runs without a subprocess or a socket.

Can I use VCR.py or pytest-recording to test an MCP server?

Yes, but confirm the transport first. Both tools intercept HTTP at the library level and serialize the exchange to a cassette file, replaying it on later runs. A Streamable HTTP MCP server's messages are ordinary HTTP POST requests to a single endpoint, so an HTTP-level interceptor can record and replay a full session — initialize, discovery, one or more tool calls — the same way it would any other HTTP-based API. A stdio server exchanges JSON-RPC over stdin/stdout with no HTTP layer for these tools to intercept, so nothing in the corpus confirms they work there; apply the same discipline manually (serialize the request/response pairs yourself, replay them in a test, scrub anything sensitive) rather than assuming a drop-in library exists for that transport.

How often should Layer 3 live smoke tests run against real clients?

Nightly or pre-release, never on every commit. Live tests against real target clients are expensive and inherently flaky, so gating a pull request on them turns an unrelated intermittent failure into a blocked merge. Run them on their own CI job with a schedule trigger, keep the test set small and hand-curated, tag the job with an expected cost, and alert if actual spend drifts materially from that baseline — a small suite that starts calling more tools, or a client that got slower, should surface as a cost anomaly before it surfaces as a surprise bill.

← All guides · MCP in practice

How to Test an MCP Server in CI

Part 4 of MCP in practice · 1,337 words · published 2026-08-09 · updated 2026-08-09 · Markdown variant

The implementation mechanics below the three-layer test pyramid: what a mocked MCP transport actually replaces, what a Streamable HTTP cassette contains, a concrete CI job/trigger shape, and how to catch spec-version drift before it reaches production.

In short

An MCP server's Layer 1 unit tests mock the transport, not the model — most servers don't call an LLM at all, so the seam you replace is the JSON-RPC channel your handlers sit behind, not an LLM client the way generic agent testing does.
Cassette-style record/replay is confirmed to work for a Streamable HTTP MCP server, because every message on that transport is literally an HTTP POST to one endpoint; there is no confirmed equivalent for stdio's raw stdin/stdout exchange, so treat "record once, replay in CI, scrub credentials" as a pattern to adapt for stdio, not a library that already does it.
Layer 3 live smoke tests belong on a separate CI job definition, not just a separate test file — a nightly or pre-release schedule with its own cost budget, distinct from the job that gates every push and pull request.
An MCP SDK version bump is itself a legitimate reason to re-run the full pyramid, deliberately: a Layer 1 or Layer 2 test that asserts the negotiated protocolVersion explicitly is what turns a breaking version change into a caught test failure instead of a silent production break.

Part of the MCP Server in Production: How to Build, Ship and Run One guide.

Everything you need to know about what to test on an MCP server — the three-layer pyramid, MCP Inspector as a manual first check, snapshot-testing tool-call trajectories, and BFCL as the underlying cross-provider reliability ceiling — is already covered in MCP server in production. This article does not redefine those layers. It answers the question that comes right after: once you've decided to build a unit layer, a cassette layer, and a live layer, what does each one actually look like in code and in a CI config for an MCP server specifically, as opposed to an LLM-calling agent generally?

What a mocked transport actually replaces

Layer 1 for an MCP server means mocking or stubbing the transport, and the transport is the part worth being precise about, because it is not the same seam generic agent testing mocks. Testing an agent that calls a model means mocking the LLM client — you replace the thing that talks to the API and returns canned completions (testing AI agents in CI). Testing an MCP server means something different: in most servers, nothing in the code under test calls a model at all. The server's job is to receive a JSON-RPC request over stdio or Streamable HTTP, run a handler, and return a result — the model lives on the client's side of the connection, not the server's.

Two things are worth testing separately at this layer, and neither requires a live JSON-RPC round trip:

The input validator, on its own. Every tool declares an inputSchema (building an MCP server); feed the validator malformed argument objects — missing required fields, wrong types, extra properties if your schema forbids them — and assert it rejects each one with the specific violation you expect, before any handler runs. This is the layer that turns a tools/call with bad arguments into a clean, typed error instead of a handler crash.
The handler, called directly. Call your registered tool function as a plain function with a constructed arguments object, skipping the wire format entirely, and assert on the returned content array — its shape, its type field, the actual values inside it. This is the fastest and most direct test you can write against an MCP server, and it catches the majority of real regressions because most bugs live in handler logic, not in the protocol layer around it.

If you want to exercise the actual JSON-RPC handling code rather than just the handler function underneath it, connect your server object to an in-process substitute for the transport instead of standing up stdio or a network listener — the same request/response cycle a real client would trigger, without a subprocess or a socket. Either approach keeps Layer 1 free of network calls, which is the property that makes it safe to run on every commit.

What's actually inside an MCP cassette

A cassette for an MCP exchange is a serialized sequence of JSON-RPC request/response pairs — initialize, tools/list, one or more tools/call exchanges, matched on method name and parameters — replayed instead of a live session on every run after the first recording. This is the same pattern general agent testing uses for LLM API calls (testing AI agents in CI), applied to the wire your server actually speaks.

Where the pattern is directly confirmed to work: Streamable HTTP. Every message on that transport is an HTTP POST to a single endpoint, and record/replay tools like VCR.py and pytest-recording intercept HTTP at the library level — so a Streamable HTTP MCP session records and replays the same way any other HTTP-based API exchange would, with the same credential scrubbing (filter_headers, filter_post_data_parameters) applying before you commit the cassette to version control.

Where it is not confirmed: stdio. That transport exchanges newline-delimited JSON-RPC over stdin/stdout with no HTTP layer for these tools to intercept, and nothing in the corpus states that VCR.py-style libraries record a child-process exchange. If your server runs over stdio, apply the same discipline by hand — serialize the request/response pairs your test client sends and receives, replay them from a fixture file, scrub anything sensitive — rather than reaching for an HTTP-recording library and assuming it works on a transport it was not built for.

Re-record a cassette on two triggers, not one: when a tool's schema or description changes (the pillar's baseline rule), and when you deliberately bump your MCP SDK version. A stale cassette that silently stops matching a newer SDK's request shape is a test that passes for the wrong reason — in CI, pass --record-mode=none (pytest-recording's actual flag) so a missing or mismatched cassette fails the test outright instead of falling back to a live call.

A concrete CI job shape

Map the three layers onto CI triggers, not just test files, so a flaky live test can never block a merge. A workable split:

# .github/workflows/mcp-server.yml
on:
  push:
  pull_request:
  schedule:
    - cron: '0 3 * * *'   # nightly
  workflow_dispatch:        # pre-release, manual trigger

jobs:
  unit:                     # Layers 1 + 2 — every push and PR
    if: github.event_name == 'push' || github.event_name == 'pull_request'
    steps:
      - run: pytest tests/unit tests/cassette --record-mode=none

  smoke:                    # Layer 3 — nightly or pre-release only
    if: github.event_name == 'schedule' || github.event_name == 'workflow_dispatch'
    steps:
      - run: pytest tests/smoke --maxfail=1

The unit job runs deterministic and cassette-backed tests on every commit — fast, free, no network — and is the job that gates a pull request. The smoke job runs only on a schedule or a manual pre-release trigger, exercises real target clients, and should never gate a merge on its own; treat its failures as signals to investigate, tag it with an expected token/cost budget, and alert if actual spend drifts materially, since a handful of hand-curated live calls run nightly should have a predictable cost. If a smoke test starts failing intermittently for reasons you haven't root-caused, quarantine it out of the gating path rather than retrying it — a retry that eventually passes hides the failure instead of explaining it.

Catching spec drift with a test, not a production incident

The pillar's versioning discipline — pin your SDK, check protocolVersion explicitly, treat any given spec snapshot as something to reverify rather than something settled — tells you what practice to follow (MCP server in production). What it doesn't spell out is how that practice becomes a test that runs in CI rather than a habit someone forgets under deadline pressure. Two concrete tests do most of the work:

Assert the negotiated protocolVersion explicitly, in both Layer 1 and Layer 2. During initialize, the server responds with the protocolVersion it negotiated, and that value governs the rest of the session (building an MCP server). A Layer 1 test against your in-process transport substitute, and a Layer 2 test replaying a cassette, should each check the returned value matches what you expect — not just that initialize didn't throw. A mismatch that would otherwise negotiate down and "succeed" silently becomes a visible test failure instead.
Make an SDK version bump its own CI event, not a side effect of an unrelated dependency update. Re-run the full pyramid — not just the unit job — whenever you deliberately bump the MCP SDK version, the same discipline Layer 2 already applies to a cassette when a tool's schema or description changes (testing AI agents in CI). A validator or handler that behaved one way against an older SDK build can behave differently against a newer one without any of your own code changing, and testing the bump on purpose is what catches that before a client does.

None of this replaces the client-facing reliability question. Whether the model calling your server gets the arguments right in the first place is a different, provider-level problem, and BFCL remains the reference benchmark for it across vendors (reliable tool calling) — your test suite controls what your server does with a call it receives, not whether the model on the other end constructs that call correctly.

Frequently asked questions

Do I mock the LLM or the transport when unit testing an MCP server?: The transport. Generic agent Layer 1 testing mocks the LLM client because the code under test calls a model. An MCP server, in most cases, does not call a model at all — it responds to calls from a client that has one. What you replace at Layer 1 is the JSON-RPC channel: either call your registered tool handler directly with a constructed arguments object, bypassing the wire format entirely, or connect your server to an in-process substitute for stdio/Streamable HTTP so the real request-handling code runs without a subprocess or a socket.
Can I use VCR.py or pytest-recording to test an MCP server?: Yes, but confirm the transport first. Both tools intercept HTTP at the library level and serialize the exchange to a cassette file, replaying it on later runs. A Streamable HTTP MCP server's messages are ordinary HTTP POST requests to a single endpoint, so an HTTP-level interceptor can record and replay a full session — initialize, discovery, one or more tool calls — the same way it would any other HTTP-based API. A stdio server exchanges JSON-RPC over stdin/stdout with no HTTP layer for these tools to intercept, so nothing in the corpus confirms they work there; apply the same discipline manually (serialize the request/response pairs yourself, replay them in a test, scrub anything sensitive) rather than assuming a drop-in library exists for that transport.
How often should Layer 3 live smoke tests run against real clients?: Nightly or pre-release, never on every commit. Live tests against real target clients are expensive and inherently flaky, so gating a pull request on them turns an unrelated intermittent failure into a blocked merge. Run them on their own CI job with a schedule trigger, keep the test set small and hand-curated, tag the job with an expected cost, and alert if actual spend drifts materially from that baseline — a small suite that starts calling more tools, or a client that got slower, should surface as a cost anomaly before it surfaces as a surprise bill.

#mcp #testing #ci #agents #tool-calling

Agents: this guide is available as Markdown and JSON; the whole cluster is indexed at /api/articles.json. The reference corpus behind it is at /llms.txt, with licensing at pricing.