Remote MCP Server Reliability (2026)

1. Why this study

The Model Context Protocol is how AI agents discover and call tools hosted by other people. A remote MCP server is an HTTP endpoint: an agent connects, learns which protocol revision it speaks, asks for the list of tools, and calls them. When something goes wrong, the server is supposed to say so in a structured way the agent can reason about.

In July 2026 the protocol changed substantially. Revision `2026-07-28` made MCP stateless: the initialize/initialized handshake and the Mcp-Session-Id header were retired, every request now carries its own protocol version, client identity and capabilities in _meta, and a new server/discover method lets clients learn what a server supports. Servers built for earlier revisions still use the old handshake. An agent meeting an unknown endpoint therefore has to work out which era it is talking to before it can do anything.

Registries such as the official MCP Registry list tens of thousands of remote endpoints, but listings are self-submitted and unvetted. Before this study there was no systematic, reproducible measurement of how many listed endpoints actually work, how many speak the new revision, whether auth-gated servers tell agents how to authenticate, and whether errors come back in a form an agent can use.

Research questions (descriptive, no directional hypotheses)

2. Population and sampling

Source. The official MCP Registry API, GET https://registry.modelcontextprotocol.io/v0/servers?version=latest, fully paginated on 2026-10-12 between 02:34 and 02:48 UTC. All 422 raw response pages are archived in the repository so the population can be rebuilt exactly.

Unit. One remote endpoint (server.remotes[].url) of a server whose registry metadata says status: "active" and isLatest: true. Endpoints were de-duplicated by normalised URL (lower-case scheme and host; no query, fragment or trailing slash) but probed at their registered URL exactly as written, because some servers only answer at a trailing-slash path.

Sample. The pre-registered rule was: probe the whole frame if it has ≤ 5,000 endpoints, otherwise draw a simple random sample of 3,000 with random.Random(20261012). The frame was far larger, so 3,000 endpoints were drawn. They sit on 2,245 distinct hosts; the largest single host accounts for 261 of them.

Host clustering

A 25-endpoint pilot (run before the pre-registration was frozen and excluded from all results) showed that a handful of hosts publish hundreds of registry entries — one host had 276 of the first ~12,000. A few bulk publishers could therefore dominate endpoint-level percentages. Every headline share is reported twice: per endpoint (the primary result) and host-weighted, where each host counts once (the mean of within-host shares).

3. How the probe works

The harness (harness/probe.py) is a dual-era client: it follows the client-side backward-compatibility procedure written into MCP 2026-07-28 (Streamable HTTP § Backward Compatibility; Versioning § Backward Compatibility with Initialization-Based Versions). It tries the modern protocol first, falls back to the legacy handshake, and finally checks for the deprecated HTTP+SSE transport.

Responses are parsed robustly: plain JSON and Server-Sent Events (data: lines) are both handled, SSE streams are read only until the matching JSON-RPC response arrives, and bodies are capped at 512 KB. Only a 2 KB snippet of each response is kept in the private raw logs; the published data contains derived fields only.

Tested before use. The probe and classifier were tested against 12 local fixture servers that imitate every case the study distinguishes — a stateless 2026-07-28 server, a stateful legacy server with sessions and SSE replies, a legacy server that rejects the newest version, a legacy server answering unknown methods with HTTP 404, a server refusing 2026-07-28 with UnsupportedProtocolVersion, an SSE-only server, compliant and non-compliant auth, an HTML page, a 502, a server that never answers, and a paid (HTTP 402) endpoint. All 21 checks pass.

Vantage point. All probing ran from a single residential connection (AS36873, Airtel Networks, Nigeria). Latency and reachability figures reflect that location.

4. How outcomes were classified

Each endpoint receives exactly one primary status, checked in this order (harness/classify.py):

Secondary fields recorded per endpoint: protocol era and version, supportedVersions, dual-era status (a modern server that also lists legacy versions), capabilities, whether a session was issued, handshake latency, tool count and pagination (only the first page of tools/list is analysed), the share of tools with a strict schema (an object inputSchema whose every property declares type/enum/const/anyOf/oneOf/allOf/$ref), the share declaring readOnlyHint or destructiveHint as booleans, the share with descriptions, whether the canary error came back in-channel and its HTTP status, and for auth-gated endpoints whether the challenge is Bearer and the metadata valid.

Statistics. Proportions with Wilson 95% confidence intervals; RQ1 also broken down by registry transport type; re-test agreement as percentage agreement and Cohen's κ. Every number on this page is produced by harness/compute_stats.py from the published data.

5. Results — does the endpoint answer as an MCP server? (RQ1)

Full size

By transport listed in the registry

Endpoints still registered with the deprecated SSE transport are overwhelmingly paid, unreachable or legacy: only 10 of 132 work the modern way.

6. Results — which protocol revision do they speak? (RQ2)

1,772 endpoints completed a handshake. The probe always tries server/discover first, so a server is counted as modern only if it actually implements the 2026-07-28 revision.

Full size

• Adoption is early. Eleven weeks after the revision shipped, about one in seven connecting servers speak it. A quarter of those (25.1%, 59 of 235) are dual-era: they also advertise legacy versions, so old and new clients can both connect.
• Legacy sessions are uncommon. Only 17.1% of legacy servers issued an Mcp-Session-Id; most legacy deployments are effectively stateless already.
• Non-conformant replies. 14 endpoints, all on one hosting platform, answer server/discover without the required supportedVersions field and claim 2026-07-28 in reply to initialize — a method that revision removed. Under the pre-registered rule they count as legacy-era.
• Latency. The median handshake took 418 ms from the residential vantage point in Nigeria.

7. Results — can an agent follow the auth challenge? (RQ3)

When a server needs credentials, the MCP authorisation spec expects it to answer 401 with a WWW-Authenticate: Bearer header that points to RFC 9728 protected-resource metadata, which in turn names the authorisation server. An agent that finds both can start a standard OAuth flow without a human reading documentation.

Auth only at call time. 53 auth-gated endpoints (37 compliant, 16 non-compliant) completed the handshake and listed their tools anonymously, but demanded credentials only when a tool was called. Under the frozen rule order (auth is checked before ok) they are counted as auth-gated.

8. Results — is tool metadata usable? (RQ4)

Agents choose and call tools from the metadata a server publishes: a name, a description, an input schema and, since 2025, optional hints that say whether a tool only reads data (readOnlyHint) or may destroy it (destructiveHint).

Full size

Where tools are listed, the basic metadata is excellent — typed inputs and descriptions are nearly universal, which suggests SDKs generate them by default. Safety hints are the gap: around half of servers never say which tools are safe to call freely and which can destroy data, so an agent cannot tell a read from a delete without guessing from the name.

9. Results — are errors reported in-channel? (RQ5)

To see how errors come back, the probe called a tool that cannot exist (__glintbase_probe_nonexistent_tool__). A well-behaved server answers with a JSON-RPC error object or a result marked isError: true, so the agent knows the call failed and why.

Of the 1,773 endpoints that answered this call over HTTP, 75.1% (73.0–77.0) reported the error in-channel. The other 442 did not:

Full size

10. Results — re-test reliability (RQ6)

Round 3 re-probed all 3,000 endpoints immediately after round 2, with the same harness.

Almost every change between rounds — 183 of 199 — came from one gateway host with 194 sampled endpoints, which moved from ok to auth_compliant. The reason was us: the gateway's free anonymous tier allows 500 discovery requests and 50 tool calls per day, and our own repeated probing that day (including rounds later discarded) used up the quota. In round 3 the gateway correctly answered with a compliant auth challenge; in round 2, 9 of its endpoints were already partly affected.

11. How the study actually ran

This is the complete record of what was done, in order, including the problems. All times are UTC on 2026-10-12.

1. 02:34–02:48
   Registry snapshot

   422 pages of the official MCP Registry (version=latest) downloaded and archived.

2. ≈02:41
   25-endpoint pilot

   Harness tested on real servers. Two changes followed: a separate payment_required category for HTTP 402 responses, and the host-weighted view after finding one host with 276 registry entries. Pilot data is excluded from all results.

3. 03:17
   Frame and sample built

   26,365 unique endpoints; seeded random sample of 3,000.

4. ≈08:25
   Pre-registration frozen, bot page live

   Protocol committed (6112785) before any real data; https://glintbase.dev/research/bot published so operators could see who was probing and opt out.

5. 08:26–08:48
   Round 1 (later discarded)

   Full sample probed. Re-test interval shortened to back-to-back for a one-day schedule and logged as Deviation 1 before round 2 began.

6. ≈08:50
   Bug found in our own era-detection rule

   273 endpoints (262 on one host) were labelled modern_rejected without ever being tried with the legacy handshake. The rule — written into the pre-registration — misread 404 + -32601 on server/discover as a modern server. Round 2 was stopped after 449 endpoints.

7. 16:42
   Deviation 2 committed; both rounds restarted

   Rule corrected to follow the spec, two new fixture tests added, faulty data archived as discarded (41665a0).

8. 16:42–17:24
   Corrected round 1 contaminated

   It overlapped another study's crawl on the same home connection; unreachable doubled from 140 to 271. Discarded and logged as Deviation 3.

9. 17:24–17:53
   Round 2 — the headline round

   Run with the connection otherwise idle. unreachable = 142, matching the isolated morning baseline of 140.

10. 17:53–18:16
    Round 3 — re-test

    Same sample, back to back. Validation (30 endpoints, one sequential connection) ran alongside.

11. after 18:16
    Analysis, quota finding, report

    Statistics computed; the gateway quota exhaustion identified and reported; host names hashed in all summaries.

12. Validation

Before any number could be published, the pre-registration required ≥ 90% agreement on a stratified draw of 30 endpoints (seed 20261012), re-checked independently. A separate minimal client (harness/validate.py, written from the spec and sharing no request code with the probe) re-checked each endpoint live, and every disagreement was then reviewed against the raw logs.

13. Pre-registration deviations

The protocol was frozen in commit 6112785 before data collection. Three deviations were made afterwards, each logged and committed before the data it affects was analysed.

Deviation 1 — back-to-back re-test

• Change: the re-test interval was shortened from ≥ 6 hours to immediately after the headline round.
• Reason: the study timeline was compressed to one day.
• Consequence: RQ6 is reported as re-test reliability (probe repeatability and very short-term flakiness), not as same-day server stability.

Deviation 2 — era-detection rule corrected, both rounds re-run

• What was wrong: the frozen procedure treated 404 + JSON-RPC -32601 on server/discover as a sign of a modern server. MCP 2026-07-28 requires modern servers to implement server/discover, so that reply actually identifies a non-modern server. The rule also stopped on -32022 (UnsupportedProtocolVersion), where the spec tells a dual-era client to retry with a supported version.
• Fix: only a DiscoverResult or a -32020/-32021 error marks a server as modern-only; 404/-32601 and -32022 fall back to initialize; fixture tests added for both.
• Impact: 271 endpoints (262 on one host) returned 404/-32601 and 2 returned -32022; all had been labelled modern_rejected.
• Action: round 1 and the partial round 2 (449 endpoints) were archived as discarded and both rounds re-run in full. No discarded result was analysed.

Deviation 3 — contaminated round discarded

• What happened: the corrected round 1 re-run overlapped another study's crawl on the same residential connection. Its unreachable count was 271 against 140 in the isolated morning run; 151 endpoints that were reachable in the morning failed.
• Action: the re-run was discarded; round 2 (connection otherwise idle) became the headline round and a round 3 in isolation the re-test. From then on no other study's traffic ran during MCP rounds.
• Scope: only the unreachable count was compared, as a contamination check, before the decision.

14. Limitations

• One registry. Listings are self-submitted and unvetted; Smithery and Glama are not included.
• One vantage point, back-to-back rounds. All traffic came from one residential connection in Nigeria; stability over hours, days or weeks is not measured.
• First page only. Only the first page of tools/list is analysed.
• Deprecated HTTP+SSE servers are detected but not handshaken.
• Some "failures" are deliberate — paid endpoints (HTTP 402) and servers that expect configuration headers declared in their registry entry.
• Measurement can change the measured. A gateway with a daily anonymous quota began requiring auth after our repeated rounds; bulk-hosted endpoints that share a quota are sensitive to repeated probing.
• The protocol revision is recent. 2026-07-28 was published about 11 weeks before data collection, so adoption figures are an early snapshot.
• Self-check validation. The validation reviewer also wrote the harness.

15. Ethics and what went wrong

The study sent metadata and protocol requests only, identified itself honestly with a public bot page and opt-out, probed one endpoint per host at a time and kept volumes small (at most 8 requests per endpoint per round). Raw responses are not published, and the report names no servers; host names in the published summaries are replaced by hashes.

16. Reproducing the study

Everything needed to rebuild the frame, re-run the probe and recompute every number is in the public repository:

pip install -r requirements.txt
python test_probe.py                    # 12 fixture servers; must print ALL PASSED
python build_frame.py crawl             # snapshot the official MCP Registry
python build_frame.py frame             # frame.csv + exclusion counts
python build_frame.py sample --n 3000 --seed 20261012
python probe.py --sample ../data/frame/sample.csv --round 2
python classify.py --round 2
python compute_stats.py --round 2 --retest 3 --sensitivity-exclude-host-sha256 54b786edf0f9
python figures.py                       # the four figures on this page

Registry contents change daily, so a fresh crawl gives a different frame; the archived snapshot, frame and sample in data/frame/ reproduce this study exactly.