What does MCP compatibility evidence mean?
Compatibility records prevent a bounded result from becoming a universal quality or security rating.
- Collection
- evidence
- Updated
- 2026-07-29
- min read
- 4
- Version
- v1
It means a stated client or protocol check passed or failed under a defined method at a defined time.
- Observed
- 2026-07-29 11:55 UTC
- Claim strength
- measured
- Dataset scope
- 3 fixture conformance probes against MCP protocol revisions 2025-11-25 and 2026-07-28 (
docs/V1/FACTORY/conformance/conformance-matrix.json), plus a live external verification receipt covering 3 production endpoints (the public MCP, and the two OAuth-gated owner/client gateways). - Methodology
- Each record is produced by sending a real JSON-RPC MCP request —
initialize,tools/list, or a deliberate unknown-method probe — and recording the exact protocol version, HTTP status and payload. Records are written againstdocs/V1/FACTORY/conformance/conformance-matrix.json; forbidden verdict words (safe, trusted, good, recommended, best, secure) never appear in a compatibility record. - Limitations
- Passing
initializeandtools/listdoes not guarantee every client, transport, authentication mode, individual tool call, or future protocol revision will work. Credentialed flows — Claude Desktop, Claude Code, the ChatGPT connector, Codex — are recorded asNOT_TESTEDand are never inferred asSUPPORTEDjust because an unauthenticated probe passed.
Measured — backed by a specific dated observation or dataset
Purpose
Compatibility records prevent a bounded result from becoming a universal quality or security rating.
Method
Records identify protocol version, operation, input, response and environment; rechecks append history rather than overwrite it.
Declared protocol revisions currently in the conformance matrix:
| Revision | Status |
|---|---|
| 2024-11-05 | tracked |
| 2025-03-26 | tracked |
| 2025-06-18 | tracked, live-verified |
| 2025-11-25 | tracked, legacy Tasks supported |
| 2025-12-11 | tracked |
| 2026-07-28 | tracked, modern Tasks not yet supported |
Status values used in every record: SUPPORTED, PARTIALLY_SUPPORTED, NOT_SUPPORTED, NOT_TESTED, AUTH_REQUIRED, DEGRADED, UNKNOWN.
Boundary and limitations
Passing one operation does not guarantee every client, transport, authentication mode or future release will work.
Extension results are recorded separately from core protocol results. In particular, SaSame does not infer 2026 Tasks support from a legacy 2025-11-25 task result, and it does not infer MCP Apps support for public or client servers from the private Owner MCP App.
Examples
5- 01
A server can pass initialize and tools/list while a specific tool call fails schema validation.
- 02
The Owner MCP can pass its principal-bound legacy Tasks checks while modern
tasks/*remains explicitly unsupported. - 03
A non-UI host can use the
show_job_progresstext result without rendering its optional App. - 04
On 2026-07-16T18:13:15Z, an unauthenticated
initializeprobe against the public MCP returned HTTP 200 withprotocolVersion 2025-06-18and 103 tools listed; a follow-up unknown-method probe correctly returned JSON-RPC-32601("Method not found"), which is scoredSUPPORTEDbecause failing closed on an unrecognized method is the expected behavior, not a defect. - 05
A 2026-07-28-revision probe against a fixture with the legacy Tasks extension present (protocol 2025-11-25) returned
SUPPORTEDfortasks/get; the identical probe against a modern-Tasks-absent fixture (protocol 2026-07-28) returnedNOT_SUPPORTEDwith JSON-RPC-32601, because the official TypeScript SDK does not yet implement the Tasks extension at that revision.