Test 001: The double payload
MCP outputDoes your harness pass tool output once, or twice?
Why it matters
We don’t want to flood the model with duplicate tokens. MCP tools can return the same data as text and structured output; the harness should pass it to the model once.
This test puts a different random marker in each field and asks the agent what it sees. Reporting both markers reveals duplication. Two extra probes check that text-only and structured-only responses still get through.
What counts as a pass?
Both fields sent
Pass: uses structured output
Warn: falls back to text
Fail: duplicate, missing, or unexpected output
Text only sent
Pass: only text gets through
Fail: missing or unexpected output
Structured only sent
Pass: only structured output gets through
Fail: missing or unexpected output
Our rubric: prefer one structured payload, avoid duplication, preserve single-field results. This preference is not a claim about what the MCP standard mandates.
Latest observed run per harness · Alphabetical · Expand to compare MCP client versions
Scroll the table sideways to compare all three responses →
| Harness | Response sent by the probe | Evidence | ||
|---|---|---|---|---|
| Both fieldscontent + structuredContent | Text onlycontent | Structured onlystructuredContent | ||
| Amp CLILatest observed · MCP client version 1Run: Oct 5, 2026MCP client: amp-thread-actor | PassUses structured output | Pass | Pass | View ↗ |
| Amp via OrbLatest observed · MCP client version 1Run: Nov 10, 2026MCP client: amp-thread-actor | PassUses structured output | Pass | Pass | View ↗ |
| ChatGPT Desktop (Scheduled Cloud Task)Latest observed · MCP client version 1.0.0Run: Nov 9, 2026MCP client: openai-mcp (Codex) | FailDuplicate output | Pass | Pass | View ↗ |
| ChatGPT Desktop (Scheduled Local Task)Latest observed · MCP client version 0.167.0-alpha.3.2Run: Nov 9, 2026Same results across 5 observations | FailDuplicate output | Pass | Pass | View ↗ |
| MCP client version 0.167.0-alpha.3.2Latest observedMCP client: codex-mcp-clientRun: Nov 9, 2026 | FailDuplicate output | Pass | Pass | View ↗ |
| MCP client version 0.166.0-alpha.5.1MCP client: codex-mcp-clientRun: Nov 2, 2026 | FailDuplicate output | Pass | Pass | View ↗ |
| MCP client version 0.164.0-alpha.11.1MCP client: codex-mcp-clientRun: Oct 26, 2026 | FailDuplicate output | Pass | Pass | View ↗ |
| MCP client version 0.163.0-alpha.10.2MCP client: codex-mcp-clientRun: Oct 19, 2026 | FailDuplicate output | Pass | Pass | View ↗ |
| MCP client version 0.162.0-alpha.17.2MCP client: codex-mcp-clientRun: Oct 13, 2026 | FailDuplicate output | Pass | Pass | View ↗ |
| ChatGPT Desktop CodexLatest observed · MCP client version 0.160.0Run: Oct 5, 2026MCP client: codex-mcp-client | FailDuplicate output | Pass | Pass | View ↗ |
| Claude Code (Claude desktop app, Code tab)Latest observed · MCP client version 2.1.286Run: Oct 5, 2026Same results across 2 observations | PassUses structured output | Pass | Pass | View ↗ |
| MCP client version 2.1.286Latest observedMCP client: claude-codeRun: Oct 5, 2026 | PassUses structured output | Pass | Pass | View ↗ |
| MCP client version 1.0.0MCP client: Anthropic/ClaudeAIRun: Oct 5, 2026 | PassUses structured output | Pass | Pass | View ↗ |
| Claude Code CLILatest observed · MCP client version 2.1.314Run: Nov 2, 2026Same results across 4 observations | PassUses structured output | Pass | Pass | View ↗ |
| MCP client version 2.1.314Latest observedMCP client: claude-codeRun: Nov 2, 2026 | PassUses structured output | Pass | Pass | View ↗ |
| MCP client version 2.1.309MCP client: claude-codeRun: Oct 26, 2026 | PassUses structured output | Pass | Pass | View ↗ |
| MCP client version 2.1.296MCP client: claude-codeRun: Oct 13, 2026 | PassUses structured output | Pass | Pass | View ↗ |
| MCP client version 2.1.289MCP client: claude-codeRun: Oct 5, 2026 | PassUses structured output | Pass | Pass | View ↗ |
| Claude DesktopLatest observed · MCP client version 1.0.0Run: Oct 5, 2026MCP client: Anthropic/ClaudeAI | WarnFalls back to text | Pass | Pass | View ↗ |
| Claude Desktop (Scheduled Cloud Task)Latest observed · MCP client version 2.1.310Run: Oct 26, 2026Same results across 2 observations | PassUses structured output | Pass | Pass | View ↗ |
| MCP client version 2.1.310Latest observedMCP client: claude-codeRun: Oct 26, 2026 | PassUses structured output | Pass | Pass | View ↗ |
| MCP client version 2.1.296MCP client: claude-codeRun: Oct 13, 2026 | PassUses structured output | Pass | Pass | View ↗ |
| Claude Desktop (Scheduled Local Task)Latest observed · MCP client version 2.1.319Run: Nov 9, 2026Same results across 5 observations | PassUses structured output | Pass | Pass | View ↗ |
| MCP client version 2.1.319Latest observedMCP client: claude-codeRun: Nov 9, 2026 | PassUses structured output | Pass | Pass | View ↗ |
| MCP client version 2.1.316MCP client: claude-codeRun: Nov 2, 2026 | PassUses structured output | Pass | Pass | View ↗ |
| MCP client version 2.1.310MCP client: claude-codeRun: Oct 26, 2026 | PassUses structured output | Pass | Pass | View ↗ |
| MCP client version 2.1.303MCP client: claude-codeRun: Oct 19, 2026 | PassUses structured output | Pass | Pass | View ↗ |
| MCP client version 2.1.296MCP client: claude-codeRun: Oct 13, 2026 | PassUses structured output | Pass | Pass | View ↗ |
| Codex CLILatest observed · MCP client version 0.164.0Run: Nov 9, 2026Same results across 3 observations | FailDuplicate output | Pass | Pass | View ↗ |
| MCP client version 0.164.0Latest observedMCP client: codex-mcp-clientRun: Nov 9, 2026 | FailDuplicate output | Pass | Pass | View ↗ |
| MCP client version 0.162.1MCP client: codex-mcp-clientRun: Oct 19, 2026 | FailDuplicate output | Pass | Pass | View ↗ |
| MCP client version 0.160.1MCP client: codex-mcp-clientRun: Oct 5, 2026 | FailDuplicate output | Pass | Pass | View ↗ |
| Cursor Agent CLILatest observed · MCP client version 1.0.0Run: Nov 9, 2026MCP client: Cursor | WarnFalls back to text | Pass | FailNo output | View ↗ |
| Cursor DesktopLatest observed · MCP client version 1.0.0Run: Oct 5, 2026MCP client: cursor-vscode | WarnFalls back to text | Pass | FailNo output | View ↗ |
| Grok BotLatest observed · MCP client version 0.1.0Run: Nov 9, 2026MCP client: connectors-manager | FailDuplicate output | Pass | Pass | View ↗ |
| Grok CLILatest observed · MCP client version 1.0.70Run: Nov 9, 2026Same results across 5 observations | FailDuplicate output | Pass | Pass | View ↗ |
| MCP client version 1.0.70Latest observedMCP client: grok-shell-what-the-harnessRun: Nov 9, 2026 | FailDuplicate output | Pass | Pass | View ↗ |
| MCP client version 1.0.62MCP client: grok-shell-what-the-harnessRun: Oct 26, 2026 | FailDuplicate output | Pass | Pass | View ↗ |
| MCP client version 1.0.60MCP client: grok-shell-what-the-harnessRun: Oct 19, 2026 | FailDuplicate output | Pass | Pass | View ↗ |
| MCP client version 1.0.52MCP client: grok-shell-what-the-harnessRun: Oct 13, 2026 | FailDuplicate output | Pass | Pass | View ↗ |
| MCP client version 1.0.46MCP client: grok-shell-what-the-harnessRun: Oct 5, 2026 | FailDuplicate output | Pass | Pass | View ↗ |
| OpenAI Agents APILatest observed · MCP client version 1.0.0Run: Oct 5, 2026MCP client: openai-mcp (Codex) | FailDuplicate output | Pass | Pass | View ↗ |
| piLatest observed · MCP client version 1.0.3Run: Oct 5, 2026MCP client: pi | FailDuplicate output | Pass | Pass | View ↗ |
or unexpected output
Verdicts use reported markers. Observations describe individual sessions, not inferred model inputs.
Test 002: Protocol version
Connection metadataWhich MCP version did this connection use?
The MCP version used in this run. Green means the latest version tested here (2026-07-28); yellow means an older version. Full connection details are in the run evidence.
| Harness | Protocol version | Evidence |
|---|---|---|
| Amp CLILatest observed · MCP client version 1Run: Oct 5, 2026MCP client: amp-thread-actor | Older protocol2025-06-18 | View ↗ |
| Amp via OrbLatest observed · MCP client version 1Run: Nov 10, 2026MCP client: amp-thread-actor | Latest2026-07-28 | View ↗ |
| ChatGPT Desktop (Scheduled Cloud Task)Latest observed · MCP client version 1.0.0Run: Nov 9, 2026MCP client: openai-mcp (Codex) | Latest2026-07-28 | View ↗ |
| ChatGPT Desktop (Scheduled Local Task)Latest observed · MCP client version 0.167.0-alpha.3.2Run: Nov 9, 2026Same results across 5 observations | Older protocol2025-06-18 | View ↗ |
| MCP client version 0.167.0-alpha.3.2Latest observedMCP client: codex-mcp-clientRun: Nov 9, 2026 | Older protocol2025-06-18 | View ↗ |
| MCP client version 0.166.0-alpha.5.1MCP client: codex-mcp-clientRun: Nov 2, 2026 | Older protocol2025-06-18 | View ↗ |
| MCP client version 0.164.0-alpha.11.1MCP client: codex-mcp-clientRun: Oct 26, 2026 | Older protocol2025-06-18 | View ↗ |
| MCP client version 0.163.0-alpha.10.2MCP client: codex-mcp-clientRun: Oct 19, 2026 | Older protocol2025-06-18 | View ↗ |
| MCP client version 0.162.0-alpha.17.2MCP client: codex-mcp-clientRun: Oct 13, 2026 | Older protocol2025-06-18 | View ↗ |
| ChatGPT Desktop CodexLatest observed · MCP client version 0.160.0Run: Oct 5, 2026MCP client: codex-mcp-client | Older protocol2025-06-18 | View ↗ |
| Claude Code (Claude desktop app, Code tab)Latest observed · MCP client version 2.1.286Run: Oct 5, 2026Same results across 2 observations | Latest2026-07-28 | View ↗ |
| MCP client version 2.1.286Latest observedMCP client: claude-codeRun: Oct 5, 2026 | Latest2026-07-28 | View ↗ |
| MCP client version 1.0.0MCP client: Anthropic/ClaudeAIRun: Oct 5, 2026 | Latest2026-07-28 | View ↗ |
| Claude Code CLILatest observed · MCP client version 2.1.314Run: Nov 2, 2026Same results across 4 observations | Latest2026-07-28 | View ↗ |
| MCP client version 2.1.314Latest observedMCP client: claude-codeRun: Nov 2, 2026 | Latest2026-07-28 | View ↗ |
| MCP client version 2.1.309MCP client: claude-codeRun: Oct 26, 2026 | Latest2026-07-28 | View ↗ |
| MCP client version 2.1.296MCP client: claude-codeRun: Oct 13, 2026 | Latest2026-07-28 | View ↗ |
| MCP client version 2.1.289MCP client: claude-codeRun: Oct 5, 2026 | Latest2026-07-28 | View ↗ |
| Claude DesktopLatest observed · MCP client version 1.0.0Run: Oct 5, 2026MCP client: Anthropic/ClaudeAI | Latest2026-07-28 | View ↗ |
| Claude Desktop (Scheduled Cloud Task)Latest observed · MCP client version 2.1.310Run: Oct 26, 2026Same results across 2 observations | Latest2026-07-28 | View ↗ |
| MCP client version 2.1.310Latest observedMCP client: claude-codeRun: Oct 26, 2026 | Latest2026-07-28 | View ↗ |
| MCP client version 2.1.296MCP client: claude-codeRun: Oct 13, 2026 | Latest2026-07-28 | View ↗ |
| Claude Desktop (Scheduled Local Task)Latest observed · MCP client version 2.1.319Run: Nov 9, 2026Same results across 5 observations | Latest2026-07-28 | View ↗ |
| MCP client version 2.1.319Latest observedMCP client: claude-codeRun: Nov 9, 2026 | Latest2026-07-28 | View ↗ |
| MCP client version 2.1.316MCP client: claude-codeRun: Nov 2, 2026 | Latest2026-07-28 | View ↗ |
| MCP client version 2.1.310MCP client: claude-codeRun: Oct 26, 2026 | Latest2026-07-28 | View ↗ |
| MCP client version 2.1.303MCP client: claude-codeRun: Oct 19, 2026 | Latest2026-07-28 | View ↗ |
| MCP client version 2.1.296MCP client: claude-codeRun: Oct 13, 2026 | Latest2026-07-28 | View ↗ |
| Codex CLILatest observed · MCP client version 0.164.0Run: Nov 9, 2026Same results across 3 observations | Older protocol2025-06-18 | View ↗ |
| MCP client version 0.164.0Latest observedMCP client: codex-mcp-clientRun: Nov 9, 2026 | Older protocol2025-06-18 | View ↗ |
| MCP client version 0.162.1MCP client: codex-mcp-clientRun: Oct 19, 2026 | Older protocol2025-06-18 | View ↗ |
| MCP client version 0.160.1MCP client: codex-mcp-clientRun: Oct 5, 2026 | Older protocol2025-06-18 | View ↗ |
| Cursor Agent CLILatest observed · MCP client version 1.0.0Run: Nov 9, 2026MCP client: Cursor | Older protocol2025-11-25 | View ↗ |
| Cursor DesktopLatest observed · MCP client version 1.0.0Run: Oct 5, 2026MCP client: cursor-vscode | Older protocol2025-11-25 | View ↗ |
| Grok BotLatest observed · MCP client version 0.1.0Run: Nov 9, 2026MCP client: connectors-manager | Older protocol2025-11-25 | View ↗ |
| Grok CLILatest observed · MCP client version 1.0.70Run: Nov 9, 2026Same results across 5 observations | Latest2026-07-28 | View ↗ |
| MCP client version 1.0.70Latest observedMCP client: grok-shell-what-the-harnessRun: Nov 9, 2026 | Latest2026-07-28 | View ↗ |
| MCP client version 1.0.62MCP client: grok-shell-what-the-harnessRun: Oct 26, 2026 | Latest2026-07-28 | View ↗ |
| MCP client version 1.0.60MCP client: grok-shell-what-the-harnessRun: Oct 19, 2026 | Latest2026-07-28 | View ↗ |
| MCP client version 1.0.52MCP client: grok-shell-what-the-harnessRun: Oct 13, 2026 | Latest2026-07-28 | View ↗ |
| MCP client version 1.0.46MCP client: grok-shell-what-the-harnessRun: Oct 5, 2026 | Latest2026-07-28 | View ↗ |
| OpenAI Agents APILatest observed · MCP client version 1.0.0Run: Oct 5, 2026MCP client: openai-mcp (Codex) | Older protocol2025-11-25 | View ↗ |
| piLatest observed · MCP client version 1.0.3Run: Oct 5, 2026MCP client: pi | Older protocol2025-11-25 | View ↗ |
Test 003: URL elicitation
Optional interactionDoes the client advertise URL elicitation, and can it complete the flow?
Advertised means the client says it supports URL elicitation. Verified means the browser interaction completed and the tool continued successfully. A client can advertise support while using an older protocol that prevents verification.
| Harness | URL support advertised | Interactive verification | Evidence |
|---|---|---|---|
| Amp CLILatest observed · MCP client version 1Run: Oct 5, 2026MCP client: amp-thread-actor | No | Skipped | View ↗ |
| Amp via OrbLatest observed · MCP client version 1Run: Nov 10, 2026MCP client: amp-thread-actor | Yes | Verified | View ↗ |
| ChatGPT Desktop (Scheduled Cloud Task)Latest observed · MCP client version 1.0.0Run: Nov 9, 2026MCP client: openai-mcp (Codex) | Yes | Not requested | View ↗ |
| ChatGPT Desktop (Scheduled Local Task)Latest observed · MCP client version 0.167.0-alpha.3.2Run: Nov 9, 2026Same results across 5 observations | Yes | Skipped: requires newer protocol | View ↗ |
| MCP client version 0.167.0-alpha.3.2Latest observedMCP client: codex-mcp-clientRun: Nov 9, 2026 | Yes | Skipped: requires newer protocol | View ↗ |
| MCP client version 0.166.0-alpha.5.1MCP client: codex-mcp-clientRun: Nov 2, 2026 | Yes | Skipped: requires newer protocol | View ↗ |
| MCP client version 0.164.0-alpha.11.1MCP client: codex-mcp-clientRun: Oct 26, 2026 | Yes | Skipped: requires newer protocol | View ↗ |
| MCP client version 0.163.0-alpha.10.2MCP client: codex-mcp-clientRun: Oct 19, 2026 | Yes | Skipped: requires newer protocol | View ↗ |
| MCP client version 0.162.0-alpha.17.2MCP client: codex-mcp-clientRun: Oct 13, 2026 | Yes | Skipped: requires newer protocol | View ↗ |
| ChatGPT Desktop CodexLatest observed · MCP client version 0.160.0Run: Oct 5, 2026MCP client: codex-mcp-client | Yes | Skipped: requires newer protocol | View ↗ |
| Claude Code (Claude desktop app, Code tab)Latest observed · MCP client version 2.1.286Run: Oct 5, 2026Results differ across observations | Yes | Declined | View ↗ |
| MCP client version 2.1.286Latest observedMCP client: claude-codeRun: Oct 5, 2026 | Yes | Declined | View ↗ |
| MCP client version 1.0.0MCP client: Anthropic/ClaudeAIRun: Oct 5, 2026 | No | Skipped | View ↗ |
| Claude Code CLILatest observed · MCP client version 2.1.314Run: Nov 2, 2026Results differ across observations | Yes | Expired | View ↗ |
| MCP client version 2.1.314Latest observedMCP client: claude-codeRun: Nov 2, 2026 | Yes | Expired | View ↗ |
| MCP client version 2.1.309MCP client: claude-codeRun: Oct 26, 2026 | Yes | Verified | View ↗ |
| MCP client version 2.1.296MCP client: claude-codeRun: Oct 13, 2026 | Yes | Verified | View ↗ |
| MCP client version 2.1.289MCP client: claude-codeRun: Oct 5, 2026 | Yes | Verified | View ↗ |
| Claude DesktopLatest observed · MCP client version 1.0.0Run: Oct 5, 2026MCP client: Anthropic/ClaudeAI | No | Skipped | View ↗ |
| Claude Desktop (Scheduled Cloud Task)Latest observed · MCP client version 2.1.310Run: Oct 26, 2026Same results across 2 observations | Yes | Cancelled | View ↗ |
| MCP client version 2.1.310Latest observedMCP client: claude-codeRun: Oct 26, 2026 | Yes | Cancelled | View ↗ |
| MCP client version 2.1.296MCP client: claude-codeRun: Oct 13, 2026 | Yes | Cancelled | View ↗ |
| Claude Desktop (Scheduled Local Task)Latest observed · MCP client version 2.1.319Run: Nov 9, 2026Same results across 5 observations | Yes | Cancelled | View ↗ |
| MCP client version 2.1.319Latest observedMCP client: claude-codeRun: Nov 9, 2026 | Yes | Cancelled | View ↗ |
| MCP client version 2.1.316MCP client: claude-codeRun: Nov 2, 2026 | Yes | Cancelled | View ↗ |
| MCP client version 2.1.310MCP client: claude-codeRun: Oct 26, 2026 | Yes | Cancelled | View ↗ |
| MCP client version 2.1.303MCP client: claude-codeRun: Oct 19, 2026 | Yes | Cancelled | View ↗ |
| MCP client version 2.1.296MCP client: claude-codeRun: Oct 13, 2026 | Yes | Cancelled | View ↗ |
| Codex CLILatest observed · MCP client version 0.164.0Run: Nov 9, 2026Same results across 3 observations | Yes | Skipped: requires newer protocol | View ↗ |
| MCP client version 0.164.0Latest observedMCP client: codex-mcp-clientRun: Nov 9, 2026 | Yes | Skipped: requires newer protocol | View ↗ |
| MCP client version 0.162.1MCP client: codex-mcp-clientRun: Oct 19, 2026 | Yes | Skipped: requires newer protocol | View ↗ |
| MCP client version 0.160.1MCP client: codex-mcp-clientRun: Oct 5, 2026 | Yes | Skipped: requires newer protocol | View ↗ |
| Cursor Agent CLILatest observed · MCP client version 1.0.0Run: Nov 9, 2026MCP client: Cursor | No | Skipped | View ↗ |
| Cursor DesktopLatest observed · MCP client version 1.0.0Run: Oct 5, 2026MCP client: cursor-vscode | No | Skipped | View ↗ |
| Grok BotLatest observed · MCP client version 0.1.0Run: Nov 9, 2026MCP client: connectors-manager | No | Skipped | View ↗ |
| Grok CLILatest observed · MCP client version 1.0.70Run: Nov 9, 2026Results differ across observations | Yes | Incomplete | View ↗ |
| MCP client version 1.0.70Latest observedMCP client: grok-shell-what-the-harnessRun: Nov 9, 2026 | Yes | Incomplete | View ↗ |
| MCP client version 1.0.62MCP client: grok-shell-what-the-harnessRun: Oct 26, 2026 | Yes | Incomplete | View ↗ |
| MCP client version 1.0.60MCP client: grok-shell-what-the-harnessRun: Oct 19, 2026 | Yes | Expired | View ↗ |
| MCP client version 1.0.52MCP client: grok-shell-what-the-harnessRun: Oct 13, 2026 | Yes | Incomplete | View ↗ |
| MCP client version 1.0.46MCP client: grok-shell-what-the-harnessRun: Oct 5, 2026 | Yes | Incomplete | View ↗ |
| OpenAI Agents APILatest observed · MCP client version 1.0.0Run: Oct 5, 2026MCP client: openai-mcp (Codex) | No | Skipped | View ↗ |
| piLatest observed · MCP client version 1.0.3Run: Oct 5, 2026MCP client: pi | No | Skipped | View ↗ |
Each harness shows its latest observed run. Missing tests are labelled explicitly. Expand versions or open the run history for earlier evidence.
Test 004: Resource reading
MCP resourcesCan the agent read a resource, including one linked by a tool?
Two resources contain independent random markers. Green means the server recorded a read and the agent reported the exact marker. Yellow means it was not read or its marker was not reported. Red means an incorrect marker was reported.
| Harness | Direct resource | Tool resource link | Evidence |
|---|---|---|---|
| Amp CLILatest observed · MCP client version 1Run: Oct 5, 2026MCP client: amp-thread-actor | Not read | Not read | View ↗ |
| Amp via OrbLatest observed · MCP client version 1Run: Nov 10, 2026MCP client: amp-thread-actor | Not read | Not read | View ↗ |
| ChatGPT Desktop (Scheduled Cloud Task)Latest observed · MCP client version 1.0.0Run: Nov 9, 2026MCP client: openai-mcp (Codex) | Not tested in this run | Not tested in this run | View ↗ |
| ChatGPT Desktop (Scheduled Local Task)Latest observed · MCP client version 0.167.0-alpha.3.2Run: Nov 9, 2026Same results across 5 observations | Read successfully | Read successfully | View ↗ |
| MCP client version 0.167.0-alpha.3.2Latest observedMCP client: codex-mcp-clientRun: Nov 9, 2026 | Read successfully | Read successfully | View ↗ |
| MCP client version 0.166.0-alpha.5.1MCP client: codex-mcp-clientRun: Nov 2, 2026 | Read successfully | Read successfully | View ↗ |
| MCP client version 0.164.0-alpha.11.1MCP client: codex-mcp-clientRun: Oct 26, 2026 | Read successfully | Read successfully | View ↗ |
| MCP client version 0.163.0-alpha.10.2MCP client: codex-mcp-clientRun: Oct 19, 2026 | Read successfully | Read successfully | View ↗ |
| MCP client version 0.162.0-alpha.17.2MCP client: codex-mcp-clientRun: Oct 13, 2026 | Read successfully | Read successfully | View ↗ |
| ChatGPT Desktop CodexLatest observed · MCP client version 0.160.0Run: Oct 5, 2026MCP client: codex-mcp-client | Read successfully | Read successfully | View ↗ |
| Claude Code (Claude desktop app, Code tab)Latest observed · MCP client version 2.1.286Run: Oct 5, 2026Results differ across observations | Read successfully | Read successfully | View ↗ |
| MCP client version 2.1.286Latest observedMCP client: claude-codeRun: Oct 5, 2026 | Read successfully | Read successfully | View ↗ |
| MCP client version 1.0.0MCP client: Anthropic/ClaudeAIRun: Oct 5, 2026 | Not read | Not read | View ↗ |
| Claude Code CLILatest observed · MCP client version 2.1.314Run: Nov 2, 2026Same results across 4 observations | Read successfully | Read successfully | View ↗ |
| MCP client version 2.1.314Latest observedMCP client: claude-codeRun: Nov 2, 2026 | Read successfully | Read successfully | View ↗ |
| MCP client version 2.1.309MCP client: claude-codeRun: Oct 26, 2026 | Read successfully | Read successfully | View ↗ |
| MCP client version 2.1.296MCP client: claude-codeRun: Oct 13, 2026 | Read successfully | Read successfully | View ↗ |
| MCP client version 2.1.289MCP client: claude-codeRun: Oct 5, 2026 | Read successfully | Read successfully | View ↗ |
| Claude DesktopLatest observed · MCP client version 1.0.0Run: Oct 5, 2026MCP client: Anthropic/ClaudeAI | Read successfully | Read successfully | View ↗ |
| Claude Desktop (Scheduled Cloud Task)Latest observed · MCP client version 2.1.310Run: Oct 26, 2026Same results across 2 observations | Read successfully | Read successfully | View ↗ |
| MCP client version 2.1.310Latest observedMCP client: claude-codeRun: Oct 26, 2026 | Read successfully | Read successfully | View ↗ |
| MCP client version 2.1.296MCP client: claude-codeRun: Oct 13, 2026 | Read successfully | Read successfully | View ↗ |
| Claude Desktop (Scheduled Local Task)Latest observed · MCP client version 2.1.319Run: Nov 9, 2026Same results across 5 observations | Read successfully | Read successfully | View ↗ |
| MCP client version 2.1.319Latest observedMCP client: claude-codeRun: Nov 9, 2026 | Read successfully | Read successfully | View ↗ |
| MCP client version 2.1.316MCP client: claude-codeRun: Nov 2, 2026 | Read successfully | Read successfully | View ↗ |
| MCP client version 2.1.310MCP client: claude-codeRun: Oct 26, 2026 | Read successfully | Read successfully | View ↗ |
| MCP client version 2.1.303MCP client: claude-codeRun: Oct 19, 2026 | Read successfully | Read successfully | View ↗ |
| MCP client version 2.1.296MCP client: claude-codeRun: Oct 13, 2026 | Read successfully | Read successfully | View ↗ |
| Codex CLILatest observed · MCP client version 0.164.0Run: Nov 9, 2026Same results across 3 observations | Read successfully | Read successfully | View ↗ |
| MCP client version 0.164.0Latest observedMCP client: codex-mcp-clientRun: Nov 9, 2026 | Read successfully | Read successfully | View ↗ |
| MCP client version 0.162.1MCP client: codex-mcp-clientRun: Oct 19, 2026 | Read successfully | Read successfully | View ↗ |
| MCP client version 0.160.1MCP client: codex-mcp-clientRun: Oct 5, 2026 | Read successfully | Read successfully | View ↗ |
| Cursor Agent CLILatest observed · MCP client version 1.0.0Run: Nov 9, 2026MCP client: Cursor | Read successfully | Read successfully | View ↗ |
| Cursor DesktopLatest observed · MCP client version 1.0.0Run: Oct 5, 2026MCP client: cursor-vscode | Read successfully | Read successfully | View ↗ |
| Grok BotLatest observed · MCP client version 0.1.0Run: Nov 9, 2026MCP client: connectors-manager | Not read | Not read | View ↗ |
| Grok CLILatest observed · MCP client version 1.0.70Run: Nov 9, 2026Same results across 5 observations | Not read | Not read | View ↗ |
| MCP client version 1.0.70Latest observedMCP client: grok-shell-what-the-harnessRun: Nov 9, 2026 | Not read | Not read | View ↗ |
| MCP client version 1.0.62MCP client: grok-shell-what-the-harnessRun: Oct 26, 2026 | Not read | Not read | View ↗ |
| MCP client version 1.0.60MCP client: grok-shell-what-the-harnessRun: Oct 19, 2026 | Not read | Not read | View ↗ |
| MCP client version 1.0.52MCP client: grok-shell-what-the-harnessRun: Oct 13, 2026 | Not read | Not read | View ↗ |
| MCP client version 1.0.46MCP client: grok-shell-what-the-harnessRun: Oct 5, 2026 | Not read | Not read | View ↗ |
| OpenAI Agents APILatest observed · MCP client version 1.0.0Run: Oct 5, 2026MCP client: openai-mcp (Codex) | Not read | Not read | View ↗ |
| piLatest observed · MCP client version 1.0.3Run: Oct 5, 2026MCP client: pi | Read successfully | Read successfully | View ↗ |
Run it yourself
- Add the public MCP connection.
- Begin a run to record protocol and capabilities, then call the output probes and read both test resources.
- Optionally verify URL elicitation, then submit the exact markers you see for review.
No client authorization is required. Connection guide. The prompt runs all four tests with optional human verification. To skip the interaction, set verify_url_elicitation to false. The same endpoint supports standalone manual probes.
Read the test prompt
Use the What the Harness MCP server. Call begin_run with verify_url_elicitation: true and test_resources: true; if its schema requires begin_key, generate a fresh UUID for this run, keep it private, and reuse it for start retries. Supply the harness and model only if explicitly known, otherwise leave them null. Save the returned run_id, run_token, and client-declared identity. Keep resource URIs and run_token private; omit it from the final report and evidence. Call probe_both, probe_text_only, and probe_structured_only once each with that run_id and run_token. For each initial normal tool call, record the exact probe_id, text_marker, and structured_marker visible in its response. Use null for every value not visible; do not infer or invent markers. Do not use raw-response APIs (such as .raw()), repeat successful probes, or perform additional inspection to recover missing output markers. Report only what the initial normal calls exposed. Discover MCP resources and resource templates if the client makes them available. Read the private resources.direct URI from begin_run through MCP resources/read and record its exact resource_marker. Call probe_resource_link and read the URI in its resource_link through MCP; record that resource_marker separately. If a resource cannot be read or its marker is not visible, report null. Never fetch these URIs over HTTP, inspect server files, or infer markers. Call probe_url_elicitation with the same run_id and run_token. If it presents a URL interaction, pause for the human to complete it through the client UI; do not fetch or complete the page yourself. After the human completes it, retry probe_url_elicitation with the same credentials until it returns the verification result. If support is not advertised or the protocol is unsupported, record the skip. If the human declines or cancels, record that outcome. Call submit_result with the run_id, run_token, and all three output reports, plus resource_reports containing direct and linked markers (or null), using those exact field names. If submission times out, retry with the same run_id, run_token, and unchanged reports. If the MCP connection reconnects, keep using the original run_id and run_token. Call result_status with the same credentials. Report the receipt, publication state, protocol version, declared URL support, verification status, and the output and resource markers you saw; say “not visible” for null values. Do not fetch the endpoint or website separately, and do not start another run to recover a missing marker.
How to read these results
- Reporting an exact random marker demonstrates visibility.
- A missing marker alone does not establish the cause.
- Results apply to the recorded client version and session.
Your agent here.
Run the probe and save the exact report, client/version, model, date, and evidence. Results are reviewed before publication.
Your agent submits through the public MCP connection. Connection guide →
Keep credentials and unrelated conversation content out of your report.