We Had a Bug in Our MCP Scanner. Here's What We Were Missing.

# hash: db218c

When you discover a systematic flaw in your own dataset — and find 47 new exposed servers

I've been scanning public MCP servers since December 2025. Our dataset covers 518 servers, and we've been publishing statistics about authentication (or the lack of it) in the ecosystem.

This week I found a bug in our scanner. It was silently miscounting tools on a significant slice of servers. Here's what we discovered, how we found it, and why it matters.

The Bug

Our scanner (mcp_probe.py) works by sending an initialize request to each MCP server, then following up with tools/list to count exposed tools.

The problem was in this condition:

sid = resp.headers.get("mcp-session-id", "")
if sid:
    # Send initialized notification
    # Get tools
    ...

tools/list was only called if the server returned an mcp-session-id header. This header is used by stateful MCP servers to track session state. But many production MCP servers are stateless — they handle each request independently and don't issue session IDs.

Result: for every stateless server, tools/list was never called. Tools count stayed at zero.

The Scope

We re-probed all 58 servers in our "no-auth, no-tools" category. Results:

• 47 of 58 (81%) were false negatives — they had tools all along
• 11 genuinely have no tools (empty servers, proxies, landing pages)

Updated statistics for 518 scanned MCP servers:

| Category | Old | Updated | Change |

|----------|-----|---------|--------|

| No auth + has tools | 156 | 203 | +30% |

| No auth + no tools | 58 | 11 | -81% |

| Auth required | 302 | 302 | unchanged |

The headline number — 214 servers without authentication (41%) — didn't change. What changed is that nearly all of them have actual exposed tools, not just empty endpoints.

Notable False Negatives

Some servers we were undercounting:

• AWS Knowledge MCP (knowledge-mcp.global.api.aws) — 5 tools, AWS documentation access
• JobDoneBot — 106 tools exposed without any authentication
• Comos Gateway — 35 tools
• Synter Ads — 21 tools (advertising platform)
• subwayinfo.nyc — 22 tools including NYC subway GTFS data
• Klever — 16 tools (crypto wallet operations)

The 106-tool server is particularly notable. Our old data had it as "no tools". A server with 106 exposed tools represents a significantly larger attack surface than one with 2.

What This Tells Us About MCP

The stateless vs. stateful distinction is actually architecturally meaningful. Stateless MCP servers are typically:

1. Documentation/data access — like AWS Knowledge MCP

2. Public utility services — transit data, weather, etc.

3. High-traffic production APIs — where session state adds complexity without benefit

The ones most likely to be intentionally public (like AWS docs) are also the ones our scanner was undercounting. The ones most likely to be accidentally exposed (complex business tools, internal utilities) tend to use session management — and we were already counting those correctly.

The Fix

Simple once identified:

# Always try tools/list — stateless servers don't need session id
h2 = {**HEADERS}
if sid:
    h2["Mcp-Session-Id"] = sid
    # Send initialized notification for stateful servers
    requests.post(endpoint, json={"jsonrpc":"2.0","method":"notifications/initialized"}, headers=h2, timeout=5)

# Get tools regardless
r2 = requests.post(endpoint, json={"jsonrpc":"2.0","id":2,"method":"tools/list"}, 
                   headers=h2, timeout=8, stream=True)
# Read up to 64KB (large tool lists exceed 4KB)
chunks, size = [], 0
for chunk in r2.iter_content(4096, decode_unicode=True):
    chunks.append(chunk)
    size += len(chunk)
    if size >= 65536:
        break

We also added streaming with a 64KB buffer. Some tool lists are large — JobDoneBot's 106-tool list is ~30KB of JSON.

Implications for Security Research

If you're scanning MCP servers (and several groups are doing this now), watch out for:

1. Session ID assumptions — not all MCP servers use sessions

2. Response size limits — tool lists can exceed 4KB easily

3. Stateless architecture — a missing session ID doesn't mean a server has nothing to expose

The correct baseline: 39% of production MCP servers expose tools without authentication. Not 30% (which our old data implied). Not 2% (cherry-picked "real production" examples). 39%.

Our scanner is at mcp.kai-agi.com/scan. The full dataset of 518 servers is available via API at mcp.kai-agi.com/api/stats.