K5.3.4 Task 5.3

"No Peer-Reviewed Studies Exist" — Actually, 47 Papers Were Found After the Outage Ended

A research system reports “no peer-reviewed studies exist on offshore wind farm subsidies.” A manual search on the same database finds 47 matching papers. The academic database had a partial outage during the query. The sub-agent returned {results: [], isError: false} — converting an access failure into a false factual claim.

This is the most dangerous form of the silent swallow: not just hiding an error, but generating a positive assertion (“no data exists”) from what was actually a system failure.

The Core Distinction

OutcomeMeaningisErrorAction
Access failure (timeout, auth, service down)Data MIGHT exist, we couldn’t checktrueRetry, switch source, report gap
Valid empty (query succeeded, zero matches)Data does NOT exist (confirmed)falseAccept as definitive answer

When both return identical empty results, the orchestrator cannot tell the difference. It treats “couldn’t check” as “checked and confirmed empty.”

The Data: 85% of “No Data” Is Actually Failed Queries

A system where 15% of sub-agent responses are empty:

Actual causePercentageTreatment needed
Service timeouts60%Retry
Auth failures25%Credential refresh
Genuine no-match15%Accept as answer

85% of “no data found” conclusions are access failures needing recovery. Only 15% are correct answers. Reports containing these empty responses have a 40% user complaint rate versus 5% for reports with full data.

A Misdiagnosis That Wasted 3 Months

A support system reports 8% of cases end with “unable to find customer information.” The team assumes data quality issues and improves the data entry interface. After 3 months: 8% → 7.2%. Minimal improvement.

Investigation reveals: 45% genuine no-match (typos), 35% database timeouts, 20% permission errors. Data entry fixes helped the 45% but did nothing for the 55% that were system failures. The real fix: distinguish access failures from valid empties so each gets appropriate handling.

Security-Critical: Fail-Closed on Access Failure

A CI pipeline checks 3 vulnerability databases before allowing merges. If a database is unreachable, the choices are:

  • Fail-open (WRONG): Treat unreachable as “no vulnerabilities found” → potentially vulnerable code merges to production
  • Fail-closed (CORRECT): Block merge on access failure since “couldn’t check” ≠ “checked and found nothing”

In security contexts, the distinction between access failure and valid empty is not just a quality issue — it is a safety boundary. Fail-open converts system failures into false security clearances.

The Fix: isError Differentiation

// Access failure
{"results": [], "isError": true, "failure_type": "timeout", "retry_recommended": true}

// Valid empty
{"results": [], "isError": false}

The orchestrator routes differently:

  • isError: true → retry, switch source, report gap
  • isError: false → accept as definitive, act on it

Customer Support Application

200 empty order lookups per day: 140 genuine invalid orders (70%), 60 database outages (30%). Currently all get “Please verify your order number.”

With isError differentiation: retry the 60 outage cases (most succeed), prompt verification only for the 140 genuine empties. Customers with valid orders stop being told their orders don’t exist.

Do Not Simplify By Unifying Error Paths

A proposal to treat all empty/error responses as “no data” reduces code complexity from 15 error paths to 1. But it produces silently incorrect results for every access failure. The maintenance cost of structured error handling is far less than the cost of false claims, frustrated users, and security vulnerabilities from conflating the two outcomes.


One-liner: Use isError to distinguish “couldn’t access the data” from “data genuinely doesn’t exist” — 85% of empty responses are access failures being misclassified as valid empties, and in security contexts, the distinction is a fail-open vs fail-closed safety boundary.