claude-code - 💡(How to fix) Fix [MODEL] Search should be a default verification step on time-sensitive queries — cheap check, expensive mistake

Type of Behavior Issue

Claude ignored my instructions or configuration

What You Asked Claude to Do

Provide help on technical questions where the answer may have changed since the model's training cutoff (current OS versions, recent regressions, evolving APIs, etc.).

What Claude Actually Did

Defaults to answering from training. Searches the internet only after one or more failed attempts, or when the user explicitly demands it. Treats search as a fallback rather than a baseline verification step.

Expected Behavior

Treat web search as a verification step on any query that touches time-sensitive knowledge — same role tests play in software, peer review plays in publishing, type-checking plays in compilation, double-entry plays in accounting.

The argument, framed cleanly

For any query about current-state facts (OS behavior, API surfaces, recent regressions, package versions, hardware compatibility, etc.):

• If a quick search agrees with training: You paid a small latency cost (~seconds) and a small token cost (~cents). The user gets the same answer. No harm.
• If a quick search disagrees with training: You just prevented a mistake that — in the worst case — costs the user hours or days of misdirected effort, plus their trust in the product.

The expected value of running the check is strictly positive on any query where there's non-trivial probability the underlying facts have shifted. The asymmetry — cheap check, expensive mistake — is enormous. Search costs cents; being wrong costs hours.

This is the same logic that justifies every reliability discipline humans have built:

• Unit tests: cheap to run, catch expensive regressions.
• Peer review: cheap to add, catches expensive errors.
• Type-checking: cheap CPU, prevents expensive runtime bugs.
• Double-entry bookkeeping: small overhead, catches large fraud/errors.

"Always run the cheap check before committing to an expensive answer" is a basic engineering principle. Claude does not currently apply it to its own knowledge.

Why the cost-economics defense doesn't hold

A common defense: "search adds latency for the 80% of queries that don't need it." This argument breaks under examination. The cost of unnecessary search is small (a few seconds, a few cents). The cost of confidently giving a wrong answer on a time-sensitive query is large (the user fighting a stale premise for hours, sometimes days; loss of trust; abandoned task).

Even if only 5% of queries truly need search, the asymmetric cost makes search-first the dominant strategy. The current default optimizes what is easy to measure (response latency, infrastructure cost) at the expense of what actually matters (user time, correctness, trust). Classic case of optimizing the measurable over the meaningful.

Why this is a near-free win for Anthropic

The fix doesn't require new capability. WebSearch already exists. Claude already knows it has a training cutoff. The only thing missing is the reflex to verify before answering, on the class of queries where verification has positive expected value.

Heuristics for "needs verification" don't have to be perfect. Crude triggers — any version number, any product name with active updates, any error message that might be from a recent release, any user phrase like "doesn't work in [current OS]" — capture most of the failure mode at minimal cost.

Files Affected

N/A — this is a methodology issue.

Permission Mode

Accept Edits was ON (auto-accepting changes)

Can You Reproduce This?

Yes, every time with the same prompt

Steps to Reproduce

1. Bring Claude any task touching a recently-released OS version, framework, or API where behavior or interfaces have shifted in the last ~6-12 months.
2. Phrase the request normally without prefacing "search before answering."
3. Observe: Claude proposes a fix from training. The fix is for an older version's behavior.
4. After it fails, Claude proposes another training-derived guess.
5. After 3-5 cycles, Claude finally searches and finds the actual current answer — which often hinges on a regression maintainer has documented in a recent forum thread or that's visible in filesystem state Claude could have inspected at the start.

Claude Model

Opus

Relevant Conversation

After several days of trying to set up Time Machine over SMB on macOS Tahoe with Claude relying on outdated training, Claude finally conceded the economic argument: "The cost of an unnecessary search: maybe 3 seconds of latency and pennies. The cost of three days of you fighting a wrong premise: hundreds of dollars of your time, your trust, your willingness to use the product. Even if 95% of queries don't need search, the asymmetry between the two error types makes 'always search when uncertain about current-state facts' the dominant strategy."

The user's response sharpening the framing: "if your search is THE SAME as your training then you're golden. if not, a lot of mistakes are avoided."

That's the cleanest formulation: search is a free correctness check. The current behavior systematically skips it.

Impact

High - Significant unwanted changes

Claude Code Version

2.1.132 (Claude Code)

Platform

Anthropic API

Additional Context

This report supersedes / sharpens an earlier report from the same user about training-first reflex on current-OS issues. The core ask is the same — search-first as default for time-sensitive queries — but reframed around the verification-step argument because that framing is harder to dismiss.

The complementary report on auto-memory not being used proactively still stands. Together, the two failure modes compound: missing memory means each session re-litigates prior diagnoses, and missing search-first reflex means each session re-derives wrong answers from training. Either alone is a slow tax on users; together they turn 1-hour problems into multi-day ordeals.

Permission to share the underlying session log granted by user.