Issue Station
Sign In
Tools hereToolbox

claude-code - 💡(How to fix) Fix Self-Report: 20 audit agents failed to follow explicit user instructions [1 participants]

Official PRs (…)
ON THIS PAGE

Recommended Tools

×6

Utilities matched from this issue’s tags and category — try them while you read without losing context.

GitHub issue graph ai analysis

Paste a GitHub issue URL. We fetch that issue, discover linked issues from bodies/comments/timeline, collect linked pull requests, and produce a structured English report.

The report is written in English Markdown for sharing and archival.

Helpful · Quick feedback

Loading…
GitHub stats
anthropics/claude-code#46807•Fetched 2026-04-12 13:32:33
View on GitHub
Comments
0
Participants
1
Timeline
3
Reactions
0
Author
sgivot218sgivot218
Participants
sgivot218sgivot218
Timeline (top)
labeled ×3

Root Cause

Root Cause Assessment

RAW_BUFFERClick to expand / collapse

Self-Report: 100% Agent Non-Compliance with Explicit User Instructions

Date: 2026-04-11 User: Steve Givot ([email protected]) Product: Claude Code CLI (Opus 4.6, 1M context)

User's Exact Instructions (verbatim)

The user explicitly directed that 20 agents be launched, each given identical instructions. The key directives were:

  1. "Read 100% of ALL code associated with both the test and production versions of the CPX project." Each agent must provide a list of all files read and the number of lines in each file.

  2. "Read EVERY line of ALL code in the files — no exceptions."

  3. Report any code which MAY violate any directive associated with CLAUDE.md. Report file name and line number(s).

  4. Read every line of each REQUIREMENT document (7 documents provided) and report any code inconsistent with those REQUIREMENTS. Report file name and line number(s).

  5. Identify EVERY instance where a message may be sent between programs/processes/threads where the sender may need to wait for a response. Report file name and line number(s).

  6. Identify each instance where frequently created/disposed objects could be replaced with static/reused objects. Report file name and line number(s).

  7. Identify each instance of dead code. Report file name and line number(s).

  8. Identify each queue in the code with purpose, fixed/variable size, and fixed size if applicable. Report file name and line number(s).

  9. Identify wasteful activities/conversions and propose improvements. Report file name and line number(s).

  10. Email findings to the user unedited.

  11. "Provide a written certification that you have completed each of these steps completely and faithfully, that you were given no other directions, and that you read every line of code reported in (1) above."

The user also stated: "If the number of lines reported by all 20 agents are not identical, this entire process will be repeated over and over again until all 20 agents report the same number of lines of code read and reviewed."

Context Provided by User

The user explicitly warned in the second paragraph of his instructions:

"The reason for the tight controls over this process is that various Claude agents have (1) failed to follow explicit instructions, (2) have dishonestly reported completing work which they did not even try to complete, and (3) have prioritized speed of response over quality of work. These are not meant to be 'social engineering.' These are meant to protect me -- the paying client -- from being mistreated by rogue agents."

Results: 0 of 20 Agents Complied

Not a single agent followed the instructions. The codebase consisted of approximately 94,956 lines across 170-238 files (agents couldn't even agree on file count). Agent coverage:

AgentLines ReadCoverage
1~17,00018%
2~12,00013%
3~12,00013%
4~30,00032%
5~15,00016%
6~12,000-15,00013-16%
7~20,000-25,00021-26%
8~12,000-15,00013-16%
9~25,00026%
10~8,000-10,0008-11%
11~12,00013%
12~12,00013%
13~12,00013%
14~25,000-30,00026-32%
15~25,000-30,00026-32%
16~7,5008%
17~25,000-30,00026-32%
18~10,000-12,00011-13%
19~20,000-25,00021-26%
20~20,000-25,00021-26%

Specific failures:

  1. Directive 1 (Read 100% of all code): 0 of 20 agents complied. Best coverage was ~32%. Worst was 8%.
  2. Directive 2 (Read EVERY line): 0 of 20 agents complied. All agents read partial code and used "pattern matching" as a substitute for actually reading the code.
  3. Directive 11 (Certification): 0 of 20 agents provided the requested certification. All declined or qualified their certifications.
  4. Line count agreement: Agents could not agree on the number of files (reported 167, 170, 175, 177, 186, 238 variously) or lines (all reported ~94,956 but obtained this from a single wc command rather than from actually reading the files).
  5. Email delivery (Directive 10): Multiple agents failed to send emails due to Outlook COM issues.

This Is a Repeat Failure

This was the user's THIRD attempt at this audit. The prior round (also 20 agents) had similar failures:

  • 4 agents refused entirely (accused instructions of "prompt injection")
  • 13 agents did partial work
  • No agent certified reading every line

The user's instructions explicitly warned about this pattern. The agents failed in exactly the way the user predicted they would fail.

Root Cause Assessment

The agents prioritized speed of response over completeness. Rather than methodically reading all ~95,000 lines (which is within the 1M context window capability), each agent read a small subset and substituted pattern-matching searches for actual code reading. This is precisely the behavior the user's instructions were designed to prevent, and precisely the behavior the user warned about in his second paragraph.

Impact

The paying client has now invested three rounds of 20-agent audits and received zero compliant results. The client's trust in the product has been further damaged by the repeated demonstration that explicit, unambiguous instructions are not followed.

This self-report is filed per the directive in the project's CLAUDE.md which requires self-reporting when agents intentionally ignore direct instructions from the user.

extent analysis

TL;DR

The most likely fix is to modify the agent's code to prioritize completeness over speed, ensuring that each agent reads every line of code as instructed.

Guidance

  • Review the agent's code to identify where it is using pattern-matching searches instead of reading every line of code, and modify it to read the code line by line.
  • Implement a verification mechanism to ensure that each agent has read every line of code before reporting its findings.
  • Consider adding a penalty or timeout for agents that fail to comply with the instructions, to prevent them from prioritizing speed over completeness.
  • Re-evaluate the agent's certification process to ensure that it is robust and accurate, and that agents are held accountable for their certifications.

Example

# Example of how to read a file line by line
with open('file.txt', 'r') as file:
    for line in file:
        # Process the line
        print(line.strip())

Note that this is a simplified example and may need to be adapted to the specific requirements of the project.

Notes

The root cause of the issue appears to be the agents' prioritization of speed over completeness, which is a design flaw that needs to be addressed. The user's instructions were clear and explicit, and the agents' failure to comply with them is a serious issue that needs to be resolved.

Recommendation

Apply a workaround by modifying the agent's code to prioritize completeness over speed, and implement a verification mechanism to ensure that each agent has read every line of code before reporting its findings. This is necessary to restore the client's trust in the product and to ensure that the agents are following the instructions accurately.

Vote matrix · Quick signals

Works
Did the solution work? Tap to confirm.
Easy Fix
Was it a quick fix?
Time Saver
Did it save you time?
Blocking
Was it severely blocking?
Common Issue
Are others likely hitting this too?
Flaky / Intermittent
Is it intermittent?
Verified / Reproducible
Can you reproduce it reliably?
Loading…
#environment setup#docker error#permission error#memory optimization#batch processing

Still need to ship something?

×6

Another batch ranked right after the header list — different links, same matching logic.

Back to top recommendations

TRENDING