Issue Station
Sign In
Tools hereToolbox

codex - ✅(Solved) Fix Compacted replacement_history retains active startup/context packets, duplicating AGENTS/developer/memory/skills context [3 pull requests, 1 participants]

Official PRs (…)
ON THIS PAGE

Recommended Tools

×6

Utilities matched from this issue’s tags and category — try them while you read without losing context.

GitHub issue graph ai analysis

Paste a GitHub issue URL. We fetch that issue, discover linked issues from bodies/comments/timeline, collect linked pull requests, and produce a structured English report.

The report is written in English Markdown for sharing and archival.

Helpful · Quick feedback

Loading…
GitHub stats
openai/codex#21269•Fetched 2026-05-06 06:23:59
View on GitHub
Comments
0
Participants
1
Timeline
5
Reactions
0
Author
macintogmacintog
Participants
macintogmacintog
Timeline (top)
labeled ×4cross-referenced ×1

Root Cause

I can provide the representative JSONL privately if useful. I am not attaching the full session publicly because it contains local paths and model-visible context, but the event paths, timestamps, counts, and reproduction checks above are exact.

Fix Action

Fix / Workaround

Local mitigation tests:

These mitigations reduce the size of my local duplicated payload. They do not address the general bug: active context should not be copied into replacement_history and re-sent as history for any user.

PR fix notes

PR #10574: core: refresh developer instructions after compaction replacement history

Description (problem / solution / changelog)

Summary

When replaying compacted history (especially replacement_history from remote compaction), we should not keep stale developer messages from older session state. This PR trims developer- role messages from compacted replacement history and reinjects fresh developer instructions derived from current turn/session state.

This aligns compaction replay behavior with the intended "fresh instructions after summary" model.

Problem

Compaction replay had two paths:

  • Compacted { replacement_history: None }: rebuilt with fresh initial context
  • Compacted { replacement_history: Some(...) }: previously used raw replacement history as-is

The second path could carry stale developer instructions (permissions/personality/collab-mode guidance) across session changes.

What Changed

1) Added helper to refresh compacted developer instructions

  • File: codex-rs/core/src/compact.rs
  • Function: refresh_compacted_developer_instructions(...)

Behavior:

  • remove all ResponseItem::Message { role: "developer", .. } from compacted history
  • append fresh developer messages from current build_initial_context(...)

2) Applied helper in remote compaction flow

  • File: codex-rs/core/src/compact_remote.rs
  • After receiving compact endpoint output, refresh developer instructions before replacing history and persisting replacement_history.

3) Applied helper while reconstructing history from rollout

  • File: codex-rs/core/src/codex.rs
  • In reconstruct_history_from_rollout(...), when processing Compacted entries with replacement_history, refresh developer instructions instead of directly replacing with raw history.

Non-Goals / Follow-up

This PR does not address the existing first-turn-after-resume double-injection behavior. A follow-up PR will handle resume-time dedup/idempotence separately.

If you want, I can also give you a shorter “squash-merge friendly” version of the description.

Codex author

codex fork 019c25e6-706e-75d1-9198-688ec00a8256

Changed files

  • codex-rs/core/src/codex.rs (modified, +45/-0)
  • codex-rs/core/src/compact.rs (modified, +396/-39)
  • codex-rs/core/src/compact_remote.rs (modified, +3/-0)
  • codex-rs/core/tests/suite/compact_remote.rs (modified, +313/-3)

PR #12252: Fix compaction context reinjection and model baselines

Description (problem / solution / changelog)

Summary

  • move regular-turn context diff/full-context persistence into run_turn so pre-turn compaction runs before incoming context updates are recorded
  • after successful pre-turn compaction, rely on a cleared reference_context_item to trigger full context reinjection on the follow-up regular turn (manual /compact keeps replacement history summary-only and also clears the baseline)
  • preserve <model_switch> when full context is reinjected, and inject it before the rest of the full-context items
  • scope reference_context_item and previous_model to regular user turns only so standalone tasks (/compact, shell, review, undo) cannot suppress future reinjection or <model_switch> behavior
  • make context-diff persistence + reference_context_item updates explicit in the regular-turn path, with clearer docs/comments around the invariant
  • stop persisting local /compact RolloutItem::TurnContext snapshots (only regular turns persist TurnContextItem now)
  • simplify resume/fork previous-model/reference-baseline hydration by looking up the last surviving turn context from rollout lifecycle events, including rollback and compaction-crossing handling
  • remove the legacy fallback that guessed from bare TurnContext rollouts without lifecycle events
  • update compaction/remote-compaction/model-visible snapshots and compact test assertions (including remote compaction mock response shape)

Why

We were persisting incoming context items before spawning the regular turn task, which let pre-turn compaction requests accidentally include incoming context diffs without the new user message. Fixing that exposed follow-on baseline issues around /compact, resume/fork, and standalone tasks that could cause duplicate context injection or suppress <model_switch> instructions.

This PR re-centers the invariants around regular turns:

  • regular turns persist model-visible context diffs/full reinjection and update the reference_context_item
  • standalone tasks do not advance those regular-turn baselines
  • compaction clears the baseline when replacement history may have stripped the referenced context diffs

Follow-ups (TODOs left in code)

  • TODO(ccunningham): fix rollback/backtracking baseline handling more comprehensively
  • TODO(ccunningham): include pending incoming context items in pre-turn compaction threshold estimation
  • TODO(ccunningham): inject updated personality spec alongside <model_switch> so some model-switch paths can avoid forced full reinjection
  • TODO(ccunningham): review task turn lifecycle (TurnStarted/TurnComplete) behavior and emit task-start context diffs for task types that should have them (excluding /compact)

Validation

  • just fmt
  • CI should cover the updated compaction/resume/model-visible snapshot expectations and rollout-hydration behavior
  • I did not rerun the full local test suite after the latest resume-lookup / rollout-persistence simplifications

Changed files

  • codex-rs/core/src/codex.rs (modified, +582/-131)
  • codex-rs/core/src/compact.rs (modified, +219/-239)
  • codex-rs/core/src/compact_remote.rs (modified, +82/-7)
  • codex-rs/core/src/context_manager/history.rs (modified, +11/-8)
  • codex-rs/core/src/context_manager/updates.rs (modified, +10/-9)
  • codex-rs/core/src/state/session.rs (modified, +14/-12)
  • codex-rs/core/src/tasks/compact.rs (modified, +4/-5)
  • codex-rs/core/src/tasks/mod.rs (modified, +0/-10)
  • codex-rs/core/src/tasks/undo.rs (modified, +2/-1)
  • codex-rs/core/src/tasks/user_shell.rs (modified, +4/-0)
  • codex-rs/core/tests/common/responses.rs (modified, +7/-5)
  • codex-rs/core/tests/suite/compact.rs (modified, +24/-14)
  • codex-rs/core/tests/suite/compact_remote.rs (modified, +20/-37)
  • codex-rs/core/tests/suite/compact_resume_fork.rs (modified, +139/-574)
  • codex-rs/core/tests/suite/model_visible_layout.rs (modified, +6/-2)
  • codex-rs/core/tests/suite/prompt_caching.rs (modified, +14/-14)
  • codex-rs/core/tests/suite/resume_warning.rs (modified, +24/-2)
  • codex-rs/core/tests/suite/snapshots/all__suite__compact__manual_compact_with_history_shapes.snap (modified, +5/-5)
  • codex-rs/core/tests/suite/snapshots/all__suite__compact__manual_compact_without_prev_user_shapes.snap (modified, +4/-4)
  • codex-rs/core/tests/suite/snapshots/all__suite__compact__pre_sampling_model_switch_compaction_shapes.snap (modified, +6/-7)
  • codex-rs/core/tests/suite/snapshots/all__suite__compact__pre_turn_compaction_including_incoming_shapes.snap (modified, +7/-8)
  • codex-rs/core/tests/suite/snapshots/all__suite__compact__pre_turn_compaction_strips_incoming_model_switch_shapes.snap (modified, +7/-8)
  • codex-rs/core/tests/suite/snapshots/all__suite__compact_remote__remote_manual_compact_with_history_shapes.snap (modified, +7/-9)
  • codex-rs/core/tests/suite/snapshots/all__suite__compact_remote__remote_mid_turn_compaction_multi_summary_reinjects_above_last_summary_shapes.snap (modified, +10/-13)
  • codex-rs/core/tests/suite/snapshots/all__suite__compact_remote__remote_mid_turn_compaction_shapes.snap (modified, +7/-7)
  • codex-rs/core/tests/suite/snapshots/all__suite__compact_remote__remote_mid_turn_compaction_summary_only_reinjects_context_shapes.snap (modified, +3/-3)
  • codex-rs/core/tests/suite/snapshots/all__suite__compact_remote__remote_pre_turn_compaction_including_incoming_shapes.snap (modified, +4/-5)
  • codex-rs/core/tests/suite/snapshots/all__suite__compact_remote__remote_pre_turn_compaction_strips_incoming_model_switch_shapes.snap (modified, +7/-7)
  • codex-rs/core/tests/suite/snapshots/all__suite__model_visible_layout__model_visible_layout_resume_override_matches_rollout_model.snap (modified, +3/-4)
  • codex-rs/core/tests/suite/snapshots/all__suite__model_visible_layout__model_visible_layout_resume_with_personality_change.snap (modified, +5/-8)
  • codex-rs/protocol/src/protocol.rs (modified, +4/-0)

PR #14313: Defer initial context insertion until the first turn

Description (problem / solution / changelog)

Summary

  • defer fresh-session build_initial_context() until the first real turn instead of seeding model-visible context during startup
  • rely on the existing reference_context_item == None turn-start path to inject full initial context on that first real turn (and again after baseline resets such as compaction)
  • add a regression test for InitialHistory::New and update affected deterministic tests / snapshots around developer-message layout, collaboration instructions, personality updates, and compact request shapes

Notes

  • this PR does not add any special empty-thread /compact behavior
  • most of the snapshot churn is the direct result of moving the initial model-visible context from startup to the first real turn, so first-turn request layouts no longer contain a pre-user startup copy of permissions / environment / other developer-visible context
  • remote manual /compact with no prior user still skips the remote compact request; local first-turn /compact still issues a compact request, but that request now reflects the lack of startup-seeded context

Changed files

  • codex-rs/core/src/codex.rs (modified, +2/-14)
  • codex-rs/core/src/codex_tests.rs (modified, +12/-0)
  • codex-rs/core/tests/suite/collaboration_instructions.rs (modified, +20/-26)
  • codex-rs/core/tests/suite/compact_remote.rs (modified, +4/-9)
  • codex-rs/core/tests/suite/personality.rs (modified, +1/-1)
  • codex-rs/core/tests/suite/snapshots/all__suite__compact__manual_compact_without_prev_user_shapes.snap (modified, +2/-5)
  • codex-rs/core/tests/suite/snapshots/all__suite__compact__pre_sampling_model_switch_compaction_shapes.snap (modified, +5/-7)
  • codex-rs/core/tests/suite/snapshots/all__suite__compact__pre_turn_compaction_strips_incoming_model_switch_shapes.snap (modified, +5/-7)
  • codex-rs/core/tests/suite/snapshots/all__suite__compact_remote__remote_compact_resume_restates_realtime_end_shapes.snap (modified, +6/-4)
  • codex-rs/core/tests/suite/snapshots/all__suite__compact_remote__remote_manual_compact_restates_realtime_start_shapes.snap (modified, +6/-4)
  • codex-rs/core/tests/suite/snapshots/all__suite__compact_remote__remote_manual_compact_without_prev_user_shapes.snap (modified, +2/-8)
  • codex-rs/core/tests/suite/snapshots/all__suite__compact_remote__remote_mid_turn_compaction_does_not_restate_realtime_end_shapes.snap (modified, +17/-14)
  • codex-rs/core/tests/suite/snapshots/all__suite__compact_remote__remote_pre_turn_compaction_restates_realtime_end_shapes.snap (modified, +6/-4)
  • codex-rs/core/tests/suite/snapshots/all__suite__compact_remote__remote_pre_turn_compaction_restates_realtime_start_shapes.snap (modified, +6/-4)
  • codex-rs/core/tests/suite/snapshots/all__suite__model_visible_layout__model_visible_layout_cwd_change_does_not_refresh_agents.snap (modified, +6/-9)
  • codex-rs/core/tests/suite/snapshots/all__suite__model_visible_layout__model_visible_layout_turn_overrides.snap (modified, +7/-8)
RAW_BUFFERClick to expand / collapse

What version of the Codex App are you using (From “About Codex” dialog)?

Version 26.429.61741 (2429)

What subscription do you have?

Pro

What platform is your computer?

Darwin 24.6.0 arm64 arm

What issue are you seeing?

Codex compaction appears to preserve active startup/context instruction packets inside payload.replacement_history as if they were ordinary conversation history. On the next turn, those same instruction surfaces are also supplied again through the normal active turn-context path.

This creates a large fixed post-compaction prompt floor before any meaningful new work is added. In one representative Codex Desktop session, the first post-compaction request started at 33,093 input tokens with only 3,456 cached, leaving about 29,637 uncached input.

This is not just “my local startup packet is large.” Larger local memory/skills payloads make the symptom easier to measure, but the underlying bug is independent of that size: active startup/context packets should not be retained in compacted history and then supplied again as active context. Any user with AGENTS/project instructions, developer context, plugins, skills, or memories can pay the same duplicated-context cost at smaller scale.

Impact is cost/headroom and possible steering risk: retained startup/context packets appear as history while current versions are also supplied through the active instruction path.

Representative session:

/Users/<user>/.codex/sessions/2026/05/05/rollout-2026-05-05T12-43-22-019df905-d546-7100-b619-4571226c3b53.jsonl

Representative cwd:

/Users/<user>/work/repo-with-agents

Representative compaction timestamp:

2026-05-05T17:22:37.986Z

In the type == "compacted" record:

  • payload.message == ""
  • replacement_history entries: 9
  • serialized replacement_history size: 98,519 bytes
  • retained developer text: about 58.9 KB
  • retained AGENTS wrapper text: about 16.1 KB
  • largest retained developer sections: memory about 26.6 KB, skills about 24.0 KB, app context about 3.5 KB

Token counts around compaction, from last_token_usage:

  • before compaction: 221,525 input, 220,032 cached
  • first nonzero post-compaction request: 33,093 input, 3,456 cached
  • first post-compaction uncached input: about 29,637

Redacted proof shape:

  • payload.replacement_history[N].role == "user" contains # AGENTS.md instructions...
  • the normalized/hash of that AGENTS body matches the adjacent turn_context.user_instructions
  • payload.replacement_history[M].role == "developer" contains <app-context>, ## Memory, and <skills_instructions>
  • the adjacent post-compaction turn_context also contains current active context surfaces

I can provide exact redacted hashes privately if useful.

What steps can reproduce the bug?

  1. Start a Codex Desktop thread in a repo with an AGENTS.md.
  2. Use normal startup/context surfaces, especially skills and/or memories if available.
  3. Drive the thread to compaction.
  4. Inspect the session JSONL type == "compacted" event.
  5. Check whether payload.replacement_history contains:
    • a user item with # AGENTS.md instructions...
    • a developer item containing <app-context>, ## Memory, <skills_instructions>, or plugin/app instructions
  6. Inspect the adjacent turn_context.
  7. Compare whether the AGENTS body is also present in turn_context.user_instructions.
  8. Inspect the first post-compaction event_msg / token_count.
  9. Confirm whether post-compaction input remains high before new work is added.

Focused scan:

  • window: 2026-04-28T00:00:00Z through 2026-05-05T23:59:59Z
  • 70 compactions across 34 session files
  • 63/70 compactions retained developer/app-context packets
  • 63/70 retained # AGENTS.md instructions... wrappers
  • 63/63 normal compactions with an AGENTS wrapper contained the same body already present in turn_context.user_instructions
  • 63/70 retained skills instruction markers
  • 36/70 retained memory sections
  • replacement history size: median 107,290 bytes, p95 1,130,774, max 1,470,356
  • first post-compaction input tokens: median 32,514, max 90,411

Broader scan across all available local sessions:

  • 1,015 compaction events across 479 session files
  • retained AGENTS/developer packet pattern observed across every week with compactions in the local corpus
  • Desktop: 938/986 compactions included retained AGENTS wrappers and developer packets
  • TUI: 22/29 compactions included retained AGENTS wrappers and developer packets
  • earliest retained AGENTS sample observed: 2026-02-27T21:18:49.602Z, Codex Desktop 0.105.0, cwd /Users/<user>/work/another-repo-with-agents

The retention pattern predates my recent local startup/config setup. Later memory/skills configuration amplified the duplicated payload size but did not create the underlying compaction behavior.

What is the expected behavior?

After compaction, replacement_history should contain compacted conversational history and/or the compaction summary.

It should not retain raw current startup/context packets when those same surfaces are already supplied through active turn context on the next request.

After compaction:

  • replacement_history does not contain raw active AGENTS/developer/memory/skills/app/plugin startup packets
  • active startup/context surfaces are still present exactly once in the next model request
  • first post-compaction prompt size reflects compacted conversation state plus one current context injection, not duplicated startup/context payload
  • disabling memories/skills may reduce baseline size, but should not be required to avoid duplication

Additional information

Related prior work:

  • #10574: https://github.com/openai/codex/pull/10574

    Closely related. It handled stale developer-role messages in compacted replacement_history and reinjected fresh developer instructions. This report shows active startup/context packets still being retained through another path or broader surface area, including contextual user wrappers such as AGENTS.

  • #12252: https://github.com/openai/codex/pull/12252

    Possibly related. It addressed compaction context reinjection and duplicate context concerns. This report is specifically about post-compaction replacement_history retaining active startup/context surfaces.

  • #14313: https://github.com/openai/codex/pull/14313

    Possibly related. It deferred fresh-session initial context insertion until the first real turn.

  • #14365: https://github.com/openai/codex/issues/14365

    Related open issue about remote compaction preserving too much replacement history and failing to create meaningful headroom.

Local mitigation tests:

Measured with:

codex debug prompt-input 'probe'

From /Users/<user>/work/repo-with-agents:

  • default: 76,270 serialized prompt bytes
  • -c features.memories=false: 49,302 bytes
  • -c memories.use_memories=false: 49,302 bytes
  • -c skills.include_instructions=false: 52,097 bytes

These mitigations reduce the size of my local duplicated payload. They do not address the general bug: active context should not be copied into replacement_history and re-sent as history for any user.

I can provide the representative JSONL privately if useful. I am not attaching the full session publicly because it contains local paths and model-visible context, but the event paths, timestamps, counts, and reproduction checks above are exact.

extent analysis

TL;DR

The most likely fix for the issue is to modify the compaction logic to exclude active startup/context packets from being retained in replacement_history when they are already supplied through active turn context.

Guidance

  • Review the compaction logic to identify why active startup/context packets are being retained in replacement_history despite being supplied through active turn context.
  • Verify that the replacement_history entries are correctly filtered to exclude duplicate context surfaces.
  • Investigate the related prior work (e.g., #10574, #12252, #14313) to determine if similar issues were addressed and how they might apply to this case.
  • Consider implementing a mitigation similar to the local tests performed, such as disabling memories or skills, to reduce the size of the duplicated payload, but note that this does not address the underlying bug.

Example

No code snippet is provided as the issue does not contain sufficient information to create a specific example.

Notes

The issue appears to be related to the compaction logic and how it handles active startup/context packets. The provided information suggests that the bug is independent of the size of the local memory/skills payloads, but the size can amplify the duplicated payload size.

Recommendation

Apply a workaround by modifying the compaction logic to exclude active startup/context packets from replacement_history when they are already supplied through active turn context, as this is the most direct way to address the issue.

Vote matrix · Quick signals

Works
Did the solution work? Tap to confirm.
Easy Fix
Was it a quick fix?
Time Saver
Did it save you time?
Blocking
Was it severely blocking?
Common Issue
Are others likely hitting this too?
Flaky / Intermittent
Is it intermittent?
Verified / Reproducible
Can you reproduce it reliably?
Loading…
#api#ssr#conversation history#dependency error#configuration error

Still need to ship something?

×6

Another batch ranked right after the header list — different links, same matching logic.

Back to top recommendations

TRENDING