Issue Station
Sign In
Tools hereToolbox

openclaw - ✅(Solved) Fix [Bug]: browser snapshot --format ai returns role refs (e1, e2, ...) instead of numeric AI refs on 2026.4.5 [2 pull requests, 1 participants]

Official PRs (…)
ON THIS PAGE

Recommended Tools

×6

Utilities matched from this issue’s tags and category — try them while you read without losing context.

GitHub issue graph ai analysis

Paste a GitHub issue URL. We fetch that issue, discover linked issues from bodies/comments/timeline, collect linked pull requests, and produce a structured English report.

The report is written in English Markdown for sharing and archival.

Helpful · Quick feedback

Loading…
GitHub stats
openclaw/openclaw#62550•Fetched 2026-04-08 03:02:42
View on GitHub
Comments
0
Participants
1
Timeline
2
Reactions
0
Author
jeugreggjeugregg
Participants
jeugreggjeugregg
Timeline (top)
cross-referenced ×1labeled ×1

The docs say AI snapshots should use numeric refs, while role snapshots use e-prefixed refs. On OpenClaw 2026.4.5, openclaw browser snapshot --format ai --json returns format: "ai" but the snapshot and refs object still use e-prefixed role refs with role/name/nth metadata.

Root Cause

I am debugging browser targeting issues on a page with duplicate/near-identical buttons. I need to compare AI snapshot resolution against role snapshot resolution, but on this build --format ai appears to fall back to role-style refs even though the JSON still says "format": "ai".

PR fix notes

PR #62607: fix(browser): use numeric refs for --format ai snapshots instead of role refs

Description (problem / solution / changelog)

Summary

Fixes #62550

Running openclaw browser snapshot --format ai reported "format": "ai" in JSON output but the snapshot text still used e-prefixed role refs ([e1], [e2], ...) instead of numeric refs ([1], [2], ...).

The refs object also contained role-mode metadata (role, name, nth) instead of the expected numeric keys.

Fix: In snapshotAiViaPlaywright(), remap e-prefixed refs to numeric refs after building the snapshot:

  • e1 → 1, e2 → 2, etc.
  • Replace [ref=e1] with [1] in snapshot text
  • Return remapped refs object with numeric keys

Test plan

  • Updated e2e test to expect numeric refs (1, 2) instead of role refs (e1, e2)
  • Test verifies snapshot text contains [1] and [2] (not [ref=e1])
  • Internal aria-ref resolution still works with e-prefixed refs (click via ref)

Changed files

  • extensions/browser/src/browser/pw-ai.e2e.test.ts (modified, +7/-2)
  • extensions/browser/src/browser/pw-tools-core.snapshot.ts (modified, +17/-1)

PR #64123: Browser: keep explicit ai snapshots out of efficient role fallback (#62550)

Description (problem / solution / changelog)

Summary

  • Problem: browser snapshot --format ai could still return role-style e-prefixed refs when caller defaults injected mode=efficient, because the CLI and Browser tool kept applying browser.snapshotDefaults.mode=efficient even after the user explicitly requested format=ai.
  • Why it matters: this made explicit AI snapshot requests look like silent fallbacks to role snapshots, which matches the confusing behavior reported in #62550.
  • What changed: explicit format=ai now opts out of config-driven efficient defaults in both the CLI and Browser tool entrypoints, so callers only get role-style refs when they explicitly ask for efficient/interactive/compact behavior. This branch also keeps the additive numeric AI ref parsing compatibility for Playwright AI snapshots.
  • Regression coverage: targeted tests now lock in that explicit AI snapshots do not inherit efficient defaults and that numeric AI refs remain visible in the public snapshot output and usable for follow-up actions.

Change Type (select all)

  • Bug fix
  • Feature
  • Refactor required for the fix
  • Docs
  • Security hardening
  • Chore/infra

Scope (select all touched areas)

  • Gateway / orchestration
  • Skills / tool execution
  • Auth / tokens
  • Memory / storage
  • Integrations
  • API / contracts
  • UI / DX
  • CI/CD / infra

Linked Issue/PR

  • Closes #62550
  • Related #64123
  • This PR fixes a bug or regression

Root Cause (if applicable)

  • Root cause: config-driven browser.snapshotDefaults.mode=efficient was still being applied after callers explicitly requested format=ai, and efficient intentionally routes through the role snapshot path with e-prefixed refs.
  • Missing detection / guardrail: tests covered config defaults and numeric ref parsing separately, but did not lock in that an explicit AI format request must override implicit efficient defaults.
  • Contributing context (if known): because the response still reported format: ai, the fallback looked like an AI snapshot regression instead of a default-mode override.

Regression Test Plan (if applicable)

  • Coverage level that should have caught this:
    • Unit test
    • Seam / integration test
    • End-to-end test
    • Existing coverage already sufficient
  • Target tests/files:
    • extensions/browser/src/cli/browser-cli-inspect.test.ts
    • extensions/browser/src/browser-tool.test.ts
    • extensions/browser/src/browser/pw-ai.e2e.test.ts
  • Scenarios locked in:
    • explicit --format ai does not inherit config-driven mode=efficient
    • explicit Browser tool snapshotFormat: ai does not inherit config-driven mode=efficient
    • numeric AI refs remain visible in the public snapshot output and still work for follow-up actions
  • Why this is the smallest reliable guardrail: the bug lives at the entrypoints that assemble snapshot requests plus the AI snapshot response path, so focused CLI/tool tests plus one Playwright seam test cover the failure without requiring a live browser run.

User-visible / Behavior Changes

  • Explicit AI snapshot requests now stay AI snapshots unless the caller also explicitly asks for efficient/role-style behavior.
  • Numeric Playwright AI refs remain accepted and usable for follow-up actions.

Security Impact (required)

  • New permissions/capabilities? (No)
  • Secrets/tokens handling changed? (No)
  • New/changed network calls? (No)
  • Command/tool execution surface changed? (No)
  • Data access scope changed? (No)

Repro + Verification

Environment

  • OS: Windows 10
  • Runtime/container: Node 24 repo test entrypoint
  • Model/provider: N/A
  • Integration/channel (if any): browser plugin / Playwright AI snapshots
  • Relevant config (redacted): simulated browser.snapshotDefaults.mode=efficient

Steps

  1. Configure browser.snapshotDefaults.mode=efficient.
  2. Request openclaw browser snapshot --format ai (or Browser tool snapshotFormat: ai) without an explicit mode.
  3. Verify the request/query no longer inherits mode=efficient.
  4. Feed snapshotAiViaPlaywright() a numeric-ref AI snapshot and verify the returned snapshot text and follow-up action path.

Expected

  • Explicit AI snapshot requests do not silently fall back to role snapshots due to config defaults.
  • Numeric AI refs remain visible in the returned snapshot text and usable in follow-up actions.

Actual before fix

  • Explicit AI snapshot requests could still inherit mode=efficient, producing role-style refs while still reporting format: ai.

Evidence

  • Failing test/log before + passing after
  • Trace/log snippets
  • Screenshot/recording
  • Perf numbers (if relevant)

Human Verification (required)

  • Verified scenarios: ran targeted regression tests for the CLI snapshot defaults, Browser tool snapshot defaults, and Playwright AI snapshot response path.
  • Commands run: pnpm test extensions/browser/src/browser/pw-ai.e2e.test.ts extensions/browser/src/cli/browser-cli-inspect.test.ts extensions/browser/src/browser-tool.test.ts
  • Extra environment note: the PR worktree initially lacked dependencies (vitest/package.json missing), so I ran pnpm install and retried the exact test command before finishing verification.

Compatibility / Migration

  • Backward compatible? (Yes)
  • Config/env changes? (No)
  • Migration needed? (No)

Risks and Mitigations

  • Risk: callers that implicitly relied on config default efficient even when explicitly asking for format=ai will now get true AI snapshots instead of role snapshots.
    • Mitigation: that new behavior matches the explicit caller intent; callers that want role-style snapshots can still request --efficient, --interactive, --compact, --depth, or equivalent tool inputs explicitly.

Changed files

  • extensions/browser/src/browser-tool.actions.ts (modified, +7/-4)
  • extensions/browser/src/browser-tool.test.ts (modified, +15/-1)
  • extensions/browser/src/browser/pw-ai.e2e.test.ts (modified, +29/-0)
  • extensions/browser/src/browser/pw-role-snapshot.test.ts (modified, +15/-1)
  • extensions/browser/src/browser/pw-role-snapshot.ts (modified, +14/-3)
  • extensions/browser/src/cli/browser-cli-inspect.test.ts (modified, +6/-1)
  • extensions/browser/src/cli/browser-cli-inspect.ts (modified, +5/-1)

Code Example

openclaw browser navigate https://app.lighter.xyz/trade/BTC

---

openclaw browser snapshot --format ai

---

openclaw browser --json snapshot --format ai

---

openclaw browser snapshot --format ai

---

openclaw browser --json snapshot --format ai

---



---

{
  "ok": true,
  "format": "ai",
  "url": "https://my_website",
  "snapshot": "- document:\n  - link \"My Website\" [ref=e1]\n  - link \"Tools1\" [ref=e2]\n  - button \"Tools2\" [ref=e6]\n  - button \"Tools3\" [ref=e20]\n  ...",
  "refs": {
    "e20": {
      "role": "button",
      "name": "Tools3"
    },
    "e27": {
      "role": "button",
      "nth": 1
    }
  }
}
RAW_BUFFERClick to expand / collapse

Bug type

Regression (worked before, now fails)

Beta release blocker

No

Summary

The docs say AI snapshots should use numeric refs, while role snapshots use e-prefixed refs. On OpenClaw 2026.4.5, openclaw browser snapshot --format ai --json returns format: "ai" but the snapshot and refs object still use e-prefixed role refs with role/name/nth metadata.

Steps to reproduce

  1. Open a page, for example: 
    openclaw browser navigate https://app.lighter.xyz/trade/BTC
  2. Run: 
    openclaw browser snapshot --format ai
  3. Or run: 
    openclaw browser --json snapshot --format ai
  4. Observe that the output still uses e-prefixed refs.

Expected behavior

When running:

openclaw browser snapshot --format ai

or

openclaw browser --json snapshot --format ai

the snapshot should use numeric refs for AI mode, not role refs. [web:164][web:111]

Actual behavior

The command reports "format": "ai" in JSON output, but the snapshot still contains refs like e1, e2, etc., and the refs object looks like role-mode output with fields such as role, name, and nth.

OpenClaw version

2026.4.5

Operating system

Ubuntu 24

Install method

No response

Model

n/a

Provider / routing chain

n/a

Additional provider/model setup details

No response

Logs, screenshots, and evidence

Impact and severity

No response

Additional information

Example JSON output

{
  "ok": true,
  "format": "ai",
  "url": "https://my_website",
  "snapshot": "- document:\n  - link \"My Website\" [ref=e1]\n  - link \"Tools1\" [ref=e2]\n  - button \"Tools2\" [ref=e6]\n  - button \"Tools3\" [ref=e20]\n  ...",
  "refs": {
    "e20": {
      "role": "button",
      "name": "Tools3"
    },
    "e27": {
      "role": "button",
      "nth": 1
    }
  }
}

Why this matters

I am debugging browser targeting issues on a page with duplicate/near-identical buttons. I need to compare AI snapshot resolution against role snapshot resolution, but on this build --format ai appears to fall back to role-style refs even though the JSON still says "format": "ai".

Notes

  • This does not appear to be a shell alias or wrong binary issue.
  • The installed binary is the expected npm binary at /root/.npm-global/bin/openclaw.
  • The docs currently describe AI snapshots as numeric-ref based, and role snapshots as e-prefixed. [web:164][web:111]

extent analysis

TL;DR

The issue can be mitigated by verifying the OpenClaw configuration and ensuring that the --format ai option is correctly implemented to use numeric refs instead of e-prefixed role refs.

Guidance

  • Review the OpenClaw documentation and code to ensure that the --format ai option is correctly implemented to use numeric refs.
  • Verify that the refs object in the JSON output is correctly formatted for AI mode, without role, name, and nth fields.
  • Check the OpenClaw version and ensure that it is up-to-date, as the issue may be related to a regression in version 2026.4.5.
  • Test the --format ai option with different inputs and scenarios to ensure that it consistently produces the expected output.

Example

No code snippet is provided as the issue is related to the OpenClaw binary and its configuration.

Notes

The issue may be related to a regression in the OpenClaw binary, and further investigation is needed to determine the root cause. The documentation and code review may help identify the issue and provide a solution.

Recommendation

Apply workaround: Use a different version of OpenClaw or modify the configuration to ensure that the --format ai option uses numeric refs correctly, as the current version 2026.4.5 appears to have a regression.

Vote matrix · Quick signals

Works
Did the solution work? Tap to confirm.
Easy Fix
Was it a quick fix?
Time Saver
Did it save you time?
Blocking
Was it severely blocking?
Common Issue
Are others likely hitting this too?
Flaky / Intermittent
Is it intermittent?
Verified / Reproducible
Can you reproduce it reliably?
Loading…

FAQ

Expected behavior

When running:

openclaw browser snapshot --format ai

or

openclaw browser --json snapshot --format ai

the snapshot should use numeric refs for AI mode, not role refs. [web:164][web:111]

#tokenizer error#prompt formatting#chain error#conversation history#tool integration

Still need to ship something?

×6

Another batch ranked right after the header list — different links, same matching logic.

Back to top recommendations

TRENDING

openclaw - ✅(Solved) Fix [Bug]: browser snapshot --format ai returns role refs (e1, e2, ...) instead of numeric AI refs on 2026.4.5 [2 pull requests, 1 participants]