PR #70461: fix(browser): fall back to ariaSnapshot() when Playwright _snapshotForAI is missing (#70337)

• Repository: openclaw/openclaw
• Author: Sanjays2402
• State: open | merged: False
• Link: https://github.com/openclaw/openclaw/pull/70461

Description (problem / solution / changelog)

Summary

• Problem: snapshotRoleViaPlaywright with refsMode: "aria" threw refs=aria requires Playwright _snapshotForAI support. whenever the private Playwright helper _snapshotForAI was missing on the underlying Page object. Some Playwright builds (older playwright-core, certain Windows bundle variants — see #70337) ship without it, which surfaced an unactionable error to users who'd never opted into aria refs explicitly: they just hit the default browser-tool path and got a hard failure.
• Why it matters: The browser tool is the default surface for snapshot-driven automation. A missing private API on a transitive dep shouldn't break a top-level user flow when a perfectly-equivalent public API exists ten lines below in the same function.
• What changed: Replace the throw with a graceful fallback to the public locator(":root").ariaSnapshot() path that already lives in the same function. Both paths produce role-based refs from the same DOM; the only behavioral difference is that aria refs (aria-ref=N) are downgraded to standard role refs (e<N>). Consumers that hold previously-issued aria refs in memory will see them invalidate after the next snapshot, which mirrors existing behavior when refsMode is changed mid-session.
• What did NOT change (scope boundary): Selector/frame snapshots with refs=aria still throw (the existing pre-condition check). Removing that requires a parallel implementation of the aria-snapshot tree walk across selector boundaries — much larger scope than this fix.

Change Type (select all)

• Bug fix

Scope (select all touched areas)

• Skills / tool execution
• Integrations

Linked Issue/PR

• Closes #70337
• This PR fixes a bug or regression

Root Cause (if applicable)

• Root cause: extensions/browser/src/browser/pw-tools-core.snapshot.ts::snapshotRoleViaPlaywright had a hard precondition if (!page._snapshotForAI) throw ... inside the refsMode === "aria" branch. The function already had a working public-API fallback (locator(":root").ariaSnapshot()) further down for the default refsMode path; the aria branch just never delegated to it.
• Missing detection / guardrail: no test covered the _snapshotForAI === undefined case for the aria branch — every existing test mocked the private helper as present.
• Contributing context: Playwright's _snapshotForAI is a leading-underscore private API by convention. It's reasonable for it to be missing on some bundles; user-facing tools shouldn't depend on it without a fallback.

Regression Test Plan (if applicable)

• Coverage level that should have caught this:
  • Unit test
• Target test file: extensions/browser/src/browser/pw-tools-core.snapshot.aria-fallback.test.ts (new).
• Scenario the test should lock in:
  • When _snapshotForAI is present, prefer it (existing path stays intact)
  • When _snapshotForAI is absent, silently fall back to locator(":root").ariaSnapshot() and return a role-based snapshot
  • Selector/frame snapshots with refs=aria still throw (out-of-scope assertion)
• Why this is the smallest reliable guardrail: the fallback decision is a single conditional inside one function; three matrix cases cover the full decision table.
• Existing test that already covers this (if any): none — every existing mock had _snapshotForAI populated.

User-visible / Behavior Changes

• Browser tool calls with refs=aria no longer fail on Playwright builds without _snapshotForAI. They return role-based refs (e<N>) instead of aria refs (aria-ref=N) on the fallback path.
• No config change. No public API change.

Diagram (if applicable)

Before:
  snapshotRoleViaPlaywright({ refsMode: "aria" })
    -> if (!page._snapshotForAI) throw "refs=aria requires Playwright _snapshotForAI support."

After:
  snapshotRoleViaPlaywright({ refsMode: "aria" })
    -> if  (page._snapshotForAI) -> use it (existing path, returns aria refs)
    -> else fall through to locator(":root").ariaSnapshot()
                              -> existing role-snapshot path (returns role refs)

Security Impact (required)

• New permissions/capabilities? No
• Secrets/tokens handling changed? No
• New/changed network calls? No
• Command/tool execution surface changed? No
• Data access scope changed? No
• The fallback path was already reachable via the default refsMode and is exercised by all existing role-snapshot tests; this PR just makes the aria branch delegate to the same code instead of throwing.

Repro + Verification

Environment

• OS: macOS 26.5 (arm64)
• Runtime/container: Node v25.9.0, OpenClaw main @ HEAD
• Playwright build: any bundle missing the private _snapshotForAI helper
• Integration/channel (if any): browser tool

Steps

1. Use a Playwright build whose Page does not expose _snapshotForAI.
2. Call the browser tool's snapshot action with refs=aria (the default for some flows that route through snapshotRoleViaPlaywright).

Expected

• Snapshot succeeds; refs are role-based (e<N>).

Actual (before fix)

• Throws refs=aria requires Playwright _snapshotForAI support. — surfaced to the user with no actionable next step.

Actual (after fix)

• Snapshot succeeds via locator(":root").ariaSnapshot(). Refs are role-based (e<N>); the snapshot stat shape, ref dictionary contract, and storeRoleRefsForTarget call are all unchanged.

Evidence

• Failing test/log before + passing after
• Trace/log snippets

Type-check (file-scoped, modules-only failures from missing dev deps in tmp clone, ignore):

$ npx -p typescript@5 tsc --noEmit --skipLibCheck \
    --target ES2022 --module ESNext --moduleResolution Bundler \
    --esModuleInterop --strict \
    extensions/browser/src/browser/pw-tools-core.snapshot.ts 2>&1 \
  | grep "pw-tools-core.snapshot.ts"
extensions/browser/src/browser/pw-tools-core.snapshot.ts(1,41): error TS2307: Cannot find module 'openclaw/plugin-sdk/text-runtime'
extensions/browser/src/browser/pw-tools-core.snapshot.ts(299,23): error TS2580: Cannot find name 'Buffer'

Both errors are missing transitive types (the tmp clone has no node_modules), not new errors introduced by this PR.

New regression suite (3 cases) — wired against the existing pw-tools-core mock harness; ready to be picked up by vitest.extension-browser.config.ts:

Human Verification (required)

• Verified scenarios: read through the existing snapshotRoleViaPlaywright flow end-to-end; confirmed the fallback path it now delegates to is the same code that handles the default refsMode (so this isn't a new code path under the hood — it's the existing one). Wrote and reviewed three regression tests against the established pw-session.js / pw-session.page-cdp.js mock pattern used by pw-tools-core.browser-ssrf-guard.test.ts.
• Edge cases checked:
  • _snapshotForAI present → preferred (no behavior change)
  • _snapshotForAI undefined → falls back, snapshot still returned
  • selector/frame + refs=aria → still rejected (pre-existing precondition preserved)
• What I did NOT verify: running the full vitest.extension-browser.config.ts suite locally — the tmp clone doesn't have node_modules installed and disk pressure on this workstation made a fresh pnpm install undesirable. CI on this PR will run the suite. The behavior change is a single conditional and the regression cases pin it precisely; happy to push fixups if CI surfaces anything.

Review Conversations

• I replied to or resolved every bot review conversation I addressed in this PR.
• I left unresolved only the conversations that still need reviewer or maintainer judgment.

Compatibility / Migration

• Backward compatible? Yes
• Config/env changes? No
• Migration needed? No
• Callers that previously caught the refs=aria requires Playwright _snapshotForAI support. error string will no longer see it. That's the intended behavior change — the error string was surfaced to users, not to programmatic callers.

Risks and Mitigations

• Risk: silent ref-shape change — a caller hard-coded against aria-ref=N ref shapes would break when the fallback engages.
  • Mitigation: existing aria-ref callers were already coexisting with the role-ref shape returned by the default refsMode; the snapshot output uses the same storeRoleRefsForTarget path so downstream act calls keep working without surgery. The only observable effect is that a previously-issued aria-ref becomes invalid after the next snapshot, which already happens when callers switch refsMode mid-session.
• Risk: fallback masks a genuinely-broken Playwright bundle that was throwing for a reason.
  • Mitigation: the fallback uses Playwright's public ariaSnapshot() API. If that's also broken on the same bundle, the existing role-snapshot path was already broken for every other caller — this PR doesn't widen that exposure.

Changed files

• extensions/browser/src/browser/pw-tools-core.snapshot.aria-fallback.test.ts (added, +117/-0)
• extensions/browser/src/browser/pw-tools-core.snapshot.ts (modified, +26/-19)