Issue Station
Sign In
Tools hereToolbox

openclaw - 💡(How to fix) Fix [Feature]: Built-in headless browser for reliable web access without external dependencies [1 comments, 2 participants]

Official PRs (…)
ON THIS PAGE

Recommended Tools

×6

Utilities matched from this issue’s tags and category — try them while you read without losing context.

GitHub issue graph ai analysis

Paste a GitHub issue URL. We fetch that issue, discover linked issues from bodies/comments/timeline, collect linked pull requests, and produce a structured English report.

The report is written in English Markdown for sharing and archival.

Helpful · Quick feedback

Loading…
GitHub stats
openclaw/openclaw#53763•Fetched 2026-04-08 01:23:43
View on GitHub
Comments
1
Participants
2
Timeline
2
Reactions
0
Author
luoziyan100luoziyan100
Participants
luoziyan100luoziyan100mrkeyianomrkeyiano
Timeline (top)
commented ×1labeled ×1

Bundle a headless Chromium instance as a first-class OpenClaw tool, so agents can reliably access any webpage (including JS-rendered and login-required pages) without depending on the user's Chrome or third-party APIs.

Root Cause

Bundle a headless Chromium instance as a first-class OpenClaw tool, so agents can reliably access any webpage (including JS-rendered and login-required pages) without depending on the user's Chrome or third-party APIs.

Fix Action

Fix / Workaround

Affected: All agents that need web access (every OpenClaw user) Severity: Blocks workflow — agents frequently fail to retrieve web content, requiring manual intervention Frequency: Daily — every session involving web search or page scraping hits at least one failure Consequence: +10-30 minutes/day of manual workarounds (copy-pasting content, retrying with different tools, opening Chrome for bb-browser)

  • web_fetch on zsxq.com: returns empty content (JS-rendered SPA)
  • web_fetch on 小红书: blocked by anti-scraping
  • Tavily extract on WeChat articles: intermittent failures
  • bb-browser: requires Chrome open + extension attached, fails ~30% of the time on dynamic pages
  • Related issues: #52190 (Patchright stealth mode), #14803 (browser launch flags)
  • Prior art: Claude Code's web-access skill uses CDP proxy to solve this same problem
RAW_BUFFERClick to expand / collapse

Summary

Bundle a headless Chromium instance as a first-class OpenClaw tool, so agents can reliably access any webpage (including JS-rendered and login-required pages) without depending on the user's Chrome or third-party APIs.

Problem to solve

Agents currently rely on a fragile three-layer stack for web access:

  1. web_fetch — no JS execution, fails on dynamic pages
  2. Third-party APIs (Tavily) — search quality varies, rate-limited, costs money
  3. bb-browser / Chrome relay — requires user's Chrome running + extension attached, slow and unstable

Real-world pain points:

  • Social media (zsxq.com) needs login + JS rendering — only bb-browser works, and it's flaky
  • WeChat articles sometimes fail on both web_fetch and Tavily
  • Dynamic SPAs return empty content through web_fetch
  • bb-browser breaks entirely on headless/server deployments

Every layer is "borrowing someone else's eyes." No tool gives the agent direct, reliable web access.

Proposed solution

Bundle a headless Chromium (via Playwright or Puppeteer) accessible through the existing browser tool interface:

  • Runs in background, no dependency on user's Chrome
  • Full JS execution, cookie/session persistence
  • Proxy support, user-agent rotation
  • Works in all deployments: macOS, Linux, Docker, headless servers

Alternatives considered

No response

Impact

Affected: All agents that need web access (every OpenClaw user) Severity: Blocks workflow — agents frequently fail to retrieve web content, requiring manual intervention Frequency: Daily — every session involving web search or page scraping hits at least one failure Consequence: +10-30 minutes/day of manual workarounds (copy-pasting content, retrying with different tools, opening Chrome for bb-browser)

Evidence/examples

  • web_fetch on zsxq.com: returns empty content (JS-rendered SPA)
  • web_fetch on 小红书: blocked by anti-scraping
  • Tavily extract on WeChat articles: intermittent failures
  • bb-browser: requires Chrome open + extension attached, fails ~30% of the time on dynamic pages
  • Related issues: #52190 (Patchright stealth mode), #14803 (browser launch flags)
  • Prior art: Claude Code's web-access skill uses CDP proxy to solve this same problem

Additional information

No response

extent analysis

Fix Plan

To implement a headless Chromium instance as a first-class OpenClaw tool, follow these steps:

  • Step 1: Choose a Library
    • Select either Playwright or Puppeteer for browser automation.
    • For this example, we'll use Puppeteer.
  • Step 2: Install Dependencies
    • Install Puppeteer using npm: npm install puppeteer
  • Step 3: Implement Browser Launch
    • Launch a headless Chromium instance using Puppeteer:
const puppeteer = require('puppeteer');

(async () => {
  const browser = await puppeteer.launch({
    headless: true,
    args: ['--no-sandbox', '--disable-setuid-sandbox'],
  });
  // ...
})();
  • Step 4: Implement Page Navigation and Content Retrieval
    • Create a new page and navigate to the desired URL:
const page = await browser.newPage();
await page.goto('https://example.com');
const content = await page.content();
// ...
  • Step 5: Handle Login and Session Persistence
    • Use Puppeteer's built-in support for cookies and sessions:
await page.setCookie({
  name: 'session_id',
  value: 'your_session_id',
  domain: 'example.com',
});
  • Step 6: Integrate with Existing Browser Tool Interface
    • Modify the existing browser tool interface to use the headless Chromium instance:
// Replace existing web_fetch implementation with headless Chromium
const webFetch = async (url) => {
  const page = await browser.newPage();
  await page.goto(url);
  const content = await page.content();
  return content;
};

Verification

To verify that the fix worked:

  • Test the headless Chromium instance with various URLs, including those that previously failed with web_fetch or third-party APIs.
  • Verify that the instance can handle login and session persistence correctly.
  • Monitor the instance's performance and reliability in different deployments (macOS, Linux, Docker, headless servers).

Extra Tips

  • Ensure that the headless Chromium instance is properly configured for proxy support and user-agent rotation to avoid anti-scraping measures.
  • Consider implementing additional error handling and logging mechanisms to improve the instance's reliability and debuggability.
  • Refer to the Puppeteer documentation for more information on advanced features and configuration options.

Vote matrix · Quick signals

Works
Did the solution work? Tap to confirm.
Easy Fix
Was it a quick fix?
Time Saver
Did it save you time?
Blocking
Was it severely blocking?
Common Issue
Are others likely hitting this too?
Flaky / Intermittent
Is it intermittent?
Verified / Reproducible
Can you reproduce it reliably?
Loading…
#api#training loop#device allocation#model download#tokenizer error

Still need to ship something?

×6

Another batch ranked right after the header list — different links, same matching logic.

Back to top recommendations

TRENDING

openclaw - 💡(How to fix) Fix [Feature]: Built-in headless browser for reliable web access without external dependencies [1 comments, 2 participants]