openclaw - ✅(Solved) Fix [Bug]: web_fetch tool fails to fetch websites when HTTP/HTTPS proxy is enabled [1 pull requests, 2 comments, 3 participants]

Official PRs (…)
ON THIS PAGE

Recommended Tools

×6

Utilities matched from this issue’s tags and category — try them while you read without losing context.

GitHub issue graph ai analysis

Paste a GitHub issue URL. We fetch that issue, discover linked issues from bodies/comments/timeline, collect linked pull requests, and produce a structured English report.

The report is written in English Markdown for sharing and archival.

Helpful · Quick feedback

Loading…
GitHub stats
openclaw/openclaw#49948Fetched 2026-04-08 01:00:56
View on GitHub
Comments
2
Participants
3
Timeline
23
Reactions
0
Timeline (top)
referenced ×15commented ×2cross-referenced ×2labeled ×2

web_fetch 在代理环境下无法获取某些网站

Root Cause

web_fetch 在代理环境下无法获取某些网站

Fix Action

Fixed

PR fix notes

PR #50650: fix(infra/net): use EnvHttpProxyAgent when proxy env vars are configured

Description (problem / solution / changelog)

Problem

When HTTPS_PROXY / HTTP_PROXY environment variables are set, web_fetch (and other tools using fetchWithSsrFGuard without an explicit dispatcherPolicy.mode) fail with fetch failed or getaddrinfo EAI_AGAIN. The pinned dispatcher connects directly to the DNS-resolved IP, bypassing the proxy entirely.

This breaks OpenClaw in:

  • OpenShell/NemoClaw sandboxes — DNS is blocked in the network namespace, and direct outbound TCP is only routable through the proxy
  • Docker containers with proxy-only network access
  • Corporate networks requiring proxy for all HTTPS traffic

The TRUSTED_ENV_PROXY mode (#45248) works correctly but requires explicit opt-in. The default STRICT mode always uses createPinnedDispatcher in direct mode, which bypasses the proxy.

Fix

Two files changed.

src/infra/net/fetch-guard.ts — When hasEnvHttpProxyConfigured() returns true and the caller hasn't already specified a dispatcherPolicy.mode or allowed private networks, use createPinnedDispatcher with mode: "env-proxy" instead of the default direct mode:

+import { hasEnvHttpProxyConfigured, hasProxyEnvConfigured } from "./proxy-env.js";
-import { hasProxyEnvConfigured } from "./proxy-env.js";

       } else if (params.pinDns !== false) {
-        dispatcher = createPinnedDispatcher(pinned, params.dispatcherPolicy, params.policy);
+        const protocol = parsedUrl.protocol === "http:" ? "http" : "https";
+        const useEnvProxy =
+          hasEnvHttpProxyConfigured(protocol) &&
+          !params.dispatcherPolicy?.mode &&
+          !params.policy?.allowPrivateNetwork &&
+          !params.policy?.dangerouslyAllowPrivateNetwork;
+        const dispatcherPolicy: PinnedDispatcherPolicy | undefined = useEnvProxy
+          ? Object.assign({}, params.dispatcherPolicy, { mode: "env-proxy" as const })
+          : params.dispatcherPolicy;
+        dispatcher = createPinnedDispatcher(pinned, dispatcherPolicy, params.policy);
       }

src/infra/net/fetch-guard.ssrf.test.ts — Updated test expectations to reflect that STRICT mode now uses EnvHttpProxyAgent (via pinned env-proxy dispatcher) when proxy env vars are configured:

-  it("ignores env proxy by default to preserve DNS-pinned destination binding", async () => {
+  it("routes through env proxy in strict mode via pinned env-proxy dispatcher", async () => {
     await runProxyModeDispatcherTest({
       mode: GUARDED_FETCH_MODE.STRICT,
-      expectEnvProxy: false,
+      expectEnvProxy: true,
     });
   });

-  it("uses env proxy only when dangerous proxy bypass is explicitly enabled", async () => {
+  it("routes through env proxy when trusted proxy mode is explicitly enabled", async () => {

What's preserved:

  • DNS pinningcreatePinnedDispatcher threads the pinned lookup into EnvHttpProxyAgent via connect: withPinnedLookup() in its existing env-proxy mode
  • SSRF validationresolvePinnedHostnameWithPolicy() still runs before the dispatcher is created
  • Explicit dispatcher policies — callers with mode: "explicit-proxy" (Telegram media), mode: "direct" (fallback recovery), or any other mode are not overridden
  • Private/local network access — callers with allowPrivateNetwork or dangerouslyAllowPrivateNetwork stay on the direct pinned path (BlueBubbles, remote memory over LAN)
  • TRUSTED_ENV_PROXY — unchanged, remains the explicit opt-in for unpinned proxy routing
  • Protocol-aware detection — uses hasEnvHttpProxyConfigured(protocol) so http:// URLs only trigger env-proxy when HTTP_PROXY is set
  • No proxy configured — falls through to existing direct mode behavior

How I found this

Deploying NemoClaw (NVIDIA's OpenClaw wrapper) inside an OpenShell sandbox. The sandbox has NODE_USE_ENV_PROXY=1, HTTPS_PROXY, and NODE_EXTRA_CA_CERTS configured — raw fetch() works natively. But web_fetch fails because the SSRF guard's pinned dispatcher in direct mode bypasses the proxy.

The env-proxy mode already existed in createPinnedDispatcher (in ssrf.ts) and correctly threads DNS pinning through EnvHttpProxyAgent. This PR just activates it when proxy env vars are detected and the caller hasn't opted out.

Testing

  • All 14 tests in fetch-guard.ssrf.test.ts pass locally
  • pnpm check passes locally (format, types, lint, all boundary checks)
  • Verified web_fetch works for google.com, httpbin.org, bbc.com, api.github.com in an OpenShell sandbox
  • SSRF validation still blocks private/reserved IPs
  • Slack API, Jira API, and Codex inference unaffected
  • No proxy configured → existing direct mode (unchanged)
  • Callers with explicit dispatcherPolicy.mode → not overridden
  • Callers with allowPrivateNetwork / dangerouslyAllowPrivateNetwork → direct pinned path preserved

Fixes

  • #47598 — web_fetch doesn't work behind HTTP proxy
  • #49948 — web_fetch fails when HTTPS proxy is enabled
  • #32947 — web_fetch always fails even for example.com
  • #46306 — web_fetch fails behind proxy with getaddrinfo EAI_AGAIN

Changed files

  • src/infra/net/fetch-guard.ssrf.test.ts (modified, +3/-3)
  • src/infra/net/fetch-guard.ts (modified, +16/-2)
RAW_BUFFERClick to expand / collapse

Bug type

Behavior bug (incorrect output/state without crash)

Summary

web_fetch 在代理环境下无法获取某些网站

Steps to reproduce

配置代理 → 启动 OpenClaw → 使用 web_fetch → 失败

Expected behavior

正常通过代理获取,或提供绕过选项

Actual behavior

连接/SSL 错误

OpenClaw version

2026.3.18

Operating system

Ubuntu 24.04

Install method

npm global

Model

bailian/kimi-k2.5

Provider / routing chain

openclaw -> bailian

Config file / key location

No response

Additional provider/model setup details

No response

Logs, screenshots, and evidence

Impact and severity

中等级别,影响需要代理的用户

Additional information

No response

extent analysis

Fix Plan

To resolve the issue of web_fetch failing to retrieve certain websites in a proxy environment, we need to configure the proxy settings correctly.

  • Update the web_fetch function to accept proxy settings.
  • Pass the proxy settings to the web_fetch function.

Example Code

import requests

def web_fetch(url, proxy=None):
    if proxy:
        proxies = {
            'http': proxy,
            'https': proxy
        }
        response = requests.get(url, proxies=proxies)
    else:
        response = requests.get(url)
    return response

# Usage
proxy = 'http://your-proxy-server:8080'
url = 'https://example.com'
response = web_fetch(url, proxy)
print(response.status_code)

Verification

To verify that the fix worked, check the response status code. If it's 200, the request was successful.

Extra Tips

  • Make sure to replace 'http://your-proxy-server:8080' with your actual proxy server URL.
  • If you're using a proxy with authentication, you'll need to modify the proxies dictionary to include the authentication details. For example:
proxies = {
    'http': 'http://username:password@your-proxy-server:8080',
    'https': 'http://username:password@your-proxy-server:8080'
}

Vote matrix · Quick signals

Works
Did the solution work? Tap to confirm.
Easy Fix
Was it a quick fix?
Time Saver
Did it save you time?
Blocking
Was it severely blocking?
Common Issue
Are others likely hitting this too?
Flaky / Intermittent
Is it intermittent?
Verified / Reproducible
Can you reproduce it reliably?
Loading…

FAQ

Expected behavior

正常通过代理获取,或提供绕过选项

Still need to ship something?

×6

Another batch ranked right after the header list — different links, same matching logic.

Back to top recommendations

TRENDING