PR #59007: fix(net): skip DNS pinning before trusted env proxy dispatch

• Repository: openclaw/openclaw
• Author: cluster2600
• State: closed | merged: True
• Link: https://github.com/openclaw/openclaw/pull/59007

Description (problem / solution / changelog)

Summary

Fixes fetchWithSsrFGuard() so trusted env-proxy mode can actually reach EnvHttpProxyAgent before any local DNS lookup is attempted.

This fixes the remaining proxy-only sandbox failure tracked in #59005. It is the same production gap described in #58034, but rebased cleanly on current main and with the fallback test tightened so host ALL_PROXY / all_proxy values cannot accidentally satisfy the proxy branch.

What changed

• move DNS pinning and pinned-dispatcher creation into the non-proxy branch
• keep trusted env-proxy dispatch as an early branch
• clear all six proxy env keys in the tests before setting expectations
• add an explicit regression test for trusted mode with no proxy env vars present

Why

#50650 fixed the strict-mode env-proxy path, but both main and v2026.3.31 still resolve DNS before entering the trusted env-proxy branch:

const pinned = await resolvePinnedHostnameWithPolicy(...);
const canUseTrustedEnvProxy =
  mode === GUARDED_FETCH_MODE.TRUSTED_ENV_PROXY && hasProxyEnvConfigured();
if (canUseTrustedEnvProxy) {
  dispatcher = new EnvHttpProxyAgent();
}

In proxy-only sandboxes, that ordering means the request fails on local DNS and never reaches the trusted proxy.

Testing

• pnpm exec vitest run --config vitest.unit.config.ts src/infra/net/fetch-guard.ssrf.test.ts
• pnpm check

Notes

• Fixes #59005
• Supersedes #58034

Changed files

• CHANGELOG.md (modified, +1/-0)
• src/infra/net/fetch-guard.ssrf.test.ts (modified, +38/-0)
• src/infra/net/fetch-guard.ts (modified, +4/-4)

PR #58034: fix(net): harden trusted env proxy fetch guard and add explicit web_fetch opt-in

• Repository: openclaw/openclaw
• Author: cosmicnet
• State: open | merged: False
• Link: https://github.com/openclaw/openclaw/pull/58034

Description (problem / solution / changelog)

This PR now layers on top of merged #59007 rather than competing with it.

When using a trusted environment proxy, web_fetch can now skip local DNS pinning only when explicitly enabled with tools.web.fetch.useTrustedEnvProxy. The default behaviour stays unchanged. In the trusted-proxy path, hostname policy checks still run before dispatch, and requests fall back to the pinned-DNS path when no matching HTTP(S) proxy env var applies or when NO_PROXY excludes the target.

Local DNS fails in sandboxed environments where UDP is blocked and all traffic must route through the proxy, but skipping local DNS needs to stay narrow and explicit so SSRF protections do not get weakened by default.

• Keep the core DNS-before-proxy fix from #59007 as the baseline
• Tighten trusted env proxy detection to match HTTP(S) proxy env semantics
• Preserve pre-DNS hostname policy checks in the trusted-proxy path
• Fall back to the pinned-DNS path when NO_PROXY excludes the target or no matching HTTP(S) proxy env var is configured
• Add an explicit tools.web.fetch.useTrustedEnvProxy opt-in for web_fetch
• Document the new config setting and update the generated config-doc baseline

Summary

• Problem: after #59007 fixed the basic DNS-before-proxy ordering, the trusted env proxy path still had important gaps. Proxy applicability did not fully match EnvHttpProxyAgent HTTP(S) semantics, hostname policy checks were skipped in the trusted-proxy branch, NO_PROXY exclusions did not cleanly fall back to the pinned-DNS path, and web_fetch still relied on an implicit opt-in model.
• Why it matters: proxy-only sandbox environments need the proxy to resolve DNS, but that path still needs explicit activation and the same hostname-level SSRF guardrails. Without those follow-up fixes, some proxy configurations still fail unexpectedly and the safety boundary is harder to reason about.
• What changed: the fetch guard now uses protocol-aware HTTP(S) proxy detection, applies assertHostnameAllowedByPolicy before trusted env proxy dispatch, falls back to the normal pinned-DNS path when NO_PROXY excludes the target or no matching HTTP(S) proxy env var exists, and exposes tools.web.fetch.useTrustedEnvProxy as an explicit web_fetch config opt-in. This PR also adds the corresponding documentation and updates the config-doc baseline hash.
• What did NOT change (scope boundary): no changes to general proxy dispatcher bootstrap, no changes to strict-mode redirect handling or response processing, and no default behaviour change for web_fetch unless the new config opt-in is enabled.

Change Type (select all)

• Bug fix
• Feature
• Refactor required for the fix
• Docs
• Security hardening
• Chore/infra

Scope (select all touched areas)

• Gateway / orchestration
• Skills / tool execution
• Auth / tokens
• Memory / storage
• Integrations
• API / contracts
• UI / DX
• CI/CD / infra

Linked Issue/PR

• Related #43821
• Related #59007
• Related #59005
• Related #46538
• Related #56564
• Related #46306
• Related #63565
• This PR fixes a bug or regression

Note: #59007 merged the core DNS-before-proxy ordering fix. This PR keeps the remaining hardening and behaviour-shaping changes on top of that: protocol-aware proxy detection, NO_PROXY fallback, pre-DNS hostname policy checks in the trusted-proxy path, the explicit web_fetch config opt-in, and the user-facing documentation for that new setting.

Root Cause / Regression History (if applicable)

• Root cause: trusted env proxy support was originally added as a narrow dispatch path, but its applicability and safety checks were incomplete. In particular, proxy detection did not fully match HTTP(S) env-proxy semantics, hostname policy enforcement did not run in the trusted-proxy branch, and NO_PROXY exclusions did not cleanly revert to the pinned-DNS path.
• Missing detection / guardrail: there was no direct coverage for protocol-aware proxy detection, NO_PROXY fallback behaviour, or the explicit config opt-in for web_fetch.
• Prior context: the SSRF guard was originally designed for direct-fetch environments. Trusted env proxy support was added later, and #59007 subsequently fixed the core DNS-before-proxy ordering issue. This PR is the follow-up hardening pass.
• Why this surfaced now: the basic ordering bug was fixed upstream first, which made the remaining edge cases clearer and allowed this branch to narrow down to the residual gaps.
• If unknown, what was ruled out: N/A. The remaining issues were isolated to the trusted env proxy path and config surface.

Regression Test Plan (if applicable)

• Coverage level that should have caught this:
  • Unit test
  • Seam / integration test
  • End-to-end test
  • Existing coverage already sufficient
• Target tests or files: fetch-guard SSRF tests, proxy-env tests, web-fetch SSRF tests, web tools fetch tests, and config schema tests
• Scenario the tests should lock in:
  1. Trusted env proxy dispatch only activates when a matching HTTP(S) proxy env var applies.
  2. NO_PROXY exclusions fall back to the pinned-DNS path.
  3. Hostname policy checks still run in the trusted env proxy path.
  4. web_fetch only opts into this path when tools.web.fetch.useTrustedEnvProxy is explicitly enabled.
  5. The runtime config schema accepts the new setting.
• Why this is the smallest reliable guardrail: these unit-level checks exercise the fetch guard and config surfaces directly, without needing a live proxy or sandbox deployment.
• Existing test that already covers this (if any): none comprehensively covered these trusted env proxy edge cases before.
• If no new test is added, why not: new coverage was added.

User-visible / Behaviour Changes

• Default behaviour is unchanged.
• When tools.web.fetch.useTrustedEnvProxy is true and a matching HTTP(S) proxy env var applies, web_fetch routes through the trusted env proxy without local DNS pinning.
• When the opt-in is unset or false, or when NO_PROXY excludes the target, or when no matching HTTP(S) proxy env var exists, the guard falls back to the existing pinned-DNS path.
• The new opt-in is documented in the Web Fetch guide and the gateway configuration reference.

Diagram (if applicable)

Before:
[web_fetch] -> fetchWithSsrfGuard -> trusted env proxy branch
                                   -> local safeguards incomplete for proxy applicability / hostname checks / NO_PROXY fallback

After:
[web_fetch] -> tools.web.fetch.useTrustedEnvProxy?
               NO  -> pinned-DNS path
               YES -> assertHostnameAllowedByPolicy
                      -> matching HTTP(S) proxy env and not NO_PROXY?
                         YES -> EnvHttpProxyAgent [proxy resolves DNS]
                         NO  -> resolvePinnedHostnameWithPolicy -> createPinnedDispatcher

Security Impact (required)

• New permissions/capabilities? No
• Secrets/tokens handling changed? No
• New/changed network calls? Yes
• Command/tool execution surface changed? No
• Data access scope changed? No
• If any Yes, explain risk + mitigation: when tools.web.fetch.useTrustedEnvProxy is enabled and the trusted path is taken, local SSRF IP pinning is delegated to the trusted proxy instead of being performed locally. The risk is contained because (1) the path is explicit and off by default, (2) it only activates when a matching HTTP(S) proxy env var applies to the request protocol, (3) NO_PROXY exclusions fall back to the pinned-DNS path, and (4) hostname policy checks still run before proxy dispatch.

Repro + Verification

Environment

• OS: Linux (WSL2 / Ubuntu)
• Runtime/container: OpenShell sandbox with CONNECT proxy
• Model/provider: any, this is a network-path issue
• Integration/channel (if any): N/A
• Relevant config (redacted): tools.web.fetch.useTrustedEnvProxy: true with HTTP_PROXY or HTTPS_PROXY set in the sandbox environment

Steps

1. Run OpenClaw inside an OpenShell sandbox where UDP DNS is blocked
2. Enable tools.web.fetch.useTrustedEnvProxy for web_fetch
3. Use the web_fetch agent tool to fetch a public URL
4. Repeat with NO_PROXY excluding the target host

Expected

• With the opt-in enabled and a matching HTTP(S) proxy env var active, the request routes through the trusted proxy and the proxy resolves DNS.
• If NO_PROXY excludes the target, or no matching HTTP(S) proxy env var exists, the request falls back to the pinned-DNS path.

Actual

• Before these follow-up fixes, trusted env proxy mode still had edge cases around proxy detection, hostname checks, and NO_PROXY fallback.
• After this PR, the trusted path is explicit, protocol-aware, and falls back cleanly when the proxy should not be used.

Evidence

• Failing test/log before + passing after
• Trace/log snippets

Verification performed for the rebased branch:

• pnpm config:docs:check passes after the schema/docs update
• pnpm test was run on updated main and on this branch after rebasing; after refreshing the stale generated base-schema file, the visible failures on this branch matched the current main baseline, with no extra visible branch-only failure identified
• live manual verification against a completely fresh bleeding-edge NemoClaw / OpenShell / OpenClaw build and deploy showed that enabling tools.web.fetch.useTrustedEnvProxy let web_fetch reach upstream services through the trusted env proxy path; remaining example.com-style failures traced to NVIDIA/OpenShell#813 rather than this OpenClaw change
• the refreshed rebased branch has been force-pushed to the fork after the final local fixups, so the remote PR branch now matches this verified state

Human Verification (required)

• Verified scenarios: code and tests were reviewed for protocol-aware proxy applicability, NO_PROXY fallback, hostname policy enforcement in the trusted-proxy path, the explicit web_fetch opt-in, and the accompanying config/docs surfaces. The trusted env proxy path was also exercised in a completely fresh bleeding-edge NemoClaw / OpenShell / OpenClaw build and deploy.
• Edge cases checked: proxy mode enabled but no matching HTTP(S) proxy env var configured; target excluded by NO_PROXY; default web_fetch behaviour when the opt-in is not enabled.
• What I did not verify: I did not repeat that full fresh-build deployment after the final local fixups, because the live verification had already been completed against a freshly built bleeding-edge NemoClaw / OpenShell / OpenClaw environment.

Review Conversations

• I replied to or resolved every bot review conversation I addressed in this PR.
• I left unresolved only the conversations that still need reviewer or maintainer judgement.

Compatibility / Migration

• Backward compatible? Yes
• Config/env changes? Yes, optional new tools.web.fetch.useTrustedEnvProxy setting
• Migration needed? No

Risks and Mitigations

• Risk: when the explicit trusted env proxy opt-in is enabled and the trusted path is taken, local SSRF IP pinning is no longer performed.
  • Mitigation: hostname policy checks still run before dispatch; the path is off by default; it only activates when a matching HTTP(S) proxy env var applies; and NO_PROXY exclusions fall back to the existing pinned-DNS path.

This PR was co-authored with an AI assistant (GitHub Copilot). All changes were reviewed and verified by the human author.

Changed files

• docs/.generated/config-baseline.sha256 (modified, +2/-2)
• docs/gateway/configuration-reference.md (modified, +6/-0)
• docs/tools/web-fetch.md (modified, +20/-0)
• src/agents/tools/web-fetch.ssrf.test.ts (modified, +17/-0)
• src/agents/tools/web-fetch.ts (modified, +9/-1)
• src/agents/tools/web-tools.fetch.test.ts (modified, +26/-1)
• src/config/schema.base.generated.ts (modified, +11/-0)
• src/config/schema.help.ts (modified, +2/-0)
• src/config/schema.labels.ts (modified, +1/-0)
• src/config/schema.test.ts (modified, +12/-0)
• src/config/types.tools.ts (modified, +2/-0)
• src/config/zod-schema.agent-runtime.ts (modified, +1/-0)
• src/infra/net/fetch-guard.ssrf.test.ts (modified, +261/-1)
• src/infra/net/fetch-guard.ts (modified, +12/-2)
• src/infra/net/proxy-env.test.ts (modified, +18/-0)
• src/infra/net/proxy-env.ts (modified, +9/-8)

PR #1755: fix(sandbox): restore Telegram media downloads in proxy-only sandbox

• Repository: NVIDIA/NemoClaw
• Author: tommylin-signalpro
• State: closed | merged: True
• Link: https://github.com/NVIDIA/NemoClaw/pull/1755

Description (problem / solution / changelog)

Summary

Restore Telegram media downloads inside NemoClaw's proxy-only sandbox. The agent receives photo messages but MediaFetchError: Blocked hostname (not in allowlist): 10.200.0.1 aborts the download before the file reaches the model. This PR makes the existing fetchRemoteMedia callsite route through the L7 proxy and stops OpenClaw's new SSRF check from rejecting the proxy hostname.

Scope: Telegram only. Discord/Slack are not part of this PR — see Implementation scope below for what does and doesn't extend to them.

Related Issue

• Extends #1252 (downstream OpenClaw env-proxy DNS workaround) to cover fetchRemoteMedia
• Builds on #1301 (Telegram /file/bot*/** policy fix, already merged)

Root Cause

NemoClaw forces all sandbox egress through the OpenShell L7 proxy (default 10.200.0.1:3128). OpenClaw's media download path requires two things to work in this environment, and OpenClaw 2026.4.2 broke the second one.

Layer 1 — wrong fetch mode (long-standing)

openclaw/dist/fetch-ClF-ZgDC.js calls

fetchWithSsrFGuard(withStrictGuardedFetchMode({ ... }))

unconditionally. Strict mode does DNS pinning + direct connect, which fails in the sandbox netns where only the proxy is reachable. The agent needs withTrustedEnvProxyGuardedFetchMode (uses EnvHttpProxyAgent) for any callsite running in a proxy-only environment.

Layer 2 — SSRF check rejects the proxy itself (regression in 2026.4.2)

OpenClaw 2026.4.2 added assertExplicitProxyAllowed() in fetch-guard-*.js:

async function assertExplicitProxyAllowed(dispatcherPolicy, lookupFn, policy) {
    if (!dispatcherPolicy || dispatcherPolicy.mode !== "explicit-proxy") return;
    ...
    await resolvePinnedHostnameWithPolicy(parsedProxyUrl.hostname, {
        lookupFn,
        policy: dispatcherPolicy.allowPrivateProxy === true
            ? { ...policy, allowPrivateNetwork: true }
            : policy
    });
}

When fetching Telegram media, OpenClaw passes { hostnameAllowlist: ["api.telegram.org"] } as the SSRF policy (the target's allowlist). The new check then validates the proxy hostname (10.200.0.1) against that same allowlist and rejects it because the proxy is not api.telegram.org. Even with allowPrivateProxy: true flipping allowPrivateNetwork, matchesHostnameAllowlist("10.200.0.1", ["api.telegram.org"]) still returns false.

This is an upstream OpenClaw design issue — a proxy is infrastructure, not a fetch target, so the target's hostname allowlist should not apply to it. NemoClaw's proxy-only architecture hits this 100% of the time.

This PR previously worked end-to-end against [email protected] (which had no assertExplicitProxyAllowed). After main bumped to 2026.4.2 (#1522), only the layer 1 fix survived; the new SSRF check started rejecting every media download.

Implementation scope

This PR contains four changes; their reach is deliberately not the same. Reviewers should know what each one actually affects.

Why Patches 1 and 2 are necessarily fetch-guard-wide: they operate on the openclaw module's exports, not on individual messaging channels. Inside NemoClaw's proxy-only sandbox, every fetch must go through the proxy and every fetch hits the same SSRF check. Narrowing these patches to "Telegram-only" would require a different patching strategy that NemoClaw maintains long-term. The wider reach is consistent with the sandbox's architectural constraint (proxy-only egress), so it is presented as intended behavior, not collateral.

Discord and Slack are explicitly out of scope. Their account configs are not touched by this PR. Any latent improvement they get from the fetch-guard patches inside the sandbox is incidental and not claimed here. If Discord/Slack media downloads also need restoring, they belong in a separate PR with their own end-to-end verification.

Fail-close design

Both sed patches use set -eu and explicit pre/post-grep verification:

• Pre: test -n "$matches" aborts if no candidate file matches.
• Post: positive grep for the inserted marker (Patch 2) and explicit if grep ...; then exit 1; fi that no withStrictGuardedFetchMode as <letter> export remains (Patch 1, written this way because set -e does not propagate failure through !-prefixed commands).

If a future OpenClaw bundle reorganizes the function or exports, the build fails at the next OPENCLAW_VERSION bump instead of silently producing broken media downloads. The next maintainer reviewing the bump PR sees the failure with full context in this Dockerfile comment.

Removal criteria

• Patch 1 can be dropped when OpenClaw deprecates withStrictGuardedFetchMode, or when all proxy-relevant callsites unconditionally pass useEnvProxy.
• Patch 2 can be dropped when OpenClaw fixes assertExplicitProxyAllowed to skip the target's hostname allowlist when validating the proxy, or exposes config to disable the check. The OPENSHELL_SANDBOX env var is set by OpenShell, not by this PR — no extra cleanup needed.

Testing

End-to-end on WSL2 + Docker Desktop, against [email protected] (CI's pinned version):

Build verification

1. Reproduced the CI build step in isolation (docker build on a minimal Dockerfile that pulls [email protected] and runs the same RUN block) — patches apply cleanly, post-patch verification passes.
2. Full docker build with the production Dockerfile (using BASE_IMAGE=ghcr.io/nvidia/nemoclaw/sandbox-base:latest) succeeds. All 19 stages pass, no set -eu aborts.
3. Inspected the resulting image to confirm both patches are persisted in the openclaw bundle:

   $ grep export ... fetch-guard-*.js
   export { withTrustedEnvProxyGuardedFetchMode as a, withTrustedEnvProxyGuardedFetchMode as i, ... };
   
   $ grep 'async function assertExplicitProxyAllowed' ...
   async function assertExplicitProxyAllowed(...) { if (process.env.OPENSHELL_SANDBOX === "1") return; /* nemoclaw: env-gated bypass */

4. Inside a running OpenShell sandbox, confirmed the env marker that triggers Patch 2 is present (set by OpenShell at pod startup, not by this image):

   $ openshell sandbox exec -n <name> -- env | grep OPENSHELL_SANDBOX
   OPENSHELL_SANDBOX=1

End-to-end Telegram round-trip

Setup: nemoclaw onboard with Telegram channel enabled and a real bot token. Inference provider: Anthropic claude-haiku-4-5 (chosen because vision support is reliable, so a successful model description confirms the file actually reached the model intact, not just that download succeeded).

1. Sent a JPEG photo from Telegram → bot received getUpdates, called POST /bot[CREDENTIAL]/getFile (ALLOWED), then GET /file/bot[CREDENTIAL]/photos/file_X.jpg (ALLOWED — the request that previously never reached the proxy).
2. OpenClaw application log confirms the file was saved and resized:

   Image resized to fit limits: file_0---*.jpg 719x1280px 126.7KB -> 136.4KB

3. Pulled the saved JPEG out of the sandbox via openshell sandbox download and visually confirmed the bytes match the source photo (motorcycle dashboard, then a Japanese curry menu — the bytes are not corrupted in transit).
4. Claude Haiku correctly described the contents of both photos in its reply, end-to-end confirming download → save → read tool → model vision pipeline works.
5. No MediaFetchError, no Blocked hostname, no Failed to download media in any log layer.
6. Reproduced the exact failure mode prior to Patch 2 on a separate onboard (Patch 1 only, no env-gated bypass):

   MediaFetchError: Failed to fetch media from
   https://api.telegram.org/file/bot.../photos/file_2.jpg:
   Blocked hostname (not in allowlist): 10.200.0.1

7. Reproduced an additional failure mode where the env gate referenced a Dockerfile-set ENV var (NEMOCLAW_SANDBOX=1) — that var did not propagate into the OpenShell sandbox runtime, so the gate stayed false and the bug recurred. Switched the gate to OPENSHELL_SANDBOX (injected by OpenShell at pod startup) and the e2e flow became reliable.

Type of Change

• Code change (feature, bug fix, or refactor)
• Code change with doc updates
• Doc only (prose changes, no code sample modifications)
• Doc only (includes code sample changes)

Verification

• npx prek run --all-files passes
• npm test passes (pre-existing CLI lifecycle-recovery test failures reproduce on origin/main — unrelated)
• Tests added or updated for new or changed behavior (not applicable — Dockerfile-only patch, validated by build + e2e)
• No secrets, API keys, or credentials committed
• Docs updated for user-facing behavior changes (not applicable — internal sandbox patch, no public surface change)
• make docs builds without warnings (doc changes only)
• Doc pages follow the style guide (doc changes only)
• New doc pages include SPDX header and frontmatter (new pages only)

AI Disclosure

• AI-assisted — tool: Claude Code

Signed-off-by: Tommy Lin [email protected]

Changed files

• Dockerfile (modified, +76/-3)