PR #78574: feat(github-copilot): add native web search for Copilot GPT models

• Repository: openclaw/openclaw
• Author: Marvae
• State: open | merged: False
• Link: https://github.com/openclaw/openclaw/pull/78574

Description (problem / solution / changelog)

Summary

• Problem: Copilot GPT models support native web search via OpenAI Responses API but OpenClaw doesn't inject the web_search tool for the github-copilot provider
• Why it matters: Users need external search API config (SearXNG/Brave) even though Copilot already supports server-side search natively
• What changed: Added self-contained native web search in extensions/github-copilot/ — eligibility check, payload injection, managed tool replacement, reasoning effort bump
• What did NOT change: No changes to extensions/openai/, no cross-plugin dependencies, no config schema changes

Change Type (select all)

• Bug fix
• Feature
• Refactor required for the fix
• Docs
• Security hardening
• Chore/infra

Scope (select all touched areas)

• Gateway / orchestration
• Skills / tool execution
• Auth / tokens
• Memory / storage
• Integrations
• API / contracts
• UI / DX
• CI/CD / infra

Linked Issue/PR

• Closes #78573
• This PR fixes a bug or regression

Real behavior proof (required for external PRs)

• Behavior or issue addressed: Copilot GPT models can now perform native web search without external search API configuration
• Real environment tested: macOS, OpenClaw installed globally via npm, GitHub Copilot provider with gpt-5.4 model
• Exact steps or command run after this patch:
  1. Patched compiled dist/stream-Dx1NfAai.js and dist/native-web-search-BgFH6yGE.js with equivalent logic
  2. Restarted gateway
  3. Ran subagent with github-copilot/gpt-5.4 model asking for current news
• Evidence after fix (terminal capture):

$ curl -s -X POST "https://api.enterprise.githubcopilot.com/v1/responses" \
  -H "Authorization: Bearer $COPILOT_TOKEN" \
  -H "Copilot-Integration-Id: vscode-chat" \
  -d '{"model":"gpt-5.4","stream":false,"input":"What is the latest stable version of React.js?","tools":[{"type":"web_search_preview"}]}'

{"output":[{"type":"web_search_call","status":"completed","action":{"query":"site:react.dev React latest version"}},{"type":"message","content":[{"type":"output_text","text":"The latest stable React version is 19.2...","annotations":[{"type":"url_citation","url":"https://react.dev/versions"}]}]}],"usage":{"input_tokens":12708,"output_tokens":437}}

$ openclaw subagent output (github-copilot/gpt-5.4, no search provider configured):

Session metadata: api=openai-responses provider=github-copilot model=gpt-5.4 stopReason=stop

Response text:
  今天是 2026年5月7日，周四。
  最近几条比较大的 AI 新闻：
  - OpenAI：5月5日 ChatGPT 默认模型又更新 (techcrunch.com)
  - Anthropic：5月4日 宣布和 Blackstone、Goldman Sachs 做企业 AI (nasdaq.com)
  - Anthropic/算力：5月6日 从 SpaceX 获得算力支持 (axios.com)
  - 行业监管：5月5日 Google、Microsoft、xAI 同意政府先测试 (tomshardware.com)

Citation URLs (native web search signature ?utm_source=openai):
  https://techcrunch.com/2026/04/23/openai-chatgpt-gpt-5-5-ai-model-superapp/?utm_source=openai
  https://www.axios.com/2026/05/06/anthropic-spacex-elon-musk-compute?utm_source=openai

$ gateway log before patch:
[tools] web_search failed: SearXNG base URL is not configured. Set SEARXNG_BASE_URL or configure plugins.entries.searxng.config.webSearch.baseUrl.

$ gateway log after patch:
(no managed search failures — native web search handled server-side)

• Observed result after fix: Native web search activated; no managed search failures in logs; real-time citations returned
• What was not tested: url_citation annotation rendering (known separate issue in stream parser — affects OpenAI direct equally)
• Before evidence: Without patch, subagent fell back to managed web_search which failed with SearXNG base URL is not configured errors

API Proof: Copilot backend supports native web_search tool

Direct curl to Copilot Responses API with {"type": "web_search_preview"} tool:

curl -s -X POST "https://api.enterprise.githubcopilot.com/v1/responses" \
  -H "Authorization: Bearer $COPILOT_TOKEN" \
  -H "Content-Type: application/json" \
  -H "Copilot-Integration-Id: vscode-chat" \
  -d '{"model":"gpt-5.4","stream":false,"input":"What is the latest stable version of React.js?","tools":[{"type":"web_search_preview"}],"include":["web_search_call.action.sources"]}'

Response (key fields):

{
  "output": [
    {
      "type": "web_search_call",
      "status": "completed",
      "action": {
        "query": "site:react.dev React latest version release official",
        "sources": [
          {"type": "url", "url": "https://react.dev/versions"},
          {"type": "url", "url": "https://github.com/facebook/react/releases"},
          {"type": "url", "url": "https://github.com/facebook/react"}
        ]
      }
    },
    {
      "type": "message",
      "content": [{
        "type": "output_text",
        "text": "The latest **stable React major/minor version** is **19.2**...",
        "annotations": [
          {"type": "url_citation", "url": "https://react.dev/versions", "title": "React Versions"},
          {"type": "url_citation", "url": "https://github.com/facebook/react", "title": "..."}
        ]
      }]
    }
  ],
  "usage": {
    "input_tokens": 12708,
    "input_tokens_details": {"cached_tokens": 4480},
    "output_tokens": 437,
    "output_tokens_details": {"reasoning_tokens": 292}
  }
}

OpenClaw e2e: Subagent input/output

Input:

Model: github-copilot/gpt-5.4
Task: "回答：今天是几号？最近有什么 AI 相关的大新闻？用中文简短回答。"
Config: No web search provider configured (SearXNG/Brave/etc all unconfigured)

Output:

今天是 2026年5月7日，周四。

最近几条比较大的 AI 新闻：
- OpenAI：5月5日 ChatGPT 默认模型又更新了一次 (techcrunch.com)
- Anthropic：5月4日 宣布和 Blackstone、Goldman Sachs 一起做企业 AI 服务公司 (nasdaq.com)
- Anthropic / 算力：5月6日 传出将从 SpaceX 获得算力支持 (axios.com)
- 行业监管：5月5日 Google、Microsoft、xAI 等同意让美国政府在模型发布前先做测试 (tomshardware.com)

Session metadata:

{"api": "openai-responses", "provider": "github-copilot", "model": "gpt-5.4", "stopReason": "stop"}

Citation URLs (all with ?utm_source=openai — native web search signature):

• https://techcrunch.com/2026/04/23/openai-chatgpt-gpt-5-5-ai-model-superapp/?utm_source=openai
• https://www.nasdaq.com/press-release/anthropic-partners-blackstone-hellman-friedman-and-goldman-sachs-launch-enterprise-ai?utm_source=openai
• https://www.axios.com/2026/05/06/anthropic-spacex-elon-musk-compute?utm_source=openai
• https://www.tomshardware.com/tech-industry/artificial-intelligence/google-microsoft-and-xai-agree-to-let-us-govenment-test-ai-models-before-public-release?utm_source=openai

Gateway logs comparison:

• Before: [tools] web_search failed: SearXNG base URL is not configured.
• After: No managed search errors — native web search handled server-side.

Root Cause (if applicable)

N/A — new feature, not a bug fix.

Regression Test Plan (if applicable)

• Coverage level that should have caught this:
  • Unit test
  • Seam / integration test
  • End-to-end test
  • Existing coverage already sufficient
• Target test or file: extensions/github-copilot/native-web-search.test.ts
• Scenario the test should lock in: Eligibility gating, payload injection, managed tool replacement, config override behavior
• Why this is the smallest reliable guardrail: Pure function unit tests cover all branching logic; e2e requires live Copilot token
• Existing test that already covers this (if any): None (new feature)

User-visible / Behavior Changes

• Copilot GPT models (gpt-4o, gpt-5, gpt-5.4, etc.) now automatically use native web search when tools.web.search.enabled is not false
• No config changes required — activates automatically for eligible models
• Can be disabled by setting tools.web.search.enabled: false or pinning a non-OpenAI search provider

Diagram (if applicable)

Before:
[user query] -> [Copilot GPT] -> [no web search tool] -> [stale/hallucinated answer]
                                  OR
[user query] -> [managed web_search] -> [SearXNG/Brave required] -> [fails without config]

After:
[user query] -> [payload patch injects {type: web_search}] -> [Copilot GPT searches server-side] -> [real-time answer + citations]

Security Impact (required)

• New permissions/capabilities? No
• Secrets/tokens handling changed? No
• New/changed network calls? No (search happens server-side via existing Copilot API connection)
• Command/tool execution surface changed? No
• Data access scope changed? No

Repro + Verification

Environment

• OS: macOS 14.6 (Intel)
• Runtime/container: Node.js v25.6.1, OpenClaw installed globally
• Model/provider: github-copilot/gpt-5.4
• Integration/channel: Telegram (via subagent)
• Relevant config: Default (no web search provider configured)

Steps

1. Configure github-copilot provider with valid auth
2. Send a query requiring current information (e.g. "What are the latest AI news?")
3. Observe response contains real-time information with citation URLs

Expected

• Response includes current information from web search
• Citation URLs contain ?utm_source=openai parameter
• No [tools] web_search failed errors in gateway logs

Actual

• ✅ All expected behaviors confirmed

Evidence

• Failing test/log before + passing after
• Trace/log snippets
• Copied live output
• Screenshot/recording
• Perf numbers (if relevant)

Before (gateway log — managed search fails without config):

[tools] web_search failed: SearXNG base URL is not configured. Set SEARXNG_BASE_URL or configure plugins.entries.searxng.config.webSearch.baseUrl. raw_params={"query":"2026年5月 AI新闻","count":5}

After (OpenClaw subagent live output with github-copilot/gpt-5.4):

Session metadata:
  api: openai-responses
  provider: github-copilot
  model: gpt-5.4
  stopReason: stop

Assistant response (verbatim from session history):
  今天是 2026年5月7日，周四。

  最近几条比较大的 AI 新闻，简短版：
  - OpenAI：5月5日 ChatGPT 默认模型又更新了一次，主打更准确、更个性化。
  - Anthropic：5月4日 宣布和 Blackstone、Goldman Sachs 一起做企业 AI 服务公司。
  - Anthropic / 算力：5月6日 传出将从 SpaceX 获得算力支持。
  - 行业监管：5月5日 Google、Microsoft、xAI 等同意让美国政府在模型发布前先做测试。

Citation URLs in response (all with ?utm_source=openai — native web search signature):
  - https://techcrunch.com/2026/04/23/openai-chatgpt-gpt-5-5-ai-model-superapp/?utm_source=openai
  - https://www.nasdaq.com/press-release/anthropic-partners-blackstone-hellman-friedman-and-goldman-sachs-launch-enterprise-ai?utm_source=openai
  - https://www.axios.com/2026/05/06/anthropic-spacex-elon-musk-compute?utm_source=openai
  - https://www.tomshardware.com/tech-industry/artificial-intelligence/google-microsoft-and-xai-agree-to-let-us-govenment-test-ai-models-before-public-release?utm_source=openai

Direct Copilot API curl (proves the API supports web_search):

$ curl -s -X POST "https://api.enterprise.githubcopilot.com/v1/responses" \
  -H "Authorization: Bearer $COPILOT_TOKEN" \
  -H "Content-Type: application/json" \
  -H "Copilot-Integration-Id: vscode-chat" \
  -d '{"model":"gpt-5.4","stream":false,"input":"What is the latest stable version of React.js?","tools":[{"type":"web_search_preview"}]}'

{
  "output": [
    {"type": "web_search_call", "status": "completed", "action": {"query": "site:react.dev React latest version release official"}},
    {"type": "message", "content": [{"type": "output_text", "text": "The latest stable React version is 19.2...", "annotations": [{"type": "url_citation", "url": "https://react.dev/versions"}]}]}
  ],
  "usage": {"input_tokens": 12708, "input_tokens_details": {"cached_tokens": 4480}, "output_tokens": 437}
}

Unit tests:

$ pnpm test:extension -- github-copilot
  Tests: 87 passed (87)

Human Verification (required)

• Verified scenarios: Native search injection for eligible model, skip for non-eligible models, config override behavior, managed tool replacement
• Edge cases checked: undefined config, pinned provider, disabled search, already-present native tool, non-object payload
• What you did not verify: Citation annotation rendering in final output (known separate issue)

Review Conversations

• I replied to or resolved every bot review conversation I addressed in this PR.
• I left unresolved only the conversations that still need reviewer or maintainer judgment.

Compatibility / Migration

• Backward compatible? Yes
• Config/env changes? No
• Migration needed? No

Risks and Mitigations

• Risk: Copilot backend could change web_search tool format in future
  • Mitigation: Strict eligibility check (only openai-responses API); easy to disable via config; follows same pattern as openai plugin

Changed files

• CHANGELOG.md (modified, +1/-0)
• extensions/github-copilot/native-web-search.test.ts (added, +193/-0)
• extensions/github-copilot/native-web-search.ts (added, +128/-0)
• extensions/github-copilot/stream.ts (modified, +4/-1)