Issue Station
Sign In
Tools hereToolbox

openclaw - 💡(How to fix) Fix Feature Request: visionOS / Apple Vision Pro node [1 comments, 1 participants]

Official PRs (…)
ON THIS PAGE

Recommended Tools

×6

Utilities matched from this issue’s tags and category — try them while you read without losing context.

GitHub issue graph ai analysis

Paste a GitHub issue URL. We fetch that issue, discover linked issues from bodies/comments/timeline, collect linked pull requests, and produce a structured English report.

The report is written in English Markdown for sharing and archival.

Helpful · Quick feedback

Loading…
GitHub stats
openclaw/openclaw#55405•Fetched 2026-04-08 01:39:56
View on GitHub
Comments
1
Participants
1
Timeline
2
Reactions
0
Author
Zera78Zera78
Participants
Zera78Zera78
Timeline (top)
commented ×1subscribed ×1

Building a native visionOS node for OpenClaw as part of LOAM STUDIO's work on spatial AI. Opening this issue to signal intent, surface protocol nuances early, and confirm the contribution path before opening a PR.

Root Cause

Building a native visionOS node for OpenClaw as part of LOAM STUDIO's work on spatial AI. Opening this issue to signal intent, surface protocol nuances early, and confirm the contribution path before opening a PR.

RAW_BUFFERClick to expand / collapse

Overview

Building a native visionOS node for OpenClaw as part of LOAM STUDIO's work on spatial AI. Opening this issue to signal intent, surface protocol nuances early, and confirm the contribution path before opening a PR.

Capabilities Planned

  • WebSocket handshake — role: node with Ed25519 keypair, protocol v3 signature, device ID as SHA-256 of public key
  • spatial.hands — HandTrackingProvider, 26-joint skeletal map per hand (simd_float4x4 → flattened 4×4 JSON array)
  • spatial.planes — PlaneDetectionProvider for flat surface detection
  • spatial.mesh — SceneReconstructionProvider with chunking protocol (SHA-256, fragmented frames)
  • device.position — WorldTrackingProvider (no privacy key required)
  • location.get — CoreLocation integration
  • canvas.present / canvas.navigate / canvas.eval / canvas.snapshot — WKWebView in a WindowGroup
  • camera.snap / camera.clip — CameraFrameProvider (Enterprise entitlement required; graceful UNAVAILABLE fallback while entitlement is pending)
  • APNs silent push — background wake / heartbeat support; register token with Gateway over active WebSocket

Architecture Note

visionOS drops WebSocket connections when the app loses foreground. Node lifecycle must be anchored to an active ImmersiveSpace — this is an OS constraint, not a design choice. Our implementation uses a .mixed ImmersiveSpace as the keep-alive anchor; losing it triggers a graceful exit frame and node-offline state.

Protocol Findings

Working through the handshake from source revealed a few visionOS-specific nuances:

  • client.id must be the constant "node-host" (GATEWAY_CLIENT_IDS enum)
  • device.id = SHA-256(rawEd25519PublicKeyBytes).hexString
  • Signature payload must be base64url-encoded (no padding) — not standard base64
  • signedAtMs = send time (Date.now()), not the challenge ts
  • deviceFamily: "headset" required in the client dict for v3 payload reconstruction

Happy to document all of this in docs/nodes/visionos.md as part of the PR.

Questions for Maintainers

  1. What are the minProtocol / maxProtocol targets we should build against?
  2. Preferred PR scope: one large PR or staged by capability group?
  3. Any prior visionOS exploration in the codebase we should be aware of?

Contributor

Nick / LOAM STUDIO — @LOAM-STUDIO Repo: LOAM-STUDIO/visionOS-node

extent analysis

Fix Plan

To address the visionOS node implementation for OpenClaw, we need to focus on the WebSocket handshake and protocol nuances. Here are the concrete steps:

  • Implement the WebSocket handshake with the correct parameters:
    • client.id set to "node-host"
    • device.id calculated as SHA-256 of the raw Ed25519 public key bytes
    • Signature payload base64url-encoded without padding
    • signedAtMs set to the current time using Date.now()
    • deviceFamily set to "headset" in the client dictionary
  • Ensure the node lifecycle is anchored to an active ImmersiveSpace to prevent connection drops when the app loses foreground.

Example Code

const crypto = require('crypto');
const base64url = require('base64url');

// Generate Ed25519 keypair
const keypair = crypto.generateKeyPairSync('ed25519', {
  publicKeyEncoding: { type: 'spki', format: 'pem' },
  privateKeyEncoding: { type: 'pkcs8', format: 'pem' },
});

// Calculate device ID
const deviceId = crypto.createHash('sha256').update(keypair.publicKey).digest('hex');

// Create signature payload
const payload = {
  client: {
    id: 'node-host',
    deviceFamily: 'headset',
  },
  device: {
    id: deviceId,
  },
  signedAtMs: Date.now(),
};

// Base64url-encode signature payload
const encodedPayload = base64url.encode(JSON.stringify(payload));

// Sign payload with private key
const signature = crypto.sign('SHA-256', encodedPayload, keypair.privateKey);

// Establish WebSocket connection with correct handshake parameters
const ws = new WebSocket('wss://example.com', {
  headers: {
    'Sec-WebSocket-Protocol': 'v3',
  },
});

ws.on('open', () => {
  ws.send(`{"type": "handshake", "payload": "${encodedPayload}", "signature": "${signature}"}`);
});

Verification

To verify the fix, test the WebSocket connection and handshake by checking the following:

  • The connection is established successfully
  • The handshake is completed with the correct parameters
  • The node lifecycle is anchored to an active ImmersiveSpace

Extra Tips

  • Ensure the Ed25519 keypair is generated and stored securely
  • Use a secure random number generator for the signedAtMs timestamp
  • Implement error handling and logging for the WebSocket connection and handshake
  • Test the implementation thoroughly to ensure compatibility with the visionOS protocol nuances.

Vote matrix · Quick signals

Works
Did the solution work? Tap to confirm.
Easy Fix
Was it a quick fix?
Time Saver
Did it save you time?
Blocking
Was it severely blocking?
Common Issue
Are others likely hitting this too?
Flaky / Intermittent
Is it intermittent?
Verified / Reproducible
Can you reproduce it reliably?
Loading…
#runtime error#dependency conflict#environment setup#docker error#permission error

Still need to ship something?

×6

Another batch ranked right after the header list — different links, same matching logic.

Back to top recommendations

TRENDING

openclaw - 💡(How to fix) Fix Feature Request: visionOS / Apple Vision Pro node [1 comments, 1 participants]