openclaw - 💡(How to fix) Fix Enable image analysis via file-based approach (bypass broken replyOptions.images) [2 comments, 2 participants]

Fix Plan

The fix involves using OpenClaw's detectImageReferences() to bypass the broken replyOptions.images pipeline. Here are the steps:

• Download the image from WhatsApp using downloadWhatsAppMedia() and save it to /tmp/whatsapp-images/{messageId}.{ext}.
• Include [Image: source: /path] in the prompt text to allow OpenClaw to auto-detect the image file path.
• Modify the code to handle images with and without captions, and implement error handling for download failures.

Example Code

import os

def process_whatsapp_image(message):
    # Download image from WhatsApp
    image_data = downloadWhatsAppMedia(message)
    # Save image to disk
    image_path = f"/tmp/whatsapp-images/{message.id}.{image_data['ext']}"
    with open(image_path, "wb") as f:
        f.write(image_data['data'])
    
    # Create prompt text with image path
    if message.caption:
        prompt = f"{message.caption} [Image: source: {image_path}]"
    else:
        prompt = f"Image [Image: source: {image_path}]"
    
    # Use OpenClaw's detectImageReferences() to send image to LLM
    return prompt

def downloadWhatsAppMedia(message):
    # Implement WhatsApp media download logic here
    pass

Verification

To verify the fix, run the following tests:

• Unit tests: Run all 151 unit tests to ensure the fix does not introduce any regressions.
• Local Docker smoke tests: Run local Docker smoke tests to ensure the fix works in a production-like environment.
• Manual testing: Test the fix with images with and without captions, and verify that error handling works correctly.