openclaw - 💡(How to fix) Fix Model fallback retries primary too aggressively before moving to next fallback [1 comments, 2 participants]

Fix Plan

To address the issue, we will introduce a configurable model.fallbackTimeoutMs and model.maxRetriesBeforeFallback to control the fallback behavior.

Configuration Changes

• Add the following configuration options:
  • model.fallbackTimeoutMs: timeout in milliseconds before falling back to the next model
  • model.maxRetriesBeforeFallback: maximum number of retries before falling back to the next model

Code Changes

# Add configuration options
model_config = {
    'fallbackTimeoutMs': 5000,  # 5 seconds
    'maxRetriesBeforeFallback': 2
}

# Update the fallback logic
def fallback_to_next_model(current_model, retry_count, timeout):
    if retry_count >= model_config['maxRetriesBeforeFallback'] or timeout >= model_config['fallbackTimeoutMs']:
        # Fallback to the next model
        return get_next_model(current_model)
    return current_model

# Example usage
current_model = 'anthropic/claude-opus-4-6'
retry_count = 0
timeout = 0
while retry_count < model_config['maxRetriesBeforeFallback']:
    try:
        # Try to use the current model
        response = use_model(current_model)
        break
    except OverloadedError:
        # Increment retry count and timeout
        retry_count += 1
        timeout += 1000  # assume 1 second timeout
        current_model = fallback_to_next_model(current_model, retry_count, timeout)

Verification

To verify the fix, test the fallback behavior with different scenarios:

• Primary model overloaded: verify that the first fallback is attempted within 5 seconds
• First fallback overloaded: verify that the second fallback is attempted within 10 seconds
• Second fallback overloaded: verify that the third fallback is attempted within 15 seconds

Extra Tips

• Monitor the fallback behavior and adjust the model.fallbackTimeoutMs and model.maxRetriesBeforeFallback configuration options as needed to achieve the desired fallback behavior.
• Consider adding logging and metrics to track the fallback behavior and identify potential issues.