openclaw - 💡(How to fix) Fix 2 local models on 2 local server ? [1 participants]

TL;DR

To resolve the issue of conflicting model activations, consider configuring OpenCLAW to merge two endpoints for separate local LLM servers.

Guidance

• Investigate using OpenCLAW to configure multiple local LLM servers with distinct endpoints, such as 192.168.0.34 and 192.168.0.134, to avoid conflicts between models.
• Explore the possibility of using one model on a GPU and the second model on a VM using the CPU to separate their operations and reduce interference.
• Verify that the issue is indeed caused by both models being activated simultaneously, and test the proposed solution to ensure it resolves the tool-calling problems.
• Consider testing with different model combinations, such as a 4b model for chat and a 20-40b model for coding, to identify the optimal setup for local LLM servers.

Example

No code snippet is provided as the issue does not contain specific code references.

Notes

The proposed solution relies on the assumption that OpenCLAW can be configured to merge multiple endpoints, which may require further investigation and testing to confirm its feasibility.

Recommendation

Apply a workaround by configuring OpenCLAW to merge multiple endpoints for separate local LLM servers, as this approach may help resolve the conflicting model activation issue without requiring significant changes to the existing setup.