ollama - 💡(How to fix) Fix Discussion: Coordinating multiple Ollama instances — peer awareness via handshake signal [1 participants]

TL;DR

Implementing a passive peer discovery mechanism could enable Ollama instances to discover each other and coordinate load without a central registry.

Guidance

• Investigate existing peer discovery protocols and algorithms that can be adapted for Ollama instances to detect and communicate with each other.
• Consider using a common channel or messaging system for instances to advertise their model availability and current load.
• Evaluate the feasibility of using the proposed 0x42-HERMES signal for peer discovery and load coordination.
• Research potential security implications of implementing a peer discovery mechanism without a central registry.

Example

No code snippet is provided as the issue is focused on conceptual discussion and design.

Notes

The implementation details of the peer discovery mechanism are not specified, and further research is needed to determine the best approach for Ollama instances.

Recommendation

Apply workaround: Implement a custom peer discovery mechanism using existing protocols and algorithms, as upgrading to a fixed version is not mentioned in the issue. This approach allows for a flexible and adaptable solution to the load coordination problem.