vllm - 💡(How to fix) Fix [Feature]: vllm 的 api 添加支持 ollama 的风格 [1 participants]

TL;DR

Modify the VLLM API interface to support the OLLAMA style, allowing compatibility with certain IDEs that only support OLLAMA.

Guidance

• Investigate the VLLM API documentation to understand the required changes for OLLAMA support.
• Check if there are any existing configuration options or workarounds that can enable OLLAMA compatibility without modifying the API.
• Consider reaching out to the VLLM API support team or community for guidance on implementing OLLAMA support.
• Review the GitHub Copilot documentation to see if there are any known limitations or workarounds for using custom models with OLLAMA-style APIs.

Notes

The solution may require modifications to the VLLM API or the use of a proxy server to translate between the VLLM API and the OLLAMA style.

Recommendation

Apply workaround: Modify the VLLM API interface to support OLLAMA style, as it seems to be the most feasible solution given the limitations of the GitHub Copilot and OpenAI API.