vllm - 💡(How to fix) Fix [Feature]: Add support for PrismML models [1 participants]

TL;DR

• Investigate adding support for PrismML models to reduce compute requirements and improve performance.

Guidance

• Review the PrismML documentation at https://prismml.com/ to understand the requirements for integration.
• Consider the benefits of supporting the gguf format to consider a more comprehensive solution.
• Search existing issues and documentation to ensure this feature request is not already addressed.
• Evaluate the potential impact of supporting PrismML models on the overall system performance and resource utilization.

Notes

• The issue lacks specific technical details, making it challenging to provide a detailed fix or workaround.
• Further discussion or clarification may be necessary to determine the best approach for implementing PrismML model support.

Recommendation

• Apply workaround: Investigate potential workarounds or proof-of-concepts for integrating PrismML models, as there is no clear indication of an existing solution or fixed version.