vllm - 💡(How to fix) Fix [Feature]: Add Eagle3 Speculative Decoding Support for Gemma 4 Target Models [2 comments, 3 participants]

TL;DR

The user is inquiring about the possibility of adding support for Eagle3 speculative decoding with Gemma 4 models as target models, and the most likely fix or workaround would involve modifying or extending the existing model support.

Guidance

• Investigate the current implementation of model support to determine if Eagle3 speculative decoding can be integrated with Gemma 4 models.
• Review the documentation and existing issues to see if there are any related discussions or solutions that can be applied.
• Consider reaching out to the development team or community for further guidance on implementing the desired support.

Notes

The provided information lacks technical details, making it challenging to provide a specific solution or code example.

Recommendation

Apply workaround: The user should attempt to find alternative solutions or workarounds, such as using a different model or decoding method, until official support for Eagle3 speculative decoding with Gemma 4 models is implemented.