vllm - 💡(How to fix) Fix [RFC]: Automatic test target determination for CI [1 comments, 1 participants]

TL;DR

Implement an automated test selection approach based on code diff to optimize CI pipeline efficiency.

Guidance

• Explore the proposed LLM-based approach using an agent instructions markdown file integrated with the CI pipeline generator to determine the feasibility of this method.
• Investigate alternative techniques such as static analysis on dependency graphs, training ML models, or code coverage report mapping to find the most suitable solution.
• Evaluate the trade-offs between complexity, accuracy, and scalability for each approach to ensure the chosen solution aligns with project requirements.
• Consider the potential impact on CI costs and test coverage when selecting an approach.

Example

No specific code example can be provided without further technical details, but a high-level example of how the LLM-based approach could work might involve:

# Agent Instructions
## Test Selection
- Run tests for modified source files
- Include dependent tests based on code diff analysis

Notes

The proposed solution relies on the effectiveness of the LLM-based approach or alternative techniques, which may require experimentation and evaluation to determine the best fit.

Recommendation

Apply a workaround by implementing the proposed LLM-based approach as a proof-of-concept to assess its viability and potential for reducing CI costs.