TL;DR

Update the PyTorch version check in flex_attention.py to require version 2.9.1 or higher to correctly import AuxRequest.

Guidance

• Verify the PyTorch version being used by checking the documentation or running torch.__version__ in Python.
• Update the version check in src/transformers/integrations/flex_attention.py to >= 2.9.1 to ensure compatibility.
• Test the updated code with PyTorch version 2.9.1 or higher to confirm the fix.
• If using PyTorch 2.9.0, consider upgrading to a compatible version to use AuxRequest.

Example

import torch

# Check PyTorch version
if torch.__version__ >= '2.9.1':
    from torch.nn.attention.flex_attention import AuxRequest
else:
    print("PyTorch version 2.9.1 or higher required for AuxRequest")

Notes

This fix assumes that the only issue is the incorrect version check and that AuxRequest is correctly used in the code.

Recommendation

Apply workaround: Update the version check to >= 2.9.1 to ensure compatibility with PyTorch versions that support AuxRequest.