pytorch - 💡(How to fix) Fix [pipelining] Add downstream tests to test pipelining integration with popular HF transformers architectures

🚀 The feature, motivation and pitch

I recently tried using the pipelining module to pipeline models with various popular architectures (GPT2, Gemma, Qwen, Llama, BERT) using implementations in the HF transformers library. I encountered many issues, which I reported. Since, in my view, training large transformer models, such as those in HF transformers, is probably the main use case of the pipelining module, it would make sense that there are some tests to ensure that those models actually work. I'm not familiar with how this would be realized, or if something like this already exists, but in my mind this type of primary use case should have associated end-to-end tests to ensure they work out-of-the-box. This was not the case from my experience starting roughly about a month ago; I had to spend quite some time troubleshooting the pipelining module to actually get a basic "will it run" script to work for all the listed models when using 2.11.

I took a brief look at the tests for the pipelining module under test/distributed/pipelining. I didn't see anything the tested actual models from the HF transformers library, only handwritten unit/integration tests. I know that dynamo/compiler have extensive tests for HF transformers and timm, so I'm wondering if something like that at a smaller scale could be implemented for the pipelining module

Alternatives

No response

Additional context

https://github.com/pytorch/pytorch/issues/184361#issuecomment-4508567998

cc @awgu @wanchaol @fegin @fduwjj @wz337 @wconstab @d4l3k @pragupta @msaroufim @dcci @aditvenk @weifengpy