Issue Station
Sign In
Tools hereToolbox

pytorch - 💡(How to fix) Fix Add support for sparse block-diagonal matrices [3 comments, 3 participants]

Official PRs (…)
ON THIS PAGE

Recommended Tools

×6

Utilities matched from this issue’s tags and category — try them while you read without losing context.

GitHub issue graph ai analysis

Paste a GitHub issue URL. We fetch that issue, discover linked issues from bodies/comments/timeline, collect linked pull requests, and produce a structured English report.

The report is written in English Markdown for sharing and archival.

Helpful · Quick feedback

Loading…
GitHub stats
pytorch/pytorch#181494•Fetched 2026-04-26 05:05:33
View on GitHub
Comments
3
Participants
3
Timeline
19
Reactions
0
Author
stingpiestingpie
Participants
geurinmccuan345-makergeurinmccuan345-makerpearupearustingpiestingpie
Timeline (top)
mentioned ×6subscribed ×6labeled ×4commented ×3
RAW_BUFFERClick to expand / collapse

🚀 The feature, motivation and pitch

Create a new type of sparse tensor: block-diagonal sparse tensor, which can be stored as a BSR tensor, but operations unsupported for BSR can be implemented on the C side of things with a loop over each block. This would increase performance as the loops would not be required on the python side.

Alternatives

The alternative is to loop over an array of matrices on the python side, which is very slow.

Additional context

I'm working on a project that involves using multiple linear layers at once, instead of sequentially. The way I've implemented this is as multiple vector-matrix multiplications done in a loop. This is incredibly inefficient however, and training is so slow it's comical. The loop could be removed if the matrices were stored in a block-diagonal matrix, and I would only have to do a single multiplication with a concatenated vector. This is an incredibly good use case for sparse BSR tensors, but they are unsupported for nearly every useful operation.

cc @nikitaved @pearu @cpuhrsch @amjames @bhosmer @jcaip

extent analysis

TL;DR

Implementing a block-diagonal sparse tensor as a BSR tensor with custom C-side operations could significantly improve performance.

Guidance

  • Investigate the feasibility of implementing unsupported BSR operations on the C side to leverage the performance benefits of block-diagonal sparse tensors.
  • Explore the possibility of storing matrices in a block-diagonal format to enable a single vector-matrix multiplication, eliminating the need for a loop.
  • Consider the trade-offs between implementing custom C-side operations and the alternative of looping over an array of matrices on the Python side.
  • Evaluate the potential performance gains of using block-diagonal sparse tensors in the context of multiple linear layers and vector-matrix multiplications.

Notes

The implementation details and potential challenges of creating a custom block-diagonal sparse tensor are not fully explored in the issue, and further investigation is needed to determine the viability of this approach.

Recommendation

Apply workaround: Implementing custom C-side operations for BSR tensors could provide a significant performance boost, making it a worthwhile workaround to explore.

Vote matrix · Quick signals

Works
Did the solution work? Tap to confirm.
Easy Fix
Was it a quick fix?
Time Saver
Did it save you time?
Blocking
Was it severely blocking?
Common Issue
Are others likely hitting this too?
Flaky / Intermittent
Is it intermittent?
Verified / Reproducible
Can you reproduce it reliably?
Loading…
#prompt formatting#chain error#conversation history#tool integration#LLM response

Still need to ship something?

×6

Another batch ranked right after the header list — different links, same matching logic.

Back to top recommendations

TRENDING