Issue Station
Sign In
Tools hereToolbox

pytorch - 💡(How to fix) Fix add support for Predictive Coding as alternative to Backpropagation [1 comments, 2 participants]

Official PRs (…)
ON THIS PAGE

Recommended Tools

×6

Utilities matched from this issue’s tags and category — try them while you read without losing context.

GitHub issue graph ai analysis

Paste a GitHub issue URL. We fetch that issue, discover linked issues from bodies/comments/timeline, collect linked pull requests, and produce a structured English report.

The report is written in English Markdown for sharing and archival.

Helpful · Quick feedback

Loading…
GitHub stats
pytorch/pytorch#177623•Fetched 2026-04-08 00:47:12
View on GitHub
Comments
1
Participants
2
Timeline
77
Reactions
0
Author
MovGP0MovGP0
Participants
MovGP0MovGP0RoaimkhanRoaimkhan
Timeline (top)
subscribed ×35mentioned ×34labeled ×7commented ×1

Error Message

  • possible energy efficiency gains through localized error estimation
RAW_BUFFERClick to expand / collapse

🚀 The feature, motivation and pitch

There is some evidence that Predictive Coding (PC) can outperform gradient estimation over Backpropagation (BP) in some use cases:

  • online learning
  • training on small datasets
  • continual learning
  • possible energy efficiency gains through localized error estimation

Thus, we should consider to implement this algorithm into torchlib as an alternative to BP.

Alternatives

Backpropagation

There has been more research done in BP, so PB currently outperforms PC on most tasks.

(Alternatives to) algorithms like Batch Normalization, Optimizers like ADAM, and Dropout do not exist (yet) for PC.

My argument is that we should consider implementing it anyway, such that those algorithms can be developed.

Additional context

Papers

Talk

cc @ezyang @albanD @gqchen @nikitaved @soulitzer @Varal7 @bobrenjc93 @vincentqb @jbschlosser @janeyx99 @crcrpar

extent analysis

Fix Plan

To implement Predictive Coding (PC) in torchlib as an alternative to Backpropagation (BP), we need to:

  • Create a new module for PC
  • Implement the PC algorithm
  • Integrate PC with existing torchlib functionality

Example Code

import torch
import torch.nn as nn

class PredictiveCoding(nn.Module):
    def __init__(self, input_dim, output_dim):
        super(PredictiveCoding, self).__init__()
        self.input_dim = input_dim
        self.output_dim = output_dim
        self.weights = nn.Parameter(torch.randn(input_dim, output_dim))

    def forward(self, x):
        # Implement PC algorithm here
        # For example, a simple PC update rule
        prediction_error = x - torch.matmul(x, self.weights)
        self.weights.data += 0.1 * torch.matmul(x.T, prediction_error)
        return torch.matmul(x, self.weights)

# Example usage
pc = PredictiveCoding(10, 10)
input_data = torch.randn(1, 10)
output = pc(input_data)

Verification

To verify the implementation, we can:

  • Test the PC module with simple inputs and verify the output
  • Compare the performance of PC with BP on small datasets
  • Monitor the energy efficiency of PC compared to BP

Extra Tips

  • Start with a simple implementation and gradually add complexity
  • Use existing torchlib functionality to simplify the implementation
  • Consider implementing Batch Normalization, Optimizers like ADAM, and Dropout for PC in the future.

Vote matrix · Quick signals

Works
Did the solution work? Tap to confirm.
Easy Fix
Was it a quick fix?
Time Saver
Did it save you time?
Blocking
Was it severely blocking?
Common Issue
Are others likely hitting this too?
Flaky / Intermittent
Is it intermittent?
Verified / Reproducible
Can you reproduce it reliably?
Loading…
#api#ssr#installation#tensor shape#autograd error#retrieval issue#search optimization#API routing#API middleware#SSR setup

Still need to ship something?

×6

Another batch ranked right after the header list — different links, same matching logic.

Back to top recommendations

TRENDING

pytorch - 💡(How to fix) Fix add support for Predictive Coding as alternative to Backpropagation [1 comments, 2 participants]