pytorch - 💡(How to fix) Fix Adadelta uses SGD in its examples

optim = torch.optim.SGD(model.parameters(), lr=3e-4)

---

model = torch.nn.Linear(10, 10)

optim = torch.optim.Adadelta(
    model.parameters(),
    lr=1.0,      # default LR
    rho=0.9,
    eps=1e-6,
)

scheduler1 = torch.optim.lr_scheduler.LinearLR(
    optim,
    start_factor=0.1,
    end_factor=1,
    total_iters=20,
)

scheduler2 = torch.optim.lr_scheduler.CosineAnnealingLR(
    optim,
    T_max=80,
    eta_min=0.01,   # scaled for Adadelta-style LR
)

lr = torch.optim.lr_scheduler.SequentialLR(
    optim,
    schedulers=[scheduler1, scheduler2],
    milestones=[20],
)

lr.load_state_dict(torch.load("./save_seq.pt"))

# now load the optimizer checkpoint after loading the LRScheduler
optim.load_state_dict(torch.load("./save_optim.pt"))