pytorch - 💡(How to fix) Fix Bernoulli(probs=0).log_prob(1) returns dtype-dependent finite values instead of -inf (Geometric special-cases this and tests it)

🐛 Describe the bug

Bernoulli.log_prob of an impossible outcome at the valid boundary parameters p=0 / p=1 returns a finite, dtype-dependent value instead of −inf:

import torch

print(torch.distributions.Bernoulli(torch.tensor(0.0)).log_prob(torch.tensor(1.0)))
# tensor(-15.9424)   float32 — exp() ≈ 1.19e-7 for an event that cannot happen

print(torch.distributions.Bernoulli(torch.tensor(0.0, dtype=torch.float64)).log_prob(torch.tensor(1.0, dtype=torch.float64)))
# tensor(-36.0437)   float64 — same well-defined quantity, different answer

−15.9424 = log(float32 eps) and −36.0437 = log(float64 eps) — the result changes by ~20 nats with precision. Symmetric case: Bernoulli(1.).log_prob(0.). Both p=0 and p=1 pass argument validation (the unit-interval constraint is inclusive) and value=1 is in the support, so validation does not catch this.

This contradicts the module's own tested semantics: Geometric.log_prob explicitly special-cases the boundary —

probs = probs.clone(memory_format=torch.contiguous_format)
probs[(probs == 1) & (value == 0)] = 0
return value * (-probs).log1p() + self.probs.log()

— so Geometric(1).log_prob(torch.tensor(1.0)) correctly returns -inf, and test/distributions/test_distributions.py asserts exactly that (assertEqual(Geometric(1).log_prob(torch.tensor(1.0)), -inf)). Exact −inf for impossible events at boundary p is tested, intended behavior in torch.distributions; Bernoulli just doesn't implement it.

Root cause: Bernoulli.log_prob always routes through logits — probs_to_logits(probs, is_binary=True) calls clamp_probs, which clamps p into (eps, 1−eps) before binary_cross_entropy_with_logits. The clamp is useful for gradient stability but leaks into the forward value.

Suggested fix: special-case exact-boundary probs in log_prob (mirroring Geometric), so the forward value is exact while gradients keep the clamped path. Possibly related (same clamp family): #72525.

For reference, scipy.stats.bernoulli(0).logpmf(1) returns -inf.

Versions

torch 2.12.0, CPU (arm64, macOS); Bernoulli.log_prob unchanged on current main.

cc @fritzo @neerajprad @alicanb @nikitaved