Distributions: True Posterior p(z|x) vs Variational q(z)
True Posterior p(z|x)
Variational q(z)
KL Divergence Area
ELBO Decomposition: Reconstruction − KL
Optimization Landscape: log p(x) ≥ ELBO
μ = 0.0
σ = 1.0
μ = 0.5
σ = 0.8
ELBO Components
Key Equation
log p(x) = ELBO + KL(q||p)
ELBO = E_q[log p(x,z)] - E_q[log q(z)]
= E_q[log p(x|z)] - KL(q||prior)
Maximize ELBO ⟺ Minimize KL(q||p)
ELBO = E_q[log p(x,z)] - E_q[log q(z)]
= E_q[log p(x|z)] - KL(q||prior)
Maximize ELBO ⟺ Minimize KL(q||p)
Detailed Metrics
log p(x) (Evidence):
-
ELBO:
-
KL(q || p):
-
H[q] (Entropy):
-
E_q[log p(x|z)]:
-