GAN Training — Step-by-Step Visualization

Algorithm Pattern

Minimax Adversarial Game

Key Idea

A GAN trains two networks in competition: G generates fake samples to fool D; D learns to distinguish real from fake. At convergence, G produces samples indistinguishable from real data.

Step-by-Step Approach

1. Step 1 (D update): maximize log D(real) + log(1 − D(fake)).
2. D(real) should be high (real looks real); D(G(z)) should be low (fake looks fake).
3. Step 2 (G update): maximize log D(G(z)) — fool D into scoring fake as real.
4. Alternate D and G updates — one step each per iteration.
5. Convergence: D(real)=0.5=D(fake) — D can no longer distinguish.

Common Gotchas

• Mode collapse: G learns to produce one type of sample that always fools D.
• Training instability: if D is too strong, G gets no gradient signal.
• Wasserstein GAN (WGAN) uses a different loss to stabilize training.