Generative models: models that output high-dimensional data (or anything involving a GAN, VAE, PixelCNN, etc). It is useful for lots of problems beyond density estimation and sampling random images.

What Can We Do

  • Data prediction
    • Use generative model to convert a Gaussian nosie to a synthesized image
    • Or a word (Conditional Generative Model)
      • Model $p(x, y)$, then do inference to get conditional $p(y|x)$
      • directly model $p(y|x)$
      • GAN loss
        • G tries to synthesize fake images that fool D: $\arg \min_G E_{x,y}[logD(G(x)) + log(1-D(y))]$
        • D tries to identify the fakes: $\arg \max_D E_{x,y}[logD(G(x)) + log(1-D(y)]$
        • G tries to synthesize fake images that fool the best D: $\arg \min_G \max_D E_{x,y}[logD(G(x)) + log(1-D(y))]$
        • D is a loss function to G
      • Patch Discriminator
        • Rather than penalizing if output image looks fake, penalize if each overlapping patch in output looks fake.
          • Faster, fewer parameters
          • More supervised observations
          • Applies to arbitrarily large images
    • Challenges
      • Output is a high-dimensional, structured objects
      • Uncertainty in the mapping, many plausible outputs
  • Domain mapping
  • Representation Learning
  • Model-based Intelligence