Bias in Generative Modeling
Generative models often produce biased statistics relative to the underlying data distribution. The bias can stem from the training dataset itself (“dataset bias”) or the model training procedure (“model bias”). In this talk, I will explore both these kinds of bias and present an importance weighting approach for bias mitigation. Our approach is broadly applicable to both likelihood-based and likelihood-free generative models. With an additional source of weak-supervision in the form of a small unbiased dataset, our technique can mitigate dataset bias without requiring any labeled data. Mitigating model bias with the technique requires no additional supervision, and our technique consistently improves standard goodness-of-fit metrics for evaluating the sample quality of state-of-the-art generative models, suggesting reduced bias. Finally, we demonstrate its utility on representative applications in a) fair data generation, b) data augmentation using generative adversarial nets, and c) model-based policy evaluation using off-policy data.
Aditya Grover is a final-year Ph.D. candidate in Computer Science at Stanford University advised by Stefano Ermon. His research focusses broadly on probabilistic machine learning, including topics in generative modeling, approximate inference, and deep learning as well as applications relating to scientific discovery and sustainable development. He is a recipient of Microsoft Research Ph.D. Fellowship, a Lieberman Fellowship, and a Data Science Scholarship. He is also a Teaching Fellow at Stanford since 2018, where he co-designed and teaches a new class on Deep Generative Models. Before joining Stanford, he obtained his bachelors in Computer Science and Engineering from IIT Delhi in 2015, where he was awarded the best experimental undergraduate thesis award.