The Thermodynamic Variational Objective and more

[  AI&Physics  ] 20 Jan 2021

This is a reading note centering around paper Vaden Masrani, et al. (2019) The Thermodynamic Variational Objective. This paper provides a way to construct a tighter bound, called TVO, to observed likelihood in VAE than ELBO (my note about ELBO), base on a statistical physics intuition. There are two main points I want to make here:

1. Mordern statistical machine learning could borrow a lot from statistical physics field, as they generally share many similar questions;
2. A tighter bound of learning objective does not always guarantee a better performance. Yes, a beautiful heuristic math sometimes doesn’t produce better result.

This is also what we choose to study on final project of Harvard 20 Fall AM207 course, we test this novel TVO on some sepcially-designed toy datesets. You can find our report here

TBC

---

Comments