black box variational inference (wip)

04 February 2017

notes on (Ranganath et al., 2014).

summary

understanding: 7/10
code: https://github.com/stan-dev/stan/blob/develop/src/stan/variational/advi.hpp

a key paper for modern variational inference. the elbo is maximized using a score function gradient estimator. the main problem is high variance of gradient estimators. two variance reduction techniques are presented:

• rao-blackwellization
• control variates

i don’t understand control variates very well.

---

references

1. Ranganath, R., Gerrish, S., & Blei, D. (2014). Black Box Variational Inference. Proceedings of the Seventeenth International Conference on Artificial Intelligence and Statistics, 814–822.
   • Bib

   @inproceedings{ranganath2014black,
     title = {Black Box Variational Inference},
     author = {Ranganath, Rajesh and Gerrish, Sean and Blei, David},
     booktitle = {Proceedings of the Seventeenth International Conference on Artificial Intelligence and Statistics},
     pages = {814--822},
     year = {2014}
   }
   

[back]