Entropy bounds on Bayesian learning

Tristan Tomala (1) , Olivier Gossner (2)

1 GREGH - Groupement de Recherche et d'Etudes en Gestion à HEC
2 CECO - Laboratoire d'économétrie de l'École polytechnique

An observer of a process View the MathML source believes the process is governed by Q whereas the true law is P. We bound the expected average distance between P(xt|x1,...,xt−1) and Q(xt|x1,...,xt−1) for t=1,...,n by a function of the relative entropy between the marginals of P and Q on the n first realizations. We apply this bound to the cost of learning in sequential decision problems and to the merging of Q to P.

Mots clés: Bayesian learning; Repeated decision problem; Value of information; Entropy

Domaines: Economie et sciences de la décision

Tristan Tomala, Olivier Gossner. Entropy bounds on Bayesian learning. Journal of Mathematical Economics, 2008, Vol.44,n°1, pp.24-32. ⟨10.1016/j.jmateco.2007.04.006⟩. ⟨hal-00464554⟩