Evidence Information in Bayesian Updating

Abstract

Bayes theorem (discrete case) is taken as a paradigm of information acquisition. As mentioned by Aitchison, Bayes formula can be identified with perturbation of a prior probability vector and a discrete likelihood function, both vectors being compositional. Considering prior, posterior and likelihood as elements of the simplex, a natural choice of distance between them is the Aitchison distance. Other geometrical features can also be considered using the Aitchison geometry. For instance, orthogonality in the simplex allows to think of orthogonal information, or the perturbation-difference to think of opposite information. The Aitchison norm provides a size of compositional vectors, and is thus a natural scalar measure of the information conveyed by the likelihood or captured by a prior or a posterior. It is called evidence information, or e-information for short. In order to support such e-information theory some principles of e-information are discussed. They essentially coincide with those of compositional data analysis. Also, a comparison of these principles of e-information with the axiomatic Shannon-information theory is performed. Shannoninformation and developments thereof do not satisfy scale invariance and also violate subcompositional coherence. In general, Shannon-information theory follows the philosophy of amalgamation when relating information given by an evidence-vector and some sub-vector, while the dimension reduction for the proposed e-information corresponds to orthogonal projections in the simplex. The result of this preliminary study is a set of properties of e-information that may constitute the basis of an axiomatic theory. A synthetic example is used to motivate the ideas and the subsequent discussion