Message passing over factor graph can be considered as generalization of manywell known algorithms for efficient marginalization of multivariate function. Aspecific instance of the algorithm is obtained by choosing an appropriatecommutative semiring for the range of the function to be marginalized. Someexamples are Viterbi algorithm, obtained on max-product semiring andforward-backward algorithm obtained on sum-product semiring. In this paper,Entropy Message Passing algorithm (EMP) is developed. It operates over entropysemiring, previously introduced in automata theory. It is shown how EMP extendsthe use of message passing over factor graphs to probabilistic model algorithmssuch as Expectation Maximization algorithm, gradient methods and computation ofmodel entropy, unifying the work of different authors.