Why Least Squares and Maximum Entropy? An Axiomatic Approach to Inference for Linear Inverse Problems

@article{citeulike:586727, abstract = {An attempt is made to determine the logically consistent rules for selecting a vector from any feasible set defined by linear constraints, when either all n-vectors or those with positive components or the probability vectors are permissible. Some basic postulates are satisfied if and only if the selection rule is to minimize a certain function which, if a "prior guess" is available, is a measure of distance from the prior guess. Two further natural postulates restrict the permissible distances to the author's f-divergences and Bregman's divergences, respectively. As corollaries, axiomatic characterizations of the methods of least squares and minimum discrimination information are arrived at. Alternatively, the latter are also characterized by a postulate of composition consistency. As a special case, a derivation of the method of maximum entropy from a small set of natural axioms is obtained.}, author = {Csiszar, Imre}, citeulike-article-id = {586727}, comment = {Referenced by Banerjee et al's "Clustering with Bregman Divergences".}, journal = {The Annals of Statistics}, keywords = {bregman\_divergence, f-divergence, max-ent}, number = {4}, pages = {2032--2066}, posted-at = {2008-04-22 01:33:59}, priority = {3}, title = {Why Least Squares and Maximum Entropy? An Axiomatic Approach to Inference for Linear Inverse Problems}, url = {http://www.jstor.org/stable/2241918}, volume = {19}, year = {1991} }

TY - JOUR ID - citeulike:586727 L3 - citeulike-article-id:586727 N2 - An attempt is made to determine the logically consistent rules for selecting a vector from any feasible set defined by linear constraints, when either all n-vectors or those with positive components or the probability vectors are permissible. Some basic postulates are satisfied if and only if the selection rule is to minimize a certain function which, if a "prior guess" is available, is a measure of distance from the prior guess. Two further natural postulates restrict the permissible distances to the author's f-divergences and Bregman's divergences, respectively. As corollaries, axiomatic characterizations of the methods of least squares and minimum discrimination information are arrived at. Alternatively, the latter are also characterized by a postulate of composition consistency. As a special case, a derivation of the method of maximum entropy from a small set of natural axioms is obtained. IS - 4 JF - The Annals of Statistics EP - 2066 TI - Why Least Squares and Maximum Entropy? An Axiomatic Approach to Inference for Linear Inverse Problems VL - 19 SP - 2032 KW - bregman_divergence KW - f-divergence KW - max-ent N1 - Referenced by Banerjee et al's "Clustering with Bregman Divergences". AU - Csiszar, Imre PY - 1991/// UR - http://www.jstor.org/stable/2241918 ER -