The Dual of the Least-Squares Method

Quirino  Paris

doi:10.4236/ojs.2015.57067

Quirino Paris

Abstract: This paper presents the dual specification of the least-squares method. In other words, while the traditional (primal) formulation of the method minimizes the sum of squared residuals (noise), the dual specification maximizes a quadratic function that can be interpreted as the value of sample information. The two specifications are equivalent. Before developing the methodology that describes the dual of the least-squares method, the paper gives a historical perspective of its origin that sheds light on the thinking of Gauss, its inventor. The least-squares method is firmly established as a scientific approach by Gauss, Legendre and Laplace within the space of a decade, at the beginning of the nineteenth century. Legendre was the first author to name the approach, in 1805, as “méthode des moindres carrés”, a “least-squares method”. Gauss, however, used the method as early as 1795, when he was 18 years old. Again, he adopted it in 1801 to calculate the orbit of the newly discovered planet Ceres. Gauss published his way of looking at the least-squares approach in 1809 and gave several hints that the least-squares algorithm was a minimum variance linear estimator and that it was derivable from maximum likelihood considerations. Laplace wrote a very substantial chapter about the method in his fundamental treatise on probability theory published in 1812.

Keywords: Least Squares, Primal, Dual, Pythagoras Theorem, Noise, Value of Sample Information

Cite this paper: Paris, Q. (2015) The Dual of the Least-Squares Method. Open Journal of Statistics, 5, 658-664. doi: 10.4236/ojs.2015.57067.

References

[1] Carolo Friderico, G. (1809) Theoria Motus Corporum Coelestium in Sectionibus Conicis Solem Ambientium. Hamburgi Sumtibus Frid, Pertheset I. H. Besser.

[2] Adrien Marie, L. (1805) Nouvelles Méthodes pour la Détermination des Orbites des Comètes, Chez Firmin Didot, Libraire pour la Mathématique, la Marine,l’Architecture, et les éditionsstéreotypes, rue de Thionville. A Paris.

[3] Charles Henry, D. (1857) Theory of the Motion of the Heavenly Bodies Moving about the Sun in Conic Sections: A Translation of Gauss’s “Theoria Motus” with an Appendix. Little Brown and Com pany, Boston.

[4] Le Comte, L.M. (1812) Théorie Analytique des Probabilités, M.me V. Courcier, Imprimeur-Libraire pour la Mathématiques, quai des Augustins. Paris.

[5] Joseph-Louis, L. (1808) Leçonssur le Calcul des Fonctions, M.me V. Courcier, Imprieur-Librairepour la Mathématiques, quai des Augustins. Paris.