We Do Mathematics / Machine Learning

The maximum likelihood method belongs to one of the most used methods when it comes to fitting of distribution functions against data. In machines learning a typical place where this method finds application is logistic regression. In this article we are looking into the detailed development and theory of likelihood estimation. Prerequisites in order to understand this article are just basic knowledge of classic probability theory. Our ultimate goal is to fit a conditional distribution function against given data. Let us first describe the structure of these data. The sample data The data are lists of records, each containing a set of feature variables $(X_i)_{1\leq i\leq n}$ and one dependent variable $Y$. This can be presented in matrix form like: $$ \begin{pmatrix} x_{1,1} & x_{1,2} & \cdots & x_{1,n} & y_1\\ x_{2,1} & x_{2,2} & \cdots & x_{2,n} & y_2\\ \vdots\\ x_{m,1} & x_{m,2} & \cdots & x_{m,n} & y_m\\ \end{pmatrix} $$ We a...