# Maximum-likelihood method

One of the fundamental general methods for constructing estimators of unknown parameters in statistical estimation theory.

Suppose one has, for an observation with distribution depending on an unknown parameter , the task to estimate . Assuming that all measures are absolutely continuous relative to a common measure , the likelihood function is defined by The maximum-likelihood method recommends taking as an estimator for the statistic defined by  is called the maximum-likelihood estimator. In a broad class of cases the maximum-likelihood estimator is the solution of a likelihood equation (1)

Example 1. Let be a sequence of independent random variables (observations) with common distribution , . If there is a density relative to some measure , then and the equations (1) take the form (2)

Example 2. In Example 1, let be the normal distribution with density where , , , . Equations (2) become  and the maximum-likelihood estimator is given by Example 3. In Example 1, let take the values and with probabilities , , respectively. Then and the maximum-likelihood estimator is .

Example 4. Let the observation be a diffusion process with stochastic differential where is a Wiener process and is an unknown one-dimensional parameter. Here (see ),  There are no definitive reasons for optimality of the maximum-likelihood method and the widespread belief in its efficiency is partially based on the great success with which it has been applied to numerous concrete problems, and partially on rigorously established asymptotic optimality properties. For example, in Example 1, under broad assumptions, with -probability . If the Fisher information exists, then the difference is asymptotically normal with parameters , and , in a well-defined sense, has an asymptotically-minimal mean-square deviation from (see , ).