Namespaces
Variants
Actions

Difference between revisions of "Confidence estimation"

From Encyclopedia of Mathematics
Jump to: navigation, search
m
m (tex done)
 
Line 1: Line 1:
{{TEX|want}}
+
{{TEX|done}}
  
 
A method in mathematical statistics for the construction of a set of approximate values of the unknown parameters of probability distributions.
 
A method in mathematical statistics for the construction of a set of approximate values of the unknown parameters of probability distributions.
  
Let <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/c/c024/c024620/c0246201.png" /> be a random vector assuming values in a set <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/c/c024/c024620/c0246202.png" /> in a Euclidean space and let the probability distribution of this vector belong to the parametric family of distributions defined by the densities <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/c/c024/c024620/c0246203.png" />, <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/c/c024/c024620/c0246204.png" />, <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/c/c024/c024620/c0246205.png" />, with respect to some measure <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/c/c024/c024620/c0246206.png" />. It is assumed that the true value of the parametric point <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/c/c024/c024620/c0246207.png" /> corresponding to the result of observations of <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/c/c024/c024620/c0246208.png" /> is unknown. Confidence estimation consists in constructing a certain set <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/c/c024/c024620/c0246209.png" />, depending on <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/c/c024/c024620/c02462010.png" />, containing the value of a given function <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/c/c024/c024620/c02462011.png" /> corresponding to the unknown true value of <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/c/c024/c024620/c02462012.png" />.
+
Let $  X $
 +
be a random vector assuming values in a set $  {\mathcal X} $
 +
in a Euclidean space and let the probability distribution of this vector belong to the parametric family of distributions defined by the densities $  p ( x \mid  \theta ) $,  
 +
$  x \in {\mathcal X} $,  
 +
$  \theta \in \Theta $,  
 +
with respect to some measure $  \mu (x) $.  
 +
It is assumed that the true value of the parametric point $  \theta $
 +
corresponding to the result of observations of $  X $
 +
is unknown. Confidence estimation consists in constructing a certain set $  C(X) $,  
 +
depending on $  X $,  
 +
containing the value of a given function $  u ( \theta ) $
 +
corresponding to the unknown true value of $  \theta $.
  
Let <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/c/c024/c024620/c02462013.png" /> be the range of values of the function <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/c/c024/c024620/c02462014.png" />, <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/c/c024/c024620/c02462015.png" />, and let <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/c/c024/c024620/c02462016.png" />, <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/c/c024/c024620/c02462017.png" />, be some family of sets belonging to <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/c/c024/c024620/c02462018.png" /> for all <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/c/c024/c024620/c02462019.png" /> from <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/c/c024/c024620/c02462020.png" />; moreover, it is assumed that for an arbitrary element <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/c/c024/c024620/c02462021.png" /> and any value of <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/c/c024/c024620/c02462022.png" /> the probability of the event <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/c/c024/c024620/c02462023.png" /> is defined. This probability is given by the integral
+
Let $  U $
 +
be the range of values of the function $  u ( \theta ) $,  
 +
$  \theta \in \Theta $,  
 +
and let $  C (x) $,  
 +
$  x \in {\mathcal X} $,  
 +
be some family of sets belonging to $  U $
 +
for all $  x $
 +
from $  {\mathcal X} $;  
 +
moreover, it is assumed that for an arbitrary element $  u \in U $
 +
and any value of $  \theta \in \Theta $
 +
the probability of the event $  \{ C (X) \ni u \} $
 +
is defined. This probability is given by the integral
  
<table class="eq" style="width:100%;"> <tr><td valign="top" style="width:94%;text-align:center;"><img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/c/c024/c024620/c02462024.png" /></td> </tr></table>
+
$$
 +
{\mathsf P} _ {C} ( u ,\  \theta ) \  = \  \int\limits _ {C (x) \ni u } p (x \mid  \theta ) \  d
 +
\mu (x) ,\ \  u \in U ,\ \  \theta \in \Theta ,
 +
$$
  
and is said to be the covering probability of the value of <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/c/c024/c024620/c02462025.png" /> by the set <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/c/c024/c024620/c02462026.png" /> for a given value of <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/c/c024/c024620/c02462027.png" />.
+
and is said to be the covering probability of the value of $  u $
 +
by the set $  C (X) $
 +
for a given value of $  \theta $.
  
If the true value of <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/c/c024/c024620/c02462028.png" /> is unknown, the set <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/c/c024/c024620/c02462029.png" /> (from the family of sets <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/c/c024/c024620/c02462030.png" />, <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/c/c024/c024620/c02462031.png" />) which corresponds to the results of observations of <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/c/c024/c024620/c02462032.png" /> is said to be a confidence set or an [[Interval estimator|interval estimator]] for the unknown true value of the function <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/c/c024/c024620/c02462033.png" />. The confidence probability <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/c/c024/c024620/c02462034.png" />, which can be expressed, in terms of the covering probability, by the equation
+
If the true value of $  \theta $
 +
is unknown, the set $  C (X) $(
 +
from the family of sets $  C (x) $,  
 +
$  x \in {\mathcal X} $)  
 +
which corresponds to the results of observations of $  X $
 +
is said to be a confidence set or an [[Interval estimator|interval estimator]] for the unknown true value of the function $  u ( \theta ) $.  
 +
The confidence probability $  {\mathsf P} _ {C} ( \theta ) $,  
 +
which can be expressed, in terms of the covering probability, by the equation
  
<table class="eq" style="width:100%;"> <tr><td valign="top" style="width:94%;text-align:center;"><img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/c/c024/c024620/c02462035.png" /></td> </tr></table>
+
$$
 +
{\mathsf P} _ {C} ( \theta ) \  = \
 +
{\mathsf P} _ {C} [ u ( \theta ) ,\  \theta ] ,\ \  \theta \in \Theta ,
 +
$$
  
is employed as the probability characteristic of the interval estimator <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/c/c024/c024620/c02462036.png" /> constructed according to the above rule. In other words, <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/c/c024/c024620/c02462037.png" /> is the probability of covering by the set <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/c/c024/c024620/c02462038.png" /> of the value of a given function <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/c/c024/c024620/c02462039.png" /> corresponding to the unknown true parametric point <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/c/c024/c024620/c02462040.png" />.
+
is employed as the probability characteristic of the interval estimator $  C (X) $
 +
constructed according to the above rule. In other words, $  {\mathsf P} _ {C} ( \theta ) $
 +
is the probability of covering by the set $  C (X) $
 +
of the value of a given function $  u ( \theta ) $
 +
corresponding to the unknown true parametric point $  \theta $.
  
If the confidence probability <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/c/c024/c024620/c02462041.png" /> is independent of <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/c/c024/c024620/c02462042.png" />, the interval estimator <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/c/c024/c024620/c02462043.png" /> is said to be similar to the sampling space. This name is due to the analogy between the formulas
+
If the confidence probability $  {\mathsf P} _ {C} ( \theta ) $
 +
is independent of $  \theta $,  
 +
the interval estimator $  C (X) $
 +
is said to be similar to the sampling space. This name is due to the analogy between the formulas
  
<table class="eq" style="width:100%;"> <tr><td valign="top" style="width:94%;text-align:center;"><img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/c/c024/c024620/c02462044.png" /></td> </tr></table>
+
$$
 +
{\mathsf P} _ {C} ( \theta ) \  = \  {\mathsf P} \{ C (X) \ni u ( \theta ) \mid  \theta
 +
\} \  = \  \textrm{ const }
 +
$$
  
 
and
 
and
  
<table class="eq" style="width:100%;"> <tr><td valign="top" style="width:94%;text-align:center;"><img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/c/c024/c024620/c02462045.png" /></td> </tr></table>
+
$$
 +
{\mathsf P} \{ X \in {\mathcal X} \mid  \theta \} \  = \  \textrm{ const } \  = \  1 .
 +
$$
  
In a more general situation <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/c/c024/c024620/c02462046.png" /> depends on the unknown <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/c/c024/c024620/c02462047.png" />, and for this reason the quality of the interval estimator is usually characterized in practical work by the confidence level
+
In a more general situation $  {\mathsf P} _ {C} ( \theta ) $
 +
depends on the unknown $  \theta $,  
 +
and for this reason the quality of the interval estimator is usually characterized in practical work by the confidence level
  
<table class="eq" style="width:100%;"> <tr><td valign="top" style="width:94%;text-align:center;"><img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/c/c024/c024620/c02462048.png" /></td> </tr></table>
+
$$
 +
P _ {C} \  = \  \inf \  {\mathsf P} _ {C} ( \theta ) ,
 +
$$
  
where the lower bound is calculated over the set <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/c/c024/c024620/c02462049.png" />. (The confidence level is sometimes called the confidence coefficient.)
+
where the lower bound is calculated over the set $  \Theta $.  
 +
(The confidence level is sometimes called the confidence coefficient.)
  
Optimization of confidence estimation is defined by the requirements to be met by interval estimators. For instance, if the objective is to construct confidence sets similar to sampling spaces, with a given confidence level <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/c/c024/c024620/c02462050.png" /> (<img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/c/c024/c024620/c02462051.png" />), the first requirement is expressed by the identity
+
Optimization of confidence estimation is defined by the requirements to be met by interval estimators. For instance, if the objective is to construct confidence sets similar to sampling spaces, with a given confidence level $  \omega $(
 +
0.5 \leq  \omega < 1 $),  
 +
the first requirement is expressed by the identity
  
<table class="eq" style="width:100%;"> <tr><td valign="top" style="width:94%;text-align:center;"><img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/c/c024/c024620/c02462052.png" /></td> </tr></table>
+
$$
 +
{\mathsf P} _ {C} [ u ( \theta ) ,\  \theta ] \  \equiv \  \omega ,\ \  \theta \in \Theta .
 +
$$
  
It is natural to look for an interval estimator that covers the true value of <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/c/c024/c024620/c02462053.png" /> with a probability at least that of covering an arbitrary value <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/c/c024/c024620/c02462054.png" />. In other words, the second requirement, known as the requirement of unbiasedness, is expressed by the inequality
+
It is natural to look for an interval estimator that covers the true value of $  u ( \theta ) $
 +
with a probability at least that of covering an arbitrary value $  u \in U $.  
 +
In other words, the second requirement, known as the requirement of unbiasedness, is expressed by the inequality
  
<table class="eq" style="width:100%;"> <tr><td valign="top" style="width:94%;text-align:center;"><img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/c/c024/c024620/c02462055.png" /></td> </tr></table>
+
$$
 +
{\mathsf P} _ {C} ( u ,\  \theta ) \  \leq  \  \omega ,\ \  u \in U ,\ \  \theta \in \Theta .
 +
$$
  
Under these conditions, the  "best"  interval estimator <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/c/c024/c024620/c02462056.png" /> may reasonably be taken as one which covers any value <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/c/c024/c024620/c02462057.png" /> other than the true value <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/c/c024/c024620/c02462058.png" /> with a smaller probability. Hence the third requirement of  "highest accuracy" : For any set <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/c/c024/c024620/c02462059.png" /> other than <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/c/c024/c024620/c02462060.png" /> and meeting the condition
+
Under these conditions, the  "best"  interval estimator $  C $
 +
may reasonably be taken as one which covers any value $  u $
 +
other than the true value $  u ( \theta ) $
 +
with a smaller probability. Hence the third requirement of  "highest accuracy" : For any set $  C ^ {\  \prime } $
 +
other than $  C $
 +
and meeting the condition
  
<table class="eq" style="width:100%;"> <tr><td valign="top" style="width:94%;text-align:center;"><img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/c/c024/c024620/c02462061.png" /></td> </tr></table>
+
$$
 +
{\mathsf P} _ {C ^ {\  \prime }  } [ u ( \theta ) ,\  \theta ] \  \geq  \  \omega ,\ \  \theta \in
 +
\Theta ,
 +
$$
  
 
the inequality
 
the inequality
  
<table class="eq" style="width:100%;"> <tr><td valign="top" style="width:94%;text-align:center;"><img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/c/c024/c024620/c02462062.png" /></td> </tr></table>
+
$$
 +
{\mathsf P} _ {C} ( u ,\  \theta ) \  \leq  \
 +
{\mathsf P} _ {C ^ {\  \prime }  } ( u ,\  \theta ) ,\ \  u
 +
\in U ,\ \  \theta \in \Theta ,
 +
$$
  
 
must be valid.
 
must be valid.
  
The task of finding interval estimators <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/c/c024/c024620/c02462063.png" /> satisfying all three requirements is equivalent to the task of constructing unbiased most-powerful statistical tests similar to the sampling space and having significance level <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/c/c024/c024620/c02462064.png" />. The problem of the existence of a solution to this problem and its constructive description form the base of the general theory of statistical hypothesis testing.
+
The task of finding interval estimators $  C $
 +
satisfying all three requirements is equivalent to the task of constructing unbiased most-powerful statistical tests similar to the sampling space and having significance level $  1 - \omega $.  
 +
The problem of the existence of a solution to this problem and its constructive description form the base of the general theory of statistical hypothesis testing.
  
Confidence estimation is most often used when <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/c/c024/c024620/c02462065.png" /> is a scalar function. Let <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/c/c024/c024620/c02462066.png" />, <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/c/c024/c024620/c02462067.png" />, be independent random variables subject to the same [[Normal distribution|normal distribution]], with the unknown parameters <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/c/c024/c024620/c02462068.png" /> and <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/c/c024/c024620/c02462069.png" />. The problem is to construct an interval estimator for <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/c/c024/c024620/c02462070.png" />. Let
+
Confidence estimation is most often used when $  u ( \theta ) $
 +
is a scalar function. Let $  X _ {1} \dots X _ {n} $,  
 +
$  n \geq  2 $,  
 +
be independent random variables subject to the same [[Normal distribution|normal distribution]], with the unknown parameters $  {\mathsf E} X _ {i} = \theta _ {1} $
 +
and $  {\mathsf D} X _ {i} = \theta _ {2} $.  
 +
The problem is to construct an interval estimator for $  u ( \theta ) = \theta _ {1} $.  
 +
Let
  
<table class="eq" style="width:100%;"> <tr><td valign="top" style="width:94%;text-align:center;"><img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/c/c024/c024620/c02462071.png" /></td> </tr></table>
+
$$
 +
\overline{X}\; \  =
 +
\frac{1}{n}
 +
\sum _ {i = 1 } ^ { n }  X _ {i} \ \  \textrm{ and } \ \  s
 +
^ {2} \  =
 +
\frac{1}{n-1}
 +
\sum _ {i = 1 } ^ { n }  ( X _ {i} - \overline{X}\; )
 +
^ {2} .
 +
$$
  
Since the random variable <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/c/c024/c024620/c02462072.png" /> is subject to the [[Student distribution|Student distribution]] with <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/c/c024/c024620/c02462073.png" /> degrees of freedom, and since this distribution does not depend on the unknown parameters <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/c/c024/c024620/c02462074.png" /> and <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/c/c024/c024620/c02462075.png" /> (<img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/c/c024/c024620/c02462076.png" />, <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/c/c024/c024620/c02462077.png" />), it follows that, for any positive <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/c/c024/c024620/c02462078.png" />, the probability of occurrence of the event
+
Since the random variable $  T = \sqrt n ( \overline{X}\; - \theta _ {1} ) / s $
 +
is subject to the [[Student distribution|Student distribution]] with $  n - 1 $
 +
degrees of freedom, and since this distribution does not depend on the unknown parameters $  \theta _ {1} $
 +
and $  \theta _ {2} $(
 +
$  | \theta _ {1} | < \infty $,
 +
$  \theta _ {2} > 0 $),  
 +
it follows that, for any positive $  t $,  
 +
the probability of occurrence of the event
  
<table class="eq" style="width:100%;"> <tr><td valign="top" style="width:94%;text-align:center;"><img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/c/c024/c024620/c02462079.png" /></td> </tr></table>
+
$$
 +
\left \{ \overline{X}\; -
 +
\frac{ts}{\sqrt n }
 +
< \
 +
\theta _ {1} \  < \
 +
\overline{X}\; +
 +
\frac{ts}{\sqrt n }
 +
\right \}
 +
$$
  
depends only on <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/c/c024/c024620/c02462080.png" />. If this interval is taken as the interval estimator <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/c/c024/c024620/c02462081.png" /> for <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/c/c024/c024620/c02462082.png" />, it will correspond to the confidence probability
+
depends only on $  t $.  
 +
If this interval is taken as the interval estimator $  C $
 +
for $  \theta _ {1} $,  
 +
it will correspond to the confidence probability
  
<table class="eq" style="width:100%;"> <tr><td valign="top" style="width:94%;text-align:center;"><img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/c/c024/c024620/c02462083.png" /></td> </tr></table>
+
$$
 +
{\mathsf P} _ {C} ( \theta _ {1} ,\  \theta _ {2} ) \  = \  {\mathsf P} \{ | T | <
 +
t \} ,
 +
$$
  
which is independent of <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/c/c024/c024620/c02462084.png" />. Such an interval estimator is known as a confidence interval, while its end points are known as confidence bounds; in this case the confidence interval is a confidence estimator similar to the sampling space. In this example the interval estimator is most accurate unbiased.
+
which is independent of $  \theta = ( \theta _ {1} ,\  \theta _ {2} ) $.  
 +
Such an interval estimator is known as a confidence interval, while its end points are known as confidence bounds; in this case the confidence interval is a confidence estimator similar to the sampling space. In this example the interval estimator is most accurate unbiased.
  
 
====References====
 
====References====
 
<table><TR><TD valign="top">[1]</TD> <TD valign="top">  S.S. Wilks,  "Mathematical statistics" , Wiley  (1962)</TD></TR><TR><TD valign="top">[2]</TD> <TD valign="top">  L. Schmetterer,  "Introduction to mathematical statistics" , Springer  (1974)  (Translated from German)</TD></TR><TR><TD valign="top">[3]</TD> <TD valign="top">  E.L. Lehmann,  "Testing statistical hypotheses" , Wiley  (1986)</TD></TR><TR><TD valign="top">[4]</TD> <TD valign="top">  L.N. Bol'shev,  "On the construction of confidence limits"  ''Theor. Probab. Appl.'' , '''10'''  (1965)  pp. 173–177  ''Teor. Veroyatnost. i Primenen.'' , '''10''' :  1  (1965)  pp. 187–192</TD></TR></table>
 
<table><TR><TD valign="top">[1]</TD> <TD valign="top">  S.S. Wilks,  "Mathematical statistics" , Wiley  (1962)</TD></TR><TR><TD valign="top">[2]</TD> <TD valign="top">  L. Schmetterer,  "Introduction to mathematical statistics" , Springer  (1974)  (Translated from German)</TD></TR><TR><TD valign="top">[3]</TD> <TD valign="top">  E.L. Lehmann,  "Testing statistical hypotheses" , Wiley  (1986)</TD></TR><TR><TD valign="top">[4]</TD> <TD valign="top">  L.N. Bol'shev,  "On the construction of confidence limits"  ''Theor. Probab. Appl.'' , '''10'''  (1965)  pp. 173–177  ''Teor. Veroyatnost. i Primenen.'' , '''10''' :  1  (1965)  pp. 187–192</TD></TR></table>

Latest revision as of 11:37, 10 February 2020


A method in mathematical statistics for the construction of a set of approximate values of the unknown parameters of probability distributions.

Let $ X $ be a random vector assuming values in a set $ {\mathcal X} $ in a Euclidean space and let the probability distribution of this vector belong to the parametric family of distributions defined by the densities $ p ( x \mid \theta ) $, $ x \in {\mathcal X} $, $ \theta \in \Theta $, with respect to some measure $ \mu (x) $. It is assumed that the true value of the parametric point $ \theta $ corresponding to the result of observations of $ X $ is unknown. Confidence estimation consists in constructing a certain set $ C(X) $, depending on $ X $, containing the value of a given function $ u ( \theta ) $ corresponding to the unknown true value of $ \theta $.

Let $ U $ be the range of values of the function $ u ( \theta ) $, $ \theta \in \Theta $, and let $ C (x) $, $ x \in {\mathcal X} $, be some family of sets belonging to $ U $ for all $ x $ from $ {\mathcal X} $; moreover, it is assumed that for an arbitrary element $ u \in U $ and any value of $ \theta \in \Theta $ the probability of the event $ \{ C (X) \ni u \} $ is defined. This probability is given by the integral

$$ {\mathsf P} _ {C} ( u ,\ \theta ) \ = \ \int\limits _ {C (x) \ni u } p (x \mid \theta ) \ d \mu (x) ,\ \ u \in U ,\ \ \theta \in \Theta , $$

and is said to be the covering probability of the value of $ u $ by the set $ C (X) $ for a given value of $ \theta $.

If the true value of $ \theta $ is unknown, the set $ C (X) $( from the family of sets $ C (x) $, $ x \in {\mathcal X} $) which corresponds to the results of observations of $ X $ is said to be a confidence set or an interval estimator for the unknown true value of the function $ u ( \theta ) $. The confidence probability $ {\mathsf P} _ {C} ( \theta ) $, which can be expressed, in terms of the covering probability, by the equation

$$ {\mathsf P} _ {C} ( \theta ) \ = \ {\mathsf P} _ {C} [ u ( \theta ) ,\ \theta ] ,\ \ \theta \in \Theta , $$

is employed as the probability characteristic of the interval estimator $ C (X) $ constructed according to the above rule. In other words, $ {\mathsf P} _ {C} ( \theta ) $ is the probability of covering by the set $ C (X) $ of the value of a given function $ u ( \theta ) $ corresponding to the unknown true parametric point $ \theta $.

If the confidence probability $ {\mathsf P} _ {C} ( \theta ) $ is independent of $ \theta $, the interval estimator $ C (X) $ is said to be similar to the sampling space. This name is due to the analogy between the formulas

$$ {\mathsf P} _ {C} ( \theta ) \ = \ {\mathsf P} \{ C (X) \ni u ( \theta ) \mid \theta \} \ = \ \textrm{ const } $$

and

$$ {\mathsf P} \{ X \in {\mathcal X} \mid \theta \} \ = \ \textrm{ const } \ = \ 1 . $$

In a more general situation $ {\mathsf P} _ {C} ( \theta ) $ depends on the unknown $ \theta $, and for this reason the quality of the interval estimator is usually characterized in practical work by the confidence level

$$ P _ {C} \ = \ \inf \ {\mathsf P} _ {C} ( \theta ) , $$

where the lower bound is calculated over the set $ \Theta $. (The confidence level is sometimes called the confidence coefficient.)

Optimization of confidence estimation is defined by the requirements to be met by interval estimators. For instance, if the objective is to construct confidence sets similar to sampling spaces, with a given confidence level $ \omega $( $ 0.5 \leq \omega < 1 $), the first requirement is expressed by the identity

$$ {\mathsf P} _ {C} [ u ( \theta ) ,\ \theta ] \ \equiv \ \omega ,\ \ \theta \in \Theta . $$

It is natural to look for an interval estimator that covers the true value of $ u ( \theta ) $ with a probability at least that of covering an arbitrary value $ u \in U $. In other words, the second requirement, known as the requirement of unbiasedness, is expressed by the inequality

$$ {\mathsf P} _ {C} ( u ,\ \theta ) \ \leq \ \omega ,\ \ u \in U ,\ \ \theta \in \Theta . $$

Under these conditions, the "best" interval estimator $ C $ may reasonably be taken as one which covers any value $ u $ other than the true value $ u ( \theta ) $ with a smaller probability. Hence the third requirement of "highest accuracy" : For any set $ C ^ {\ \prime } $ other than $ C $ and meeting the condition

$$ {\mathsf P} _ {C ^ {\ \prime } } [ u ( \theta ) ,\ \theta ] \ \geq \ \omega ,\ \ \theta \in \Theta , $$

the inequality

$$ {\mathsf P} _ {C} ( u ,\ \theta ) \ \leq \ {\mathsf P} _ {C ^ {\ \prime } } ( u ,\ \theta ) ,\ \ u \in U ,\ \ \theta \in \Theta , $$

must be valid.

The task of finding interval estimators $ C $ satisfying all three requirements is equivalent to the task of constructing unbiased most-powerful statistical tests similar to the sampling space and having significance level $ 1 - \omega $. The problem of the existence of a solution to this problem and its constructive description form the base of the general theory of statistical hypothesis testing.

Confidence estimation is most often used when $ u ( \theta ) $ is a scalar function. Let $ X _ {1} \dots X _ {n} $, $ n \geq 2 $, be independent random variables subject to the same normal distribution, with the unknown parameters $ {\mathsf E} X _ {i} = \theta _ {1} $ and $ {\mathsf D} X _ {i} = \theta _ {2} $. The problem is to construct an interval estimator for $ u ( \theta ) = \theta _ {1} $. Let

$$ \overline{X}\; \ = \ \frac{1}{n} \sum _ {i = 1 } ^ { n } X _ {i} \ \ \textrm{ and } \ \ s ^ {2} \ = \ \frac{1}{n-1} \sum _ {i = 1 } ^ { n } ( X _ {i} - \overline{X}\; ) ^ {2} . $$

Since the random variable $ T = \sqrt n ( \overline{X}\; - \theta _ {1} ) / s $ is subject to the Student distribution with $ n - 1 $ degrees of freedom, and since this distribution does not depend on the unknown parameters $ \theta _ {1} $ and $ \theta _ {2} $( $ | \theta _ {1} | < \infty $, $ \theta _ {2} > 0 $), it follows that, for any positive $ t $, the probability of occurrence of the event

$$ \left \{ \overline{X}\; - \frac{ts}{\sqrt n } \ < \ \theta _ {1} \ < \ \overline{X}\; + \frac{ts}{\sqrt n } \right \} $$

depends only on $ t $. If this interval is taken as the interval estimator $ C $ for $ \theta _ {1} $, it will correspond to the confidence probability

$$ {\mathsf P} _ {C} ( \theta _ {1} ,\ \theta _ {2} ) \ = \ {\mathsf P} \{ | T | < t \} , $$

which is independent of $ \theta = ( \theta _ {1} ,\ \theta _ {2} ) $. Such an interval estimator is known as a confidence interval, while its end points are known as confidence bounds; in this case the confidence interval is a confidence estimator similar to the sampling space. In this example the interval estimator is most accurate unbiased.

References

[1] S.S. Wilks, "Mathematical statistics" , Wiley (1962)
[2] L. Schmetterer, "Introduction to mathematical statistics" , Springer (1974) (Translated from German)
[3] E.L. Lehmann, "Testing statistical hypotheses" , Wiley (1986)
[4] L.N. Bol'shev, "On the construction of confidence limits" Theor. Probab. Appl. , 10 (1965) pp. 173–177 Teor. Veroyatnost. i Primenen. , 10 : 1 (1965) pp. 187–192
How to Cite This Entry:
Confidence estimation. Encyclopedia of Mathematics. URL: http://encyclopediaofmath.org/index.php?title=Confidence_estimation&oldid=44400
This article was adapted from an original article by Yu.V. LinnikN.M. Khalfina (originator), which appeared in Encyclopedia of Mathematics - ISBN 1402006098. See original article