Invariance of a statistical procedure

The equivariance (see below) of some decision rule in a statistical problem, the statement of which admits of a group $ G $ of symmetries, under this group $ G $. The notion of invariance of a statistical procedure arises in the first instance in so-called parametric problems of mathematical statistics, when there is a priori information: the probability distribution $ P ( d \omega ) $ of the outcomes $ \omega $ of an observation belongs to a known family $ \{ {P _ \theta } : {\theta \in \Theta } \} $. A statistical decision problem is said to be $ G $-equivariant under a group $ G $ of measurable transformations $ g $ of a measurable space $ ( \Omega , B _ \Omega ) $ of outcomes if the following conditions hold: 1) there is a homomorphism $ f $ of $ G $ onto a group $ \overline{G} $ of transformations of the parameter space $ \Theta $,

$$ f : g \rightarrow \overline{g} \in \overline{G} ,\ \forall g \in G , $$

with the property

$$ ( P _ \theta g ) ( \cdot ) = P _ {\overline{g} ( \theta ) } ( \cdot ) ,\ \ \forall g \in G ; $$

2) there exists a homomorphism $ h $ of $ G $ onto a group $ \widehat{G} $ of measurable transformations of a measurable space $ ( D , B _ {D} ) $ of decisions $ d $,

$$ h : g \rightarrow \widehat{g} \in \widehat{G} ,\ \forall g \in G , $$

with the property

$$ L ( \overline{g} ( \theta ) , \widehat{g} ( d ) ) = L ( \theta , d ) , $$

where $ L ( \theta , d ) $ is the loss function; and 3) all the additional a priori information on the possible values of the parameter (the a priori density $ p ( \theta ) $, the subdivision into alternatives $ \Theta = \Theta _ {1} \cup \dots \cup \Theta _ {s} $, etc.) is $ G $-invariant or $ G $-equivariant. Under these conditions, the decision rule $ \delta : \omega \rightarrow \delta ( \omega ) \in D $, whether deterministic or random, is called an invariant (more precisely, a $ G $-equivariant) procedure if

$$ \delta ( g ( \omega ) ) = \widehat{g} ( \delta ( \omega ) ) ,\ \ \forall \omega \in \Omega ,\ \forall g \in G . $$

The risk

$$ r _ \delta ( \theta ) = {\mathsf E} _ \theta L ( \theta , \delta ( \omega ) ) $$

of an equivariant decision procedure $ \delta $ is $ G $-invariant; in particular, it does not depend on $ \theta $ if the group $ G $ acts transitively on $ \Theta $.

In parametric problems there is, in general, no guaranteed optimal decision procedure which minimizes the risk for each value of the parameter $ \theta \in \Theta $. In particular, a procedure may lead to very small values of the risk for certain values of $ \theta $ at the expense of worsening the quality for other equally-possible a priori values of the parameter. Equivariance guarantees to some extent that the approach is unbiased. When the group $ G $ is sufficiently rich, there is an optimal invariant procedure with a uniformly minimal risk among the invariant procedures.

Invariant procedures are widely applied in hypotheses testing (see also Invariant test) and in the estimation of the parameters of a probability distribution. Thus, in the problem of estimating an unknown vector of means for the family of $ m $-dimensional normal distributions

$$ p ( \mathbf x , \pmb\alpha ) = \frac{1}{( 2 \pi ) ^ {m/2} } \mathop{\rm exp} \left [ \frac{- \sum _ {j} ( x _ {j} - \alpha _ {j} ) ^ {2} }{2} \right ] $$

with unit covariance matrix and quadratic loss function $ \sum _ {j} ( \delta _ {j} - \alpha _ {j} ) ^ {2} $, the optimal equivariant estimator is the ordinary sample mean

$$ \mathbf x ^ {*} = \frac{\mathbf x ^ {(} 1) + \dots + \mathbf x ^ {(} N) }{N} . $$

Here the group $ G $ is given by the product of the group $ S _ {N} $ of permutations of the observations and the group $ \mathop{\rm Ort} ( m ) $ of motions of the Euclidean space $ \mathbf R ^ {m} $; $ \overline{G} = \widehat{G} = \mathop{\rm Ort} ( m) $. For $ m \geq 3 $, there exist for this problem non-equivariant estimators leading to a smaller risk than for $ \mathbf x ^ {*} $ for all $ \pmb\alpha $; however, the region of essential "superefficiency" turns out to be insignificant and diminishes without bound as the size $ N $ of the sample increases. The possibility of superefficient procedures is connected with the non-compactness of $ G $.

Equivariant statistical procedures also arise in a number of non-parametric statistical problems, when the a priori family of distributions $ P $ of outcomes is essentially infinite-dimensional, as well as in the construction of confidence sets for the parameter $ \theta $ of the distribution in the presence of nuisance parameters.

References

[1]	E.L. Lehmann, "Testing statistical hypotheses" , Wiley (1986)