# Bayesian approach

to statistical problems

An approach based on the assumption that to any parameter in a statistical problem there can be assigned a definite probability distribution. Any general statistical decision problem is determined by the following elements: by a space of (potential) samples , by a space of values of the unknown parameter , by a family of probability distributions on , by a space of decisions and by a function , which characterizes the losses caused by accepting the decision when the true value of the parameter is . The objective of decision making is to find in a certain sense an optimal rule (decision function) , assigning to each result of an observation the decision . In the Bayesian approach, when it is assumed that the unknown parameter is a random variable with a given (a priori) distribution on the best decision function (Bayesian decision function) is defined as the function for which the minimum expected loss , where and is attained. Thus, In searching for the Bayesian decision function , the following remark is useful. Let , , where and are certain -finite measures. One then finds, assuming that the order of integration may be changed,   It is seen from the above that for a given is that value of for which is attained, or, what is equivalent, for which is attained, where But, according to the Bayes formula Thus, for a given , is that value of for which the conditional average loss attains a minimum.

Example. (The Bayesian approach applied to the case of distinguishing between two simple hypotheses.) Let , , , ; , , . If the solution is identified with the acceptance of the hypothesis : , it is natural to assume that , . Then  implies that is attained for the function The advantage of the Bayesian approach consists in the fact that, unlike the losses , the expected losses are numbers which are dependent on the unknown parameter , and, consequently, it is known that solutions for which and which are, if not optimal, at least -optimal , are certain to exist. The disadvantage of the Bayesian approach is the necessity of postulating both the existence of an a priori distribution of the unknown parameter and its precise form (the latter disadvantage may be overcome to a certain extent by adopting an empirical Bayesian approach, cf. Bayesian approach, empirical).