of a function , defined on a domain with values in the Riemann sphere , at a point with respect to a set ,
The set of values for which there exists a sequence of points , , , such that
Every number is called a cluster value of at with respect to . The theory of cluster sets is a branch of function theory in which boundary properties of functions are studied in terms of topological and metric properties of various cluster sets.
If the entire domain is taken for , one obtains the full cluster set ; if the inclusion is strict, the corresponding set is sometimes called a partial cluster set. A full cluster set is closed; if is continuous on a set that is locally connected at , then the cluster set is either degenerate, i.e. consists of a single point, or is a non-degenerate continuum. If coincides with , then it is called a total cluster set. A number belongs to the set of recurrent values of at with respect to if there is a sequence of points , , such that , . One always has . If for some there is a path : , , in ending at a point , , , and such that , then is called an asymptotic value of at (along ). The asymptotic set is the set of all asymptotic values of at .
The notion of a cluster set was clearly formulated for the first time by P. Painlevé in 1895 (he called it the "region of indeterminacy" , cf. ) in connection with studying an analytic function near one of its singular points and with classifying singularities of such functions. At that time one basically studied three, geometrically most simple, cases in the theory of cluster sets: a) is an isolated point of the boundary or an interior point of ; b) is the unit disc or, in general, a Jordan domain, and is a point on the boundary ; and c) the boundary is an everywhere-discontinuous compactum in the plane (i.e. a totally-disconnected compact set) and . A number of classical results in complex function theory have a formulation in terms of cluster sets. E.g., the Sokhotskii theorem, in a somewhat stronger form, states: If is an isolated point of an everywhere-discontinuous compactum and is a meromorphic function on , then the cluster set is either degenerate or total. The Picard theorem, supplementing it, states that if is total, i.e. if is an essential singular point, then the set contains at most two different values. Also, in this case
(the Iversen theorem).
The main result related to the theory of the behaviour of meromorphic functions near "thin" boundaries (the Painlevé theory) is (cf. , ): If a set has linear Hausdorff measure zero, , and the function is meromorphic in , then for every point the cluster set is either degenerate or total; moreover, in the first case is also meromorphic at . Thus, a point for which the cluster set is degenerate is a removable singular point of ; the study of removable sets of various function classes can be regarded as a branch of the theory of cluster sets.
Golubev's theorem is an important strengthening of the theorem of Picard: If , and is meromorphic in , then the set has analytic capacity zero at every essential singular point (hence its plane measure ).
The work of P. Fatou (1906) on boundary values of functions holomorphic in the unit disc was the starting point for the theory of cluster sets in the case of continuous boundaries. If such a function is bounded in , then almost-everywhere (in the sense of the Lebesgue measure) on the circle it has radial and angular (non-tangential) boundary values (Fatou's theorem). Let be an arbitrary point; denote by the chord of ending at and forming with the radius at an angle , . Let be the angular domain with vertex , consisting of those points of lying between the chords
A point is called a Fatou point, and belongs to the set , if the union
over all angular domains consists of a single value , which is called the angular boundary value of at . Another formulation of Fatou's theorem: For a bounded holomorphic function in the decomposition , , holds. This result is supplemented by the F. and M. Riesz uniqueness theorem (1916): If is holomorphic and bounded in and if on some set , , it has angular boundary values , , then . This statement was proved, independently, by N.N. Luzin and I.I. Privalov (1919), who obtained an essential generalization of it to the case of arbitrary meromorphic functions. In the same year they published a boundary uniqueness theorem for the case of radial boundary values: If a function , holomorphic in , has the same radial boundary value on a set of the second category and metrically dense on some arc , i.e. if , , then .
Privalov, in 1936, noted that the statement remains true also when the values are not necessarily equal at the points , but belong to a set of (logarithmic) capacity zero. The basic idea and the elements of the proof of the Luzin–Privalov theorem are applicable in the general case of continuous mappings of , which was subsequently used in many papers.
A point is called a Plessner point, and belongs to the set , if the intersection
over all angular domains with vertex coincides with . A.I. Plessner proved (1927) that for a meromorphic function in almost-all points of the boundary belong either to or to , i.e. , . A point is called a Meier point, and belongs to , if and if the intersection of the chordal cluster sets, , over all chords drawn at , coincides with . K. Meier established (1961) the following analogue of Plessner's theorem in terms of Baire categories: If is meromorphic in , then all points of the boundary , with the possible exception of a set of the first category, belong to the union . A more precise statement of Meier's theorem has been obtained, in which is a set of the first category and of type (cf. –, in which generalizations of Plessner's and Meier's theorems have been obtained, and in which a converse of Meier's theorem and a characterization of have been given).
The work of Fatou served as an original source for the development of fundamental research on boundary properties of analytic functions. The studies of F. and M. Riesz, Luzin, Privalov, R. Nevanlinna, Plessner, V.I. Smirnov, and others were conducted independently of the ideas of Painlevé, and the use of methods related to measure and integration theory, including the notion of Baire categories, is characteristic for them (cf. –).
The basic objects of study for F. Iversen and W. Gross were meromorphic functions in domains with a Jordan boundary . At an arbitrary point , the boundary cluster set is defined as follows: If denotes the closure of the union over all points
then . One of the main theorems obtained, independently, by them asserts that, under the conditions stated, the set
is open (for any ), and all values , with possibly two exceptions, belong to the set of recurrent values . Moreover, every exceptional value (if existing) is an asymptotic value of at .
The research of Iversen and Gross obtained a further development in the work of A. Beurling, W. Seidel (who in 1932 also introduced the term "cluster set" ) and others (cf. –). They basically considered the case when belongs to a "small" set on the boundary , having zero linear measure or zero capacity, and studied the cluster set , defined analogously to . Methods of potential theory are also used in these studies.
The most recent results in this direction are stated below for the case of the disc . Suppose a set on an arc of the boundary of having is fixed, and let . To every point one assigns a Jordan arc ending at . Let be the closure of the union over all points
Then the set
is open, the set has capacity zero, and every value is an asymptotic value of either at or at every point of some sequence , , . If has capacity zero, then for every connected component , of the set consists of at most two distinct values.
Lindelöf's theorem has been proved using normal families (cf. Normal family): If a holomorphic function is bounded in and has asymptotic value at , then it has at this point as angular boundary value. Normality of a family of meromorphic functions in a domain can be characterized in terms of the so-called spherical derivative
To be precise, is a normal family if and only if the spherical derivatives , , are uniformly bounded inside , i.e. if for every compactum there is a constant such that
However, the most important occurrence of normal families in the theory of cluster sets is in the notion of a normal function. A function , meromorphic in a simply-connected domain , is called a normal function in if the family , where runs through the family of all conformal automorphisms of , is normal; is normal in a multiply-connected domain if it is normal on the universal covering surface of . A function , meromorphic in , is normal if and only if there is a constant , , such that
Here, the left-hand side is the line element in the so-called chordal metric on the Riemann sphere for the mapping , while the expression is the hyperbolic metric of . Bounded holomorphic functions and meromorphic functions not taking three distinct values are normal, and certain properties of functions of the classes indicated carry over to arbitrary normal functions. E.g., the conclusion of Lindelöf's theorem holds for arbitrary normal functions. The class of all normal meromorphic functions in has some resemblance to the class of functions of bounded characteristic (cf. Function of bounded characteristic). There are, however, essential differences. E.g., there exist normal meromorphic functions without asymptotic values, hence without radial boundary values, a fact which cannot hold for functions of bounded characteristic. G.R. MacLane ,  conducted important studies on asymptotic values. MacLane's theory allows one to obtain new proofs of already known properties of normal functions. E.g., the set of points at which a normal holomorphic function has asymptotic values, hence angular boundary values, is dense on .
The value distribution of meromorphic functions is closely connected with the notion of normality. A sequence of points in with is called a -sequence for a meromorphic function in if for every infinite subsequence and every the set
contains at most two values. It has been proved that has at least one -sequence if and only if
Thus, the value distribution of the meromorphic function is related to the structure of the cluster set of the continuous function .
Substantial progress has been made on the theory of cluster sets of general mappings , . Already in 1955 the ambiguous point theorem was proved: Let be an arbitrary mapping; then the points at which one can draw two continuous curves and such that
form a set that is at most countable. Collingwood's maximality theorem: Let be an arbitrary continuum in such that , let be the continuum obtained from by rotation over around the coordinate origin and let be an arbitrary mapping; then the points at which
form a set of the first category on . A point is said to belong to the set if the cluster set coincides with the intersection
over all angular domains with vertex . It has been proved  that
for an arbitrary mapping , where is a set of the first category of type . Conversely, for an arbitrary set of the first category and of type there exists a function , holomorphic and bounded in , for which . The set is a subset of the set of all at which
for any two angular domains and . Let and . For a given let denote the length of the largest open arc on contained in the -neighbourhood of and not having points in common with ; if such an arc does not exist, . A set is called porous on if for any point ,
a -porous set is a union of at most countably many porous sets. Every -porous set is of the first category and has linear measure zero. The equality is valid for any mapping , where is a -porous set of type . Conversely, for an arbitrary -porous set there exists a function , holomorphic and bounded in , such that .
|||P. Painlevé, "Leçons sur la théorie analytique des équations différentielles, professées à Stockholm (1895)" , Paris (1897)|
|||B. Zoretti, "Leçons sur la prolongement analytique" , Gauthier-Villars (1911)|
|||V.V. Golubev, "Univalent analytic functions. Automorphic functions" , Moscow (1961) (In Russian)|
|||I.I. [I.I. Privalov] Priwalow, "Randeigenschaften analytischer Funktionen" , Deutsch. Verlag Wissenschaft. (1956) (Translated from Russian) MR0083565|
|||K. Noshiro, "Cluster sets" , Springer (1960) MR0133464 Zbl 0090.28801|
|||E.F. Collingwood, A.J. Lohwater, "The theory of cluster sets" , Cambridge Univ. Press (1966) pp. Chapt. 9 MR0231999 Zbl 0149.03003|
|||G.R. MacLane, "Asymptotic values of holomorphic functions" , Rice Univ. Studies, Math. Monographs , 49 : 1 , Rice Univ. , Houston (1963)|
|||A.I. Markushevich, G.Ts. Tumarkin, S.Ya. Khavinson, , Studies on comtemporary problems in the theory of functions of a complex variable , Moscow (1961) pp. 100–110 (In Russian)|
|||A. Lohwater, "The boundary behaviour of analytic functions" Itogi Nauk. Mat. Anal. , 10 (1973) pp. 99–259 (In Russian)|
|||E.P. Dolzhenko, "Boundary properties of arbitrary functions" Izv. Akad. Nauk SSSR Ser. Mat. , 31 : 1 (1967) pp. 3–14 (In Russian)|
|||E.P. Dolzhenko, "The metric properties of singular sets of holomorphic functions of several variables" Ann. of Math. , 2 (1976) pp. 191–201 (In Russian) (English summary)|
|||V.I. Gavrilov, "Behavior of holomorphic functions along a chord in the unit disk" Soviet Math. Dokl. , 15 : 3 (1974) pp. 725–728 Dokl. Akad. Nauk SSSR , 216 : 1 (1974) pp. 21–23|
|||A.N. Kanatnikov, V.I. Gavrilov, "Characterization of the set for meromorphic functions" Soviet Math. Dokl. , 18 : 2 (1977) pp. 270–272 Dokl. Akad. Nauk SSSR , 233 : 1 (1977) pp. 15–17 MR437770 Zbl 0377.30022 Zbl 0373.30029|
|||A.N. Kanatnikov, "A converse to Meier's theorem on meromorphic functions" Soviet Math. Dokl. , 19 : 1 (1978) pp. 162–165 Dokl. Akad. Nauk SSSR , 238 : 5 (1978) pp. 1043–1046 MR477055|
|||W. Rudin, "Function theory in polydiscs" , Benjamin (1969) MR0255841 Zbl 0177.34101|
|||G.M. Khenkin, E.M. Chirka, "Boundary properties of holomorphic functions of several complex variables" J. Soviet Math. , 5 (1976) pp. 612–687 Itogi Nauk. i Tekhn. Sovrem. Probl. Mat. , 4 (1975) pp. 13–142|
|||W. Rudin, "Function theory in the unit ball in " , Springer (1980) MR601594 Zbl 0495.32001|
Cluster set. Encyclopedia of Mathematics. URL: http://www.encyclopediaofmath.org/index.php?title=Cluster_set&oldid=24398