Rigidity of Harmonic Maps of Maximum Rank

James A. Carlson and Domingo Toledo

June 24, 1992

Contents

1. Introduction. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 2. Outline of the proof . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 3. Existence of adapted root systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 4. Root systems. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14 5. Combinatorial arguments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18 6. Centralizers of semi-simple elements. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29 7. Rigidity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34 8. Existence. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37 9. Concluding Remarks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40

§ 1. Introduction.

Fix an irreducible symmetric space X = G/K of non-compact type which is not Hermitian symmetric, and where the group G is a classical group. The purpose of this paper is to study the “largest” possible mappings of compact Kähler manifolds to quotients of X by discrete groups of isometries.

To make the meaning of “largest” precise, let us define the homotopical rank of a continuous map between compact manifolds to be the smallest dimension of the image of a map in the same homotopy class. Our goal is then to find the maximum homotopical rank of maps f : M → N, where M is a compact Kähler manifold and N is a compact, locally symmetric manifold covered by X. Since it is known that this number is strictly smaller than the dimension of X, cf. Cor. (3.3) of [5], it remains to find the sharp upper bound, and to classify the maps that attain this bound. A reasonable way to do this is to answer the following.

Suppose X_{1} = G_{1}/K_{1} is a Hermitian symmetric space that can be totally geodesically embedded in X, and is of maximum dimension among all such Hermitian symmetric spaces.

Question 1: Is the homotopical rank of f at most the dimension of X_{1}?

Question 2: If f : M → N has homotopical rank equal to the dimension of X_{1}, must X_{1} have a compact locally symmetric quotient N_{1} which admits a totally geodesically

Research partially supported by the National Science Foundation immersion g : N_{1} → N so that f factors as f = gh for some holomorphic map h : M → N_{1}?

Both questions have affirmative answers when the symmetric space X_{1} is embedded in X as the fixed point set of an involution, provided that a small list of exceptions is excluded. More precisely, we prove the following:

1.1. Theorem. Let X = G/K where G is one of the groups SL(2n, ℝ), n ≥ 3, SO^{+}(2p, 2q), p, q ≥ 3, (p, q) ≠ (3, 3), SU^{∗}(2n), n ≥ 3 and Sp(p, q), p, q ≥ 2. Then the answer to both questions is affirmative, where X_{1} = G_{1}/K_{1} is embedded in X by the natural inclusion of G_{1} = Sp(n, ℝ), U(p, q), SO^{∗}(2n) and U(p, q) in G, respectively.

Question 1 has an affirmative answer for all X as in the first paragraph, provided that dim(X_{1}) is replaced by dim(X_{1}) + 2. Then the sharp bound is either dim(X_{1}) or dim(X_{1})+2. Just which alternative holds cannot, however, be decided by the methods of this paper. Problems of a global nature which bear on this question are discussed in §9. In any case, the bound that we obtain is much lower than the dimension of X, usually about 1/2 dim(X), and so is a considerable improvement on [5], Cor. (3.2).

Observe that if in Theorem (1.1) we had also assumed that M is locally symmetric, then the conclusion could easily have been derived from the rigidity theorem of Margulis [16]. The point of Theorem (1.1) is therefore that the only assumption on M is that it is a compact Kähler manifold. Consequently this theorem is in the same relation to the Margulis rigidity theorem as Siu’s rigidity theorem is to Mostow’s rigidity theorem, cf. [18]. [23].

The method of proof follows the pattern pioneered by Siu in the proof of his rigidity theorem [23]. Since N has non-positive curvature, the existence theorem of Eells and Sampson [9] applies and the map f may be assumed to be harmonic. Its rank therefore gives an upper bound on the homotopical rank. Since M is compact Kähler and the curvature tensor of N is rather special, any harmonic map of M to N is actually pluriharmonic, i.e., the restriction of f to any local complex curve in M is harmonic [19]. If dim(M) > 1, the resulting system of partial differential equations satisfied by f is overdetermined, and can be very effectively studied by just considering the two-jets of a solution. The technical content of this paper is in fact mostly the study of the two-jet of a pluriharmonic mapping of a complex manifold to a symmetric space whose one-jet has maximum possible rank. The results of our study will then have immediate topological consequences to the study of homotopical rank of continuous maps.

A key first-order consequence of pluriharmonicity is the following [19]. For each x ∈ M, df(T_{x}^{1,0}M) is an isotropic subspace of T_{f(x)}^{ℂ}N with respect to the complex multilinear extension of the curvature tensor R of N, i.e., R(X, Y, X, Y) = 0 for all X, Y ∈ df(T_{x}^{1,0}M). Thus, if 𝔤 = 𝔨 ⊕ 𝔭 is a Cartan decomposition of the Lie algebra of G, where 𝔨 is the isotropy algebra at f(x), then 𝔭 is naturally isomorphic to T_{f(x)}N, and df(T_{x}^{1,0}M) corresponds to a subspace W of 𝔭^{ℂ}. The above isotropy condition is then equivalent to the assertion that W is abelian: [W, W] = 0. It is therefore natural to study pluriharmonic maps of maximum rank via the following three-step program:

1) For each real simple Lie algebra 𝔤 with Cartan decomposition 𝔤 = 𝔨 ⊕ 𝔭, find the maximum dimension of W ⊂ 𝔭^{ℂ}, W abelian.

2) Classify, up to automorphism of 𝔤, all abelian subalgebras of maximum dimension.

3) Use (2) to find normal forms for the pluriharmonic maps of maximum possible rank.

We carry out step 1 for all the classical real simple Lie algebras for which the problem has not been previously solved. These are the classical real simple Lie algebras whose complexification is also simple (i.e., corresponding to the symmetric spaces of type III in Helgason’s terminology, [13], p. 379), which means that the corresponding symmetric space is not Hermitian symmetric and has rank at least two. These are the Lie algebras of the groups SL(n, ℝ), n > 2, SO(p, q), p, q > 2, SU^{∗}(2n), n > 2, and Sp(p, q), p, q > 1.

It turns out that for some of these algebras an abelian subalgebra of 𝔭^{ℂ} of maximum dimension is obtained in a very geometric way: it is the space of (1, 0) tangent vectors to a Hermitian symmetric subspace G_{1}/K_{1} which is fixed by an involution of G/K. (The integrability condition for the complex structure on G_{1}/K_{1} is equivalent to assertion that the space of (1,0)-tangent vectors is abelian). Such algebras can be locally realized by pluriharmonic maps, and can be realized globally by taking suitable quotients by discrete subgroups, where for this last step one uses [2]. We call these “algebras of Type I” (cf. §2 ) and carry out steps 2 and 3 only for the subclass listed in the statement of Theorem 1.1.

The remaining algebras in our list we call “algebras of Type II”, cf. §2 for the precise definition. For these algebras (with the exception of SO(6, 6)) the largest dimension of an abelian subalgebra of 𝔭^{ℂ} is one more than the largest dimension of a Hermitian symmetric subspace, and in most cases this subspace is no longer fixed by an involution. The question of realizability of the largest abelian subalgebra by the tangent space of a pluriharmonic map is discussed in §9.

So far we have stated the consequences of our local results in terms of mappings between compact manifolds. However, the present state of the existence theory of harmonic mappings (cf. §2 of [8] for a recent survey) makes it clear that the compactness of the target is not essential, and, more generally, that the map can be replaced by a harmonic section of a flat G/K-bundle. In particular, the existence theorem of Corlette for harmonic sections [6], combined with our results, gives immediate applications to representations of fundamental groups of compact Kähler manifolds into the Lie groups in question. Thus, if ρ : π_{1}(M) → G is a representation, where G is as in Theorem 1 and the harmonic section of ρ has rank = dim(X_{1}), then the image of ρ must be contained in an embedding of an appropiate subgroup G_{1}.

In this context it is natural to use characteristic classes to give lower bounds on the rank of a harmonic section (even though a notion of homotopical rank still applies). We treat an example for the group SO^{+}(2p, 2q): given ρ : π_{1}(M) → SO^{+}(2p, 2q), the associated flat bundle E with fibre ℝ^{2p+2q} is isomorphic, as a vector bundle, to a direct sum E_{+} ⊕ E_{−} of bundles with fiber ℝ^{2p} and ℝ^{2q} respectively, namely the maximal positive and negative sub-bundles for the indefinite inner product. Neither of these bundles is necessarily flat. In particular, if e_{+} ∈ H^{2p}(M) denotes the Euler class of E_{+}, then e_{+} does not necessarily vanish, and its powers can be used to give lower bounds for the rank of a harmonic section:

1.2. Theorem. Let M be a compact Kähler manifold, let ρ : π_{1}(M) → SO^{+}(2p, 2q), p, q ≥ 3, (p, q) ≠ (3, 3), be a representation, and let e_{+} ∈ H^{2p}(M) be the cohomology class associated to ρ as above. If e_{+}^{q} ≠ 0, then the image of ρ is contained in some embedding of U(p, q) in SO^{+}(2p, 2q).

The reason we restrict ourselves to the algebras listed above is that all the other classical cases are known. If the Lie algebra is that of the automorphism group of a Hermitian symmetric space then the type of problems that we study have all been answered by Siu [23] and are equivalent to his rigidity theorem. If 𝔤 is simple but 𝔤^{ℂ} is not simple, then 𝔤 is a simple complex Lie algebra regarded as a real Lie algebra. The corresponding symmetric spaces are those of the form G/G_{u}, where G is a simple complex Lie group and G_{u} is its compact real form, i.e., the symmetric spaces of type IV ([13], p. 379). In this case the abelian subalgebras of 𝔭^{ℂ} are the same as the abelian subalgebras of the complex simple Lie algera 𝔤. In this case problems 1 and 2 have long been solved, namely by Schur [20] for 𝔰𝔩(n, ℂ) and by Malcev [15] for all the complex simple Lie algebras. The solution of problem 3 above is contained in the paper of Corlette [7] (for the classical 𝔤 and two of the exceptional ones). Finally, if 𝔤 is classical, of real rank one, and not already included in the above cases, then either G = SO^{+}(1, n), n ≥ 4, or G = Sp(1, n), n ≥ 2, which are treated in [19], [5].

From the point of view of thoroughness, there remain only some of the exceptional Lie algebras. For the twelve exceptional real Lie algebras whose complexification is simple, two are Hermitian symmetric, hence included in [24], while real form of F_{4} corresponding to the Cayley plane is treated in [3]. Consequently there remain nine algebras for which the questions 1, 2, 3 above have not been studied. For the exceptional simple complex Lie algebras there remain three for which question 3 has not been studied.

We emphasize that in this paper we study only the pluriharmonic mappings which have maximum possible rank. Other classes of pluriharmonic maps are: (1) those that factor through any totally geodesic Hermitian subspace, (2) those that factor through Riemann surfaces, and (3) those that factor through totally geodesic real tori. It is natural to ask if there are additional, nonobvious classes. It turns out that there is at least one other construction, which comes from algebraic geometry: the theory of variations of Hodge structure gives natural examples of mappings of algebraic surfaces into quotients of symmetric spaces of suitable groups of type SO(p, q) that do not fall into any of the above categories.

Our interest in variations of Hodge structure originates with our observation in [5], §6, that they are a substitute, for non-Hermitian symmetric targets, for holomorphic maps to Hermitian symmetric targets. In particular, for each non-Hermitian X one would expect a number r(X) so that every pluri-harmonic map to X of rank larger than r(X) lifts to a variation of Hodge structure, extending our result in [5], Theorem (6.1) for X the quaternionic hyperbolic space. The reason for this expectation is its analogy with similar rank results of Siu for Hermitian targets, conjectured in §6 of [24] and proved in [25]. We present a precise result of this nature in §9.

Finally, we mention that Simpson [22] has recently constructed other classes of harmonic mappings that are neither variations of Hodge structure nor fall into any of the three obvious classes mentioned above. The resulting abelian spaces consist of semisimple elements and are therefore quite different in nature from the cases we study.

The paper is organized as follows. In §2 we give a detailed outline of the proofs of steps 1-3, and carry out the details in §3–7. In §8 we present the example from Hodge theory, and in the final section §9 we discuss further results and open problems.

As we have already mentioned, this paper and Corlette’s paper [7] have complementary results concerning the classification of harmonic mappings. We have also borrowed methods from each other in obtaining and presenting the results, and we are very grateful to Corlette for his contributions to this paper. We thank O. Gabber for explaining to us the relevance cohomological methods and Kostant’s theorem to rigidity. We also thank H. Hecht and P. Trombi for help in Lie algebra matters, A. Beauville, P. Deligne, M. Gromov, and R. Varley for enlightening remarks concerning the examples in §8, and Carlos Simpson and Kevin Corlette for their insightful comments regarding Theorem 9.3. Finally we are grateful to the IHES for its hospitality while a good portion of this work was in progress.

§ 2. Outline of the proof

Let 𝔤 be a real simple Lie algebra as in the introduction. In particular, 𝔤^{ℂ} is simple and 𝔤 is not the Lie algebra of the automorphism group of a symmetric domain. In this section we sketch the methods that we use to carry the steps 1 to 3 described in the introduction.

Step 1: Dimension bounds. Let W be an abelian subalgebra of 𝔤^{ℂ} which lies in 𝔭^{ℂ}. Without loss of generality we may assume that W is also maximal with respect to inclusion among such algebras. The problem is to bound the dimension of W, and this can be accomplished by an elaboration of the general outline of Malcev [15] , which consists of three steps:

Part a: Reduction to nilpotents. The space W has a Jordan decomposition

W = W_{σ} ⊕ W_{ν},

where the first space consists of semisimple elements and the second of nilpotents. In addition, we may assume, after possible conjugation by an element of K^{ℂ}, that W_{σ} is defined over the reals, cf. [5], Lemma (4.2). Then the centralizer of W_{σ} is the complexification reductive Lie subalgebra of 𝔤, invariant under the Cartan involution, which decomposes as 𝔷 ⊕ 𝔩, where 𝔷 is the center and where 𝔩 is semisimple. Furthermore W_{σ} ⊂ 𝔷^{ℂ}, and if 𝔩 = ⊕𝔩_{i} is a decomposition into simple ideals, then each 𝔩_{i} is the Lie algebra of a sub-symmetric space, with Cartan decomposition 𝔩_{i} = 𝔨_{i} ⊕ 𝔭_{i}, and W_{ν} = ⊕W_{i}, where W_{i} = W_{ν} ∩ 𝔭_{i}^{ℂ}. We then have

W = W_{σ} ⊕ W_{1} ⊕ ⋯ ⊕ W_{k}

where each W_{i} ⊂ 𝔭_{i}^{ℂ} consists of nilpotent elements, cf. §4 of [5] for details.

In §6 we find all the possible subalgebras 𝔩_{i} that can occur for an algebra 𝔤 in our list. It turns out that possible subalgebras 𝔩_{i} are also essentially in our list ( we only have to add some of the cases excluded as “already known”). Thus,the problem is reduced to bounding the dimension of the abelian subalgebras of 𝔭^{ℂ} consisting entirely of nilpotent elements for the algebras 𝔤 of the following groups: SL(n, ℝ), n ≥ 2, SU^{∗}(2n), n ≥ 2, SO(p, q) and Sp(p, q), p, q arbitrary.

Part b: Reduction to commutative systems of non-compact roots. Let us assume, therefore, that W consists entirely of nilpotent elements. By Lemma 3.2, there is a Cartan subalgebra 𝔱 of 𝔨 with the following properties. First the eigenvalues of the adjoint action of 𝔱 on 𝔤^{ℂ} form a root system Δ ⊂ 𝔱^{∗}. For each α ∈ Δ let 𝔤_{α} denote the corresponding eigenspace. Then there is a system of positive roots Δ^{+} ⊂ Δ such that W is contained in the sum of the positive root spaces:

W ⊂ ∑_{α∈Δ^{+}} 𝔤_{α}.

A root system Δ with this property will be called adapted to W. Because the dimension of 𝔱 may be smaller than the rank of 𝔤^{ℂ}, the adapted root system may be a restricted one. In particular, the spaces 𝔤_{α} may have dimension greater than one. However, since 𝔱 is invariant under the Cartan involution, so are the 𝔤_{α}, and consequently these spaces decompose into compact and noncompact parts: 𝔤_{α} = 𝔨_{α} ⊕ 𝔭_{α}, where 𝔨_{α} ⊂ 𝔨^{ℂ} and 𝔭_{α} ⊂ 𝔭^{ℂ}. Roots have multiplicity at most two, and when the multiplicity equals two, both terms in the above decomposition of 𝔤_{α} are nonzero. In any case, there are sets

\begin{matrix}Δ(𝔨) = \{ α ∈ Δ | 𝔨_{α} ≠ 0 \} \\ Δ(𝔭) = \{ α ∈ Δ | 𝔭_{α} ≠ 0 \}, \end{matrix}

where the first is a root system for the compact subalgebra 𝔨, and where

𝔭^{ℂ} = ∑_{α∈Δ(𝔭)} 𝔭_{α}.

The sets Δ(𝔨) and Δ(𝔭) are disjoint if and only if the all roots have multiplicity one. We shall use the term “compact” and “noncompact” to refer to elements of either set, bearing in mind that a root can be both compact and noncompact. Let Δ^{+}(𝔭) = Δ^{+} ∩ Δ(𝔭), and let

𝔭^{+} = ∑_{α∈Δ^{+}(𝔭)} 𝔭_{α}.

Then W ⊂ 𝔭^{+}.

Now choose a total ordering of the roots which is compatible with addition and with the partial ordering defined by Δ^{+}. Using Gaussian elimination, one can construct a basis { Z_{i}, i = 1, ⋯ d } for W which has the form Z_{i} = X_{i} + Y_{i}, where

i) X_{i} ∈ 𝔭_{α(i)}, ii) α(i) < α(j) for i < j. iii) Y_{i} is a linear combination of vectors X_{β} ∈ 𝔭_{β} with β > α(i) and β ∉ { α(1), ⋯, α(d) }.

Let C = {α(i) : i = 1, ⋯, d}. The elements of C are called the leading roots of W; they depend upon the choice of a total order.

Now form the Lie bracket of two basis vectors:

[Z_{i}, Z_{j}] = [X_{i}, X_{j}] + ([X_{i}, Y_{j}] + [Y_{i}, X_{j}]) + [Y_{i}, Y_{j}].  (2.1)

By construction, the first term in the right-hand side belongs to the space 𝔭_{α(i)+α(j)}, while the remaining terms belong to the space

∑_{β>α(i)+α(j)} 𝔭_{β}.

Therefore the vanishing of [Z_{i}, Z_{j}] implies the vanishing of [X_{i}, X_{j}]. Thus if we let W_{C} denote the space spanned by the X_{i}, then W_{C} is an abelian subspace of 𝔭^{ℂ} of the same dimension as W. Moreover, W_{C}, being a direct sum of the root spaces 𝔭_{α}, α ∈ C, is abelian if and only the set C of leading roots is commutative, meaning that the sum of any two elements of C is not a root. Thus we are reduced to the purely combinatorial problem of finding commutative systems of non-compact roots of maximum cardinality.

In tables 1 and 2 of §4 we list the root systems Δ and positive roots Δ^{+} for all the algebras listed in (a), and in §5 we solve the problem of finding the commutative systems of maximum cardinality for each Δ^{+}(𝔭). There is only one technical complication in §4,5. Given a root system Δ, all choices Δ^{+} of a system of positive roots are equivalent by an automorphism of Δ. But this automorphism may not preserve the decomposition of Δ into compact and non-compact roots, ie, the corresponding automorphism of 𝔤 need not commute with the Cartan involution. Thus for each root system Δ we have to consider all the choices of Δ^{+} modulo automorphisms that preserve the Cartan decomposition. The equivalence classes of choices are parametrized by Aut(Δ)/Aut(Δ(𝔨)). This set has cardinality one for SL(n, ℝ) and SU^{∗}(2n), which we list in Table 1; but has larger cardinality for SO(p, q) and Sp(p, q), which we list in Table 2. The main technical difficulty in §5 arises from the fact that, for the groups SO(p, q) and small values of p or q, the maximum cardinality of C is attained in non-obvious cosets, which do not follow the same pattern as for p, q large.

Part c: Combination of (a) and (b). Given these results one can compute a bound on the dimension of any abelian space W which decomposes as

W = W_{σ} ⊕ W_{1} ⊕ ⋯ ⊕ W_{k},

where the first term consists of semisimple elements and the remaining ones consist of nilpotents for an irreducible symmetric pair (G_{i}, K_{i}). The required bound is the maximum of these over all possible decompositions. What one finds a posteriori is that the maximum is always achieved by a space consisting entirely of nilpotents in the single irreducible symmetric pair G/K. Moreover, this maximum is achieved only in this way except for SL(4, ℝ), SL(2n + 1, ℝ) and SO(2p + 1, 2q + 1). In these cases there is also a space with a single semi-simple element achieving the maximum. In the case of SL(3, ℝ) there is, in addition, a two-dimensional space of semi-simple elements which achieves the maximum dimension.

To explain Steps 2 and 3, we first need to define a division of the root sysmes into two types, which we call types I and II, defined as follows. Let C be a commutative system of maximum cardinality, and let W_{C} be the corresponding commutative space. Let 𝔤′ denote the subalgebra of 𝔤 generated by the real points of the subspace W_{C} + W̄_{C} ⊂ 𝔭^{ℂ}. Then a case by case verification shows that there are only two possibilities, with all the properties now listed:

Type I. 𝔤′ corresponds to a Hermitian symmetric subspace. This happens precisely when W_{C} + W̄_{C} = 𝔭_{1}^{ℂ}, where 𝔭_{1} ⊂ 𝔭 is a Lie triple system, thus corresponds to a totally geodesic subspace, which is necessarily Hermitian symmetric, since 𝔭_{1}^{ℂ} contains an abelian subalgebra of half its dimension, namely W_{C}. Thus for type I, W_{C} = 𝔭_{1}^{1,0}, the space of (1,0)-tangent vectors to an invariant complex structure on 𝔭_{1}, which for ease of notation we denote by 𝔭_{1}^{+}. Moreover, this symmetric subspace is the fixed point set of an involution. Thus, the Lie algebra 𝔤 has two commuting involutions σ, τ, where the first is the Cartan involution, which give a ℤ/2 × ℤ/2-grading of 𝔤, i.e., a four-term decomposition

𝔤 = 𝔨_{1} ⊕ 𝔪 ⊕ 𝔭_{1} ⊕ 𝔮,  (2.2)

refining the Cartan decomposition of 𝔤 where the terms have (σ, τ)-types (1, 1), (1, −1), (−1, 1), and (−1, −1), respectively. If we let 𝔤_{1} = 𝔨_{1} ⊕ 𝔭_{1}, we have 𝔤′ ⊂ 𝔤_{1}, with equality except in the cases where 𝔤′ = 𝔰𝔲(p, q), in which case 𝔤_{1} = 𝔲(p, q). The groups in question are Sp(p, q), SO(2p, 2q) for p, q ≥ 3, SL(2n, ℝ) for n ≥ 2, SU^{∗}(2n) for n ≥ 2.

Type II. 𝔤′ does not correspond to a Hermitan symmetric subspace. In these cases 𝔤′ is very large, usually equal to 𝔤, and the commutative system C of maximum cardinality contains a subsystem C′ of cardinality one less, so that the corresponding commutative space W_{C′} is of the form 𝔭_{1}^{+} for a Lie triple system 𝔭_{1} ⊂ 𝔭, which is necessarily Hermitian symmetric but, in most cases, is no longer the fixed point set of an involution. The groups in question are complementary, in our list, to those listed under Type I, except for the group SO(6, 6). The latter has two inequivalent systems of largest cardinality 9, one which falls under Type I, the other under Type II.

In the remaining steps we assume that the algebra is of Type I.

Step 2: First order rigidity. Let W ⊂ 𝔭^{ℂ} be an abelian subalgebra of maximum dimension and consisting entirely of nilpotent elements (the second assumption is unnecessary for all groups of Type I except SL(4, ℝ)). Let C be its commutative system of leading roots as in Step 1b. Using the notation established there, let W_{C} ⊂ 𝔭^{ℂ} be the direct sum of the corresponding root spaces, with basis {X_{i}}, where {Z_{i} = X_{i} + Y_{i}} is a basis for W. Then, as in the definiton of Type I, W_{C} = 𝔭_{1}^{+}, and property (iii) of Step 1b implies that Y_{i} ∈ 𝔮^{ℂ}. From the evident multiplicative properties of the ℤ/2 × ℤ/2-grading (2.2), it follows that the second term of the right-hand side of (2.1) belongs to 𝔪^{ℂ} and the third term belongs to 𝔨_{1}^{ℂ}, hence each term vanishes. Thus, if we define a linear map ϕ : 𝔭_{1}^{+} → 𝔮^{+}, where 𝔮^{+} = 𝔮^{ℂ} ∩ 𝔭^{+}, by letting ϕ(X_{i}) = Y_{i}, then

W = {X + ϕ(X) : X ∈ 𝔭_{1}^{+}},

and the vanishing of the second term in the right-hand side of (2.1) is equivalent to the following cocyle condition for ϕ:

[X, ϕ(Y)] + [ϕ(X), Y] = 0 for all X, Y ∈ 𝔭_{1}^{+}.  (2.3)

This equation means geometrically that W_{t} = {X + tϕ(X) : X ∈ 𝔭_{1}^{+}} is a deformation of W to W_{C} = 𝔭_{1}^{+}, and cohomologically means the following. Let V ⊂ 𝔤^{ℂ} be a subspace which is stable under the Cartan involution, is a 𝔭_{1}^{+}-module, i.e., [𝔭_{1}^{+}, V] ⊂ V, and which contains the image of ϕ. Then ϕ ∈ Hom(𝔭_{1}^{+}, V) = C^{1}(𝔭_{1}^{+}, V) and (2.3) is equivalent to ϕ being a cocycle. Suppose that its class in H^{1}(𝔭_{1}^{+}, V) is zero. This means that there is and X_{0} ∈ V such that

ϕ(X) = [X, X_{0}] for all X ∈ 𝔭_{1}^{+}.  (2.4)

Moreover, it is clear that X_{0} ∈ V ∩ 𝔨^{ℂ}, thus exp(X_{0}) gives a K^{ℂ}-conjugacy between W and 𝔭_{1}^{+}, i.e., 𝔭_{1}^{+} is rigid as an abelian subspace of 𝔭^{ℂ}.

A module V that satisfies these properties is 𝔪^{ℂ} ⊕ 𝔮^{ℂ}, and in §7 we use essentially this module thogether with a well-known theorem of Kostant [10], to prove that for most algebras of Type I the cohomology class of ϕ vanishes. We also show that the K^{ℂ}-conjugacy provided by (2.4) is actually a K-conjugacy. We conclude that if 𝔤 is of Type I and f : M → G/K is a pluri-harmonic map of maximum rank, then for each x ∈ M there is a totally geodesic Hermitian symmetric subspace X_{x} ⊂ G/K containing f(x) so that df(T_{x}M) = T_{f(x)}X_{x}. We call this first-order rigidity of f.

We remark that at this point the restriction to Type I may be mostly for convenience, in order to apply cohomological methods to replace the more involved combinatiorial arguments of [15], where algebras analogous to our Type II (e.g., B_{n}) also occur, and are handled directly. Since the next and final step is out of reach of the present methods, this is at the moment not a serious restriction.

Step 3: Global rigidity. This is the only step where we use the second order information in the pluri-harmonic equation. We assume that f : M → G/K is a pluriharmonic map of maximum rank for which first-order rigidity has been proved in Step 2. At each point it is tangent to a totally geodesic Hermitian symmetric subspace, so second-order rigidity is equivalent to local rigidity or global rigidity.

We consider first the local problem. Suppose that U ⊂ M is a connected open set, let X = G/K, and suppose that f : U → X is a smooth map. We regard X as the manifold of isotropy subalgebras {𝔨_{x} ⊂ 𝔤 : x ∈ X}. Then there are tautological subbundles 𝔨_{X}, 𝔭_{X} of the trivial bundle 𝔤_{X} = X × 𝔤, with fibre over x the isotropy subalgebra 𝔨_{x} and its canonical complement 𝔭_{x} respectively. Moreover the flat connection on 𝔤_{X} corresponding to the trivialization splits (by projection to the two components) as the direct sum of a K-connection and a one-form with values in 𝔭_{X}. The bundle f^{∗}𝔤_{X} splits as f^{∗}𝔨_{X} ⊕ f^{∗}𝔭_{X}, and the induced flat connection splits accordingly. If U is simply connected, the datum of the induced bundle and its decomposition and the decomposition of the flat connection determine the map f uniquely up to left translation by an element of G. Thus in studying the local problem we will replace the map f by a simply connected open set U, a flat Lie algebra bundle 𝔤_{U} over U, a (non-flat) decomposition 𝔤_{U} = 𝔨_{U} ⊕ 𝔭_{U}, and a decomposition of the flat connection d on 𝔤_{U} as

d = D + θ  (2.5)

where D is a K-connection on 𝔤_{U} and θ ∈ A^{1}(U, 𝔭_{U}). Then θ replaces df, and the pluriharmonic equation D′′d′f = 0, cf. [23], [19], [5], for f becomes

D′′θ′ = 0,  (2.6)

where D = D′ + D′′ and θ = θ′ + θ′′ are the decompositions of D and θ into (1,0) and (0,1) components respectively.

Now our assumption is that for some x ∈ U, θ′(T_{x}^{1,0}U) has maximum possible dimension. Since, by (2.6), θ′ is holomorphic, θ′ has maximum possible rank in the complement of a complex analytic subvariety of U. Call this set V and observe that it is connected, open, and dense in U. Thus if G is of Type I, and first-order rigidity holds, for each x ∈ V, θ′(T_{x}^{1,0}V) = 𝔭_{1}^{+}(x), where 𝔭_{1}(x) is the tangent space to a totally geodesic Hermitan symmetric subspace of G/K fixed by an involution. For each x ∈ V we get a decomposition (2.2) of 𝔤_{x}, and a corresponding decomposition of the bundle 𝔤_{V}, denoted by a subscript V under each term of (2.2).

The geometric interpretation of this decomposition is the following. There is a fibration G/K_{1} → G/K, and the fibre over y ∈ G/K parametrizes the tangent spaces at y to totally geodesic embeddings of G_{1}/K_{1} in G/K passing through y. The pull-back to G/K_{1} of the bundle 𝔤_{X} over G/K splits according to (2.2), and the sum of the last three terms is isomorphic to the tangent bundle of G/K_{1}. The map f : V → G/K lifts to a map F : V → G/K_{1} by letting F (x) = df(T_{x}V), which makes sense since by assumption df(T_{x}V) is the tangent space to a totally geodesic embedding of G_{1}/K_{1} passing through f(x), hence is a point of G/K_{1} lying over f(x). The connection D in (2.5) splits as D = D_{1} + μ, where D_{1} is a K_{1}-connection and μ ∈ A^{1}(V, 𝔪_{V}). Thus the original flat connection d of (2.5) splits further as

d = D_{1} + μ + θ.  (2.7)

Observe that in this notation, dF = μ + θ.

Now the orbits of G_{1} on G/K_{1} foliate this manifold into leaves isomorphic to G_{1}/K_{1} which project to the different geodesic embeddings of G_{1}/K_{1} in G/K. We want to show that f(U) is contained in a single such embedding. Since V is dense in U, this is equivalent to showing that F (V) is contained in a single leaf of the foliation. Since V is connected, this is equivalent to F (V) being tangent to the foliation. By construction the one-form θ has values in (𝔭_{1})_{V}, which is the tangent bundle to the foliation, so it suffices to show that μ = 0. This is accomplished by the equations that μ must satisfy. First the pluriharmonic equation (2.6) splits into two equations:

D′′_{1}θ′ = 0 and [μ′′, θ′] = 0  (2.8)

and the flatness of the original connection d in (2.5), when decomposed further into types and also according to (2.2) gives the further equation

[μ′, θ′] = 0.  (2.9)

In §7 we show that the second equation (2.8) together with (2.9) have a cohomological interpretation, and that the vanishing of μ can be proved by appealing again to Kostant’s theorem. This shows the local rigidity of f, from which the global rigidity is immediate (cover M by simply connected open sets U as above with connected intersections).

§ 3. Existence of adapted root systems

We now prove the technical result that allows us to put W in upper triangular form in a way that is compatible with the Cartan involution. We owe the following lemma to Henryk Hecht.

3.1. Lemma. Let 𝔤 be a real semisimple Lie algebra, let θ denote its Cartan involution, and let W be a θ-stable subalgebra of 𝔤^{ℂ}, each element of which is nilpotent. Then there is a fundamental Cartan subalgebra 𝔥 of 𝔤 and a system of positive roots Δ^{+} ⊂ Δ(𝔤, 𝔥) such that

W ⊂ 𝔫^{+} = ∑_{α∈Δ^{+}} 𝔤_{α}.

We recall that a Cartan subalgebra 𝔥 of 𝔤 is called fundamental if it is stable under the Cartan involution, and, if 𝔥 = 𝔥_{+} ⊕ 𝔥_{−} denotes its decomposition into ±1 eigenspaces for θ, then its compact part 𝔥_{+} has maximum possible dimension; i.e., 𝔥_{+} is a Cartan subalgebra for 𝔨. Similarly, a Cartan subalgebra 𝔥 of 𝔤^{ℂ} is called fundamental if 𝔥_{+} is a Cartan subalgebra of 𝔨^{ℂ}.

Let 𝔥 be a fundamental Cartan subalgebra of 𝔤, let 𝔱 = 𝔥_{+}, and let Δ(𝔤, 𝔥) denote the roots for the adjoint action of 𝔥 on 𝔤^{ℂ}. It is a standard and simple fact that the maximality of 𝔥_{+} implies that no root α ∈ Δ(𝔤, 𝔥) restricts to zero on 𝔱. From general principles it follows that the set of restrictions to 𝔱 of the elements of Δ(𝔤, 𝔥) forms a root system on 𝔱, which we will denote by Δ(𝔤, 𝔱). Decomposing the resulting root spaces into compact and noncompact parts as 𝔤_{α} = 𝔨_{α} ⊕ 𝔭_{α}, and observing that the restriction of a positive system of roots for 𝔥 is a positive system of roots for 𝔱, we obtain the following related form of the lemma. We state it only for abelian subalgebras of 𝔭^{ℂ}, rather than more general θ-stable subalgebras, since this is the form that we use.

3.2. Lemma. Let W be an abelian subspace of 𝔭^{ℂ} which consists entirely of nilpotent elements. Then there is a Cartan subalgebra 𝔱 of 𝔨 and a system of positive roots Δ^{+}(𝔤, 𝔱) such that

W ⊂ ∑_{α∈Δ^{+}(𝔤,𝔱)} 𝔭_{α}

Proof of Lemma A: Since W is a subalgebra consisting entirely of nilpotent elements, it is contained in the nil-radical 𝔫 = [𝔟, 𝔟] of some Borel subalgebra 𝔟 of 𝔤^{ℂ}. Applying Theorem 1(i) of Matsuki’s paper [17] to the affine symmetric space (G^{ℂ}, K^{ℂ}, θ), it follows easily that there is a Cartan subalgebra 𝔥 of 𝔤^{ℂ} in 𝔟 which is θ-stable. Write 𝔥 = 𝔥_{+} ⊕ 𝔥_{−}, where the first term is in 𝔨^{ℂ} and the second is in 𝔭^{ℂ}. Let Δ^{+} be the system of positive roots in Δ(𝔤, 𝔥) corresponding to 𝔟. Since W ⊂ 𝔫 = [𝔟, 𝔟], one can write a general element X ∈ W as

X = ∑_{α∈S} c_{α}X_{α},

where the S ⊂ Δ^{+}, X_{α} ∈ 𝔤_{α}, and c_{α} ≠ 0 for all α ∈ S. Roots that appear in some such S are said to belong to W. We wite Δ_{W} for the collection of all such roots. Now θX ∈ W ⊂ 𝔫 and

θX = ∑_{α∈S} c′_{α}X_{θα},

where c′_{α} ≠ 0 for all α ∈ S. Thus, if α belongs to W, then so does θα. Define a special collection of positive roots which includes all of those belonging to W by setting

(Δ^{+})′ = { α ∈ Δ^{+} | θα ∈ Δ^{+} }.

We proceed to modify to modify these choices to obtain a new Borel subalgebra of 𝔤^{ℂ} containing W which is defined by a fundamental Cartan subalgebra of 𝔤.

To begin, we observe that there is a θ-stable real form 𝔥_{ℝ} of 𝔥 such that Δ(𝔤, 𝔥) ⊂ 𝔥_{ℝ}^{∗}: take 𝔥_{ℝ} = { H ∈ 𝔥 | α(H) ∈ ℝ for all α ∈ Δ }. Let H be a regular element of 𝔥_{ℝ}, i.e., one not contained in any root hyperplane, and write H = H_{+} + H_{−}, with H_{±} ∈ (𝔥_{ℝ})_{±}. Choose H so that

α(H_{+}) > 0 for all α ∈ (Δ^{+})′ .

Recall that a root is real if θα = −α, or, equivalently, if α(𝔥_{+}) = 0. Let Δ_{ℝ} denote the set of real roots, let Δ_{ℝ}^{+} = Δ_{ℝ} ∩ Δ^{+} and define a set P ⊂ Δ(𝔤, 𝔥) by the conditions

if α ∈ Δ_{ℝ}^{+} then α ∈ P,
if α ∉ Δ_{ℝ} but α(H_{+}) > 0, then α ∈ P.

Then P ⊂ Δ is a system of positive roots. Replacing Δ^{+} by this new system P of positive roots, we may thus assume that the original Δ^{+} satisfies the following conditions:

if α ∈ Δ^{+} \ Δ_{ℝ} then θα ∈ Δ^{+} \ Δ_{ℝ},
if α ∈ Δ^{+} \ Δ_{ℝ}, β ∈ Δ_{ℝ}, and α + β ∈ Δ, then α + β ∈ Δ^{+} \ Δ_{ℝ},

(2.3)

in addition to the obvious inclusion Δ_{W} ⊂ Δ^{+} \ Δ_{ℝ}.

Now define θ-stable Lie algebras

\begin{matrix}𝔩 = 𝔥 ⊕ ∑_{α∈Δ_{ℝ}} 𝔤_{α}, \\ 𝔲 = ∑_{α∈Δ^{+}\Δ_{ℝ}^{+}} 𝔤_{α}, \end{matrix}

where 𝔩 is reductive, 𝔲 is nilpotent and 𝔮 = 𝔩 ⊕ 𝔲 is parabolic. Then W ⊂ 𝔲, and by the second condition (2.3), 𝔩 normalizes 𝔲. Now, let 𝔥̃ be a fundamental Cartan subalgebra of 𝔩. Since 𝔩_{+} = 𝔩 ∩ 𝔨^{ℂ} and 𝔨^{ℂ} have the same rank, 𝔥̃ is a fundamental cartan subalgebra of 𝔤^{ℂ}, and since 𝔥̃ normalizes 𝔲, 𝔲 is still a sum of root spaces for 𝔥̃.

Let Δ denote now the root system for 𝔤^{ℂ} relative to 𝔥̃. Then Δ = Δ_{0} ∪ Δ_{1}, where Δ_{0} is the set of roots of 𝔥̃ belonging to 𝔩 (a sub-system of Δ) and Δ_{1} = Δ \ Δ_{0}. Define now a positive root system Δ^{+} ⊂ Δ by

Δ^{+} = Δ_{0}^{+} ∪ Δ_{1}^{+},

where Δ_{0}^{+} is a positive root system for Δ_{0} satisfing (2.3), and Δ_{1}^{+} is the set of roots in Δ belonging to 𝔲. Then Δ^{+} also satisfies the conditions(2.3). Let 𝔟̃ be the Borel subalgebra of 𝔤^{ℂ} corresponding to this choice of 𝔥̃ and Δ^{+}. Then 𝔟̃ is a Borel subalgebra of 𝔤^{ℂ} containing W and defined relative to a fundamental Cartan subalgebra of 𝔤^{ℂ}. It only remanins to show that 𝔟̃ is defined relative to a fundamental Cartan subalgebra of 𝔤.

To see this last point, observe that 𝔟̃ is defined relative to a fundamental Cartan subalgebra of 𝔤^{ℂ} and a positive system Δ^{+} satisfying the first condition (2.3). By Proposition 2 of [17] , these are precisely the conditions for the K^{ℂ}-orbit of 𝔟̃ to be closed in the flag variety G^{ℂ}/B of Borel subalgebras of 𝔤^{ℂ}. But this is equivalent to saying that the K-orbit of 𝔟̃ is the same as its K^{ℂ}-orbit. Since by a K^{ℂ}-equivalentce we can make 𝔥̃ defined over ℝ, it follows that 𝔟̃ contains a fundamental Cartan subalgebra of 𝔤. Finally, let 𝔥 ⊂ 𝔤^{ℂ} now denote this subalgebra, and let Δ^{+} denote the positive root system of 𝔤^{ℂ} relative to 𝔥 corresponding to 𝔟̃, and the proof of the lemma is complete.

§ 4. Root systems.

In this section we establish and record in the tables given below the information on the adapted root systems constructed by the procedure of §3 that we shall need for the proof of the main theorem. For technical reasons we divide the systems into two classes, according to whether cardinality of Aut(Δ)/Aut(Δ_{𝔨}) is one or greater than one. The first class, treated in table 1 corresponds to the groups SL(2n), SL(2n + 1), and SU^{∗}(2n). The second class, treated in table 2, corresponds to the groups Sp(p, q), SO(2p, 2q), SO(2p, 2q + 1), for which |Aut(Δ)/Aut(Δ_{𝔨})| = \binom{p+q}{p}.

The tables are organized as follows. The first line in each entry identifies the group G, the type of root system, and the multiplicities of the roots. The second line gives the type of the compact part of the root system, as well as an explicit representation of the positive compact roots corresponding to a choice of positive system of roots for 𝔤. The third line lists the positive non-compact roots corresponding to the same system used to obtain the previous line. In the fourth line we exhibit a commutative set C^{+} of positive noncompact roots which is, for all but very low values of the parameters, of maximum cardinality; when there is such an exception, conditions are given for C^{+} to be maximal. The last line identifies the relevant real form of the group G′ generated by W + \overline{W}, where W is the abelian space associated to the commutative system C^{+} found in the previous line. Here there is an alternative (§2, Step 1b). In the first case (“type I”) G′ is the group of a geodesically imbedded Hermitian space and W can be identified with its holomorphic tangent space. In the second case (“type II”), G′ is not the group of a Hermitian symmetric space and in fact is equal to G itself, except in the case of SO(2p + 1, 2q + 1) where it is equal to SO(2p, 2q + 1) or SO(2p + 1, 2q). The bounds on the cardinality of C will be derived in the following section §5.

The root systems and their orderings can be determined concretely in the following way. Choose a maximal torus T of K, and let V be the “defining” representation space for G, i.e., V = ℝ^{n} for SL(n, ℝ), ℍ^{n} for SU^{∗}(2n), etc. The compexification of V decomposes into eigenspaces V_{λ} for the action of 𝔱. The eigenvalues, or weights, are linear functionals on 𝔱 with pure imaginary values, and the set Φ of these weights is stable under complex conjugation, which acts as does multiplication by -1. Choose a vector ξ in i𝔱, and let Φ^{+}, Φ^{−}, and Φ^{0} be the sets of weights which have positive, negative, and zero inner product with ξ. If ξ is chosen generically, Φ^{0} is either empty or consists of zero alone. The set Φ^{+} consists of n linearly independent elements which we shall label as λ_{1} , …, λ_{n}, and Φ^{−} = −Φ^{+}.

The roots of 𝔤 relative to 𝔱 are the linear combinations of the basic weights just described, and have the property that 𝔤_{λ} maps V_{μ} to V_{λ+μ} if λ + μ is a weight in Φ, and to zero in the contrary case. To compute a particular root system it suffices to choose a basis for V^{ℂ} which diagonalizes T. The standard basis vectors are then weight vectors, and root vectors are matrices with a small number of nonzero entries. It is therefore an elementary, if time consuming problem to determine the roots and their multiplicities. The results are recorded in the two tables already mentioned, and displayed below.

As noted already, the systems in Table 1 are characterized by the condition that Aut(Δ)/Aut(Δ_{𝔨}) has cardinality one. In the case of SL(2n + 1) and SU^{∗}(2n) this is evident, since for these the Weyl groups of G^{ℂ} and K^{ℂ} relative to 𝔱 coincide. In the case of SL(2n) the two groups may differ by a factor of 2, since when n is odd, the symmetry which reverses the signs of all the λ_{i} is in the larger of the two Weyl groups, but not the smaller. This transformation is, however, induced by an outer automorphism of G^{ℂ} preserving K^{ℂ}.

Table 1. Root Systems with |Aut(Δ)/Aut(Δ_{𝔨})| = 1
Group  RS  Remarks
SL(2n)  C_{n}  m(short) = 2, m(long) = 1, long: noncompact
D_{n}  Δ^{+}(𝔨) = { λ_{i} ± λ_{j} | i < j }
Δ^{+}(𝔭) = { λ_{i} ± λ_{j} | i < j } ∪ { 2λ_{i} }
C^{+} = { λ_{i} + λ_{j} | i ≤ j }, |C| ≤ n(n + 1)/2
G′ = Sp(n, ℝ)
SL(2n + 1) BC_{n} m(short, medium) = 2, m(long) = 1, long: noncompact
B_{n}  Δ^{+}(𝔨) = { λ_{i} } ∪ { λ_{i} ± λ_{j} | i < j }
Δ^{+}(𝔭) = { λ_{i} } ∪ { λ_{i} ± λ_{j} | i < j } ∪ { 2λ_{i} }
C^{+} = { λ_{i} + λ_{j} | i ≤ j } ∪ { λ_{1} }, |C| ≤ n(n + 1)/2 + 1
G′ = SL(2n + 1, ℝ).
SU^{∗}(2n)  C_{n}  m(short) = 2, m(long) = 1, long: compact
C_{n}  Δ^{+}(𝔨) = { λ_{i} ± λ_{j} | i < j } ∪ { 2λ_{i} }
Δ^{+}(𝔭) = { λ_{i} ± λ_{j} | i < j }
C^{+} = { λ_{i} + λ_{j} | i < j }, |C| ≤ n(n − 1)/2
G′ = SO^{∗}(2n)

It remains, therefore, to consider the systems with |Aut(Δ)/Aut(Δ_{𝔨})| > 1. These are distinguished by the fact that K is reducible, splitting into two factors K_{(0)} and K_{(1)}. Any maximal torus T splits accordingly, as does the set of weights Φ. To be precise, we say that a weight belongs to one factor of 𝔱 if that factor acts nontrivially while the other one acts trivially. Viewing the indexing on the factors of K as an indexing modulo two, we obtain a parity function on weights, ϵ : Φ ⟶ ℤ/2. Some care must be taken when 0 is a weight, i.e., when G is SO(2p, 2q + 1) or SO(2p + 1, 2q + 1). In the first case it is a weight of multiplicity 1, and its parity is that assigned to SO(2q + 1). In the second case it is a weight of multiplicity 2, with both even and odd weight vectors. We shall therefore view the parity function as multiple-valued. Regardless of the fine points, we there is an induced parity function on roots, which satisfies ϵ(λ + μ) = ϵ(λ) + ϵ(μ). Compact roots have parity 0 and noncompact ones have parity 1. For example, in the case of SO(2p + 1, 2q + 1), λ_{i} and 0 are weights, the latter with multiplicity two and both parities. Thefore the root λ_{i} = λ_{i} + 0 has multiplicity two and both parities, hence has root vectors of both compact and noncompact types.

It is evident that the Weyl group of K^{ℂ} acts trivially on parity functions, as does the sign-reversing element which may be an outer automorphism of K^{ℂ}. Therefore parity is an invariant of the equivalence class of the positive systems which we have in mind. Since all the Weyl groups contain the permutations on the index set I = { 1 … n }, it is clear that the full Weyl group acts transitively on the set of parity functions. Cardinality arguments then imply that parity is a complete invariant. Therefore the systems given in table 2 are representative of all positive systems.

Table 2. Root Systems with |Aut(Δ)/Aut(Δ_{𝔨})| > 1
Group  RS  Remarks
Sp(p, q)  C_{p+q}  m = 1, long: compact
C_{p} + C_{q} Δ^{+}(𝔨) = { λ_{i} ± λ_{j} | i < j, ϵ(λ_{i}) = ϵ(λ_{j}) } ∪ { 2λ_{i} }
Δ^{+}(𝔭) = { λ_{i} ± λ_{j} | i < j, ϵ(λ_{i}) ≠ ϵ(λ_{j}) }
C^{+} = { λ_{i} + λ_{j} | i ≤ p < j }, |C| ≤ pq
G′ = SU(p, q)
SO(2p, 2q)  D_{p+q}  m = 1
D_{p} + D_{q} Δ^{+}(𝔨) = { λ_{i} ± λ_{j} | i < j, ϵ(λ_{i}) = ϵ(λ_{j}) }
Δ^{+}(𝔭) = { λ_{i} ± λ_{j} | i < j, ϵ(λ_{i}) ≠ ϵ(λ_{j}) }
C^{+} = { λ_{i} + λ_{j} | i ≤ p < j }, |C| ≤ pq (p, q ≥ 3)
G′ = SU(p, q) (ϵ(λ_{1}) ≠ ϵ(0))
SO(2p, 2q + 1)  B_{p+q}  m = 1
D_{p} + B_{q} Δ^{+}(𝔨) = { λ_{i} ± λ_{j} | i < j, ϵ(λ_{i}) = ϵ(λ_{j}) } ∪ { λ_{i} | ϵ(λ_{i}) = ϵ(0) }

Δ^{+}(𝔭) = { λ_{i} ± λ_{j} | i < j, ϵ(λ_{i}) ≠ ϵ(λ_{j}) } ∪ { λ_{i} | ϵ(λ_{i}) ≠ ϵ(0) } C^{+} = { λ_{i} + λ_{j} | i ≤ p < j } ∪ { λ_{1} }, |C| ≤ pq + 1 (p, q ≥ 3)

G′ = SO(2p, 2q + 1)
SO(2p + 1, 2q + 1) B_{p+q}  m(short) = 2, m(long) = 1
B_{p} + B_{q} Δ^{+}(𝔨) = { λ_{i} ± λ_{j} | i < j, ϵ(λ_{i}) = ϵ(λ_{j}) } ∪ { λ_{i} }
Δ^{+}(𝔭) = { λ_{i} ± λ_{j} | i < j, ϵ(λ_{i}) ≠ ϵ(λ_{j}) } ∪ { λ_{i} }

C^{+} = { λ_{i} + λ_{j} | i ≤ p < j } ∪ { λ_{1} }, |C| ≤ pq + 1 (p, q ≥ 3)

G′ = SO(2p, 2q + 1) ≠ G

We remark that to give a parity function is to give a decomposition of the interval I = { 1 … n } into consecutive subintervals I_{j}, where j = 0 , …, k, where successive intervals are assigned different parities. The interval I should be thought of as an index set for the positive weights { λ_{j} }. To take a concrete example, consider the decomposition

I = I_{0} ∪ I_{1} ∪ I_{2} = { 1, 2 } ∪ { 3, 4 } ∪ { 5, 6 },

then the functional λ_{1} − λ_{2} is even, hence compact, while λ_{2} − λ_{3} is odd, hence noncompact. Now consider the structure which the associated parity function defines on the positive roots { λ_{i}±λ_{j} | 1 ≤ i < j ≤ 6 }. The simple roots are associated to a Dynkin diagram of type of type D_{6}. Coloring the noncompact ones black, we obtain the following diagram of type D_{6} that describes an adapted root system for SO(4, 8):

Figure 1.

Thus, classes of positive root systems modulo Aut(Δ_{𝔨}) are also indexed by certain colored Dynkin diagrams.

§ 5. Combinatorial arguments

We shall now use the information on root systems compiled in the last section to determine the maximum cardinality ν of a commutative set of noncompact roots:

5.1. Proposition. Let C ⊂ Δ^{+}(𝔭, 𝔱) be a commutative set of roots for one of the groups listed in table 3 below. Then its cardinality is at most ν(G), provided that p, q ≥ 6 if G = SO(p, q). The bounds are realized by the systems C^{+} given in tables 1 and 2. Moreover, any other system of cardinality ν is equivalent to C^{+} under the action of Aut(Δ_{k}), except that if G = SO(p, q) we require in addition to the previous condition that (p, q) ≠ (6, 6). When (p, q) = (6, 6) there are two equivalence classes, each represented by a system of cardinality 9.

Table 3. The Invariant ν(G)

Group  ν
SL(2n)  n(n + 1)/2
SL(2n + 1)  n(n + 1)/2 + 1
SU^{∗}(2n)  n(n − 1)/2
Sp(p, q)  pq
SO(p, q)  [\frac{p}{2}][\frac{q}{2}] + ϵ(p, q)

ϵ(p, q) is 0 if p and q are even, is 1 otherwise

We treat first the first three lines of the table, for which |Aut(Δ_{𝔤}/Aut(Δ_{𝔨})| = 1, and then treat the more intricate cases in which |Aut(Δ_{𝔤})/Aut(Δ_{k})| > 1. In the course of our analysis we shall find that G = SO(p, q), for min(p, q) < 6 admits pathological systems of excess dimension, e.g., of dimension 2q + 1 for SO(4, 2q). These systems, which constitute more than one equivalence class under the action of Aut(Δ_{𝔨}), are mainly responsible for the difficulty of the final part of the proof.

The case SL(2n).

According to the table, the relevant root system is of type C_{n}, and one system of positive noncompact roots is

Δ^{+}(𝔭) = { λ_{i} ± λ_{j} | i < j } ∪ { 2λ_{i} }.

Any other system is equivalent to it by an automorphism of G preserving 𝔱. Consider, therefore, the set of elements free of minus signs, namely,

C^{+} = { λ_{i} + λ_{j} | i < j } ∪ { 2λ_{i} }.

It is commutative and of cardinality n(n + 1)/2. Any other commutative subset C must contain an element of the form μ = λ_{a} − λ_{b}. The elements of Δ^{+} in which λ_{b} appears with positive coefficient cannot be in C, since when added to μ they form a root. Therefore C is lies in the set

C′ ∪ C′′,

where C′ is free of the index b and where C′′ = { λ_{i} − λ_{b} | i < b } “contains” the index b with negative coefficient. Arguing by induction, we may assume that |C′| ≤ n(n − 1)/2, so that |C| < n(n + 1)/2.

This gives the required bound, and shows that any commutative set which attains the bound is equivalent to C^{+}. Note, however, that there is an automorphism of G which acts on 𝔱 by the transformation which fixes all roots except λ_{i} ± λ_{n} and ±2λ_{n}, for which it interchanges signs. Application of this automorphism to C^{+} yields an equivalent commutative system of the same cardinality. The equivalence is not, however, by an inner automorphism of SL(2n, ℝ).

Decompositions of the kind C = C′ ∪ C′′ considered above will be used repeatedly in what follows. The characteristic feature is that C′ is free of a certain index, while C′′ “contains” it with negative coefficient. We call such a decomposition a reduction of C relative to the given index.

The case SL(2n + 1).

For BC_{n} we have
Δ^{+}(𝔭) = { λ_{i} } ∪ { λ_{i} ± λ_{j} | i < j } ∪ { 2λ_{i} }

As before all positive systems are equivalent, so that it suffices to consider the one displayed above. We argue by reduction to the previous case, writing C = C′ ∪ C′′, where the first term consists of medium and long roots, the second term of short ones. It is clear that C′′ has at most one element, since the sum of any two short positive roots is a root. For the other term we already have the bound C′ ≤ n(n + 1)/2, so that |C| ≤ n(n + 1)/2 + 1.

The case SU^{∗}(2n).

The noncompact roots are Δ^{+}(𝔭) = { λ_{i} ± λ_{j} | i < j }, and the arguments given for SL(2n) apply to show that C = { λ_{i} + λ_{j} | i < j } is a system of cardinality n(n − 1)/2 , that this is the maximum cardinality, and that all other systems of maximum cardinality are equivalent to it.

The case Sp(p, q).

We show that |C| ≤ pq by induction on the quantity min(p, q). In the course of the proof we shall see that all commutative systems of maximum cardinality are equivalent.

To begin, we assume min(p, q) = 1, and, without loss of generality we may take p = 1. Let I = { 1 … q + 1 } be the index set, and let r be the index distinguished by the fact that its parity is different from all the others. Then

Δ^{+}(𝔭) = { λ_{i} ± λ_{r} | i < r } ∪ { λ_{r} ± λ_{i} | i > r }

Let C_{i} be the subset of C for which the nondistinguished index is i. It is a subset of { λ_{i} ± λ_{r} } if i < r, and a subset of { λ_{r} ± λ_{i} } in the contrary case. These two sets are not commutative, so C_{i} has at most one element. This already establishes the bound |C| ≤ q.

Finer analysis yields the conjugacy assertion. To begin, note that if C contains λ_{i}−λ_{r}, then it cannot contain any element of the form λ_{r}±λ_{j}, and so |C| < q if r < q+1. Thus, if |C| = q, then C = { λ_{i} − λ_{q+1} | i = 1 , …, q } or C = { λ_{i} + λ_{q+1} | i = 1 , …, q } These systems are equivalent under an automorphism of G^{ℂ} preserving 𝔱 which acts on the fundamental weights by λ_{i} ↦ ϵ_{i}λ_{i}, where ϵ_{i} = ±1 and ϵ_{i} = 1 for i ≤ q + 1.

Consider next the inductive step. Without loss of generality we assume that there are p indices of even parity, and that the first subinterval in the decomposition I = I_{0} ∪ ⋯ ∪ I_{k} has even parity. Thus,

\begin{matrix}p = ∑_{j even} |I_{j}| \\ q = ∑_{j odd} |I_{j}| \end{matrix}

The set of positive noncompact roots is

Δ^{+}(𝔭) = { λ_{i} ± λ_{j} | i < j, ϵ(i) ≠ ϵ(j) },  (5.2)

and it has an obvious commutative subset,

C^{+} = { λ_{i} + λ_{j} | ϵ(i) ≠ ϵ(j) },  (5.3)

which has cardinality pq. If C is another commutative set it must contain an element of the form λ_{i} − λ_{r}. Arguing as before, we shall reduce C relative to the root with negative coefficient, decomposing it as

C = C′ ∪ C′′,

where C′ is free of the index r, and where

C′′ ⊂ { λ_{i} − λ_{r} | i < r, ϵ(i) ≠ ϵ(r) }

Let us analyze the case in which ϵ(r) = 0. The other alternative can be handled in the same way, with the same conclusion. The first case to consider is that of p ≤ q. Then C′ is a commutative system with a smaller value of min(p, q), so, by the induction hypothesis, C′ ≤ (p − 1)q. As for C′′, we observe that it is the sum of the quantities |I_{j}|, for j odd and less than b, where r ∈ I_{b}. Therefore |C′′| ≤ q, with equality possible only if I_{b} is the rightmost interval. Consequently |C| ≤ pq. In the case of equality there is an automorphism which carries C to C^{+}, namely, one which acts on the fundamental weights by λ_{i} → ϵ_{i}λ_{i}, where ϵ_{i} = 1 if i is not in the rightmost interval.

For the case in which p > q we use a subsidiary induction argument. Let C(p, q) denote a commutative system with parameters p and q. For each reduction relative to a root λ_{i} − λ_{r} there is an inequality of the form |C(p, q)| ≤ |C(p − 1, q)| + q, with strict inequality unless r is in the rightmost interval. Apply reduction repeatedly until the resulting system has no roots of the form λ_{i} − λ_{j} or until the quantity min(p, q) decreases. However this happens one obtains an inequality of the form

|C(p, q)| ≤ |C(p − k, q)| + kq

or

|C(p, q)| ≤ |C(p − k, q − 1)| + kq + (p − k)

In either case the right-hand side can be evaluated, giving the required bound. As before the inequality is strict unless the original system is equivalent to C^{+}.

The case SO(2p, 2q).

We now begin the analysis of the most difficult of all the cases, SO(2p, 2q). The plan is to proceed as in the previous case, by induction on min(p, q). Because of certain unpleasant pathologies, we must consider the cases of min(p, q) = 1, 2, and 3 separately. If we assume, as we may without loss of generality, that p ≤ q, these bounds are 2q, 2q + 1, and 3q, respectively. Given this much, a small modification of the previous induction argument shows that |C| ≤ pq for min(p, q) ≥ 3. The pathologies alluded to are the systems of excess dimension 2q + 1 which occur for SO(4, 2q). Their existence is due to the fact that SO(2p, 2q) has fewer compact roots than does Sp(p, q). Indeed, the long roots 2λ_{i} are absent, so that certain previously forbidden pairs, e.g., λ_{i} ± λ_{j}, now commute.

Let us begin with the induction argument, assuming the bounds just given for low values of min(p, q) and assuming min(p, q) > 3 for the system under study. The argument is identical to that used for Sp(p, q), except that the system C′′ used in the decomposition relative to a root α = λ_{i} − λ_{r} requires more care. The positive noncompact roots which commute with α and contain the index r are of the form β = λ_{j} − λ_{r}, with j ≠ i, or γ = λ_{i} + λ_{r}. These two possibilities are mutually exclusive, since β + γ is a root. If a β occurs, then C′′ is as before. If a γ occurs, it is a subset of the two-element set { λ_{i} ± λ_{r} }. However, because min(p, q) > 3, the sets C′′ which arise in this way are always smaller than the other kind. Consequently the argument used for Sp(p, q) still applies, yielding both the bound and the conjugacy assertion. Modulo the proof of the bounds for low values of min(p, q), we have therefore established the following.

5.4. Proposition. Let C be a commutative system of positive noncompact roots for SO(2p, 2q), where p, q > 3. Then |C| ≤ pq, and equality holds if and only if |C| is equivalent under the action of Aut(Δ_{𝔨}) to the standard commutative system { λ_{i} + λ_{b} | ϵ(i) ≠ ϵ(j) }.

We turn next to the special cases, beginning with an analysis of the case in which the decomposition of I given by the parity function has just two subintervals.

5.5. Proposition. Let C be a commutative subset of the system of positive noncompact roots associated to the decomposition I = I_{0} ∪ I_{1}, where we assume p ≤ q. Then |C| ≤ 2q if p = 1, and |C| ≤ pq if p > 1. If p = 1 and equality holds then C is equivalent to the set

C^{∗} = { λ_{1} ± λ_{j} | j ∈ I_{1} },

and if p > 2 and equality holds then C is equivalent to the set

C^{+} = { λ_{i} + λ_{j} | i ∈ I_{0}, j ∈ I_{1} }

If p = 2 there is are two equivalence classes, represented by C^{∗} and C^{+}. These correspond to geodesic imbeddings of the symmetric spaces for SO(2, 2q) and SU(2, q) in the symmetric space of SO(4, 2q), respectively.

The positive root systems corresponding to the partitions just described are called Borel-Siebenthal systems. They are characterized by having a unique noncompact simple root. For SO(2p, 2q) this is the root that bridges the gap between the two intervals, namely, λ_{p} − λ_{p+1}.

Figure 2. Borel-Siebenthal system

For the proof, as well as for later work, we shall need the following technical results.

5.6. Lemma A. Let S = { λ_{i} ± λ_{r} | i ∈ A }, where r ∉ A, and let C be a commutative subset of S. Then C is a subset of one of the following commutative sets:

a) { λ_{i} ± λ_{r} }, where i is fixed, or
b) { λ_{i} + λ_{r} | all i ∈ A }, or
c) { λ_{i} − λ_{r} | all i ∈ A }.

Proof: If there exist elements λ_{i} − λ_{r} and λ_{j} + λ_{r} in C, then i = j and C is as in case (a). Otherwise C must be as in (b) or (c).

5.7. Lemma B. Let S and C be as above. Then |C| ≤ max(2, |A|). In particular, if |C| > 2, then C must be a subset of one of the sets (b) or (c) above.

The proof is immediate.

5.8. Lemma C. Let S = { λ_{i} ± λ_{j} | i ∈ A, j ∈ B } be a subset of a root system of type D_{n}, where the index sets A and B are disjoint. Then

\begin{matrix}|C| ≤ 2|B| if |A| = 1, \\ |C| ≤ |A||B| if |A| > 1. \end{matrix}

Moreover, these bounds are sharp because the sets below are commutative:

a) { λ_{1} ± λ_{j} | j ∈ B }
b) { λ_{i} + λ_{j} | i ∈ A, j ∈ B }

If |A| = 1 then a commutative system of maximum cardinality must be of type (a). If |A| ≥ 3, then a system of maximum cardinality must be equivalent to one of type (b). If |A| = 2, then there are 2 equivalence classes.

Proof: Let C_{j} = C ∩ { λ_{i} ± λ_{j} | i ∈ A } and observe that |C| = ∑ |C_{j}|. The inequalities of the proposition follow from the inequalities of the previous lemma applied to the C_{j}. If |A| = 1 then it is clear that system (a) is the only one which attains the bound. If |A| = 2 then each C_{j} must be one of the four sets { λ_{1} ± λ_{j} }, { λ_{2} ± λ_{j} }, { λ_{1} + λ_{j}, λ_{2} + λ_{j} }, { λ_{1} − λ_{j}, λ_{2} − λ_{j} }. If one C_{j} has one of the types just listed, then commutativity forces the types of the remaining C_{j}’s to be the same. This gives four possible commuting sets of maximal dimension, or two equivalence classes. If |A| > 2 then each C_{j} must have cardinality |A| > 2, so that it is of the form { λ_{i} +ϵ_{j}λ_{j} | i ∈ A }. The automorphism which sends λ_{j} to ϵ_{j}λ_{j} and which fixes the remaining λ’s sends the given C to the one displayed in the proposition.

The proposition with which this discussion began is a special case of the last lemma.

Analysis of the case min(p, q) = 2.

We turn at last to the analysis of the difficult case of min(p, q) = 2. Positive root systems satisfying this condition correspond to one of the decompositions of I = { 1 , …, n }

below, where a subinterval written Ĵ has cardinality one, while ˆ Ĵ has cardinality 2.

1) I_{0} Î_{1} I_{2} Î_{3} I_{4}
2) I_{0} Î_{1} I_{2} Î_{3}
3) Î_{0} I_{1} Î_{2} I_{3}
4) Î_{0} I_{1} Î_{2}
5) I_{0} ˆ Î_{1} I_{2}
6) ˆ Î_{0} I_{1}
7) I_{0} ˆ Î_{1}

We shall need one additional technical result:

5.9. Lemma D. Consider disjoint sets A, A′, B and subsets

\begin{matrix}S = \{ λ_{i} ± λ_{j} | i ∈ A, j ∈ B \} \\ S′ = \{ λ_{i} ± λ_{j} | i ∈ A′, j ∈ B \} \end{matrix}

in a root system of type D_{n}. Suppose further that |A| = |A′|, and let C ⊂ S ∪ S′ be commutative. Then

|C| ≤ 2|A||B|.

Proof: Let ϕ : A ⟶ A′ be a 1-1 correspondence. Define a 1-1 correspondence Φ : S ⟶ S′ by

Φ(λ_{i} ± λ_{j}) = λ_{ϕ(i)} ∓ λ_{j}

Then S∪S′ splits into equivalence classes { x, Φ(x) }, where x+Φ(x) is a root. Therefore C can contain at most one element form each class, hence the result.

We must now analyze the commutative systems of positive roots associated to each of the above seven partitions. For the last two there is nothing to prove, since they are of Borel-Siebenthal type. We take the remaining ones in turn. The first is the most difficult to treat.

Partition 1. In this case p = |I_{0}|+|I_{2}|+|I_{4}| and q = 2. Write I_{1} = { α } and I_{3} = { β }, and organize the positive noncompact roots as in the table below:

\begin{matrix}I′_{0} & I′′_{0} & I′_{2} & I′_{4} & I′′_{2} & I′′_{4} \\ λ_{i} + λ_{α} & λ_{i} + λ_{β} & λ_{α} + λ_{j} & λ_{α} + λ_{k} & λ_{j} + λ_{β} & λ_{β} + λ_{k} \\ λ_{i} − λ_{α} & λ_{i} − λ_{β} & λ_{α} − λ_{j} & λ_{α} − λ_{k} & λ_{j} − λ_{β} & λ_{β} − λ_{k} \end{matrix}

Indices i, j, k will be used systematically to refer to the index sets I_{0}, I_{2}, and I_{4}. Roots in the first row of the table are said to have type I′_{0}(+), I′′_{0}(+), while those in the second row have type I′_{0}(−), etc. We shall use { I′_{r} }, { I′′_{r} }, { I_{r} } = { I′_{r} } + { I′′_{r} } to refer to the number of roots of the indicated type that lie in the system C under consideration. Applying Lemma 5.9 above, we see that only half the I_{4} roots can appear, so that { I_{4} } ≤ 2|I_{4}|. To bound { I_{2} }, partition the I_{2} roots into sets with j constant:

C′_{j} = C ∩ { λ_{α} ± λ_{j} }
C′′_{j} = C ∩ { λ_{j} ± λ_{β} }
C_{j} = C′_{j} ∪ C′′_{j}

If { I_{2} } > 2|I_{2}| then there is a j such that |C_{j}| > 2. Call this index j_{0}. Commutativity then forces

C_{j_{0}} = { λ_{α} + λ_{j_{0}}, λ_{j_{0}} + λ_{β}, λ_{j_{0}} − λ_{β} }.

Moreover, the presence of λ_{j_{0}} ± λ_{β} in C excludes all roots of type I′′_{0}, and the presence of λ_{α} + λ_{j_{0}} excludes elements of type I′_{0}(−). Therefore { I_{0} } ≤ |I_{0}|. In addition, commutativity forces all sets C_{j} for j ≠ j_{0} to be subsets of { λ_{α} ± λ_{j} }. Consequently { I_{2} } ≤ 2|I_{2}| + 1, and so

|C| ≤ |I_{0}| + 2|I_{2}| + 2|I_{4}| + 1.

Since

pq = 2|I_{0}| + 2|I_{2}| + 2|I_{4}|,

we have |C| < pq unless |I_{0}| = 1, in which case |C| ≤ pq. When I_{0} has cardinality 1, we shall write it as I_{0} = { λ_{0} }.

It remains, then, to analyze the case { I_{2} } ≤ 2|I_{2}|. Note that the relation { I_{4} } ≤ 2|I_{4}| obtained in the previous paragraph is still in force. Now if |I_{0}| > 1, then { I′_{0} } ≤ |I_{0}| and so by Lemma 5.8, { I′_{0} } ≤ |I_{0}| and { I′′_{0} } ≤ |I_{0}|. Consequently { I_{0} } ≤ 2|I_{0}|, and so |C| ≤ pq. If |I_{0}| = 1, then |C| ≤ pq unless { I_{0} } > 2. The roots of type I_{0} are

\begin{matrix}λ_{0} + λ_{α} & λ_{0} + λ_{β} \\ λ_{0} − λ_{α} & λ_{0} − λ_{β} \end{matrix}

We must study the cases in which { I_{0} } ≥ 3. If both λ_{0} − λ_{α} and λ_{0} − λ_{β} are in C, then commutativity forces { I′_{2} } = 0, { I′_{4} } = 0, { I′′_{4} } = 0, and { I′′_{2}(+) } = 0. Consequently |C| ≤ 4 + |I_{2}| ≤ pq. There remain two possiblities: the roots in C of type I_{0} are { λ_{0} + λ_{α}, λ_{0} + λ_{β}, λ_{0} − λ_{α} } or { λ_{0} + λ_{α}, λ_{0} + λ_{β}, λ_{0} − λ_{β} }. In the first case commutativity forces { I′_{2} } = 0, { I′_{4} } = 0, and { I′′_{2}(−) } = 0, so that |C| ≤ 3+|I_{2}|+2|I_{4}| < pq. In the second case { I′′_{2} } = 0, { I′′_{4} } = 0. The biggest set that one can form under these restrictions is the one displayed below, which is commutative. Note that its cardinality is pq + 1.

Table 4. Exceptional commutative system for partion 1.

λ_{0} + λ_{α}  λ_{0} + λ_{β}  λ_{α} + λ_{j}  λ_{α}  +  λ_{k}
λ_{0} − λ_{β}  λ_{α} − λ_{j}  λ_{α}  −  λ_{k}

To summarize, if C is commutative for a system of positive roots coming from partition 1, then |C| ≤ pq + 1. Moreover, equality is possible only if |I_{0}| = 1 and C has the form given in table 4. This system is not Hermitian.

Partition 2. By ignoring the I_{4} roots wherever they appear, one may apply analysis of the preceding case, obtaining the same conclusion, with the obvious modification to table 4 above.

Partition 3. In this case p = 2, q = |I_{1}| + |I_{3}|, and the roots under consideration are as below:

\begin{matrix}I′_{1} & I′′_{1} & I′_{3} & I′′_{3} \\ λ_{α} + λ_{i} & λ_{i} + λ_{β} & λ_{α} + λ_{j} & λ_{β} + λ_{j} \\ λ_{α} − λ_{i} & λ_{i} − λ_{β} & λ_{α} − λ_{j} & λ_{β} − λ_{j} \end{matrix}

Apply Lemma 5.9 to get { I_{3} } ≤ 2|I_{3}|. If { I_{1} } ≤ 2|I_{1}|, then |C| ≤ pq. Assume, therefore, that |C| > pq, hence that { I_{1} } > 2|I_{1}|. Let

C_{i} = { λ_{α} ± λ_{i}, λ_{i} ± λ_{β} } ∩ C.

The hypothesis implies that there is an i such that |C_{i}| > 2. Without loss of generality we may assume that this i = 2 (note α = 1), and that

C_{2} = { λ_{α} + λ_{2}, λ_{2} + λ_{β}, λ_{2} − λ_{β} }.

In that case { I′′_{3} } = 0 and

C_{i} ⊂ { λ_{α} + λ_{2}, λ_{2} − λ_{α} }

for i > 2. Therefore C is contained in the set displayed in the table below, where j ∈ I_{3} is arbitrary and where i ∈ I_{1} satisfies i > 2. This sytem is commutative and of cardinality pq + 1. As before, the bound is achieved only the system of the table, and the resulting system is non-Hermitian.

Table 5. Exceptional system.

λ_{α} + λ_{2}  λ_{2} + λ_{β}  λ_{α} + λ_{j}
λ_{2} − λ_{β}  λ_{α} − λ_{j}

λ_{α} + λ_{i} λ_{α} − λ_{i}

Partition 4. By ignoring the I_{3} roots wherever they occur, the analysis of the preceding case applies to give |C| ≤ pq + 1, with a single system of that cardinality, namely, the one obtained by supressing the last column of the previous table.

Partition 5.

In this case p = |I_{0}| + |I_{2}| and q = 2, and the roots to be considered are

\begin{matrix}I′_{0} & I′′_{0} & I′_{2} & I′′_{2} \\ λ_{i} + λ_{α} & λ_{i} + λ_{β} & λ_{α} + λ_{j} & λ_{β} + λ_{j} \\ λ_{i} − λ_{α} & λ_{i} − λ_{β} & λ_{α} − λ_{j} & λ_{β} − λ_{j} \end{matrix}

Lemma 5.9 implies that { I_{2} } ≤ 2|I_{2}|. If I_{0} > 1 then Lemma 5.8 implies that { I_{0} } ≤ 2|I_{0}|, so that |C| ≤ pq. If |I_{0}| = 1, then the I_{0} roots are { λ_{1} +λ_{α}, λ_{1} +λ_{β}, λ_{1} −λ_{α}, λ_{1} − λ_{β} }. If { I_{0} } ≤ 2, then we are done. If { I_{0} } = 4 then { I_{2} } = 0, since λ_{1} − λ_{α} kills the λ_{α} ± λ_{j} and λ_{1} − λ_{β} kills the λ_{β} ± λ_{j}. In this case |C| ≤ 4 ≤ pq, with equality only in the case |I_{0}| = |I_{1}| = 1.

If { I_{0} } = 3 and both λ_{1} − λ_{α} and λ_{1} − λ_{β} appear,then |C| = 3 < pq. If just one of the roots just discussed does not appear, then there are two possible maximal commutative systems. The first is

C′ = { λ_{1} + λ_{α}, λ_{1} − λ_{β}, λ_{1} + λ_{β} } ∪ { λ_{α} ± λ_{j} }.

The second is obtained from this one by replacing α with β. In this final case |C| ≤ pq + 1, realized by the system just given, and systems conjugate to it.

This completes the analysis of the case min(p, q) = 2.

The case min(p, q) = 3.

Consider at last a commutative system C with min(p, q) = 3. The usual arguments based on reduction along a root λ_{i} − λ_{r} apply and function as expected, except that when min(p, q) decreases, C′ may be a system of excess dimension. It suffices to treat this new possibility. If p is the smallest parameter, then the relevant inequality is

|C| ≤ (p − 1)q + 1 + |C′′| = pq + (1 + |C′′| − q).

Since p, q ≥ 3, it suffices to show that |C′′| ≤ 2. To do this, consider one of the exceptional systems E described above, and let I be its index set. Let γ be a new index, and let Ẽ be a commutative system containing E, built on the index set I ∪ { γ }, and containing a root of the form λ_{γ} − λ_{r} or λ_{r} − λ_{γ}, where r ∈ I. We give a complete argument for the first exceptional system; the others are treated similarly. Referring to Table 4, we see that λ_{γ} − λ_{r} cannot occur. In the second case, r must be different from any of the j’s, any of the k’s, and from β. Only the indices 0 and α remain, leaving the roots λ_{0} ± λ_{γ}, λ_{α} ± λ_{γ} as candidates. The only possible sets C′′ are therefore subsets of one of the two-element sets { λ_{0} − λ_{γ}, λ_{α} − λ_{γ} }, { λ_{0} ± λ_{γ} }, { λ_{α} ± λ_{γ} }. Thus |C′′| ≤ 2, as required. The other cases are dealt with similarly.

The argument just given in fact proves somewhat more. If q ≥ p = 3, and if C contains a root of the form λ_{i} − λ_{r} in which a fundamental weight occurs with negative coefficient then we have shown |C| ≤ pq. However, strict inequality holds if q > p. Thus, if min(p, q) = 3 and C has maximum cardinality, it must be “standard”, i.e., equivalent to C^{+}, unless (p, q) = (3, 3). In this latter case there are, up to equivalence, just two commutative systems of maximum cardinality, namely C^{+} and the system

C^{∗} = { λ_{0} + λ_{1}, λ_{0} ± λ_{3}, λ_{0} ± λ_{5}, λ_{1} ± λ_{2}, λ_{1} ± λ_{4} }

The parities of the indices are the natural ones, namely i modulo 2. If W is the abelian space corresponding to C^{∗}, then W + W̄ generates 𝔤_{ℂ}, i.e., G′ = G. Thus W is not the holomorphic tangent space of any geodesically imbedded hermitian symmetric space.

We note in passing that all of the exceptional systems contain a system of the form

E = { λ_{0} + λ_{α}, λ_{0} ± λ_{β}, λ_{α} ± λ_{j} }

for a suitable labeling of the indices, and where all indices are fixed except j, which ranges over a set of elements of the same parity.

The cases SO(2p, 2q + 1) and SO(2p + 1, 2q + 1).

In these cases the root system is of type B_{n} and contains the substem D_{n} analyzed above together with the short roots λ_{i}. One sees easily that the commutative system of largest cardinality is that obtained by adjoining a single short root to a system of largest cardinality for SO(2p, 2q).

§ 6. Centralizers of semi-simple elements.

In this section we describe the centralizers of semi-simple elements for the algebras in our list, and use this information to find the maximum dimension μ of an arbitrary abelian space, thus completing all the dimension bounds outlined in Step 1 of §2. In the course of our analysis we will also determine an invariant ν_{0} such that if dim W > ν_{0} then W consists entirely of nilpotents. The values of μ and ν_{0} are given in table 6 below. The case μ = ν_{0} is interpreted to mean that there are two kinds of abelian subalgebras of dimension μ, one consisting entirely of nilpotents, the other containing a semisimple element.

Table 6. The Invariants μ and ν_{0}.

Group  μ  ν_{0}  Remarks
SU^{∗}(2n)  n(n − 1)/2  1 + (n − 1)(n − 2)/2 n ≥ 3
SL(2n)  n(n + 1)/2  1 + n(n − 1)/2  n ≥ 3
SL(2n + 1) n(n + 1)/2 + 1  μ
Sp(p, q)  pq  1 + (p − 1)(q − 1)  q ≥ p ≥ 2
SO(p, q)  [\frac{p}{2}] [\frac{q}{2}] + ϵ(p, q) [\frac{p−1}{2}] [\frac{q−1}{2}] + 2  p and q not both odd
p, q ≥ 6  p, q ≥ 7
SO(p, q)  [\frac{p}{2}] [\frac{q}{2}] + 1  μ  p,q odd ≥ 3

To begin the proof, we recall that by part (b) of Step 1, it suffices to consider centralizers of real elements. These are described as follows:

6.1. Proposition. Let 𝔞_{0} ⊂ 𝔭 be an abelian subspace, and 𝔩 be the sum of the noncompact simple subalgebras in the semi-simple part of the centralizer of 𝔞_{0}. Then 𝔩 is as follows:

1. If 𝔤 = 𝔰𝔩(n, ℝ), then 𝔩 = 𝔰𝔩(n_{1}, ℝ) ⊕ ⋯ ⊕ 𝔰𝔩(n_{k}, ℝ), where n_{1} + ⋯ + n_{k} ≤ n.

2. If 𝔤 = 𝔰𝔲^{∗}(2n), then 𝔩 = 𝔰𝔲^{∗}(2n_{1}) ⊕ ⋯ ⊕ 𝔰𝔲^{∗}(2n_{k}), where n_{1} + ⋯ 𝔫_{k} ≤ n.

3. If 𝔤 = 𝔰𝔬(p, p + r), r ≥ 0, then 𝔩 = 𝔰𝔩(p_{1}, ℝ) ⊕ ⋯ ⊕ 𝔰𝔩(p_{k}, ℝ) ⊕ 𝔰𝔬(q, q + r), where p_{1} + ⋯ + p_{k} + q ≤ p.

4. If 𝔤 = 𝔰𝔭(p, p + r), r ≥ 0, then 𝔩 = 𝔰𝔲^{∗}(2p_{1}) ⊕ ⋯ ⊕ 𝔰𝔲^{∗}(p_{k}) ⊕ 𝔰𝔭(q, q + r), where p_{1} + ⋯ + p_{k} + q ≤ p.

Proof: Let 𝔞 be a maximal abelian subspace of 𝔭 containing 𝔞_{0}, and let Σ ⊂ 𝔞^{∗} denote the root system of 𝔤 relative to 𝔞. Thus Σ consists of the restrictions to 𝔞 of the roots of 𝔤^{ℂ} relative to a Cartan subalgebra of 𝔤^{ℂ} containing 𝔞. This restricted system together with the multiplicities of the simple roots determines 𝔤 (see Table VI, pp 532-534 of [13]) . To prove the Proposition, one first observes that from the definition of 𝔩, the root system of 𝔩 is Σ ∩ 𝔞_{0}^{⊥} From this it follows that any system of positive roots for Σ ∩ 𝔞_{0}^{⊥} can be enlarged to a system of positive roots for Σ. Therefore the associated simple roots for Σ ∩ 𝔞_{0}^{⊥} are also simple roots for Σ. Thus the Dynkin diagram of 𝔩 is a subdiagram of the Dynkin diagram of 𝔤. To complete the proof, one simply looks in [13] , Table VI, at the Dynkin diagrams in question, and verifies that the possibilities listed in the Proposition correspond precisely all the subdiagrams, together with their multiplicities. These are the diagrams A I for case 1, A II for case 2, B II for case 3, r odd, D I for case 3, r even, and finally C II for case 4.

Consider now an abelian space W = W_{σ} + W_{ν}, where W_{σ} is the complexification of a real abelian space, and W_{ν} = W_{1} + ⋯ + W_{k}, where W_{i} ⊂ 𝔭_{i}^{ℂ}, and where 𝔩_{i} = 𝔨_{i} ⊕ 𝔭_{i} is the Cartan decompositions of the Lie algebras 𝔩_{i} as in the proposition.

The dimension of each of these terms is, by definition, bounded by ν(𝔩_{i}), the invariant described in Proposition 5.1 and Table 3. To bound the dimension of W_{σ}, note that the split rank of 𝔷 is the same as the split rank ρ of 𝔤, a fact which we can write as ρ(𝔤) = dim(W_{σ}) + ∑_{i=1}^{k} ρ(𝔩_{i}). Therefore

dim W ≤ ρ(𝔤) − ∑_{i=1}^{k} ρ(𝔩_{i}) + ∑_{i=1}^{k} ν(𝔩_{i}).  (6.2)

The maximum of the right-hand side over all centralizers is the invariant μ. We must compute it for the each of the groups SU^{∗}, SL, Sp, and SO. In so doing we will also determine a minimal integer ν_{0} such that dim W > ν_{0} implies that W consists entirely of nilpotent elements. In some cases, namely, SL(n) for n odd or SO(p, q) for both p and q odd, an abelian space of maximal dimension can have semisimple elements. We signal this fact by writing ν_{0} = μ. The relevant information is summarized in Table 6 above.

Case SU.

We begin with this case, since it is the easiest. By the proposition, the summands 𝔤_{i} are of the form 𝔰𝔲^{∗}(2n_{i}), which have split rank n_{i} − 1. Therefore the inequality (6.2) can be written as

dim W ≤ k − 1 + ∑_{i=1}^{k} ν(𝔩_{i}).

Setting f(n) = ν(n) + 1, this reads

dim W ≤ −1 + ∑_{i=1}^{k} f(n_{i}).

It will therefore be sufficient to maximize the expression on the right-hand side, which we write as F (n_{1}, ..., n_{k}), for all partitions of numbers m ≤ n. In fact, it is enough to maximize F (n_{1}, ..., n_{k}) for all partitions of n. For this it is enough to observe that the function f(x) = x(x − 1)/2 + 1 is superadditive, i.e., f(x + y) ≥ f(x) + f(y), for x, y ≥ 1. Therefore F, which is a sum of superadditive functions, is also superadditive. Consequently it takes its maximum for the trivial partition (n), and so μ = n(n − 1)/2. Moreover, since f is strictly superadditive if x > 1 or y > 1, the trivial partition is the only partition for which the maximum is attained, except that F (2) = F (1, 1). Therefore an abelian space of maximum dimension is necessarily nilpotent, except when n = 2. In the extreme case, SU^{∗}(4), the split rank is one, and ν = 1, so an abelian space of maximal dimension consists entirely of semisimple elements or entirely of nilpotents.

Suppose that W does not consist entirely of nilpotents. Then it lies in a nontrivial centralizer, hence is bounded by the value of F on a nontrivial partition. Superadditivity implies that it is in fact bounded by F (n − 1, 1), which is always less than F (n) if n > 2. Set

ν_{0} = F (n − 1, 1) = 1 + (n − 1)(n − 2)/2.

We have shown that an abelian space of dimension d > ν_{0} consists entirely of nilpotent elements.

Case SL.

The analysis is similar to that of the previous, case, except that the formula for f(n) depends on the parity of n. Elementary arguments show that it is superadditive, and strictly superadditive except for the cases

f(4) = f(3) + f(1)
f(4) = f(2) + f(2)
f(2m + 1) = f(2m) + f(1)

Therefore

μ(n) = \frac{1}{2} [\frac{n}{2}] ([\frac{n}{2}] + 1) + ϵ(n),

where ϵ(n) = 0, 1 according to whether n is even or odd. When n is odd, there is always an abelian space of maximal dimension which has the form 𝔞′ + W′, where 𝔞′ is a one-dimensional space of semisimple elements, and where W′ is an abelian space of maximum dimension in a copy of SL(n − 1), namely, the centralizer of 𝔞′ in 𝔤. When n is even, μ = ν except for n = 2, 4.

Suppose now that W is an abelian subspace for SL(2n) which does not consist entirely of nilpotents. Then it lies in a nontrivial centralizer and so, by superadditivity, is bounded in dimension by F (2n − 1, 1) = n(n − 1)/2 + 1, which quantity we denote by ν_{0}. Thus, if dim W > ν_{0}, then W consists entirely of nilpotent elements.

Case Sp.

We shall write the symplectic group as Sp(p, p + r) as in the proposition. The decomposition of 𝔷 then takes the form

𝔞 + 𝔩_{1} + ⋯ + 𝔩_{k} + 𝔩_{k+1},  (6.3)

where the 𝔩_{i} for i < k+1 are copies of 𝔰𝔲^{∗}(p_{i}), 𝔤_{k+1} = 𝔰𝔭(q, q+r), and p_{1}+⋯+p_{k}+q ≤ p. Since the split rank of Sp(p, p + r) is p, the bound (6.2) can be written

dim W ≤ F (p_{1}, ..., p_{k}) + q(q + r) + 1,

where F is the function used in arguing the case SU^{∗}. The relation remains valid when k = 0, provided that one defines F (∅) = −1. Using the bounds for SU^{∗}, we find that

\begin{matrix}g(q) = (p − q)(p − q − 1)/2 + q(q + r) + 1 for q < p, \\ g(p) = p(p + r) \end{matrix}

This function is convex on the interval [0, p − 1] and so takes its local maxima at endpoints. In fact, it takes its local maxima at the endpoints of the larger interval [0, p], from which we conclude that W is always bounded in dimension by g(p) = p(p + r). This is the value of the invariant μ. Except for the case 𝔰𝔭(1, 1) ≅ 𝔰𝔬(1, 4), the value of g at the right-hand endpoint exceeds the value at the left-hand endpoint. Nontrivial abelian spaces are then one-dimensional, and can be either of semisimple or nilpotent type. Thus, if (p, p + r) ≠ (1, 1), then an abelian subspace contained in a nontrivial centralizer must have dimension no larger than ν_{0} = g(p − 1) = (p − 1)(p − 1 + r) + 1. Any abelian subspace of larger dimension consists entirely of nilpotent elements.

Case SO.

The decomposition of a typical centralizer takes the form (6.3) used above, except that now 𝔩_{i} = 𝔰𝔩(p_{i}) for i < k + 1, and 𝔩_{k+1} = 𝔰𝔬(q, q + r), where p_{1} + ⋯ p_{k} + q ≤ p. Arguing as before, we find that

dim W ≤ F (p_{1}, ..., p_{k}) + ν(q, q + r) + 1,

where now F is the partition function for SL, and ν(q, q + r) is the dimension of a maximal abelian subalgebra of 𝔰𝔬(q, q +r) which consists entirely of nilpotents. For low values of p there are only a small number of cases to consider. Computing them by hand, we obtain the following for μ(p, q) with q ≥ p:

μ(1, q) = 1 μ(2, q) = q μ(3, q) = q + 1 μ(4, q) = 2q + 1 μ(5, q) = 2q + 2

μ(6, q) = 3 [\frac{q}{2}] + ϵ(q),

For the remaining cases we proceed by induction, with inductive hypothesis

μ(p, q) = [\frac{p}{2}] [\frac{q}{2}] + ϵ(p, q),

where ϵ = 0 if both p and q are even, and ϵ = 1 otherwise, and where q ≥ p ≥ 6. Because of the superadditivity of the partition function which computes the dimension of the 𝔰𝔩(n)-type terms, it is enough to argue the cases 1) 𝔷 = 𝔤, 2) 𝔷 = 𝔞 + 𝔤_{1}, and 3) 𝔷 = 𝔞+𝔤_{1} +𝔤_{2}, where in each 𝔞 is one-dimensional. Suppose first that (p, q) = (2a, 2b). In case (1) the bound is that for spaces of nilpotents, namely, ab. In case (2) the centralizer is of the form 𝔞 + 𝔰𝔬(2a − 1, 2b − 1), for which the bound is (a − 1)(b − 1) + 2. This quantity is strictly less than ab, since for the puposes of induction we assume b ≥ a ≥ 4. For case (3) The centralizer is of the form 𝔞 + 𝔰𝔩(2c − 1) + 𝔰𝔬(2a − 2c, 2b − 2c) or 𝔞 + 𝔰𝔩(2c) + 𝔰𝔬(2a − 2c, 2b − 2c). We compute the relevant bound in each case, and see that the second is always larger, namely, g(c) = (a − c)(b − c) + c(c + 1)/2 + 1. This function is convex, and so takes its local maxima at endpoints of the relevant interval, which is [1, a]. Now g(1) = (a − 1)(b − 1) + 2, which is a value already considered, and g(a) = a(a + 1)/2 + 1. We find that ab > g(a) for b ≥ a > 2. The induction step is therefore complete in this case (p and q both even). Moreover, we see that if W is an abelian subspace of dimension greater than

ν_{0} = (a − 1)(b − 1) + 2,

then it must consist entirely of nilpotent elements.

There is a similar analysis for the other cases. If (p, q) = (2a, 2b + 1), with a, b ≥ 3, then μ = ab + 1. Moreover, if W has dimension greater than ν_{0} = (a − 1)b + 2, then it consists entirely of nilpotents. If (p, q) = (2a + 1, 2b + 1), then μ = ab + 1, but one can make no statement about nilpotents, since there is an abelian space of the form 𝔞 + W′ with 𝔞 one-dimensional and semisimple, and W′ maximal abelian for a copy of 𝔰𝔬(2a, 2b), the centralizer of 𝔞.

The induction argument, modulo the verifications indicated above, is now complete. We have also established the following. Let W be an abelian subspace for 𝔰𝔬(p, q), where p, q ≥ 4, and at least one of p, q is even. If

dim W ≥ ν_{0} =_{def} [\frac{p − 1}{2}] [\frac{q − 1}{2}] + 2,

then W is abelian.

§ 7. Rigidity

In this section we carry out the arguments necessary to establish infinitesimal and global rigidity that were sketched in §2 for groups of type I. The ultimate goal is to show that f takes values in the image of a quotient of a Hermitian symmetric space G_{1}/K_{1} by a discrete group. Here is a precise statement:

7.1. Rigidity Theorem.. Let X = G/K be a symmetric space, where G is one of SL(2n), SU^{∗}(2n), SO(2p, 2q), or Sp(p, q), subject to the restrictions n ≥ 3, in the first two cases, p, q ≥ 3, (p, q) ≠ (3, 3) in the third, and p, q ≥ 2 in the last. Let f : M ⟶ X/Γ be a harmonic map of maximum rank, as given in Table 6. Then f factors as gh, where h : M ⟶ X_{1}/Γ_{1} is holomorphic, X_{1} = G_{1}/K_{1} is a hermitian symmetric space, Γ_{1} is a subgroup of Γ acting discretely on it. Moreover, the map g is a geodesic immersion. Finally, the groups G_{1} corresponding to G are given in Table 7 below.

Proof: According to the sketch presented in §2, the rigidity argument comes in two parts, first order and global (=local). In the last four sections we have completed the arguments sketched in Step 1 of §2, and proved that for the groups G of Theorem (7.1) any maximum dimension abelian subalgebra consists entirely of nilpotent elements, and that its space of leading roots is unique, up to isomorphism, and is 𝔭_{1}^{+} for the Hermitian symmetric subspace corresponding to G_{1}. We conclude the remaining details of Steps 2 and 3 of §2.

Let W be an abelian subalgebra of maximum dimension μ as in Table 6. Then W = {X + ϕ(X) : X ∈ 𝔭_{1}^{+}} for some linear map ϕ : 𝔭_{1}^{+} → 𝔮^{+} satisfying (2.3). We let 𝔮′ = 𝔮^{+} + 𝔮^{−}, and observe that 𝔮′ ⊂ 𝔮^{ℂ}, with equality for the groups SO and Sp, but strict containment for SL and SU. (In the latter cases 𝔮^{ℂ} also contains the non-compact part of the fundamental Cartan subalgebra of Lemma (3.1).) We view ϕ as taking values in 𝔪 + 𝔮′, which is a 𝔭_{1}^{+}-module. From this perspective ϕ is a cocycle for Lie algebra cohomology, with class [ϕ] ∈ H^{1}(𝔭_{1}^{+}, 𝔪 + 𝔮′). Observe next that 𝔭_{1}^{+} is a 𝔨_{1}-module as are the terms of the decomposition 𝔪^{+} + 𝔪^{−} + 𝔮′. Therefore the space of 1-cochains

C^{1} = Hom(𝔭_{1}^{+}, 𝔪 + 𝔮′),  (7.2)

decomposes into 𝔨_{1}-modules as

Hom(𝔭_{1}^{+}, 𝔪^{+}) ⊕ Hom(𝔭_{1}^{+}, 𝔪^{−}) ⊕ Hom(𝔭_{1}^{+}, 𝔮′).

and this decomposition passes to cohomology. Indeed, one finds that

δ(𝔪^{+}) = 0
δ : 𝔪^{−} ⟶ Hom(𝔭_{1}^{+}, 𝔮′)
δ : 𝔮′ ⟶ Hom(𝔭_{1}^{+}, 𝔪^{+}),

so that

H^{1} = Hom(𝔭_{1}^{+}, 𝔪^{+})_{0}/δ𝔮′ ⊕ Hom(𝔭_{1}^{+}, 𝔪^{−})_{0} ⊕ Hom(𝔭_{1}^{+}, 𝔮′)_{0}/δ𝔪^{−},  (7.3)

where the subscript denotes the space of cocycles. Using a theorem of Kostant [10], we shall show, under the hypotheses of Theorem 7.1, that only the first term in the preceding decomposition is nonzero: the 𝔪^{−} and 𝔮′ components of H^{1} vanish. Thus ϕ is a coboundary, so that there is an element X_{0} of 𝔪^{ℂ} such that ϕ(X) = [X, X_{0}]. As noted in §2, this implies that W′ is K^{ℂ}-conjugate to 𝔭_{1}^{+}.

To obtain the stronger K-conjugacy result, we compare the K and K^{ℂ} orbits of 𝔭_{1}^{+}. The first can be identified with K/K_{1} and the second with K^{ℂ}/F, where F is a parabolic subalgebra with Lie algebra 𝔣 = 𝔨_{1}^{ℂ} + 𝔪^{+}. From this last assertion it follows that the two orbits have the same dimension. Indeed, the real tangent space of K/K_{1} is the set of real points in 𝔪, whereas that of K^{ℂ}/F is 𝔪^{−}, viewed as a real vector space. It follows that the inclusion of the K-orbit in the K^{ℂ}-orbit is open. But the K orbit is compact, so that in view of the connectivity of the K^{ℂ}-orbit, the inclusion must be an isomorphism.

As explained already Step 3 of §2, from first-order rigidity one obtains local liftings of a harmonic map f of maximum rank from G/K to G/K_{1}. Local rigidity follows once this map is known to be horizontal, meaning that μ = 0, where d = D_{1} + μ + θ gives the reduction of the flat G-connection to a K_{1}-connection. Since μ is real-valued, it suffices to establish the vanishing of its (1,0) component μ′. This we do using the equations which μ′ satisfies. First, since df maps T^{1,0}M onto 𝔭_{1}^{+}, we may reagrd μ′ as defined on 𝔭_{1}^{+}. Equation (2.9) implies that μ′ : 𝔭_{1}^{+} ⟶ 𝔪, viewed as is a map from 𝔭_{1}^{+} to 𝔪 + 𝔮′, defines a class in H^{1}(𝔭_{1}^{+}, 𝔪 + 𝔮′). By the application of Kostant’s theorem, already referred to above, we see that [μ′] = [μ′_{+}] + [μ′_{−}], with [μ′_{−}] = 0. In fact, μ′_{−} vanishes as a cocycle, since, by (2.3), there are no coboundaries of type 𝔪^{−}. To kill the remaining part of μ, we use the conjugate of the second equation of (2.8), which implies that μ′_{+}(X) is a vector in 𝔪^{+} commuting with 𝔭_{1}^{−}. From Table 7 below, we see that any such vector must vanish.

It remains to justify the assertion made earlier, that all H^{1}(𝔭_{1}^{+}, 𝔪 + 𝔮′) is entirely of type 𝔪^{+}. For this we use a special case of Theorem 5.14 of [10], which may be stated as follows:

7.4. Theorem (Kostant).. Let (𝔤_{1}, 𝔨_{1}) be a Hermitian symmetric pair, and let V be an irreducible 𝔤_{1}-module with highest weight λ and highest weight vector v_{λ}. Consider a system of simple roots for 𝔤_{1} such that 𝔭_{1}^{+} is a sum of positive root spaces. Let σ be a simple reflection such that all roots of Δ^{+}(𝔨_{1}) are positive. Let ω be an element of the space dual to 𝔭_{1}^{+}. Then highest weight vectors of H^{1}(𝔭_{1}^{+}, V) have the form ω ⊗ v_{σ(λ)}.

Using the criterion of the theorem and the explicit form of the roots for 𝔤_{1} and 𝔨_{1} given in Table 7, it is easy to determine the (unique) distinguished reflection σ of the preceding theorem. Under the hypotheses of Theorem 7.1 the highest weight vectors of 𝔪 + 𝔮′, which are always of type 𝔪^{+}, are invariant under σ. This completes the proof.

Table 7.

Group  Roots  Remarks
SL(2n)  Δ^{+}(𝔨_{1}) = { λ_{r} − λ_{s} | r < s }  𝔨_{1} = 𝔲(n)
Δ^{+}(𝔭_{1}) = { λ_{r} + λ_{s} | r ≤ s }  𝔤_{1} = 𝔰𝔭(n, ℝ)
Δ^{+}(𝔪) = { λ_{r} + λ_{s} | r < s }  𝔨 = 𝔰𝔬(2n)

Δ^{+}(𝔮) = { λ_{r} − λ_{s} | r < s } 𝔪 + 𝔮′ = V^{λ_{1}+λ_{2}}

σ = σ_{α_{n}}  λ_{n} ⟶ −λ_{n}
SU^{∗}(2n)  Δ^{+}(𝔨_{1}) = { λ_{r} − λ_{s} | r < s }  𝔨_{1} = 𝔲(n)
Δ^{+}(𝔭_{1}) = { λ_{r} + λ_{s} | r < s }  𝔤_{1} = 𝔰𝔬^{∗}(2n)
Δ^{+}(𝔪) = { λ_{r} + λ_{s} | r ≤ s }  𝔨 = 𝔰𝔭(n)

Δ^{+}(𝔮) = { λ_{r} − λ_{s} | r < s } 𝔪 + 𝔮′ = V^{2λ_{1}}

σ = σ_{α_{n}}  λ_{n} ↦ −λ_{n}, λ_{n−1} ↦ −λ_{n−1}
SO(2p, 2q) Δ^{+}(𝔨′_{1}) = { λ_{a} − λ_{b} | a < b }  𝔨′_{1} = 𝔲(p) in general, a, b ≤ p
Δ^{+}(𝔨′′_{1}) = { λ_{i} − λ_{j} | i < j }  𝔨′′_{1} = 𝔲(q) in general, i, j > p
Δ^{+}(𝔭_{1}) = { λ_{a} − λ_{i} }  𝔤_{1} = 𝔲(p, q)
Δ^{+}(𝔪′) = { λ_{a} + λ_{b} | a < b }  𝔨′ = 𝔰𝔬(2p)
Δ^{+}(𝔪′′) = { −λ_{i} − λ_{j} | i < j }  𝔨′′ = 𝔰𝔬(2q)

Δ^{+}(𝔮) = { λ_{a} + λ_{i} }

𝔪 + 𝔮′ = V^{λ_{1}+λ_{2}} ⊕ V^{−(λ_{n−1}+λ_{n})} n = p + q
σ = σ_{α_{p}} = (p, p + 1)  λ_{p} ↔ λ_{p+1}
Sp(p, q)  Δ^{+}(𝔨′_{1}) = { λ_{a} − λ_{b} | a < b }  𝔨′_{1} = 𝔲(p) in general, a, b ≤ p
Δ^{+}(𝔨′′_{1}) = { λ_{i} − λ_{j} | i < j }  𝔨′′_{1} = 𝔲(q) in general, i, j > p
Δ^{+}(𝔭_{1}) = { λ_{a} − λ_{i} }  𝔤_{1} = 𝔲(p, q)
Δ^{+}(𝔪′) = { λ_{a} + λ_{b} | a ≤ b }  𝔨′ = 𝔰𝔬(2p)
Δ^{+}(𝔪′′) = { −λ_{i} − λ_{j} | i ≤ j }  𝔨′′ = 𝔰𝔬(2q)

Δ^{+}(𝔮) = { λ_{a} + λ_{i} }

𝔪 + 𝔮′ = V^{2λ_{1}} ⊕ V^{−2λ_{n}}  n = p + q
σ = σ_{α_{p}} = (p, p + 1)  λ_{p} ↔ λ_{p+1}

§ 8. Existence.

The purpose of this section is to demonstrate the existence of non-obvious harmonic mappings, namely, ones that do not come from sources 1–3 mentioned in the introduction (geodesic immersions, maps which factor through a curve, and maps which factor through totally geodesic real tori). Our constructions, based on the following lemma, come from Hodge theory:

8.1. Lemma.. Let F : M ⟶ D/Γ be a variation of Hodge structure, and let p : D/Γ ⟶ X/Γ be the canonical projection to the associated locally symmetric space. Then p ∘ F is pluriharmonic.

With this in hand, we can construct pluriharmonic maps whose domain is, in an essential way an algebraic surface:

8.2. Theorem.. There exist pluriharmonic mappings f : M ⟶ X/Γ, where M is an algebraic surface and X = G/K with G = SO(2p, q). The differential of these mappings is generically injective, and the image of the fundamental group is a lattice Γ in G.

Proof of (8.1). Recall that D = G/V and X = G/K, where K is maximal compact and V ⊂ K is the centralizer of a torus. A map from f : M ⟶ D/Γ is given by a flat G-connection ∇ and a reduction of it to a V -connection, where D = G/V. Write ∇ = D + θ, where D is the V -connection and θ is a 𝔤-valued 1-form representing df. Write the decomposition into forms of type (1, 0) and (0, 1) as θ = θ′ + θ′′. If θ′ is 𝔤^{−1,1}- valued, where 𝔤_{ℂ} = ∑ 𝔤^{p,−p} is the natural Hodge decomposition on the Lie algebra, then f is, by definition, a variation of Hodge structure. Now write out the equation ∇^{2} = 0, separating it into components relative to the decomposition

𝔤 ⊗ A^{k} = ∑ 𝔤^{−p,p} ⊗ A^{r,s}.

One finds that D^{′′2} = 0, D′′(θ′) = 0, and [θ′, θ′] = 0. These same equations hold if one views D as a K-connection and θ as a 𝔭-valued one-form, i.e., if one views the projection f of F to the associated locally symmetric space. But these are the pluriharmonic equations for p ∘ F.

It remains to construct variations of Hodge structure with compact base. For this it would suffice to construct a smooth family of algebraic varieties with compact parameter space M; the “period mapping”, which associates to t ∈ M the Hodge structure on H^{k}(V_{t}) then gives a variation of Hodge structure. Unfortunately, it is difficult to construct nontrivial families with compact parameter space. We therefore construct first a smooth family V ⟶ U, where U is a Zariski open in a projective manifold T, and with the property that the local monodromy representation is finite. To explain this, let D = T − U be the divisor at infinity, let p be a point of D, let B be a small ball centered at p, and let b be a point of B − D. “The” local monodromy representation is the natural representation ρ_{p} : π_{1}(B − D, b) ⟶ GL(H^{k}(V_{b})). By pulling back the original family on U to a suitable unramified cover U′, we obtain a family with trivial local monodromy. The new parameter space sits as a Zariski open in a projective variety M; appealing to a theorem of Griffiths that asserts that period mappings behave locally as do holomorphic maps to a bounded domain, we find that the lifted map extends (as a variation of Hodge structure) to M. We have therefore reduced the construction to a) filling in the details of the sketch just given and b) constructing nontrivial families with finite local monodromy.

Let us complete the sketch of the proof, then construct the required families. To this end we suppose given a family of algebraic varieties V ⟶ T which is smooth over a Zariski open set U = T − D, where D is a normal crossing divisor, i.e., one whose local equations are of the form z_{1} ⋯ z_{k} = 0, where z_{1} , …, z_{n} are local analytic coordinates. Let ρ : π_{1}(U, b) ⟶ Γ be the monodromy representation. Its image is a finitely generated matrix group and so has a normal subgroup Γ′ which is torsion-free and of finite index in Γ. We may take this subgroup to be defined by suitable congruences [2]. Let U′ be the unramified cover corresponding to the kernel of the natural map π_{1}(U, b) ⟶ Γ/Γ′, and let ρ′ : π_{1}(U′, b′) ⟶ Γ′ be the induced monodromy representation. According to lemma 8.3 below, the cover U′ has the form M − D′, for a suitable projective manifold M. Now suppose that the local monodromy representations for the family V over T are finite. The same is true for the pullback of this family to M. However, these representations must in fact be trivial, since they take values in a torsion-free group. Griffiths’ extension theorem [12] then applies to show that the period map for this family extends from U′ to M.

We turn now to the compactification result:

8.3. Lemma.. Let Y be a projective variety, D a hypersurface in Y, and f′ : U′ ⟶ Y − D a finite unramified cover. Then there is a projective variety X, and an algebraic map f : X ⟶ Y which extends f′.

Proof. By resolution of singularities we may assume that D has normal crossings. Thus, a p is a point of D has a coordinate neighborhood V such that V^{∗} = V ∩ Y − D is isomorphic to a product of disks and punctured disks: V^{∗} = Δ^{p} × Δ^{∗q}. The unramified covers of these are given by maps

(s_{1} , …, s_{p}, t_{1} , …, t_{q}) ⟶ (s_{1} , …, s_{p}, t_{1}^{n_{1}} , …, t_{q}^{n_{q}})

Therefore one can locally form a smooth compactification of the cover in question. The local divisor at infinity is a normal crossing variety. These local compactifications patch together to give a global compactification which, a priori, is merely an analytic space. Using the Nakai-Moishezon criterion, however, one sees that it is in fact projective. We give the complete argument in the two-dimensional case, which is all we need. Let L be a bundle on Y which projectively imbeds Y, and let L̃ be its pullback to X. Take two sections of L whose zero sets Z_{1} and Z_{2} meet transversely have no common points which lie on the branch locus. Then the intersection number (Z_{1} · Z_{2}) is positive. If Z̃_{i} denotes the zero set of the pullback, then (Z̃_{1} · Z̃_{2}) = n(Z_{1} · Z_{2}) > 0. Here n is the degree of the cover. Next, let W be an irreducible curve on X which does not lie in the ramification locus. Its projection to Y meets a suitably chosen zero set Z of a section of L transversely and away from the branch locus. Since W meets the inverse image of Z in the same way, (W · Z̃) is positive. It remains to consider the case of W irreducible and contained in the branch locus. Since that locus is a normal crossing variety, we can still find a section Z of L whose zero set meets the projection of W tranversely. Again, we conclude that (W · Z̃) is positive.

Finally, we construct two-parameter families whose global monodromy group is arithmetic, but whose local monodromy groups are finite. To this end consider the family of all hypersurfaces of degree d and dimension n, parametrized by ℙ^{N}. Let D be the discriminant locus, i.e., the set t ∈ ℙ^{N} such that the corresponding hypersurface X_{t} is singular. Let Γ̃ = O(H^{n}(X_{t})_{0}) be the orthogonal group on the primitive integral cohomology which preserves the cup-product. The monodromy group Γ of this family is a subgroup of Γ̃ and, according to a result of Beauville [1] Theorem 2, it is of index two, provided that n is even and d ≥ 4. If n ≥ 4, it is enough to require that d ≥ 3. Consider now a generic 2-plane 𝕃 ⊂ ℙ^{N}, and let C = L ∩ D be its intersection with the discriminant locus. This curve contains a Zariski-open set of smooth points corresponding to hypersurfaces with a single ordinary double point and no other singularities. In suitable local analytic coordinates x_{1} , …, x_{n+1} the equation of such a hypersurface near such a point is x_{1}^{2} + ⋯ + x_{n+1}^{2} = 0. It is well-known that the local monodromy group for these singularities is of order two, given by a Picard-Lefschetz reflection. It is also fairly wellknown that for when L is generic, the singular locus of C consists of two types of points, those for which X_{t} has two ordinary double points, and those for which X_{t} has one A_{2} singularity and is otherwise smooth. Recall that an A_{2} singularity has local equation x_{1}^{2} + ⋯ + x_{n}^{2} + x_{n+1}^{3} = 0. In either case, the local monodromy group is finite. Moreover, the inclusion of L−C in ℙ^{N} −D induces an isomorphism on fundamental groups, so that our family restricted to L satisfies both of the local and global conditions we have sought to impose.

We were unable to find a reference for the fact that the singular locus of C, for generic L, has the asserted properties, and so reproduce here an argument due to Robert Varley. To begin, we note that the singular locus of D is the union of two pieces D′ and D′′, the first consisting of points t for which X_{t} has more than one singular point, the second consisting of points t for which X_{t} has a singular point whose tangent cone is nongeneric. To understand D′, consider the locus L(p, q) of points t in D for which X_{t} has two singularities at fixed locations p and q in ℙ^{n+1}. This is a linear subspace of ℙ^{N}, since it is defined by the equations ∇F (p) = 0, ∇F (q) = 0, where F is a homogeneous form with undetermined coefficients, and where ∇F is its gradient. The linear system of hypersurfaces parametrized by L(p, q) has a base locus B, namely the set of points common to all of the hypersurfaces of the system. By Bertini’s theorem, a general member of this system of hypersurfaces is smooth outside B. However, if d is large enough, e.g., larger than 3, B consists of p and q alone. (One may argue with reducible hypersurfaces to establish this point). Similar arguments apply to D′′. Consider the locus L(p, v) consisting of hypersurfaces { F = 0 } where ∇F (p) = 0 and H(F, p)v = 0, where H(F, p) is the (Hessian) matrix of second partials at p. This set is again a projective linear subspace, and Bertini’s theorem applies in the same way: if d is large enough, then the base locus consist of p and nothing else.

It remains to show that the differential of the pluriharmonic map f is generically injective. By Griffiths’ local Torelli theorem [11], dF is generically injective if the 2-plane L is generically transverse to the orbits of P GL(n) acting on ℙ^{N}, which is the case if L is chosen generically. Since F is horizontal, the result follows.

§ 9. Concluding Remarks

In this section we prove some of the results that were mentioned in the introduction and which follow from our methods. We also discuss some open problems.

First we recall the statement of theorem 1.2 from the introduction, concerning representations into SO^{+}(2p, 2q):

9.1. Theorem. Let M be a compact Kähler manifold, let ρ : π_{1}(M) → SO^{+}(2p, 2q), p, q ≥ 3, (p, q) ≠ (3, 3), be a representation, and let e_{+} ∈ H^{2p}(M) be the Euler class of a maximal positive sub-bundle of the flat vector bundle associated to ρ. If e_{+}^{q} ≠ 0, then the image of ρ is contained in some embedding of U(p, q) in SO^{+}(2p, 2q).

For the proof, we note that if the representation ρ is reductive, then by a Theorem of Corlette [6], it follows that the associated flat bundle has a harmonic section, equivalently, that there is a ρ-equivariant harmonic map f : M̃ :→ X, where X is the symmetric space for SO^{+}(2p, 2q). The characteristic class in question is represented by the pull-back by f of an invariant form on X of degree 2pq, hence its non-vanishing implies that f has rank at least 2pq = dim(X_{1}), where X_{1} is the symmetric space for U(p, q). The compactness of M implies that f is pluriharmonic, hence by the analogue of Theorem 1.1 for sections, it follows that the image of f is contained in an embedded copy of X_{1}. This implies that the image of ρ is contained in the subgroup of SO^{+}(2p, 2q) which leaves X_{1} invariant, and this group is isomorphic to U(p, q). Thus it suffices to prove that ρ is reductive, and this follows from the following Lemma:

9.2. Lemma. Let ρ : M → SO^{+}(2p, 2q) be a representation such that the characteristic class e_{+} satisfies e_{+}^{2} ≠ 0. Then the Zariski closure of the image of ρ is a reductive subgroup of SO^{+}(2p, 2q).

Proof: Otherwise, the Zariski image of the image of ρ is contained in a proper ℝ-parabolic subgroup of SO^{+}(2p, 2q), and ρ can be deformed into a representation into the (semisimple) Levi factor H of this subgroup. Now the possiblities for H can be read off from the third line of Proposition 6.1. Geometrically this means that the indefinite inner product space ℝ^{2p,2q} splits as an orthogonal direct sum

ℝ^{2p,2q} = ℝ^{a_{0},a_{0}} ⊕ ℝ^{a_{1},a_{1}} ⊕ ⋯ ⊕ ℝ^{a_{k},a_{k}} ⊕ ℝ^{b,b+c},

where a_{0} + a_{1} + ⋯ a_{k} + b = 2p, c = 2q − 2p, H acts trivially on ℝ^{a_{0},a_{0}}, acts by the standard embedding of SL(a_{i}, ℝ) in SO^{+}(a_{i}, a_{i}) on ℝ^{a_{i},a_{i}}, and acts as SO^{+}(b, b + c) on ℝ^{b,b+c}. The bundle E associated to ρ splits accordingly, and maximal positive and negative sub-bundles E_{+}, E_{−} of E can be chosen to also split accordingly:

E_{+} = E_{+}^{0} ⊕ E_{+}^{1} ⊕ ⋯ E_{+}^{k} ⊕ E′_{+},

similarly for E_{−}. Now the Euler class e is multiplicative, so

e_{+} = e(E_{+}) = e(E_{+}^{0})e(E_{+}^{1}) ⋯ e(E_{+}^{k})e(E′_{+}).

For H to be a proper subgroup of SO^{+}(2p, 2q) we must have that for at least one i ∈ {0, ⋯, k} the summand E_{+}^{i} ≠ 0. If the summand E_{+}^{0} ≠ 0, since it corresponds to a trivial sub-bundle, e(E_{+}^{0}) = 0, and consequently e_{+} = 0. If one of the summands E_{+}^{i} ≠ 0, since the bundle E_{+}^{i} ⊕ E_{−}^{i} is flat and E_{+}^{i} ≈ E_{−}^{i} as real vector bundles (because the embedding of SL(a_{i}) in SO^{+}(a_{i}, a_{i}) preserves a symplectic form on ℝ^{a_{i},a_{i}} which dually pairs E_{+}^{i} and E_{−}^{i} ), we see that e(E_{+}^{i})^{2} = 0 and consequently e_{+}^{2} = 0, and the proof is complete.

From the discussion of §8 we note that harmonic maps which arise from variations of Hodge structure are nilpotent, meaning that df(T_{x}^{1,0}M) ⊂ 𝔭^{ℂ} is nilpotent for all x ∈ M. As noted in the introduction there is a partial converse to this. The number ν_{0} = ν_{0}(G/K), is as in §6, Table 6, and 2ν_{0}(X) = r(X), where r(X) is as in the introduction.

9.3. Theorem. Let f : M ⟶ N be a harmonic map. If rank(d_{x}f) > 2ν_{0} for some x ∈ M, then f lifts to a variation of Hodge structure.

Proof: As in the proof of Theorem 7.1, we can view f as defined by a flat connection ∇ and a decomposition D + θ, where D is a K connection and the 𝔭-valued one-form represents df. Let D = D′ + D′′ and θ = θ′ + θ′′ be the decompositions into type. The pluriharmonic equation D′′θ′ = 0 and its consequences D^{′′2} = 0 and [θ′, θ′] = 0 assert that the pair (D′′, θ′) constitute a Higgs bundle. (These notions were introduced by Hitchin [14] and extended by Simpson [21]. Here we shall follow the excellent exposition of [8]). Now there is a moduli space of Higgs bundles and a circle action on it, the fixed points of which are variations of Hodge structure:

D + θ′ + θ′′ ↦ D + e^{it}θ′ + e^{−it}θ′′

The infinitesimal generator of this action is, relative to the natural symplectic form defined on the space of Higgs bundles, the gradient of the energy ([14], second equation of §7). The latter is defined as E(θ) = iη(θ′, θ′′), where

η(ϕ, ψ) = ∫_{M} B(ϕ, ψ) ∧ ω^{m−1}

is the symplectic form. Here B is the Killing form and ω is the Kähler form. Now, by a remark of Corlette, the energy is a plurisubharmonic function on the space of Higgs bundles. The relevant computation is

\frac{∂^{2}E}{∂t∂t̄} iη(θ′ + tϕ′, θ′′ + tϕ′′) = iη(ϕ′, ϕ′′)

where

∇_{t} = ∇ + tζ = (D + tψ) + (θ + tϕ)

is a family of Higgs bundles. Since ϕ is real, ϕ′′ = ϕ̄′, and so the Levi form is positive on the space of connections. It is also basic, and so descends to the moduli space of Higgs bundles.

It remains to show that the component of the moduli space in which the Higgs bundle associated to f lies is compact. For then the energy, which is plurisubharmonic, is constant, and so its gradient, which generates the circle action, is zero, hence the circle action is trivial and f is necessarily a fixed point. But our hypothesis on the rank (at some point x ∈ M, hence on the complement of an analytic subvariety) implies that all deformations θ′_{t} of the given Higgs field are nilpotent, i.e., θ′_{t}(X_{p}) is a nilpotent endomorphism for any holomorphic tangent vector X_{p}. But, according to Lemma 3.17 of [21] (cf. also §4 of [14], the map which associates to a Higgs field its characteristic polynomial is proper, and so the result follows.

Finally we make some remarks on problems that are more global in nature.

The first concerns the algebras of Type II, where the largest-dimensional abelian subalgebra (consisting of nilpotents) does not correspond to a Hermitian symmetric subspace. The space of leading roots is a maximum commutative system C whihc contains a subsystem C′ of cardinality one less which is the (1,0)-tangent space of a Hermitian symmetric subspace. We first observe that for the groups SO(2p, 2q + 1) these algebras can be realized as tangent spaces of pluriharmonic maps. This follows from the example of a variation of Hodge structure constructed in §7, IV, of [4], which, by projection to the symmetric space produces, by Lemma 8.1, a pluriharmonic map. Comparing the description of C^{+} in Table 2 with the correspondence between tangent vectors and root vectors displayed in p. 685 of [4], we see that this pluriharmonic map realizes the abelain space W_{C} corresponding to the maximum commutative system C = C^{+}. The domain of this map is a Siegel domain of the third kind which fibres over the unit disk in ℂ, with fibre the Hermitain symmetric space corresponding to the subsystem C′.

It seems unlikely that these domains admit compact, or even quasiprojective quotients. It also seems possible that these examples are the only ones in the sense that any other factors through one of the“standard” ones on the level of universal covers. If both these statements turn out to be true, they would imply that that these algebras cannot be realized by a pluriharmonic map of a compact complex manifold to a quotient of G/K. It would then follow that the maximum homotopical rank of a map of a compact Kähler manifold to a quotient of G/K would still be the largest dimension of a Hermitian symmetric subspace.

It also seems possible that the maximum-dimensional abelian subalgebra of the remaining groups of Type II (and of the exceptional 9-dimensional commutative system for SO(6, 6), which makes this group, in our terminology, both of Type I and Type II) can also be realized by similar pluriharmonic maps of Siegel domains. If this is the case, then we could also attempt the more global approach suggested above in all these cases.

Finally we would like to point out one interesting problem concerning the group Sp(n, 1). We proved in Theorem (3.5) (c) of [5], and it also follows from our results here (in §7), that first-order rigidity holds in this case. But local rigidity clearly cannot hold, since there are as many local pluriharmonic mappings as there are horizontal mappings into the associated twistor space Sp(n, 1)/Sp(n) × U(1), and the latter are integral manifolds of a holomorphic contact structure, hence form a space of infinite dimension. The question of global rigidity (for maps of compact Kähler manifolds into quotients of the symmetric space) is open and we have no conjectures on its possible resolution.

Bibliography

[1] Beauville, A., Le groupe de monodromie d’hypersurfaces et d’intersections completes. Lect. Notes. Math. 1194, 1–18 (1986).

[2] Borel, A., Compact Clifford-Klein forms of symmetric spaces, Topology 2 (1963) 111-122.

[3] Carlson, J.A., and L. Hernández, Harmonic maps from compact Kähler manifolds to exceptional hyperbolic spaces, Jour. Geom. Analysis 1 (1991), 339 - 357.

[4] Carlson, J., A. Kasparian, and D. Toledo, Variations of Hodge structure of maximal dimension, Duke Math. J. 58 (1989), 669–694.

[5] Carlson, J. and D. Toledo, Harmonic mappings of Kähler manifolds to locally symmetric spaces, Pub. Math. IHES 69 (1989), 173–201.

[6] Corlette, K., Flat G-bundles with canonical metrics, Jour. Diff. Geom. 28 (1988), 361-382.

[7] Corlette, K., Rigid representations of Kahlerian fundamental groups, Jour. Diff. Geom. 33 (1991), 239 - 252.

[8] Corlette, K., Nonabelian Hodge theory, Preprint, Univ. of Chicago, 1990.

[9] Eells, J., and J.H. Sampson, Harmonic mappings of Riemannian manifolds, Amer. J. Math. 86 (1964), 109–160.

[10] Kostant, B., Lie algebra cohomology and the generalized Borel-Weil Theorem, Ann. of Math. 74 (1961) 329-387.

[11] Griffiths, P.A., Periods of rational integrals, I, II, Ann. of Math. 90 (1969), 469–526 and 805–865.

[12] Griffiths, P.A., Periods of integrals of algebraic manifolds, III, Pub. Math. I.H.E.S. 38 (1970), 125–180.

[13] Helgason, S., Differential Geometry, Lie Groups, and Symmetric Spaces, Academic Press, 1978.

[14] Hitchin, N. J., Self-duality equations on a Riemann surface, Proc. London Math. Soc. (3) 55 (1987), 59-126.

[15] Malcev, A.I., Commutative subalgebras of semisimple Lie algebras, Izvestia Ak. Nauk USSR (Russian) 9 (1945), 125–133; English: Amer. Math. Soc. Translations No. 40 (1951).

[16] Margulis, G. A., Discrete groups of motions of manifolds of non-positive curvature (in Russian), Proc. Int. Cong. Math., Vancouver, 1974, vol. 2, 35 -44; AMS Transl. (2), Vol. 109 (1977), 33 - 45.

[17] Matsuki, T., The orbits of affine symmetric spaces under the action of minimal parabolic subgroups, J. Math. Soc. Japan 31 (1979), 331–357.

[18] Mostow, G. D., Strong Rigidity of Locally Symmetric Spaces, Annals of Math. Studies 78, Princeton Univ. Press, 1973.

[19] Sampson, J.H., Applications of harmonic maps to Kähler geometry, Contemp. Math. 49 (1986), 125–133.

[20] Schur, I., Zur Theorie der vertauschbaren Matrizen, Jour. Reine und Andgew. Math. 130 (1905), 66-76.

[21] Simpson, C. T., Higgs bundles and local systems, Preprint, Princeton Univ., 1989.

[22] Simpson, C. T., Some families of local systems over smooth projective varieties, Prépubl. Univ. Toulouse III, 1991.

[23] Siu, Y.T., Complex analyticity of harmonic maps and strong rigidity of compact Kähler manifolds, Ann. Math. 112 (1980), 73–111.

[24] Siu, Y. T., Strong rigidity of compact quotients of exceptional bounded symmetric domains, Duke Math. Jour. 48 (1981), 857 - 871.

[25] Siu, Y. T., Complex analyticity of harmonic maps, vanishing and Lefschetz theorems, Jour. Diff. Geom. 17 (1982), 55 - 138.

Authors’ address:

Department of Mathematics University of Utah Salt Lake City, Utah 84112

Electronic mail:

carlson@math.utah.edu toledo@math.utah.edu