Operator Algebras, Unitary Representations, Enveloping Algebras, and Invariant Theory: Actes du Colloque en L'honneur de Jacques Dixmier 0817634894, 3764334894

Citation preview

Progress in Mathematics

Operator Algebras, Unitary Representations, Enveloping Algebras, and Invariant Theory - Actes du colloque en l’honneur de Jacques Dixmier |

À. Connes M. Duflo

A. Joseph

R. Rentschler Editors

Birkhauser —

"| ROBERT MANNING ff | STROZIER LIBRARY |:

Digitized by the Internet Archive in 2023 with funding from Kahle/Austin Foundation

https://archive.org/details/ison_817634894

Progress in Mathematics Volume 92

Series Editors J. Oesterlé A. Weinstein

Operator Algebras, Unitary Representations, Enveloping Algebras, and Invariant Theory Actes du colloque en l’honneur de Jacques Dixmier

Edited by

Alain Connes Michel Duflo Anthony Joseph Rudolf Rentschler

1990

Birkhauser Boston + Basel ¢ Berlin

Michel Duflo

Alain Connes

Institut des Hautes Etudes Scientifiques

Université Paris 7

91440 Bures-sur-Yvette France

75251 Paris Cedex 05

Anthony Joseph

Rudolf Rentschler

Université Pierre et Marie Curie Paris, France, and

Université Pierre et Marie Curie Laboratoire de Mathématiques Fondementales

Department of Mathematics

75230 Paris Cedex 05

The Weizmann Institute of Science

France

France

Rehovot 76100 Israel

Library of Congress Cataloging-in-Publication Data Operator algebras, unitary representations, enveloping algebras, and invariant theory : actes du colloque en l’honneur de Jacques Dixmier / editors, Alain Connes . . . [et al.]. p. cm. — (Progress in mathematics ; v. 92) “Articles contributed . . . at the colloquium celebrating the sixtyfifth birthday of Professor Jacques Dixmier held during 22-26 May 1989 at the Institute Henri Poincaré in Paris” — Pref. Includes bibliographical references. ISBN 0-8176-3489-4. — ISBN 3-7643-3489-4 1. Lie groups—Congressés. 2. Lie algebras— Congresses. 3. Operator algebras—Congresses. 4. Invariants— Congresses. I. Dixmier, Jacques. II. Connes, Alain. mathematics (Boston, Mass.) ; vol. 92.

QA387.064 512’.55 — dc20

III. Series: Progress in

1990 90-49716

Printed on acid-free paper.

© 1990 Birkhäuser Boston.

All rights reserved. This work may not be translated or copied in whole or in part without the written permission of the publisher (Birkhauser Boston, 675 Massachusetts Avenue, Cambridge, Massachusetts 02139, USA), except for brief excerpts in connection with reviews or scholarly analysis. Use in connection with any form of information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now known or hereafter developed is forbidden. Permission to photocopy for internal or personal use, or the internal or personal use of specific clients, is granted by Birkhauser, Boston for libraries registered with the Copyright Clearance Center (CCC), provided that the base fee of $0.00 per copy, plus $0.20 per page is paid directly to CCC, 21 Congress St., Salem, MA 01970, USA. Special requests should be addressed directly to Birkhauser Boston, 675 Massachusetts Avenue, Cambridge, Massachusetts 02139, USA.

Camera-ready copy provided by the editors. Printed and bound by Edwards Brothers, Incj, Ann Arbor, MI. Printed in the United States of America. |

Paced

FLORIDA STATE

9987-65403 21

*

eee

ISBN 0-8176-3489-4 ISBN 3-7643-3489-4

UNIVERSITY LIBRARIES

SEP

3

1992

@+> Sexe >
0, where p is the function J — P;(M) described at the beginning of the section. p(x) = 7o(hx), x € M, then by conditions (i) and (ii) for p,

Thus if

fo(4) = T0(X(a,00)(h)) = To(P(F(4))) = F(a), proving surjectivity. Using the properties of p once more we obtain: Theorem

3.2.

The map p — fy,

p € M}, determines a bijective map

of M+/ ~ onto the set of decreasing L'-functions f :Ry — J, which are continuous from the right. Furthermore, if y,y € MY then

d([e], [UD = Îvol aa CICA The above theorem is different from the original characterization of Powers for J,-factors and its generalizations to semifinite factors in that the functions f, maps Ry into J and not the other way around. To accomplish the original characterization we use the distribution function f of fy defined by *

f(a) = Î eee

ae.

6

ERLING STORMER

Then f% is a nonnegative decreasing function on J continuous from the right, and the map f, — ff is an L}-isometry. Then y — fj is the known

characterization of Mj / +. We next consider the case when M is a factor of type III,, 0 < A < 1, with separable predual. Then there are a factor P of type IL, with trace tr and an automorphism a of P for which tr oa = A tr, and

M =P

xXq Ze

Let w be the dual weight of tr on M. Then its modular automorphism o” satisfies of, = id, where to = — cos Put

No

=

M

X gw

(R/toZ).

Then No is generated by operators mo(x), x € M, and Ad(t), t € R, defined by

(ro(z)é)(s) = o% (2 )&(s) Le€ L?(R/toZ, H). (Ao(t)E)(s) = &(s — t) Furthermore M, is a factor of type II,,, and each normal semifinite weight won

M has a canonical dual weight D on No.

No also has a canonical

trace 7) scaling the dual action of a by À. If we put

fo(a) =) AG (too (Æ))

a > 0,

then we have a nonnegative decreasing function on Ry which is continuous from the right and satisfies

(i) Afo(Aa) = fo(a),

a>0,

(ii) (1) = fy fe(a) da. Modifying the arguments from the semifinite case we then show fy = fy if and only if » ~ #, and obtain the following characterization of Mj/ =. Theorem 3.3. Let M be a factor of type I), predual, then

0 < À < 1, with separable

(i) the map [y] — f, is a bijection of MX / = onto the set of decreasing functions from Ry into R} which are continuous from the right and satisfy AS(AG) =" Ta)

ea A0.

NORMAL

STATES OF VON NEUMANN

ALGEBRAS

7

(ii) For p,Ÿ € MY we have

d([el, [¥]) = Î Lfo(a) — fy(a)| da. 4. The continuous crossed product

Let M be a von Neumann algebra acting on a Hilbert space H. Let w be a faithful normal semifinite weight on M with modular group o”. Then the crossed product N = M x,.R is defined as follows. Let a and À be the representations of M and R, respectively, on the Hilbert space L?(R, H):

€ € L?(R, H) s ER.

(x(z)€)(s) = o%,(z)é(s), (A()E)(s) = Es — +),

Then À is a unitary representation of R on L?(R, H) such that

A(t)a(x)A(t)* = r(o*(x)), N is the von It should be on w. But as below are up We define properties

ze M,tEeR.

Neumann algebra generated by a(x), x € M, and X(t), tER. remarked at this point that the way N is defined it depends pointed out by Takesaki [15] both N and the concepts defined to isomorphism independent of the particular choice of w. the dual automorphism group (4;)seR of o” on N by the

8,(r(x))= (x),

B,(A(t))=e "tA(t),

EM,

sER

s,tER.

Then (M) is the fixed point algebra of 6 in N. By [7] there is a faithful normal semifinite operator valued weight T from N on (M) given by

ry) = | O5(y)ds, ye Nt, where ds denotes the Lebesgue measure on R. Then for any normal semifinite weight on y on M its dual weight ¢ on N is given by G=poR

"oT.

8

ERLING STORMER

By [15] there is a positive self-adjoint operator affiliated with N such

that \(t) = hit, and the weight 7 defined by

T(y) = @(h-"y) is a faithful normal semifinite trace on N such that

Tob,=6 °T,

SER.

We call 7 the canonical trace on N. i If y is a normal semifinite weight on M we put hy = ae

Then hy is

a positive self-adjoint operator affiliated with N such that @(y) = T(Ayy), ye Nt. Put Ep = X(1,00)(hy),

PE My.

Then it is not hard to show r(e,) = (1). Definition 4.1. If p € Mj we denote by $ € Z(N)+, Z(N) being the center of N, the functional defined by

B(2) = t(epz), 2 € Z(N). Then by the above ¢(1) = y(1). Furthermore from the definition of ¢ it is immediate that hugue = T(u)hym(u)*,

uEeU(M),

hence €uyus = 7(u)eym(u)*, and therefore (upu*)* = ¢. It follows from our next result that ¢ = ÿ whenever y = y. Lemma

for 6,

4.2. The map y — ¢ of MY into Z(N)} preserves order, and

€ MY we have

Il — dll < d([e], (6). Proof. From the arguments in the proof of Lemma 3.1 it follows that if P,YE MX, p N,

ve [I°n-(v® [I] °vr) oo ((Ad(us) oa) @id)ox? k=N+1

k=N+1

= ||p — po Ad(u,) o || < 1/2" — 0 as n — oo. This implies that

jim on o(Ad(un) 001 Qid)os = id.

Q.E.D. Lemma

1.17.

Let M be an AFD factor of type Ill, and a € Cnt(M).

For any À, 0 < À < 1, there exists a € U(M) and a tensor product factorization: M = P,@P2 with the following properties:

(i)

Ad(a)oa =a

@ az relative to PSP):

THE STRUCTURE OF THE AUTOMORPHISM

GROUP

35

(ii) DP, is an AFD factor of type Ill); therefore a, is of the form: Ad(u) oof} with u € U(P,), TER, and y a faithful normal state on P,; (iii) Pp 1

the full Fock space and if h € H let €(h)NE (€,).e7 is an orthonormal basis in H, let A be by the s(e,) = 1/2(€(e,) + £(e,)*), « € I and y is a faithful trace state and (s(e,)),e7 is a

= h@E for € € TH. If the C*-algebra generated let y(a) = (al,1). Then semicircular family. The

von Neumann algebra generated by A is isomorphic to the free group factor

on card I generators and (-1,1) is its normal trace state. Moreover, as an algebra of operators on TH it is in standard form.

We conclude this section with some facts from [5] concerning the connection between Gaussian random matrices and free random variables.

1.12. The natural framework for random matrices is the following. (X, do) is a standard non-atomic measure space with a probability measure do and

L=

MN LP(E) is a +-algebra endowed with the state E : L — C given p2>1 by Ef = fy f(s)do(s). Let further M, be the complex n x n matrices endowed with the normalized trace 7, : M, — € (i.e. T = 1/n Tr). We shall denote by e;; or e(i, j;n) when n needs to be emphasized, the n x n matrix with zero entries except for the (2, j)-entry which is 1. The algebra of random n x n matrices is the algebra M, = M,(L) = My @ L and is equipped with a trace state y, : M, — C given by Yn = T ® E. My can be naturally identified with M, ®1C

in Mh. By A, CM,

Mn, i.e. with the constant matrices

we shall denote the constant diagonal matrices.

1.13. THEOREM ([5]). Let Y(s,n)=

>

a(t,j;n,s)e(t,j;n) € M, be

1.

THE MINIMAL REPRESENTATION OF SO(4,4)

117

5. The Vanishing of Scalar Curvature, Einstein’s Equations and 7 5.1 The vanishing of scalar curvature and a remarkable property of 6manifolds. We return to the conformal structure M on X. We recall (see Proposition 1) that the cone T?(X)4 C ['-?(X) bijectively corresponds to the cone M of metrics on X by the map f — ys. See (16) and (17). But now since p = 3 we have a G-submodule H of T-?(X) and consequently if we put Hy, = HNT-?(X); we get a G-stable subcone M, of metrics on X defined by putting

Mo = {nlf € He}

(137)

One of the key points of this paper is that we can characterize the metrics in M, purely in terms of the machinery of Riemannian geometry. Assume that M is any C® pseudo-Riemannian manifold. Let g be the metric tensor. Now let g' be any C® metric tensor on M which is conformally equivalent to g so that there exists a smooth positive function f on M such that

g = fg

(138)

One refers to f as the conformal factor. If €,7 are vector fields on M let (£,n) be the function on M defined by taking the inner product of € and n with respect to g. Since f is positive we may write f = e?°. Let ¢, be the vector field on M defined so that for any vector field € on M one has £a = (€,¢,). Let A be the Laplace-Beltrami operator on M with respect to g. It is straightforward to see that

Af = (240 — 4(¢,,Co))f

(139)

Now let Scal g be the scalar curvature of M with respect to g and let Scal g’ be defined similarly. Then if dim M = n one readily computes that

Scal g! = e~?7(Scal g + 2(n — 1)Ao — (n—1)(n—2)(C4,¢c))

(140)

See e.g [Go], (3.9.4), p.115. Multiplying both sides of (140) by e4” = f? one has, by (139),

f?Scal g’ = (n—1)Af + f(Scal g—(n—1)(n—6)(C0,67))

(141)

But (141) clearly implies the following property of 6-dimensional manifolds.

Theorem 26. Assume M is a 6-dimensional be pseudo-Riemannian metric tensors on M such a positive function on M — so that g and g' are If Scal g and Scal g’ denote the scalar curvatures

manifold. Let g and g' that g' = fg where f is conformally equivalent. ofM with respect to g

118

BERTRAM KOSTANT

and g' respectively and Scal g = 0 one has Scal g' = 0 if and only if the conformal factor f is harmonic with respect to g. That is, in the notation above, if and only if Af = 0. Remark 27. Note that (141) also implies, conversely, that if dim M > 1 and the statement of Theorem 26 is true for all conformally equivalent metrics then one necessarily has dim M = 6. 5.2 A view of the metrics in the conformal class M. Now with the assumption (45) let u, y€ R® be such that (u,u) = (y,y) = 0 and (u,y) = 1/2. Let Z be the orthocomplement Ru + Ry in R® so that Z has a basis

este

Lilt: 2760 wheres

eg; e7)us Ont

jg and (e; je)

lait 46033 and

(e;,e;) = —1 if i> 3. Now any v € R® may be uniquely written

v=au—by+z

(142)

for a = a(v),b = b(v) ER and z = z(v) € Z. Let U, be the open set in R® defined by putting U, = {v € R®|a(v) 4 0}. Let C*

= U, NC* so that C*

is clearly, using the notation (142) the open subcone of C* defined by

C* = {v €ER®|ab = (z, 2), a $ 0}

(143)

Recalling (7) let X, = a(Cÿ) so that X, is open in X. Let 8 : U, —> C* be the map defined by putting

B(v) = au—(z,z)/a-y+z

(144)

using the notation of (142) and put 6 = ao. Then we note that there exists a unique coordinate system u;, i = 1,...,6, on X, such that if &; = u; 06

then for any v € U, one has, using the notation of (142)

z(v) = a: Ÿ ü(v)e;

(145)

il

Now let Z, = {v € C*|a(v) = 1} so that a|Z, is a diffeomorphism mapping Ze onto Xo. Let a : Xo —

Z, be the inverse of a|Zp.

Recalling the notation of Proposition 2 we can be very explicit about the restriction to X, of the metric tensor y; defined by any f € Rar. 6)Bre

Proposition 28.

Let f ET-?(X);.

defined by putting f, = foa,.

Let fo be the function on X,

Then on X, one has 3

6

i=l

j=4

14 = fo: (D> du? — ST du?)

(146)

THE MINIMAL REPRESENTATION Proof.

0F SO(4,4)

119

Let x € X, and put v = a,(z) € Z,. Then using the notation

of (142),(145) and Proposition 2 we may find &; € T,(X),i=1,...,6, such that, mod Rv,

É(v) = —2p;ti;(v)y + e; € vt

(147)

where p; = 1 if 1 < 3 and p; = —1 for i > 3. But then by (16)

15 (64565) = fo(z)pibi

(148)

On the other hand if we identify R® with the tangent space T,(R®) then clearly ef = —2p;ü;(v)y + e; is tangent to Z, at v and

ACIER i

(149)

But since in (145) a = 1 on Z, one clearly has, by (145), elü; = 6;;. Thus

€; = 0/0u; But this together with (148) implies (146).

(150) QED

5.3 The cone Hy and the G-invariance of the vanishing of scalar curvature. Now let D,, Dy and for i = 1,...,6, let D; be the translation invariant vector fields on R® corresponding respectively to u,y, and to e;.

Then using the notation of (147) and (23.5) we note that 6

Bee

140. Ds

Sa

Now let f € T-?(X).

By continuity it is obvious that if Ax f

=he€

T-#(X) then h = 0 if and only if h|X, = 0. Let f’ be the function on U, defined by putting f’ = fof.

Clearly f’ is an extension of f|C* and

f'(rw) = r~?f'(w) for r € R* and w € U,. It follows from the argument of §1.4 that h|X, is given by Af'|C3. But by the definition of8 one has

Dyf'=0 Thus

(151)

;

OO

Dan 10 0e

(152)

s=1

Now let A, = >; pi(0/du;)? so that —A, is the Laplace-Beltrami operator on X, with respect to the metric is pidu?.

KOSTANT

BERTRAM

120

Lemma 29. Let f ET-2(X) and let f, be the function on X, defined (146). Then Ax f= 0 if and only if A,f, = 0. That is if and only if in as fo ts harmonic with respect to the metric g, = Nes pidu?. Proof.

Let 0; be the vector field on Z, such that a,0; =

0/üui.

But then by (149) and (150) it follows that upon replacing v in (147) by any point v’ in Z, that at v’ one has 0; = D; + cD, for some scalar c, depending on v’. Thus D;f' = 0;f on Z, by (151). But also D, D;f' = 0. Hence D?f' = 6? f. By (152) this implies

AS

6

Zoe ye post

(153)

t=1

But Ax f = 0 if and only if the left side of (153) = 0. But by (153) one bas Aloe QED We can now characterize geometrically the set of metrics M, which are in the G-submodule. The condition is just the vanishing of scalar curvature.

Theorem 30. Let the G-invariant natural 6-dimensional projective metric is uniquely of the the G-submodule defined Ay

=

HSE

M be the set of pseudo-Riemannian conformal class (signature + + + — variety X — so that (see Proposition form y; for f ET-?(X)4. Let H as in (45.5) and let M, = {vs|f €

metrics in ——) on the 1) any such CT~?(X) be H+} where

Then

Me = {y

EMI Scat 7; —0}

(154)

In particular the set of all metrics in the conformal class M which have vanishing scalar curvature ts stable under the conformal group G. Proof. Clearly, in the notation of Lemma 29, the metric g, on X, has vanishing scalar curvature. Indeed the curvature tensor for g, vanishes. But now if f € [~?(X)4 and Scal ys = 0 then by Proposition 28 and Theorem 26 one has A,f, = 0. But this implies Ay f = 0 by Lemma 29. That is f € H. Conversely if f € H then A,f, = 0 by Lemma 29. But then Scal y; = 0 by Proposition 28 and Theorem 26. QED

5.4 The vanishing of scalar curvature on X and Einstein’s equations on compactified Minkowski space. The existence of a special family of metrics on the 6-dimensional manifold X raises the question as to whether there is any application in physics for these metrics. Two related structures which do have strong connections with physical theory come to mind.

THE MINIMAL REPRESENTATION

OF SO(4,4)

121

(1) The group SO(4,4) contains SO(2)x the “conformal group” SO(2,4) as a maximal subgroup and correspondingly (2) X contains compactified Minkowski space as a submanifold. In this section we will present a result from the doctoral thesis, [B], of Witold Biedrzycki which relates Einstein’s field equations in general relativity with the metrics in M,. Assume that M is a 4-dimensional manifold and assume that h is a Lorentzian metric tensor on M (i.e. a pseudo-Riemannian metric of signature + — ——). Let R = Scal h be the scalar curvature on M with respect to h. Then using classical coordinate notation for tensor fields one has a space-time model in the sense of general relativity in case Einstein’s field equations ki; =

R/2 © hi; =

8x1;;

(155)

are satisfied where R;; is the Ricci tensor for the metric h and T;; is the

energy-momentum tensor for certain matter fields. There is of course a vast literature on these equations. One rather delicate point is how Tj; arises in each of the various examples of matter fields (e.g. an electromagnetic field, a charged scalar field, a perfect fluid, ...). See e.g. [H — E], Chapter 3, for a discussion of the variational machinery which gives rise to the energymomentum tensor. A mathematical elaboration of some of the ideas in

[H-E] can be found in [Ko 3]. One of the simplest instances of a matter field which gives rise to an energy-momentum tensor is the case of an electromagnetic field or more simply a closed 2-form w on M. But now if M happened to be a submanifold of our 6-dimensional manifold X and, say, h = g|M where g € M = T-2?(X)4 then besides the metric tensor M also inherits a closed form w from the embedding M —

X

(156)

Indeed if N is the normal 2-plane bundle of M in X then g|N defines the structure of a complex line bundle L on M and that furthermore g defines a connection in L and hence w would arise from the curvature of L. In the case of a perfect fluid scalar functions on M enter into the construction of T;;. On the other hand an embedding M — X gives rise to scalar functions on M in a number of ways. The sectional curvature of N with respect to g is one such way. Since in fact there exists special metrics g on X — namely those in M,(1.e. those in the Hilbert space of our representation x) — the following daring but far-out possibility suggested itself to us; namely, matter fields on space-time M should arise from an embedding M —> X where the metric g on X is an element in M,. In particular one has Scal g = 0. For this to be the case one would have to construct T;; from g so that (155) is satisfied in case h = g|M. In a special

122

BERTRAM

KOSTANT

case having to do with compactified Minkowski space Witold Biedrzycki has constructed such a tensor T;;. Biedrzycki’s tensor T;; is made up of the sectional curvature of N together with components of the second fundamental form associated to

the embedding (156). Let g € M and assume M is a 4-dimensional submanifold of X such

that h = g|M is a Lorentzian metric on M. Let Rin be the curvature tensor on X with respect to g. If v is a tangent vector on X let V, denote covariant differentiation by v with respect (Levi-Civita connection) to g. Thus if 0; are the local vector fields on X given by the coordinate system and V; = Va, then with the usual summation notation

(VeVi — ViVa)d; = Riu : Oi With the usual notation for raising and lowering indices the sectional curvature K(II) on 2-dimensional subspaces II of tangent spaces of X where g|II is definite is such that if g|II,;; is definite on the planes Il;; spanned by 0; and 0;,1# j, then

K (Tlij) = Rijiz /(gii955 — gi). Let K be the scalar function on M defined by putting for any p € M

K(p) = K(Np)

(157)

A field S of tangent space endomorphisms is a tensor field of type Si. The curvature tensor can be regarded as a map S —> R(S) from the space of such tensor fields to the space of covariant tensor fields U of type Uzi (i.e. fields U( , ) of tangent space bilinear forms where U(ô4,@) = Uxi). The map in coordinate notation is given by

R(Sui = Ry,1,5;

(158)

It is easy to see that if S is symmetric with respect to g then R(S) is a field of symmetric bilinear forms. Note also that if S is the field of identity

operators I then R(J) is the Ricci tensor. Next one defines a distinguished cross-section H of the normal bundle N as follows. Let p € M and let x € Np. One defines a symmetric operator C, on T,(M), using inner products, by the relation :

(Cru,v)= (x, Vin)

(159)

THE MINIMAL

REPRESENTATION

OF SO(4,4)

123

where u,v € T,(M) and where 7 is any vector field on X whose value at p is v. Then there exists a unique vector H, € N, such that (onal

tO,

(160)

One then defines a scalar function L on M by putting

L(p) = (Hp, Hp)

(161)

Then Biedrzycki’s tensor T on M as a symmetric bilinear form on 7,(M) at any p € M is given by

1/87 - T(u,v) = R(Pp)(u,v)+ (K(p) — 3/16 - L(p)) - (u,v)

(162)

where u, v € T,(M) and P, : T,(X) — N, is the orthogonal projection. Now recalling (see (8)) the coordinate system r; on R® let

Gi = {9 €Glg-m1

=11, 9-72 = 72}

so that the covering map (39) induces a covering

G1 —

SO(2,4).

(163)

of the “conformal group”. Furthermore with regard to the action of G; on X one notes there is exactly one closed orbit M, that M is 4-dimensional and the covering map (46) induces a double covering

S'x

Si —.M

(164)

MoreoverM may be identified with the usual compactification of Minkowski space and if À = g|M where g € M then h is a Lorentzian metric in the conformal class on M for which SO(2,4) is the proper conformal group. For this choice of M one has

Theorem

31.

[Witold Biedrzycki].

Let g € M

and let h be the

Lorenizian metric on compactified Minkowski space M C X given by the restriction h = g|M. Let T be the tensor field on M defined by (162). Then Einstein’s equations

Ry — R/2- hi = 8xl;;

(165)

are satisfied on M if and only if the scalar curvature Scal g vanishes on M.

In particular one has (165) if g € M.

In fact the Einstein equation (165)

is satified for all G transforms of g if and only if g € Mog. The proof of Theorem 31 will be published by Biedrzycki elsewhere.

124

BERTRAM

KOSTANT

REFERENCES [B] Biedrzycki, W., MIT Doctoral Thesis in preparation. [G] Garfinkle, D., A New Construction of the Joseph Ideal, MIT Doctoral Thesis, 1982. [Go] Goldberg, S., Curvature and Homology, Academic Press, New York and London, 1962.

[J] Joseph, A., The Minimal Orbit in a Simple Lie Algebra and its Associated Mazimal Ideal, Ann. Scient. Ec. Norm. Sup. (4) 9 (1976), 1-30. [H-E] Hawking, S. W., and Ellis, G. F. R., The Large Scale Structure of Space-Time, Cambridge Monographs on Math. Phys., Cambridge Univ. Press, 1973.

[Ka] Kazhdan, D., The Minimal Representation of D4, Submitted to the Proceedings of the Colloque en l’Honneur de Jacques Dixmier, mai

1989. [Ko 1] Kostant, B. Lie Group Representations on Polynomial Rings, Am. J. Math., 85 (1963), 327-404. [Ko 2] Kostant, B. Eigenvalues of a Laplacian and Commutative Lie Subalgebras, Topology, 13 (1965), 147-159. [Ko 3] Kostant, B., À Course in the Mathematics of General Relativity, Ark Publications Inc., Newton, 1988.

[Ko] Kostant, B., The Principle of Triality and a Distinguished Representation of SO(4,4), Differential Geometrical Methods in Theoretical Physics, edited by K. Bleuler and M. Werner, Series C: Math.

and

Phy. Sci. 250 Kluwer Acad. Pub. (1988), 65-109. Received January 25, 1990

Department of Mathematics MIT

Cambridge, MA 02139

The Minimal Representation of D,

DAVID KAZHDAN! Dedicated to Jacques Dizmier on his 65th birthday Introduction

In this paper we study the minimal representation of quasi-split groups of type D4 over a local field, F. We give an explicit realization of such a representation and show that it can be used to prove an explicit realization of the lifting from multiplicative characters of a cubic extension E of F to irreducible representations of GL3(F). I want to express my gratitude to Joseph Bernstein for a number of very useful and clarifying discussions and to the referee of the paper who did a marvelous job. 1.

Let G be an adjoint semisimple quasisplit group over a local field F of

characteristic zero, B = T U a Borel subgroup of G, X(T) a Hom(T,, G,,) the lattice of weights, ® C X(Z) the subset of roots and + the positive roots and © C ®+ the subset of simple roots. We will write G,B,... , instead of G(F),B(F),.... For simplicity we assume that ® is a reduced root system — that is, for any a € ©, 2a ¢ ®. Then for any a € Ot there exists a finite extension Fy of F and an algebraic group homomorphism

ja : SLo(Fx) — G such that jo a à CT Ve € F* and jy defines an isomorphism of the subgroup ae 1)} of SL2(F.) with the subgroup U, of U where

U, = {u € U|ln(tut—*) = a(t)In u

for all t ET).

The homomorphism 7, is uniquely defined up to a conjugation by an element t € T. We fix 7, for all a € © and for any a € F, a € & define

(a)

defocinfil: = jn

)@ def sufiloy0 death 0/00 4 1 Era) ike à! and sa(a) = 7

al a

As is well known (see [T1]), {sa} a € Uo satisfy the braid relations: for any a,B EX,

(*a,8)

a F B, we have Sasg

:..

Nap

1Partially supported by an NSF Grant.

=

SBSa...

Na,p

DAVID KAZHDAN

126 where

;

Nag =2

if (a,8) =0

= 3) if(a;p)=—1 Apt

(a, B) =

—2

OIL

(a,

—3.

)=

For any a € 5, we denote by Py = La: U, the minimal parabolic sub-

group corresponding to a and define Ba TU,

Cc B. It is clear that sq

normalizes B, for all a € X.

Let rx : G — Aut W be an irreducible unitary representation.

We say

that 7 is small if for any a € E the restriction 7, of x on Bg is irreducible.

LEMMA 1. Let 71,72 be small representations ofG such that the restrictions of 7; and m2 on B are equivalent. Then 7; and 72 are equivalent. PRooF: Since the restriction of 11,72 on B are equivalent we may assume that there exists a unitary representation tr : B — Aut W and small representations 7,72 : G — Aut W such that for any b C B,

m1(b) = m2(b) = 7(b). We want to show that 7; = m2. consider operator A;

Ti(sa)

Ba, weshaveed:t(b)Ay

=

€ Aut W,i

=

1,2.

Fix a € X and

Since s, normalizes

1(s,08,) tor all b € By, 1 =

1,2)

,omee

representations 71,72 are small, the restriction of 7 on Bg is irreducible and we conclude that A; = cA2 for c € C*. Since s2 € T C By we

have A? = 43 = 1(s2).

c = +1. To prove that c = 1 we 3 use the equality te i)€ 1)) = Id. Therefore (A17(E.(1)))? = (A2T(Ea(1))}* = Id.

Therefore

Therefore

c = 1. So we see that for any a € ¥,

T1(Sa) = %2(Sq). Since B and {sq}, a € © generate G we see that 7, = 72. Lemma 1 is proved.

We can reformulate Lemma 1 in the following way: given an irreducible unitary representation of B such that the restriction of rT on Bg is irreducible for all a € X there exists at most one way to extend 7 to a representation of G on W. It is natural then to ask: given an irreducible unitary representation T of B when is it possible to extend it to a representation of G. For any representation 7 of B we denote by 7,,7, the representations

of Ba given by 7a(b) = 7(b), Ta(b) = T(Sabs5') for b € By. LEMMA 2. Let 7: B-— Aut W bea unitary representation such that a) the representation Ta, Ta : Ba — Aut W are irreducible and equivalent. b) For any a € Y there exists A, € Aut W such that Agta (bd)ASS

Ta(b),

b€ Ba, A2 = T(W2), and for any a € F*, =

Aar(Ba(a))40=rUie{(~G

9, )Bal }Aer( a) Bao)

==Ch

MINIMAL REPRESENTATION

OF D4

127

c) For any a, 38 € X,a # B we have

eae!

(a,8)

me fau.

Na,B

Na,B

Then there exists a representation x : G — Aut W such that x(b) = r(b) forbE BCG. PROOF:

For any field E let

H C SL2(E) the subgroup of diagonal ma-

trices, N be the normalizer of H in SL2(E) and Q > H be the subgroup of upper-triangular matrices. Let S be the free product of Q and N with

H as an amalgamated subgroup. We have a natural map $ — SL2(E). As is well known (see [JL], p. 7), the kernel of this map is generated as a normal subgroup by elements (suas) !hau—asu_g-1, a € F*, where

Lda

is

ia

|

5) € Hand

cl

us =

fon“Oa:

D

(4 at

fair" 0

»

ea. There-

fore there exists a representation og : SLl2(Fa) — Aut W such that ; (iy 2 t 1 clear that we can is It A,. = ) che ), ia = T(Ea(t) a( extend og to a representation og : Py — Aut W. Now Lemma 2 follows

from Proposition 13.3 in [T2], which says that G is a quotient of the free product G of groups P,, amalgamated by B by a normal subgroup in G generated by relations (*),g8, a, BEX, a #8.

How to construct representations 7 of B satisfying the conditions of Lemma 2? We do not have a general answer but we can propose a “quasiclassical” picture. Let Lg, £Lp,... be Lie algebras of G, B,... and L6, Lp, ... be the dual spaces. The imbeddings By < G induce projections pa : LG — LR...

Given a coadjoint G-orbit

2 C L& we say that Q is small if there exists

a Bg-orbit Qa C L'. such that Pa(Q) lies in the closure of Qg and the

restriction of p, on

2N pz}(Qq) is an isomorphism QN pz'(Qa) —> Qa.

Let 4,qa be the projections q : LG — Ly, qa: Lip, — Ly. Let GED

be any U-orbit in the image qa(Qa) C Ly. It is clear q(Q) > Ad(T)Q and Ad(T)Q is open and dense in g(Q). Let T= C T be the normalizer of Q in T and pa : U — Aut R be the unitary irreducible representation of U corresponding to 2 by Kirillov’s theory. As is well known (see [H]), we can extend pz to a representation Pa : Tq: U — Aut R which is unique up to a multiplication by a character of T>. Let T be the unitary induced representation of B, 7 = IndZ_y Pa

CONJECTURE: All representations 7 of B satisfying the conditions Lemma 2 can be constructed in this way.

of

128

DAVID KAZHDAN

The case when G is a split simply laced group and Q is the smallest nonzero coadjoint orbit is analyzed in a joint work with G. Savin. In this paper we consider the case when G = G® is a quasisplit group of type D4 corresponding to an arbitrary cubic extension E of F’. Let F be a field of characteristic zero. From now on we restrict our attention to the case when G is a quasisplit adjoint simple F-group of type D4. Since the group of outer automorphisms of split adjoint group G° of type DA is isomorphic to the symmetric group S3 the adjoint quasisplit groups G of type D4 are parametrized by commutative semisimple 3-dimensional F-algebras E. To describe this correspondence we fix a Borel subgroup B° Cc G° a maximal torus T° C B° and imbedding S3 — Aut G° such that S3 preserves B°,T° and Aut G° = S3 x G° Let F be the algebraic closure of F. For any 3-dimensional commutative semisimple algebra E over F there

exists an algebraic isomorphism E @ F —> F°. Since Aut(F°) & S3 we see that E defines a homomorphism Tg : G — S3 uniquely up to a conjugation

where G © Gal(F : F). Since we have fixed an imbedding S3 — Aut G° we can consider Tg as a homomorphism

Tg : G —

Aut G°. We consider

Tg as an element of H!(F/F, Aut G°) and denote by G” the form of G°

coresponding to Tg (see [S], IIL.5). It is easy to see that the map E — G® is well defined and provides a parametrization of adjoint quasisplit F-groups of type D4. We denote by BÊTE the Borel subgroup and a maximal torus of G” such that B® (hes

B°(F) and TË(F) = T°(F). To describe the group GE of F-points of GË we consider the action of

G on G°(F) given by yog der TE(y)(g*) where g — g” is defined by the action of G on F. It is easy to see that GE = {g € G°(F)|y og = g for all y €G}. The Dynkin diagram of (en has the form

1

: FA We denote by a;, 1 < i < 3, Bo the corresponding simple roots aj,fo : def 4

T > Gn and define

a = a; + a + a3. We consider Bo, à as characters of

B” trivial on the maximal uniponent radical U° of B®. Since the characters Bo,@ are G-invariant they define characters Bo,a : BE —

G,,- It is clear

that Bo, Yo def Bo +a and wi! Yo + B are roots of (GE, BE TE). ~0 Let fad eo Cc G

be the parabolic subgroups containing B° and cor-

responding to subsets {So} and {a1,@2,a3} of the set of simple roots of (G°, BTS) correspondingly.

Since those subsets are $3-invariant P°,P°

MINIMAL REPRESENTATION

OF D,

129

then define parabolic SMPEONPARE P”, PÉoG: Let PE = L* x H, pe ==L x H be the Levi decompositions such that

Te Len inaLOC. 7 Cc fis be the commutator subgroups. It is easy to see that there exist isomorphisms j : SL2(E) —> L°, j : SL [° which map diagonal subgroups inside T” and upper triangular subgroups

C C SLE), ES

C C SL» inside BE.

We define s Sale

-

à c PE. As is well known (ss)°= (s5)°.

We will need the following description of the group GE. Let GE be the free product of PE and PE with the amalgamated subgroup BË. We have a natural surjection €: GE =, GE.

LEMMA 3. ker € is generated as a normal subgroup by the element(s5)%($s)-3. PROOF:

Follows from [T2], 13.1 and the standard description of the Weyl

group of GE in terms of generators and relations ([T2], 2.14). For any (7,j), 1 < i#j < 3 we denote by E;, the 3 x 3 matrix with 1 in the place (i,j) and 0 in all other places and for any a € F define eij(a) = Id+ aË;; E SLa(F).

LEMMA

4.

a) There exists a group imbedding i: SL, — G such that

i(e12(a)) = Eg(a), t(€23(2)) = E,,(a), i(e1s(a)) = E(a) i(en(a)) = E_-g(a), i(es2(a)) = E_, (a), i(ex1(a)) = E_, (a) where E1p,,E2+,,E44

are root subgroups in (Chas

b) The centralizer of i(SL;) in G” is isomorphic to the kernel E’ of the norm map N : E* —> F", where E*Rer(Gm).

ProoF:

a) As follows from the Corollary 1 to Theorem 7 in [St] there

exists an algebraic homomorphism 7: SL; — Ge satisfying the condition of Lemma 2. For any root v of G® we denote by h, the corresponding coroot hy Gt CE GË. The center of SL, is isomorphic to the group p3 of

cubic roots of unity and for an € € 3 C SL we have i(e) = hg(e)hy,(€7"). To show that i: SL; — G® is an imbedding it is sufficient to check that

Ad(i(e)) # Id for € € 3 — {1}. But for any 7, 1 < i < 3, a € F we have i(e)Ea(a)i(e)7! = Ea(e?a). Thereforei:SL, > G® is an imbedding. b) For any root v of G® we can find p € (+80, +70) such that v + p is also a root. Therefore the centralizer Z of i(SL3) in GE lies in the Cartan

subgroup H of GE. Moreover Z = {h € H|fo(h)= yo(h) = e}. Leth: Gn — H be the coroots over F corresponding to simple roots a;, 7 = 1,2,3,

130 ka : E* — clear that h, defines ka defines is proved. We will SLANE =

DAVID KAZHDAN H be the map given by ha(a1, a2, 43) hy(a1)h2(a2)h3 (a3). It is hy commutes with the action of the Galois group G. Therefore, a morphism of algebraic f-groups he RE — H. It is clear that an isomorphism of the algebraic group hg : E’—Z. Lemma 4

consider SL, and E’ as subgroups of GË. It is easy to see that p3 — the group of cubic roots of unity, and à extends to a group

monomorphism

: (SL; x E)/p3 —

GË, where we imbed y3 in SL3 x E,

€ — (e Id,e7'). Let YC GL, x E* be the subgroup of elements (g,e) such that detg N(e) = 1. We can imbed G,, into ¥ by Mold Aad): It is clear that the quotient Y of ÿ by the image of ¢ is isomorphic to (SL; x E’)/psTherefore we can consider 7 as an imbedding of Y into GE weet IViy be

the groups of F-points of Y and Ÿ, and GE C GLa(F) be the image of Y under the natural projection Y + GL3(F). Assusme that FE,F are a local fields, and x : Y — AutW

is a smooth

representation (see [BZ]). For any character x : E* — C* of E* denote by

W, € W the subspace of vectors w € W such that 7(1 x e)w = x(e)w for all e € E’ and define the representation 7, : GE — AutW, by the formula (9) = x~*(e)m(g,e)W, for any e € E* such that detg- N(e) = 1. It is clear that the operator T,(g) does not depend on a choice of e € E* and the map g — T,(g) is a representation of GZ. We denote by TX the induced

representation of GL3(F). We can give an alternative construction of the representation 7, which is applicable in the case when E is any semisimple 3-dimensional F-algebra.

We consider 7 as a representation of Ÿ and denote by 7 : GL3(F) x E* > AutW the induced representation (7, W) = end PIXE (y, W). For any —_ character y of E* we denote by W, the quotient of W by the subspace spend by vectors of the form 7(1 x e)w—y(e)w, w € W,e € E*. We denote by 7, the natural action of the group on Wy. Now we can formula the main results of the paper. Let QC LEE be the minimal nonzero coadjoint orbit of GE. It is easy tosee that the centralizer

of any element of Q in GE is unipotent and therefore 2LQ(F)) is a GEorbit. Then Q defines a unitary, irreducible representation 7 of B uniquely up to a multiplication by a character. THEOREM A. There exists a unique unitary irreducible representation 79 : GE —, AutW2 of GE such that the restriction of 72 on B is isomorphic to a representation T corresponding to (1. Moreover we give an explicit construction of (7, W.). Let WC W the subspace of smooth vectors and 7 be the restriction of 7. on W.

be

MINIMAL REPRESENTATION

THEOREM

OF D,

131

B. For any character y of E* the representation 7, of GL3(F)

on Wy, is nonempty and irreducible and it corresponds to x by the local

lifting (see [J-PS-S]). We can interpret Theorem B by saying that x : GE — AutW provides an explicit realization of the lifting : characters E* — representations of GL3(F) in the same way in which the metaplectic representation of Sp(4, F) provides the lifting : characters K* => representations of GL2(F) for any quadratic extension K of F. We end this introduction with the description of the place of x in the Langlands picture. The Levi component fit can be described as the quotient of the group GL, x E” by the subgroup G,, which is imbedded in it by À — (AId, A~?).

By the Hilbert 90 theorem we see that LE = GL2(E) x E*/F*. The group $3 has unique irreducible representation y : S3 — GL2(C). Combining it with the morphism pg : G — S3 we obtain a 2-dimensional

representation jig of the Galois group G. By ([JL]) it corresponds to an irreducible unitary representation p?, of GL2(F) such that 1%(AId) = eg(A)Id where eg is the quadratic character of F* corresponding to the character det up : G — (+1) C C*. We denote by pg the representation of the group GL2(F) x E* given by

PE(g, e)='p2(g)en(N(e)), where N : E* — F* is the norm map. It is clear that for all À € F*, pe(AId, A~!) = Id and therefore we can consider pg as

a representation of LE.

Let 0: LE — F* be the character given by 0(g, e) "(det g)°>N(e)? and IE be the representation of GË unitary induced from PE by the representation pe ® |6|. As follows from the Langlands theory II” has unique irreducible : —E quotient which we denote by II . THEOREM ;

C. The representation

of GE is the equivalent to the repre-

—E

sentation of GE on Il . If E is not a field we can give a more elementary description of the group G¥ and its representation 7. In this case E = F @ K where K is a semisimple 2-dimensional F algebra. The norm Nx : K — F can be considered as a quadratic form on the 2-dimensional F-space K. Let yx be the direct sum of Nx and a six-dimensional split form and let SOx be the proper orthogonal group of yx. It is easy to see that for E = F @ K, GE is isomorphic to the quotient of SOx by the center.

We have an imbedding SOx x SL2(F) — Sp(16,F) and the representation 7 cooincides with the action of SOx on the space SL2(F')-invariant functionals on the space of the metaplectic representation of Sp(16, F). In the case when F = R the representation x was discussed in [Ko] in detail.

DAVID KAZHDAN

132

2. In this section we give an explicit description of the parabolic subgroup PE C GE. We denote by N,T the norm and trace maps E — F. We denote

by Bg the bilinear form (e1,e2) — T(e1e2) on E and by dbg/r € PAR the discriminant of the bilinear form Bg.

For any a € F* we denote by

@a the homomorphism a, : G — {+1} given by aa(y)Va = (Va) for all 7 €G. We will interpret a, as an element of H'(G, {+1}). It is clear that @a depends only o the image of a in F*/F*?. For any a,b € F*/F*? we define (a, b) aa a

€ H?(G, {+1}).

The algebra EES Fr F is isomorphic to the direct sum F° of three copies of F. The permutation group S3 acts naturally as the group of 4 —3 = : À . =. R. ; automorphisms of F over F and an isomorphism 2: E = F is unique up to a composition

with an element

in AutzF

we have a homomorphism ee : G —

S3 which

conjugation. Consider the map LOF (a20@3, @13, 01a) and define 0:E—

oF

=

S3.

In particular,

is well defined

up to a

given by 63) (a, a2, a3) =

E by 6 = 17100) oi. It is clear that

8 is a quadratic map from E into itself (that is, a linear map from symZE

into E) which does not depend on a choice of an isomorphism i: E + F° and commutes with the action of G. Therefore the restriction of 8 on E

defines a quadratic map 0: E — E. It is clear that 8(e) = e~!N(e) for all invertible e € EF.

Let A

F ® E, for any t € E we denote by Q; the quadratics form on

A given by Q:(x0,x) af N(t)x2 + zoT(0(t)z) + T(t0(z)). Let d, € F*/F*?, S € H?(G, {+1}) be the discriminant and the Hasse invariant of Q; (see

[0], §58). LEMMA 1. (a) For all xo € F,x,t € E we have x0Q:(x0,x) = N(x+2ot)— N(z). a For all t € E* the form Q; is nondegenerate and d; = di. (c) Ss = Si(SE/F, N(t)) fort € E*.

Proor:

Let À = Ar @p F and ay : A — F be quadratic forms obtained

from Q; by the extension of scalars.

Using an isomorphism i:

£ + F

we can consider Q, as a quadratic form on F°. It is clear that for AG = (t1,t2,t3) we have oQ,(ro, 21, 22, %3) = (21 + 2ot1)(22+2ot2)(x3 + Lotz) — 212223 for all (x1, 22,23) € Fr. to € F. The part (a) is proved. To prove (b) we compute the discriminant Ag of Q; in terms of the standard basis

in F. We find A; = N(t)? and therefore Q; is not degenerate if N(t) 4 0. Since the identification id @ à : A ~ F° does not depend on t we see that

d; = di for all { € E*. Part (b) is proved.

We first prove (c) in the case when t € F* C E*. Let E° = (e € E,Te =

MINIMAL REPRESENTATION

0). We have E =

F @ E° 2 A=

F@F@

OF D4

133

E® and Q:(z0, yo, €0) =

(Pi(zo, vo) +tP(e°), to,yoC F,e° € E° where P,(x0, yo) = t323 + t?royo+ ty§ and

P : E° — F is the quadratic form P(e°)= T8(e°).

It is clear

that the quadratic form P, is equivalent to the form tP where P(z0, Yo)= 28 +Toÿo +y8. So Q; is equivalent to the form t(P@P). Since dim A = 4 it

follows from the definition of S; that S; = Si(t, d1) where di = d(P)-d(P) = 3d(P). To compute d(P) we observe that we have T6(e°) = —$T((e°))?

for all e® € E°. Really let (a1,a2,a3) = i(e°) € F°. Then i(8(e°)) = (203, 0143, 0102) and TO(e°) = aj a2 + a103 + a2a3 = —4(a? + a3 + a)

+5(a1 + a2 + as)? = —4T(e°)? since Te® = 0. Now it is clear that d; = 6g/r and (c) is proved in the case when t €

Pe CE, To prove c) for the general t € E* we will use the following result.

CLAIM.

If t= uv?t where u,v,t €E*, N(u) = 1 then S; = oF

PROOF OF THE CLAIM: The map (20,2) — (zg, 2’) given by zg = N(v)z0, x! = O(v)v-lu-x defines an equivalence of quadratic forms Q; and Q-The Claim is proved.

uv?t where u =

Since any t € E* can be written in the form ¢ Mier

+

tee

and

t =

N(t)

E

PE

we

have

Sn.

St

Lemma

1

is proved. Let A’ Hom(A, F) be the dual space to A, V LT À @ A’. We consider V as a symplectic vector space over F where the symplectic form [ , ] : V x V — F is defined by

[(A1, 4), (Az, 9) = XQ(A1) —M(A2),AnAz€ A, M,M € AY. We will write elements v € V as v = (xo,r,x',rt),

EX Homp(E,F),

ro

€ F, rEE,z'E€

2, € Homp(F,F) = F. We denote by 8 : À — A’

the linear map such that (zo, z)(yo, y) = —royo — T(xy) for all ro, yo € F,z,y € E. For any te E we denote by a; : À — A the map a;(z0,2) = (to,x + tot), by a, : A’ — A’ the dual map and by & : V — V the

map given by £,(A, Y Fe = (a@(À),(a,)71(X' + Q()) for (A, A’) € V, where Q::À — A’ the symmetric morphism such that (Q1(A))(A) = 2Q,(A) for all À € A. For any e € E* we define re : V — V by re(xo,t,y, yo) = (N(e)z0, O(e)z, ey, Yo), To,Yo € F,x,u € E. Let GSp(V) be the algebraic

f-group of linear transformation g : V — V such that there exists w(g) € F* such that [v1g,v2g] = w(g)[v1, v2] for all v1,v2 € V. Let E",GL,(E) as before be algebraic F-groups obtained from G,, and GL, by the on of scalars from E to F, N be the norm mapN : E" —> G,, and Ee ker IN:

134

DAVID KAZHDAN

LEMMA

2. There exists a group homomorphism r : GL,(E) — GSp(V)

such that et

(0

i) =%

and

te,

e2p0

AG

2.

i)=reeE

*

(4 0)= (5)

PRooF: To prove the lemma we may assume that F = F and fix an isomorphism E = F°. Let 01, 02,03 : GL,(E) — GL, be the natural projections.

We consider o; as two-dimensional representations of GL,(E) and define o : GL,(E) — Aut V to be the tensor product r = 0; @o2®a3. We choose bases (e;, f;), 1 < i < 3 in the spaces of representations o; and define an isomorphism À : V — V by A((ro, 2), (y, yo)’) = toe1 ® e2 © e3 + ti fi ©

€2 @e3t+ r2€1 © fo Des + Lz€1 © €2 © fat yiei © fo © fat yofi © e2 © fat

ysfi © fo ® e3 + yofi ® fo ® fz where for all zo, yo € F,x = (21, 22,23), y = (y1, y2,y3) € E where we consider (y, yo)’ as an element in A’ given by

3

(y, yo)’ (ro, x) = D

At-o( 4

01 |

, it —zxoyo. It is clear that 2, = A7! e( 1 Jarre=

)4and (ee

als

: A x)

0

for allt € E,e€

E*. So we can define the group homomorphism r : GL,(E) — GSp(V) by r= A7 lo : À. Lemma 2 is proved.

Let V

EV@rF..

We denote by QC V the subset of nonzero decomposable vectors. Using the isomorphism À : V — V we can consider { as a subvariety of V — {0}. It is clear that (2 is G-invariant. Therefore there exists an F-subvariety A C V — {0} such that Q = Q(F). Let q : A — A be the restriction of the projection V — A on (1.

LEMMA 3. a) dimQ=4

b) Q is an L-orbit c) Q'is an LE-0rbit d) For any ve V —-Q dim Lv > 4 e) ACV-{0}V=F@E@EEF Q = {(z0,2, y, Yo) ENT

can be defined by the equations

{O}|zy = Toyo, O(x) = Zoy, O(y) = Yor

f) q¢:0— A is birational and for any (zo,z)

€

A= F @ E,x0 #0,

q7+(xo,z) = (x0, 2,2 (x), 25 °N(a)). PROOF: a), b), d), e), f) are obvious. c) follows easily from b) and e).

MINIMAL REPRESENTATION

OF D4

135

COROLLARY. Assume that E is a field. a) If (xo, 2,y,yo) € Q is such that y = 0, then x = 0.

b) There is no point (ro, 2,y, PROOF:

yo) € Q such that ro = yo = O.

Clear.

Let M C GL,(E) be the image of the imbedding E’ — GL,(E) given by À — Ald and LE be the quotient group LE “ GL,(E)/M. It is clear that r defines an imbedding L” — GSp(V). The morphismw : GSP(V) — G,, defines the character w : L” — G,,. The intersection M2 un SL,(E). M

is a finite FSSRRETOND in the center of SL,(E).

We denote by LÉ be

the quotient group LEE * SL,(E)/M°. Let p : GL,(E) — GL, given by À —

LF be the natural projection. The imbedding Cm — &

therefore map E*— LE.

a defines an imbedding E* — GL,(E) and It is clear that « is an imbedding and it induces

the imbedding E’ = LF. _

Consider the morphism À : G,, — GS, (V) given by

h(a)(x0, 2, y, Yo) = (a?x0, az, y, a” yo). It is easy to see that AiG.) C LE andwoh= Id. As before, we denote by H the group H(F) of F points of an algebraic F-group H. For example, Gn = F*. LEMMA

4. a) The group L® is generated by its normal subgroup LE and

the image R(F*) of h. b) The map w : LE — F* is surjective. c) The image p(GL2(E)) in LE consists of £€ L® such that w(£) C N(E*). d) LE is generated by its normal subgroup p(SL2(E)) and the image of E’ ae PROOF:

a) and b) follow immediately from the equalities w oh = Id and

Ly kere). c) Let G = Gal(F :F).

It is clear that p(GL2(E)) is a normal sub-

group in LE and we have an imbedding of the quotient LE /p(GL2(E)) into H'(G,M(F)) = H'(G,E'(F)). From the exact sequence 0 —+ E’ — EE — G,, we have an for any g € a morphism

— 0 we conclude that H'(G, E'(F)) = F*/N(E*). Therefore, imbedding L” /p(GL2(E)) — F*/N(E*). On the other hand, GL2(E) we have w(p(g)) = N(detg) € N(E*) and w defines & : LE/p(GL2(E) — F*/N(E*). As follows from b) & is

surjective. Therefore it is an isomorphism. c) is proved.

136

DAVID KAZHDAN

d) As follows from c) for any £ € LY we can find g € GL2(E) such that -1 gE 1) 6 and E’ € À that £=p(g). Let À = det g € E”. It is clear Omer SL2(E). Then g = k(A)p(s). Lemma 4is proved. We can identify LË with the kernel of the restriction of w on LP and LE is the semidirect product of LÉ and A (& G,,), where A is the image of h. Let H be the Heisenberg group associated to V. That is, H is isomorphic to V @G, as an algebraic variety and the maultiplication on H is given by (v1;a1) - (v2;a2) = (v1 + V2; ai + a2 + $[v1, v2). It is clear

that the map a —> z def (0;a) defines an isomorphism of G, with the center Z of H. The maps À — ((À,0);0), A’ — ((0,A’);0) identify A, A’ with commutative subgroups of H and the product defines an isomorphism Ax A’ x Z — H of algebraic varieties. The group GSp(V) acts naturally on H.

In particular, we have the action of LE on H, h —

h£, where for

h = (v;a), ht€ (vt;w(£)a). Let P® c G® be the parabolic subgroup as in §2. PROPOSITION

1. PE is isomorphic to the semidirect productLE x H.

Let P = PPP), LL (D) = acts naturally on P, L and H. isomorphism Lx H +

HP),

= VF).

The’ group /¢

It is sufficient to construct an algebraic

P compatible with the action of G.

Let Bo,@1,a2,a3 be the simple roots on G°, y oe Bota, +ao+ a3. Then y is a root and (Bo, a;, B;,Yi,Yo,) is the set of positive roots where Bi = Bo + ai, Yi = Yo -—%, w = yo + B, 1 < i < 3. For any root of G° we denote by the same letter the root subspace in the Lie algebra of G°. The Lie algebra of the nilpotent radical of P is spanned by B,Ya,w,0 < a < 3, and is isomorphic to the Heisenberg Lie algebra. k = V @ F -w where V is the 8-dimensional symplectic F-space spanned by By,7a,0 < a < 3 and the symplectic form ( , ) and V is given by [v1, v2] = (vi, v2)w. Let A,A’ C V be subspaces spanned by 8, and Ya, respectively, 0 < a < 3. It is easy to see that A, A’ are maximal isotropic G-invariant subspaces of V. The Levi subgroup of P is generated by the Cartan subgroup of G and one-parameter subgroups corresponding to roots ta;, 1 < i < 3. So this Levi subgroupis isomorphic to 78, It is easy to see that P is isomorphic to the semidirect product of L and H and this isomorphism is compatible with the action of G. We leave to the reader to check the action of the Levi component of P on its unipotent radical coincides with the action of L on H constructed earlier. Proposition 1 is proved.

MINIMAL REPRESENTATION OF D4 3.

137

In this section we consider the case when F is a local field of character-

istic zero. Then H?(G,{+1}) = {+1} if F 4 C and H?(G, {+1}) = {1} if F = C. Therefore, for local fields we can consider symbols {a, 3},a,@ € F*

as elements on {+1} C C*. We fix a non-trivial additive character Ÿ : F —

C*. Let P° be the semidirect product P° © SL2(E) x H. LEMMA 1. (a) There exists unique (up to an equivalence) irreducible unitary representation 02,4 : H — Aut W? such that o24(za) = ¥(a)Id for alla eé F. (b) There exists unique extension of 72,4 to a representation o2,y : P° > Aut We. PROOF: simpler.

We consider only the case when E is a field. Other cases are even

(a) is well known (see [W]). As follows from [W] there exists a double covering Sp +

Sp(V) and

~

a representation © : Sp — Aut W? such that for any ÿ € Sp, h € H we

have &(9-!)o2,y(h)F(G) = o2,y(h")). The restriction of r to SL2(E) — Sp(V) defines a double covering on SL2(£) or, in other words, an element

a € H?(SL2(E),Z/2Z).

Since [E : F] = 3(= 1(mod 2)) it follows from

[M], Th. 14) that a = f*B where f* is the transfer homomorphism and B € H?(SL2(F),Z/2Z) is the restriction of a on SL2(F). In other words,

B corresponds to the restriction of r on the image of SL2(F). It is easy to see that 8 = 0. Therefore a = 0 and there exists a group homomorphism 7: SL2(E) — Sp such that i = roi. Now we can extend o2,y to SL2(E) by o2y = © oi. This proves b). To give an explicit realization of o24 we will use the notion of Fourier transform. let A be a finite-dimensional F-space, A’ - the dual space dA a

Haar measure on A. Let W°(A) be the space of Schwartz-Bruhat function? on A, W'(A) the dual space of distributions and W}(A) = L?(A,d)) (see [W]). The Haar measure dd defines the imbeddings W°(A) C W3(A) C W'(A). Let F : W°(A) — W°(A’) be the Fourier transform given by F(f)(’) æ ‘|W(A‘(A)) f(A) dA. As is well known, F extends to continuous A

isomorphisms F : W2(A) => W2(A’) and F : W'(A) —> W'(A') and we can choose a Haar measure dA in such a way that F is unitary. For any nondegenerate quadratic for Q on A there exists a symmetric

invertible linear map Q : A — A’ such that Q(A) = 34(A)(). We denote

by Q-! the quadratic form on A’ given by Q~1(1') = $\’(Q71()’)). One may consider the locally constant function 7(Q) on A as a distribution and define its Fourier transform F(#(Q) € W'(A'). As is well known, 1]n the case when F is a local nonarchimedean field and X is an F-variety by SchwartzBruhat functions on X we understand locally constant functions on X with compact support.

DAVID KAZHDAN

138

([W], Th. 2 and Prop. 3) F(#(Q)) = voca : #(Q-!) where cg € Rt,

xq € C*, lol = 1 and cg and yq depend only on the image of Q in the Witt group of F.

LEMMA

2.

(a) There exists a unitary representation

02.4 : H —

Et ch

Aut W;

HOO) vO +I)

th

(g2,uQo) AOA) = ¥QAdANFA) (72,y(2a) f(A) = ¥(a) FA) for all f €EW8, À LEA AH E Aa E F. (b) The representation 02,4 is irreducible. (c) There exists unique extension of o2,y to a representation ©2,4 : P° —> Aut W®2 and for all f € W8,t € F, we have 2

cu (Q à)fleo.2) = W(Qu(20,2))f 20,2+at) oy(_9G) S202) =CFUNG0,2) l= 1. (d) The subspace W° C W? is o2,y invariant. PRooF:

Lemma 2 follows immediately from Lemma 1 and [W], n. 12 and

n. 13. We denote by oy the restriction of 72,y on W° Cc W2.

LEMMA 3. (a)

oy(he)f (20,2) = (N(e), S/F )IIN(e)II-*F(N(e)~*20, N(e)~*e72) where h, def (b)

; ah) e € E*.

C= YQ::

PROOF: It follows from Lemma 1 and [W] n. 13 that there exists a character x : E* — C* such that |x| = 1 and

oy (he) f(z0,2) = X(e)NIN(e)IIT FN (e) ro, Ne) te2x). Let we © ( Of —e

1 We have 0

Deda ee piltes) -(3 {)

MINIMAL REPRESENTATION

OF D,

139

The representation o° defines the representation oy, of SL2(E) on the space W’ of distributions (that is, linear functionals on W°). Let 6 € W’ be given by 6(f) = f(0). We have '

pe

TOO

LENS

oe

1

o4(j

—e-!

1

AAG

Tyne

a

=o4(4 1)FQ) where Y(Q:) def (Q:(zo0, z)) is considered as a distribution on A. It follows

from [W], Theorem 2, that o’(w.)(6) is a locally constant function on A‘ and o’(we)(6)(0) = y(Q_--1) - c where c is a positive number. The part (b) is proved. To prove (a) we remark that h.w; = we. Therefore we find x(e) = 7(Q_--1)y(Q_1)~!. Lemma 3 follows from Lemma 3.1. Theorem 63.23 in [0] and Propositions 3 and 4 in [W]. CoRoLLARY.

¢? = (-1, bg)F).

Let P be the semidirect product PE = LE x H. We extend w : LE > Gm to the character w : PE — G,, trivial on H and define P° = ker w. It is clear that P° is the group of F-points of a connected algebraic group P°.

Since LE is isomorphic to the semidirect product LE = h(G,,) x SL,(E) we have PE = WG.) x P° and we can identify P?/P° with F*. Let dy be an additive Haar measure on F 2 F*. We denote (°c, °W) the subspace of the induced representation indPo w° which consists of smooth functions f : PE — W° which satisfy two conditions

SP Py Ap )/(D), p © P

pe PE

b) The function h*(f) on F* given by h*(f)(y) = f(h(y)) belongs to the space of Schwartz-Bruhat functions on F*. As follows from Lemma 3 the equivalence class of (°¢,°W) does not depend on a choice of w. We can realize °W as a subspace of W: def L?(F x F x E, dydrodz) which consists of Schwartz-Bruhat functions on F* x F x E. It will be convenient to conjugate this representation with an operator ® : f(y, 20,2) — f(y, yzo, yz). Then we obtain the following realization of

the representation (°c, °W). LEMMA

4.

The representation °o : P

—+

Aut °Wo is given by the

140

DAVID KAZHDAN

formulas

20,2)

(o(za)f)(y, 20,2) = ¥(ay) f(y,

(Co(to,t')F)(y, cox) = ¥(toro + t’(z)) f(y, Zo, x) (Co(to,t)f)(y, 0,2) = f(y, zo + Yto,

+ yt)

20,2+Zot) Co( 5 1)Nlerz0.2)=HOT *Qu(a0,2))f(ur Ca LT 4)Alu 0e) = rai 6m) i

FxE

f(y, Zo, Z)0(y~ *(zo®o + T(xz))dzodz

(a(h(c)) £)(y, 20,2) = (cy, c~*20, 2) °a(h(e) f(y, 20,2) = (N(e), bzyr)IN(e)If(y, N(e)ro, Nee 72) (Co(h(u))f)(y, 20,2) = f(y, t0,u- 2) for allto,a € f, cE F*, F, t' € Homp(E, F). PROOF:

tE E, ue

E’, e € E*, th € Homr(F,F)

=

Clear.

Let W2 def DEX EX E , dydzodr). We denote by °o2 : P — Aut Wp the unitary representation of P given by the same formulas. Define operators A, R € Aut W2, a € F by formulas (Af)(y, 20, 2) 2

(Res)

Ÿ (2)

f(—z0,y, 2)

¢ ee ERA EAU)

Let €: GE — GE be asin §1. Since F is a local field, we may consider GE

as a topological group. Let 3 C L° C GE be as in $1, C = T0NB.

Then

L° = SLA(F), C= (6 al hi PROPOSITION

1.

a) The representation 02 : P — Aut W2 can be extended

to a representation ? : GE — Aut W such that (6) = A,F (G és he= Te. se b) Let 7’ be a representation of G on W2 such that f'|p = o2. 1 = 3: PROOF:

Then

a) As well known the group L° is the augient of the free product

of groups {52} and C by the relations

(a) x(§ ali.

ne

MINIMAL REPRESENTATION

SW

OF D:

141

oad la ep Os

COMENT sae in)

(eeJecer

It is easy to see that Âo2(h(c)) = _o2(h((e71)) A, A? = o2(h(-1)) and o2(c,0)Ao2(c!, 0) A- *o2(c,0) = o2(h(c))A. Therefore there exists unique representation

Po

: L° —

Aut W2 such that Tlrong

#(3)) = À. It is easy to see that sü(

=

G2lLong

and

1) = R, and 7|;, extends

uniquely to a representation 7|z of L on W2 such that |z(t)= o(t) for all

Le LE, For any h € H we have (5h31) = Ao2(h)A- 1, Since PE = LE x H we see that o~ extends uniquely to a representation Mee : P® — Aut Wa such

that 7|5(b) = o2(b) for all B € B®. Therefore there exists a representation 6 : GE — Aut WA such that F($) = À and *G()

S))uh

b) Let # : GE — Aut W» be a representation such that nt’|p = o and AS? (5). The subgroup Be TH € Beis normalized by S. Therefore

A satisfies the conditions.

Ao2(b)A~!

= o2(sbs—') for all b € B.

This

implies the existence of a function a on F* such that (Af)(y,2%0,z2)

=

a((yz0)4 (-X Xe) — f(—xo,y,z) and a(N(t)to)= a((to) for all to € F*, t € E*.

The relation (c) in [° implies that for any zo,y,t € F we have

a((zo — ty)y)a(t~12o(xo — ty)) = a(zoy). Since the image Im N of the norm map N : E* — F* is an open subgroup in F* we have 1—t € Im N.

If |t\1, take ro

= y = 1. We find a(1)a(t~!) = a(1) for all sufficiently

small t. This implies

a = 1. Proposition 1 is proved.

We can formulate the main results of this paper more precisely. THEOREM 1. The representation Il’ of GF is unitarizable and 7 is equivalent to the representation iis o € where Tl, is the unitary completion of

Ter

REMARK: We will denote ne by (7,W) and its unitary completion by (12, W2). It is clear now that Theorem 1 implies Theorems A and C. Let A C Aut W be the operator given by A def = o “((a

a

CoroLLarY1. (AA)?= (AA). REMARK: It would be interesting to find a direct proof of Corollary 1. As follows from Theorem 1.2, Corollary 1 implies Theorem A.

142

DAVID KAZHDAN

PROPOSITION 2. Theorem C is true in the case when E = F®. ProoF: In this case the representation pg of the Levi subgroup LPs the representation unitary induced from the trivial character of its Borel subgroup. Therefore the representation Tl’ isa quotient of a representation of the principal series of G® and Proposition 2 follows immediately from

Theorem 4 in [KS]. 4. In this section F will be a global field of characteristic zero, © the set of places of F, ©’ C ¥ the subset of nonarchimedean places, A the adele ring of F, E > F a cubic extension, Ag the adele ring of E. For any v € © we denote by F, the completion of F at v. Then E, def E@r F, is a separable 3-dimensional F,-algebra. If v € &’ we denote by O, C F, the ring of integers in F,.. We fix a nontrivial additive character Ÿ on A such that Ÿ|r = 1. For any v €d the restriction of Ÿ on F, defines a nontrivial additive character w,

on Fy. Let

= I)

en = © vey

= Pies) ONU ANAL

ate

the corresponding adelic groups. The isomorphism PE = G,, x P° defines an isomorphism PR = Ia « PS where Ia C A is the group of ideles. For

any v € © we have a representation o? : P9 —+ Aut W? and for almost all vy€ Z’ we have dim(W2)Po. = I where PS, C PL is the subgroup of O,-points. Let of : Pg —> Aut Wa be the restricted tensor product of

of ve. Let Wy be the space of smooth functions f on Haq such that f(yzah) = v(a)f(h) for all y € Hr, h € Ha, a € F and |f| has compact support on

Ha/HpZn.

We define an L?-norm || llwe on Wy in a natural way. The

group Ha acts on Wy by right shifts

(ay (ho)f)(h) = f(hho), f € Wy, ho,h€ Ha. LEMMA

PROOF:

1.

The restriction of of on Ha is equivalent to oy.

Follows from [W], §18.

We denote by a : Wi— Wa an intertwining operator. a is unique up to a multiplication by a scalar. We extend oY, to a representation oy : PR —

Aut Wy} by où 10 on Oa. It is clear that the action of the subgroup S L2(E) of SLo(Ag)) on Ha preserves Hp and therefore defines a natural

action of SL2(E) on W2 : g : f — f9 where f%(h) = f(h?). LEMMA

2.

For any g € SL2(E), f € Wy we have

oulg)f = ff. PROOF: f —

For any g € SL2(E) consider the map B : Wy — Wy given by Bf def = oy(g~')f%. It is; clear that B commutes with the action of

MINIMAL REPRESENTATION

OF D,

143

Hq. Since the representation où is irreducible we have B = À, : 1 for some Ag = C*.Itis clear that the map g —> À, defines a group homomorphism

SL2(F) — is proved.

C*. Since the group SL2(F) is perfect we have À = 1. Lemma 2

Let Wy be the space of functions f : Pa’ — W{ such that a) f(p°p) =

oy(p°)f(p), p° € Pa, p € PE,

b) there exists a smooth function f :

A — W} such that f(¢) = f(h(£)), for all £ € In C Aand

c) [fllw, €

L?(A,dzq) where dxa is a Haar measure on A. Pa acts on Wy by right shifts. Let x : Pa —

Aut D be a representation and D’ be the dual space to

D. For any linear functional d’ € D’ we define a map yw : D — C(Pa) by

pa(d)(p) € d'(x(p}d),

dE D,p € Pa. Let Dp = {d' € D'Id'(r(y)d) = d'(d) Vd € D, y € Pr}.

For any d’ € Dp we define d,, € D’ by di,(d) def PROC LEMMA 3. Let d' € D} be such that ker yw dE ker Yai, we have m(zqa)d = d for allacA. PRooF:

Fix

d

€

ker

Pa!

and

any

=

character

{0}.

Then for any

Ÿ of A/F

define

d~ a | p(—a)m(za) da. We claim that for any Ÿ #~ Id we have A/F dz = 0. Really there exists x € F* such that ÿ(a) = (za) Va EA. By Lemma 3.3 we can find + € Pr such that y~!zgy = z,-1g

Vae A. We

have ga (dz)(p) = ¢a',(d)(7p) for all p € Pa. So dy € ker pq = {0}. Since d= = 0 for all nontrivial characters ~ of A/F we have r(z,)d = d VaeA. Lemma 3 is proved.

Suppose that for each v € © we have an irreducible unitary representation Ty : G, — Aut D2, such that dim De = 1 for almost all v € &’. We

choose d? € Dee — {0} for such v € ZX’. Let D, C Day be the subspace of smooth vectors and (7, Da) be the restricted tensor product of (7,, D,). That is, the space Da is the space of finite linear combinations of vectors of the form d = @&,exd,,d, € D,,d, = d° for almost all v € Z'. The group Ga acts naturally on Da. Let Xo be the set of places v € Y such that E splits at v. PROPOSITION

1.

Let 7m : Ga

—

Aut Da, 7 =

@,ex®,

be the tensor

product of irreducible representations of G,. Suppose that Di, ¢ {0} and that for some vo € Xo the restriction of (n,, D2,) to P, is equivalent to C2. Then the restriction of (Ty, Div) to Py, is equivalent to o2, for all vex.

DAVID KAZHDAN

144

Fix d’ € D},d' # 0. Let ya : D — C(Pa) be the map as in

Proor:

Lemma 1.11. LEMMA

4.

gq is an embedding.

PROOF

OF LEMMA

4:

Let dy

: D —

C(Ga) be the map

defined by

ba (d)(g) jus d'(x(g)d), d € D,g € Ga. It is clear that yq intertwines the actions of Ga on D and on C(Ga). Therefore ker ga is a Ga-invariant subspace of D. Since D = @, D, and the representations 7, : G, — Aut D,

are irreducible, we see that ker x

=

0.

Take d € kery,, and define

fa ef Par(d) € C(Ga). It is clear from the definition that fa(yg) = fa(g) for all y € Gr, g € Ga. Since ¢q = 0 we have falp, = 0. Therefore

alé

0:

The subset

Gr Pa C Ga is dense and we conclude that fg = 0. Since x

is an imbedding, we conclude that d = 0. Lemma 4 is proved.

Define a linear functional d/, on D as in Lemma 3, and write ¢ instead of Par, : D — C(Pa). For any p € Pa, d € D defines a function wa(p) on

Ha by wa(p)(h) = p(d)(hp). It is clear that wa(p) € Wy}. LEMMA

5.

For any d € D, p° € P&, p €

Pa

we have wa(p°p)

=

oy,(p°)wa(p). PrRooF:

Fix d € D, p € Pa and define a function © : en —

Wy by

®(p°) a oy (p°)~ wa(p°p). By the assumptions of. Proposition 1 we have ®(p°p?) = &(p°) for any p?, C Pa. On the other hand, it follows from Lemma 2 that ®(yp) = ®(p) for all y € Pr. Since the set PL - Be is dense in Pg we concluded that y = constant. Lemma 5 is proved. It follows from Lemmas 3 and 4 that ç is an imbedding. Now Proposition 1 follows from Lemma

1.

Now we can prove Theorem 3.1. Let F be a local field dimpE = 3 be a commutative semisimple F-algebra. We can field F, a non-Galois cubic extension E and a place vp € DEP 9 D3 ytd diy Let G be the Galois group G = Gal(F : F) and tg : G

and E > F, find a global © such that

—

S3 be the

morphism corresponding to the cubic extension E over F. Since E is not normal, Tg is surjective. Let wz : G — GL2(C) be the representation as in 81. We can associate with yr a cuspidal automorphic representation pg of

the group La in a subspace ME C C(La/Lr). For any good function f on Gk /pr Un with values in ME we denote by E;(g,s).the corresponding Eisenstein series of the group GE (see [L]). It is easy to see that the corresponding L-funciton L(pg,s) has a pole of

the first order at s = 1. Therefore (see [L]) functions E;(g,s) have a first order pole at s = 1. We define y} def lim,_1(s — 1)E;(g,s). As follows

MINIMAL REPRESENTATION

OF D4

145

from [L] #; are automorphic forms from GZ/GZ they span a nonzero Ga-

invariant subspace Wa of L?(GË\GX) and the representation Iq of GE on Wa is irreducible.

Moreover it follows from arguments in [J] §2.4 that TIq is

equivalent to the restricted tensor product Ovens where representations

I,=»

of GE» where defined in §1.

As follows from Proposition 4.3 the representation Ila satisfies assumptions of Proposition 1. Therefore for all v € ¥ the restriction of II, on P, is equivalent to the representation Il>,. In particular, we have a representation o,, : GÈ — Aut W,, such that the restriction of o,, on PE is

equivalent to a. As follows from Proposition 3.1, ¢ = o,,ov. Theorem 3.1 is proved.

5. In this section we fix a local nonarchimedean field F of characteristic zero, a field E D F, dimpE = 3, and will write G, P instead of GE, PE.

The restriction of (7,W) (see §3) to P we denote by (¢, W) or simply o. The goal of this section is to get some information about co. For any algebraic F-group H we denote by A(H) the category of smooth

representations of H (see [BZ] 2.1) and by €(H) the set of equivalence classes of algebraically irreducible smooth representations of H. Let U be a unipotent F-group, 8 : U — C* a character of U. For any smooth representation 7 : U — Aut R we denote by R(U,8) C R the

subspace generated by vectors (r(u)r — 8(u)r), the quotient space Ry» de R/R(U,8).

uEU, r € Rand by Ru»

We will write Ry instead of Ru ra.

LEMMA 1. For unipotent group U and a character 0 ofU the functor R — Ry» from A(U) to the category of vector spaces is exact.

PRooF:

See ([BZ] 2.30).

We start with some general results on representation of the group P.

For simplicity, we will write A instead of A(P) and define subcategories Apo, A1, 42 of A as follows.

Ao = {(r,À) € AÏRz = {0}}

A, = {(r,R) € Alrz = Id and Ry = {0}} and A2 = {(7,R) € Alm H and Z.

= Id} where 77,7

are the restrictions of x on

For any representation (7, À) C A we define subspaces Ro,À of R by

Ro © ker(R —

Rz) and R: “ ker(R —

Ry) where R — Rz, R — Ry

are natural projections. It is clear that Ro, R1 are P-invariant subspaces of Rand Ro C Ry. mt Let (70, Ro) € A be the restriction of 7 on Ro, (ñn1, Ri) € A the natural action of P on R; = Ri/Ro and (12,2) € A be the action of P on

146

DAVID KAZHDAN

LEMMA

2.

(7, Ro) € Ao, (ri, 21) € Ai and (71, Re) € A2.

PrRooF:

Follows from Lemma

1.

For any v € V we define by 6, the character of V defined by

6,(5) = H((v, 5). Since V is a quotient of H we can and will consider 6, as a character of H.

Let Q C V — {0} be a subvariety as in Lemma 3.4. We fix vo € 2 and denote by So the stabilizer of vo in L and by @& the character 0,,. It is clear that So is isomorphic to the semidirect product Sg = E* x E, where

E* is the multiplicative group of E. For any character x of E* we extend it to a character y on So trivial on E and extend 09 to a character 6%

of the semidirect product So K H by 8X(s,h) a 8o(h)x(s). We denote by

ps € Ai(P) the unitary induced representation pj tf ind(P, Sox H, 6%). For any representation (7,R) € A the action of So on R preserves R(H,6o) and therefore defines the action of So on Ry», : (s,7) — s7, s € So, FT € Rx. We denote by 79, the representation of So on Ry 6,

given by 79,(s)F = |s|!/2s7, where for s = (e*,e) € So, |s| def le*|. For any representation (7, R) € A we define

X(r) = {v EV — {0}|Ru,o, # {0}}. LEMMA 3. a) X(r) is an L-invariant subset ofV — {0}. b) Let (7, R) € A be such that X(r) = Q, dim Ry», = 1. Then T = px, where x Is the restriction of 69 on E* C So.

PROOF:

a) is clear. b) follows from the Mackey theory (see [BZ]§5).

Let o be the restriction of t on P,0

C Wo C W, CW

be the filtration

as in Lemma 2, °W © S(F* x F x E) C W as in the end of §3. We denote by 01,02 —

def

W

=

the representations of P on W, def W,/Wo

and

W/W.

PROPOSITION 1. a) Wo = °W. b)o1 = po” where xg is the character of E* given by xg(c) = (N(c), 65/F) ce E*. c) If E C F is not normal then W = {0}. d) If E C F is normal, then N(E*) is a subgroup ofindex 3 in F*. Let € be a nontrivial character on F*/N(E*).

Then

W = Cg @ Ceo where Ce is

a 1-dimensional space on which P acts by the character p — |w(p)|E(w(p)) for allp € P.

PROOF: a) It is clear that °W is a P-invariant subspace of Wo C W. To prove a) we have to show that Z acts trivially on W/°W. In other words, we have to show that for any f € W and y € F we have o(E(y))f —f E°W. But this follows immediately from formulas for OjH:

?

MINIMAL REPRESENTATION

OF D4

147

b) Let X(o1) be the subset of V = F@ E@E@® F as in Lemma 3, V1 C V be the affine subspace N = (20,2,%,1), ro € F,x,% € E and A

def

X(o1)NV.

LEMMA 4. a) X = {(N(x),@(x),x,1)},x € E and b) for any v € X, dim Wy, = 1 and Xa,(c) = (N(c),6g/r) for all ce E*. PROOF:

def

a) Let H = 5H5-1. For any v € N we denote by 6, the character

of À given by 6,(h) © 0,(5-1h5).

Let U © A’Z C H. It is clear that

SU a = For any x € E we denote by y, the character of U given by X2(Z, Zo, a) e w(T(xz) — Zo) and by ÿ,; the character of U given by Xz(u) = xz(Su5-!). As follows from Proposition 1 and formulas for oly we have dim

W, y,y, = 1 for all x € E and the natural projection W —

Wu.x, = € is given by the formula f — f(—1,0,x) W. Therefore for any v € N C V we have Wy v = (N(z),6(x),z,1) for some x € E and in this Therefore X = {N(zx),0(x),z,1} and dim Wy, = easy to check that m,(c) = (N(c),6g/r) for v € X.

for f = f(y,z0,2) 5, # 0 if and only case dim Wy 4 = 1 for all v € X. It Lemma 4 is proved.

Now we can prove Proposition 1b). As follows from Lemma

€ if 1. is

1 we have

X(o1) = X(c). Therefore X(o,) is an L-invariant subset of V — {0} and by Lemma 4, X(0,)QNN =QNN. So X(o1) D Q and X(o;) — Q is an L-invariant subset of V such that (X(o1) -Q)NN = 0. It is easy to see that this implies X(o1) = Q. Proposition 1b) follows from Lemmas 1, 2, 3, and 4 and formulas for a.

c) We have Wy = W/W,.

(In other words, the representation of L on

W/W, is the value of Jacquet functor T2(W).) As follows from Theorem 3.1, 7 is a subquotient of a representation II induced from a cuspidal

residual representation of P = L-U. Therefore F£(W)= {0}. d) The group L acts naturally on W and it follies from Theorem 3.1 that W has exactly two 1-dimensional quotients which are isomorphic to Ce and Cg2. The following result implies the validity of Proposition 1d). Let wg be the additive character of E given by #g(e) = Y(Trgyr(e)). We can identify Æ with the root subgroup E, of G?. We extend eg from E, to the unipotent radical Ug of B in such a way that Yglu = 1.

LEMMA

5. Wu,vx = {0}.

PROOF: Let Ur sUpsk7} C GE and Ÿ be the character of Up given by p(n) = ~p(5—!nS). It is sufficient to show that Wee Seg= {0}. But this

follows immediately from Proposition 1b) and Corollary to Lemma 2.3. Proposition 1 is proved.

148

DAVID KAZHDAN

Let U; be the subgroup of G® generated by root subgroups Ep, Hy.) and Beet Ÿ : U3 — C* is a character of U3 we say that piis nondegenerate if the restriction on 7 on the subgroups Eg, and E,, is nontrivial. Let Ÿ be a nondegenerate character of U3. Then there exists an element ay € F* such

that for any a € F, Y(Ep,(a)) = (s Eg,(aya))s~!) where the element s € LE is as in Lemma 1.3. It is easy to see that the image @y of ay in the group F*/N(E*) does not depend on a choice of the element s. Let E’ be the kernel of the norm map N : E* — F* and yx be a character

of E*. Let i be the imbedding i : (SL3(F) x E’)/p3 — GE as in §1. We will identify U3 with the maximal unipotent subgroup of SL3(F). Let W, C W be the subspace of w € W such that (i(1 x e’))w = x’(e’)w for all e’ € E’. The natural action of SL3(F) on W, extends (see §1) to a representation 7m, of the group GF on Wy. LEMMA

6. Let Ÿ be a nondegenerate character of U3. We have 1

dim(Wy)y, 5 = {0 PRrooF:

1.

a4

otherwise.

Follows from Proposition 1b) and Lemma 2.3.

PROPOSITION 2. Let (my,W.x) be the representation of GL3(F) induced from the representation (7, W,) of GZ. Then my is an irreducible representation.

PROOF: As follows from Lemma 6 for any nondegenerate character of Us we have dim(W, )u,,y = 1. Therefore there exists an irreducible nondegenerate (see [BZ]) subrepresentation We C W, such that the quotient W, is degenerate. As follows from te 1 b) and Corollary to Lemma . 4 that (Wy )us,ra = Ce ® C2 if E D F is normal. This implies immediately

that W, = {0}. Proposition 2 is proved. Assume now that F is a normal extension of F and € : F*/N(E*) — C* is a nontrivial character.

Consider the function y on F* x E given by

pe(ro,x) = Cry (22). We can easily extend $£ to a distribution on F x E. As in §3 we can define the Fourier transform F on the space of distributions on F x E.

PROPOSITION 3. F(pe) = pe for some y € C*. PROOF: As follows from Proposition 1d) there exists a functional A+ on W such that A+(pw)= E(w(p))|w(p)|At(w) for all we W,pe P. Let 2 ate = C GE be the parabolic opposite to P. The subgroupsP and P~ of G® are conjugate. Therefore, there exists a nonzero functional

MINIMAL REPRESENTATION

OF D4

149

À € W' such that 7'(h-)A = À and 7'(£)X = E-1(w(L))|w(C)|-1A for £2 € Pur EU. Let °W C W2 be the subspace of locally constant functions on F* x Fx E with compact support. As follows from formulas for 72 we have °W C W.

LEMMA 7. Let °X be the restriction of Àon °W. Then ° F 0. PROOF: Suppose that Maw = 0. Then we could consider À as a functional on W/°W. In this case, as follows from Proposition la), À is Z-invariant. But P~ and Z generate G. Therefore À is a G-eigenfunctional. Since W is a nontrivial irreducible representation of G we have a contradiction.

Therefore °\ 4 0. Lemma 7 is proved. Since L preserves °W, it acts on °W’ and °) is an L-eigenfunctional.

Consider the subspace

W = S(F* x F* x E) C °W = S(F* x F x E) and

denote by À the restriction of °A on W. Since ° À is an L-eigenfunctional we see that ©) is invariant under the Fourier transform A on °W. This implies

that À £0.

The subgroup By, SE BNL

of P- preserves W. The condition n'(b)A =

|w(b)|-1\ E(w(b))—! for all b € By shows the existence of a function H on F*/N(E*) such that \(f) = f(An-f)(y, ro, 2)dydzodz for all f € W where Ax is a locally constant function on F* x F* x E given by Ax(y, 20,2) =

H(y2o) Sey (FE). YZo

Let f € W be a function such that for any Zo € F*,x € E we have

f(x0,%0,x) fly, rae)

=

0.

Consider the function f on F* x F* x E given by

(ZS) f(y— 20,20, x). We have fe CPx

Ee xo) =

W. Since À is an E_g,(1)-invariant functional we have \(f) = A(f) for any f € W such that f(xo,20,2) = 0 for all zo € F*,x € E. Therefore, for any 2o,y € F*,20 # yo we have H(y(y — zo)) = H(yzo). As in the proof of Proposition 3.1b) this implies H = constant. Now it is clear that °A coincides with the functional ye. Proposition 3 is proved. 6. In this section we consider the case when F is a local nonarchimedean field and E is a direct sum E = F @ K where K is a semisimple 2dimensional F-algebra. We denote by z — Z the nontrivial involution of

K. It is clear that for any z € K, N(1,z) = 27 and T(0,z) = z @Z. Since E=F@K we have Lo = SL2(F) x SL2(K) and we can consider «1,71 as roots of G. Let Eza,, E47, : Ga subgroups.

G be the corresponding one-parametric

LEMMA 1. a) There exists an algebraic group homomorphism 2: SL3(F) >

DAVID KAZHDAN

150

ps

,

G such that

i(e12(a)) = Ea, (a), i(e1(a)) =

PRE (a), i(e23(a))

= E,,(a), t(ez(a)) =

24, (a);

acer.

Moreover, we can extend 7 to a group homomorphism 2: Sla(F) x K* > G such that for all u € K*

iw =3((9 yor) #0),

where hg, : Gm — T is the coroot corresponding to Bo.

b) Let i: SL3(F) x E’ + G be the morphism as in the end of §2. Then the morphisms i,i : SL3(F) x E’ — G are conjugate under an action of an element from Aut G. PROOF:

Clear.

Our immediate goal is to describe the unitary representation of SL3(F) x E’ on Wa obtained by the composition of 72 and 7. Remember that we have E' = K*. Let R be the subspace of Schwartz-Bruhat functions on F* x K.

LEMMA 2. such that

There exists a unitary representation T : GL2(F)xK* — Aut R

(4 1)elu2) = vOrte NE)o(2) (2, pJews =r [Po TE Dy 8

(5 os Jone) = xtoielf(u es (5 1Ja) = ex (els(euee) (T(u)p)(y, 2) = IN (u)IF(N(u)y, uz), a € F,c € F*,u € K*

where y € C* is as in [JL] $1, and ex : F* —

C(+1) C C* is the character such that ker ex = Im(N : K* — F*). PROOF: See [JL] §1. Let

Q =

MU

m * (ETS

C SLa(F)

lee

be the standard

parabolic subgroup

Q =

GL2(F). We extend r to a representation of Q x K*

trivial on the unipotent radical of Q and denote by (r2,.R2) the represen-

tation of SL3(F) x E’ unitary induced from Q x E’ the representation 7 to SL3(F)

Ce

MINIMAL

PROPOSITION

1.

REPRESENTATION

OF D4

151

The representation o2 01 of SL3(F) x E’ is equivalent

to the representation T2.

PROOF:

Since morphism 7,7 : SL3(F) x E’ — G are conjugate, it is suffi-

cient to show that o2 07 is equivalent to 72. Let || || : GL2(F) — C* be the map given by ||m|| = |det m|°/?. We extend it to a character | |: Q — C*. By the definition R2 consists of functions y : SL3(F) — Lo(F x K) such that f(¢ g) = T(qg)|lal|f(g) for all q € Q,g € G. Therefore we can consider

R2 as a subspace of the space of functions on SL3(F) x F* x K. We construct a map

y —

yy from W into the space of functions on

SLI3(F) x F* x K by the formula

#4 (9.9,2) = (o(9)f)(y, 0,0,2). It is easy to see that for any f € °W CW C Wa, vy C Ro and |lyy|lz, = l|fllw,. Let po : We — R2 be the closure of vy. It is clear that v2 is an isomorphism of Hilbert spaces. Proposition 1 is proved.

Let 7 : GL2(F) x E’ — Aut W be the image of Jacquet r™ in respect to the unipotent radical U of Q applied to (oi, W). For any character x of

K* we can define representation (T,,W,) and (T,, Ry) of M = GL2(F) as quotient of W and À by subspaces spanned by vectors {7(u)w — x(u)w},

u inE',w € W and vectors 7(u)r— x(u)7, u € K*,r € R correspondingly. If K is a field, we say that a character y of K* is regular if x}x, # 1 where Klatne k=l} MIRE PF we" sayithatiacharactersy = (v1, x9) of K* is regular if XX 7 lll, 7 Sot where et =1LET] 0710 ‘and

x3 : F* — C* is given by x3 = (x1X2)7’LEMMA 3. a) If K is a field and y is a regular unitary character of K* then the representation T, and T, are equivalent. b) If K = F? and x = (x1,X2) is a regular character of K*, then

™ = 7T(x1,x2) BT (x1,x3) D M(x2,x3): Proor: We consider only the case a), so we assume that K is a field. Since the image i(U) is a normal subgroup of P the group P > M acts naturally on W. As follows from Proposition 6.1 there exists an imbedding of S(F* x K) in W as a Bg,-invariant subspace and the subgroup i(K”) C G : ; TOR 0.) EA acts trivially on the quotient space. Since § = ibe a normalizers Bg, we can find explicitly the action of § on W(x) for any regular character x of K*. Since i(M) is generated by § and i(M)/M Bg,, Lemma 38a) is proved. We leave the case b) for the reader. Lemma 3 is proved.

152

DAVID KAZHDAN

PROPOSITION

Proor:

2.

Theorem B is true in the case when E=F@K.

In our case the norm map N : E* — F* is surjective and G? =

GL3(F). On the other hand, we have E* = (F* x {1}) x E’. Therefore for any character 6 of F* we can extend x! to a character x9 of E* defining xo to be 0 on F* x {1}. So to prove Theorem 4.3 in our case we have to construct for every 8 : F* — C* the extension of 7, to a representation 7,

of GL3(F) and to show that 7, corresponds to yg.

LEMMA 4. ProoF:

(r,y,Wy) = indg

(Fy, Ry):

As follows from Proposition 1 the representation of SL3(F) on Re

does not have supercuspidal subquotients. Therefore the representation 7,, of SLa(F) on Wy also does not have supercuspidal subquotients and zy, is completely determined by its image under the Jacquet functor r¥ (Cary

Since the functor r¥ is exact, we have rM(r,,) = 7. By Lemma 3 and [BZ] we have rif(xs) = rf (indg7,:). Lemma 4 is proved. Let 8 be a character of F*.

We extend the representation T,, : M —

Aut Fe to a representation T, : M — Aut R, where

F,x({1} x c) = 8(c)Id, c € F*.

M © M x F* and

We consider M as a Levi subgroup of a

parabolic Q C GL3(F), Q = MU

indé

and define io

ry.

As follows from Lemma 4, 7, = me Proposition 2 is proved.

6. In this section we consider the case when F is a local nonarchimedean field and E is a direct sum E = F @ K where K is a semisimple 2dimensional F-algebra. We denote by z — Z the nontrivial involution of K. It is clear that for any z € K, N(1,z) = 27 and T(0,z) = z @7. Since E=F@K we have Lo = SL2(F) x SL2(K) and we can consider a1, 71 as

roots of G. Let E44,,E+,, subgroups.

: Ga — G be the corresponding one-parametric

LEMMA 1. a) There exists an algebraic group homomorphism? : SL3(F) > G such that

L

i(e12(a)) = Eg, (a), 1(e21(a)) = E_a,(a), i(e23(a))

= E,,(a), i(e(a)) =

E°;(a),

ae

F.

Moreover, we can extend i to a group homomorphism : : Sla(F) x K* > G such that for all u € K*

= (( 21) 2.07 ,

where hg, : Gm — T is the coroot corresponding to Bo.

MINIMAL REPRESENTATION

OF D,

153

b) Let i: SL3(F) x E’ > G be the morphism as in the end of §2. Then the morphisms 1,1 :SL3(F) x E’ — G are conjugate under an action ofan element from or GC: PROOF:

Clear.

Our immediate goal is to describe the unitary representation of SL3(F) x E’ on W2 obtained by the composition of o2 and 7. Remember that we have E”’ = K*. Let R be the subspace of Schwartz-Bruhat functions on F* x K. LEMMA 2. such that

There exists a unitary representation T : GL2(F)xK* — Aut R

(5 1)e2) = 90a Ne))elW.2) cra

5)Pte z)= pes Yu T(z 2))f(y, Zdz

(2 Je = ex (eels aes) )edz) = (ou ez)

(1

(T(u)y)(y, 2) = |IN(w)IF(N(u)y, uz), a € F,c € F*,u € K* where y € C* is as in [JL] §1, and ex : F* — C(+1) C C* is the character such that ker ex = Im(N : K* — F*).

PROOF:

See [JL] 81.

Let Q =

MU

C SL3(F) be the standard parabolic subgroup Q =

if ne us A ),m € GL2(F). We extend 7 to a representation of Q x K* et m

trivial on the unipotent radical of Q and denote by (72, R2) the represen-

tation of SL3(F) x E’ unitary induced from Q x E’ the representation T to SI3(F)

SC Pee

PROPOSITION 1. The representation o2 0% of SL3(F) x E’ is equivalent to the representation 72.

PROOF: Since morphism 2, 1: SL3(F) x E’ > G are conjugate, it is sufficient to show that a2 07 is Dole to T2. Let || || :GL2(F) — C* be the map given by ||m||= |det m[°/2. We extend it to a character | |:Q — C*. By the definition R2 consists of functions y : SL3(F) — L2(F x “à such that f(¢ 9) = T(q)|lallf(g) for all q € Q,g € G. Therefore we can consider R2 as a subspace of the space of functions on SL3(F) x F* x K. We construct a map @ — yy from W into the space of functions on

SL3(F) x F* x K by the formula

vs(9,y,2) = (o(9)f)(y, 0,0, 2).

154

DAVID KAZHDAN

It is easy to see that for any f € °W CWC Wa, vy C Ro and llesile. = Ifllw,. Let yo : W2 — R2 be the closure of y. It is clear that y2 is an isomorphism of Hilbert spaces. Proposition 1 is proved.

Let 7: GL2(F) x E' — Aut W be the image of Jacquet rf in respect to the unipotent radical U of Q applied to (oi, W). For any character x of

K* we can define representation (T,,W,) and (Ty, Ry) of M = GL2(F) as quotient of W and R by subspaces spanned by vectors {7(u)@ — x(u)w},

u inE',w € W and vectors T(u)r — x(u)F, u € K*,r € R correspondingly. If K is a field, we say that a character x of K* is regular if x;~» # 1 where K' = {z € K*|zz = 1}. If K = F? we say that a character x = (x1, x2) of K* is regular if XX; INI livres} < iwhere. € = itl or Diana

x3: F* — C* is given by x3 = (x1X2)7?LEMMA

3.

a) If K is a field and x is a regular unitary character of K*

then the representation T, and 7, are equivalent.

b) If K = F? and x = (x1,X2) is a regular character of K*, then

=

T(xaxa) OT (x1,x3) D Txa.x3):

Proor: We consider only the case a), so we assume that K Since the image i(U) is a normal subgroup of P the group P naturally on W. As follows from Proposition 6.1 there exists an of S(F* x K) in W as a Bg,-invariant subspace and the subgroup acts trivially on the quotient space. Since § = i( a0

is a field. > M acts imbedding i(K’) C G

normalizers Bg,

we can find explicitly the action of § on W(x) for any regular character y

of K*. Since i(M) is generated by $ and i(M)N Bg,, Lemma 3a) is proved. We leave the case b) for the reader. Lemma 3 is proved. PROPOSITION

ProoF:

2.

Theorem B is true in the case when

E= FQ K.

In our case the norm map N : E* — F* is surjective and GE =

GL3(F). On the other hand, we have E* = (F* x {1}) x E’. Therefore for any character 8 of F* we can extend y! to a character x» of E* defining

xe to be 8 on F* x {1}. So to prove Theorem 4.3 in our case we have to construct for every 0 : F* — C* the extension of 7,’ to a representation Ty of GL3(F) and to show that x, corresponds to y4.

Lemma 4. (ty, Wy:) = indg°F,Ry). PROOF:

As follows from Proposition 1 the representation of SLa(F) on Ro

does not have supercuspidal subquotients. Therefore the representation Ty! of SL3(F) on W, also does not have supercuspidal subquotients and Ty!

MINIMAL REPRESENTATION

OF D4

155

is completely determined by its image under the Jacquet functor Pe (tp).

Since the functor rf is exact, we have rM(x,,) = 7. By Lemma 3 and [BZ] we have r(x.) = ré (indgTy:). Lemma 4 is proved. Let 4 be a character of F*.

We extend the representation T,, : M —

Aut se to a representation Ty : M — Aut Rx where M © M x F* and

T,({1} x c) = O(c)Id, c € F*. We consider M as a Levi subgroup of a parabolic Q C GL3(F), Q = MU and define a HS Te As follows from Lemma 4, x, = TË . Proposition 2 is proved. 7. In 86 we have proved Theorem B in the case when E is not a field. In this section we consider the case when E > F are local fields, dimpE = 3, and x is a multiplicative character of £*. We fix a global field E > F and a place vo € X such that F,, = F, ER E and the extension E > F is not normal. Let on : GE — Aut Wa be the representation as in the end of §5 and

ae

GE —

Aut WA be the dual representation.

We have og = Sdo, ved

where ©, : GE (Fy) — Aut W, are representations as in Theorem 3.1. Since ca is an automorphic representation, there exists a nonzero functional

7 € W! such that o’(y)w’ = w’ for all y € G*(F). Define wy € Wa by wy = Ses, X(u)o’(i(u))w'du where as before i is an imbedding E’ + GE. LEMMA

1.

ProoF:

wy # 0.

The same as for Proposition 1 in [K].

For any character x’ of EA/E" we define a quotient space W, of Wa as before, Wy

= &)(W)x.-

As follows from Lemma 1 there exists a

ved

character x’ of EA/E" such that X py = X py vO

Wy ey

and

wy # 0 where

air]

x(u)o'(i(u))w'du.

u€eE, /E'

The group SL3(A) acts naturally on the space W,,. We choose an extension x on x’ toa unitary character on EA/E*. Then, as in §4 we can define a def

representation ne : GEL? — Aut W, where Gee wipe GL3(A)| det gC Im N : Eq, — A*}. In other words, G3" = {g = (gv) € GLa(A)|g € Ge Vv EX}. Let x, be the induced representation of the group GL3(A) and Yo CE be the subset of places v such that F, is not a field.

DAVID KAZHDAN

156

PROPOSITION 1. a) my is an automorphic representation. b) my = @venty, where my, is a representation of GL3(F,) as in Theorem 4.3. c) For all v € Xo, my, corresponds to the character x, of E;.

PROOF: a) Since the extension £ D F is not normal, the norm map N induces a surjection of the idele class group of E to the one of F. Therefore

we have GL3(F)G2" = GL3(A) and

GA NGLa(F)\ GS" —> GL3(F) \ GLa(A). Consider the map y, : Wy — C(GL3(F) \ GL3(A)) given by 9, (w)(g) = wi)(72 (g)w) for g € GR. By Proposition 5.2 we know that W, is irreducible. Since y, # 0 it is an imbedding. Therefore 7, is automorphic. a) is proved.

b) Follows from the definition of Wy. c) Follows from Proposition 6.2. Proposition 1 is proved. Now we can prove Theorem B. Consider the representation 7, from Proposition 1. Since the density of

Xo is equal to 2/3 > 1/2 it follows from [G] and Proposition 1 that ,,, corresponds to x,,. Since x,,|&, = X|g,, therefore there exists a character

6 of F* such that ¥ = y,,-9. It is clear then that my = Ty,,, @ (det). Since T,,, corresponds to x,,. We see that my corresponds to x. Theorem B is proved. 8. In the conclusion we discuss some open problems. 1) Let F be a local field, V be a symplectic vector space over F, L C

Sp(V) be a semisimple subgroup such that the double covering Sp(V) — Sp(V) splits over L, o : L — Aut W be the restriction of the metaplectic representation on L. Suppose that there exists an L-orbit 2 C V — {0}

which is a closed Lagrangian subvariety. Could we define yp € (W’)4? Proposition 5.3 shows how to define a in the case when L = SL2(E), V=A@A.

2) For any n € Z* letA, =F"

OFF? NA =, 8. Unaa, Lise

7)

Let.

y be a distribution given on (F*)"~? x F” by p(yi,... ,Yn,T1,... )2n42) = rare (ae) transform.

It is easy to see that F(~) = vy where F is the Fourier

Let E,K be cyclic extensions of F of degree n +2 and n. A= K@® E. Could we find a multiplicative character € on K* x E* such that the distribution

def ye(k,e)=E(k,e)-

aol Ve (5e) is invariant under the Fourier transform?

MINIMAL

REPRESENTATION

OF D4

157

It is true in the case n = 0 by A. Weil’s theorem and in the case n = 1 by Proposition 5.3. 3) In the case when F is a finite field we can consider Ÿ (#2) as a locally free sheaf of rank 1 on K* x E. Let be the perverse extension of this sheaf to K x E. It is easy to see that F(~) = y. Is it possible to use the technique of perverse sheaves to prove that (*) in the case where E, K are

unramified extensions of F? 4) We have proven Theorem 3.1 using the global arguments. Is it possible

to prove (*) if we assume that it is known in the unramified case? 5) It will be very interesting to compute a character t, of the representation a. One could hope that t, is given by a nice explicit formula and the

restriction of t, on SL3(F) x E”’ generalizes the formulas in [GGPs]2.5 from SL» to SL3.

REFERENCES [BZ]

[G] GGP] [H]

J. Bernstein, A. Zelvinsky, Representation of the group GL(n, F), where F 1s a local nonarchimedean field, Uspekhi Mat. Nauk, 31 No. 3 (1976), 5-70.

G.T. Gilbert, Multiplicity theorems for GL(2) and GL(3), Contemp. Math. 53, Amer. Math. Soc., Providence, R.I. (1986), 201-206. I.M. Gelfand, M.I. Graev and II. Pyatetskii-Shapiro, Representation Theory and Automorphic Forms, Philadelphia, W.B. Saunders Co., 1969. R. Howe, The Fourier transform on nilpotent locally compact groups I.

Pacific J. Math. 73 (1977), 307-327. [J] H. Jacquet, On the residual spectrum of GL,, Springer-Verlag 1041, 1984. [JL] H. Jacquet and R. Langlands, Automorphic forms on GL(2), SpringerVerlag 114, 1970. PS-S] H. Jacquet, LI. Piatetskii-Shapiro et J. Shalika, Relévement cubique non

normal, C.R. Acad. Sci., Paris, 292 (1981), 567-571. [K]

D. Kazhdan,

Some

applications of the Weil representation.

Journal

d’Analyse Math., 32 (1977), 235-248. [KS] D. Kazhdan and G. Savin, On the smallest representation of simply placed groups over local fields, preprint.

[Ko]

B. Kostant,

The principle of triality and a distinguished unitary rep-

resentation of SO(4,4), Diff. Geom. Methods in Theoretical Physics, Edited by K. Bleuler and M. Werner, Series C: Math. and Phy. Sci. Vol.

250, Kluwer Academic Pub. (1988), 65-109.

158

DAVID KAZHDAN

[L] R. Langlands, On the functional equation satisfied by Eisenstein series.

Springer Lecture Notes in Math. 544 (1976). [M] J. Milnor, Introduction

to Algebraic K-theory,

Princeton

Univ.

Press

1971. [O] O.T. O'Meara, Introduction to quadratic forms, Berlin, Springer, 1963. [S] J.-P. Serre, Cohomologie Galoisienne, Collège de France, Paris, 1963. [St] R. Steinberg, Lectures on Chevalley groups, Yale, 1967.

[ T1] J. Tits, Normalisateurs de Tores. J. of Algebra 4 (1966), 96-116. [T] J. Tits, Buildings of Special Type and Finite BN-pair, Springer-Verlag 386, 1974. [W] A. Weil, Sur certains groupes d’opérateurs unitaires, Acta Math. 111, 143-211. Received January 5, 1990

Department of Mathematics Harvard University Cambridge, MA 02138

Kazhdan-Lusztig Conjecture for Symmetrizable Kac-Moody Lie Algebra. IT Intersection Cohomologies of Schubert Varieties MASAKI KASHIWARA TOSHIYUKI TANISAKI

Dedicated to Professor Jacques Dizmier on his sixty-fifth birthday

0. Introduction

0.0. This article is a continuation of Kashiwara [K3]. We shall complete the proof of a generalization of the Kazhdan-Lusztig conjecture to the case of symmetrizable Kac-Moody Lie algebras.

0.1. The original Kazhdan-Lusztig conjecture [KL1] describes the characters of irreducible highest weight modules of finite-dimensional semisimple Lie algebras in terms of certain combinatorially defined polynomials, called Kazhdan-Lusztig polynomials. It was simultaneously solved by two parties, Beilinson-Bernstein and Brylinski-Kashiwara, by similar methods ([BB|, [BK]). The proof consists of the following two parts. (i) The algebraic part — the correspondence between D-modules on the flag variety and representations of the semisimple Lie algebra. (ii) The topological part — the description of geometry of Schubert varieties in terms of the Kazhdan-Lusztig polynomials. Note that the topological part had been already established by Kazhdan

and Lusztig themselves ([KL2]). 0.2. Our proof of the generalization of the Kazhdan-Lusztig conjecture in the symmetrizable Kac-Moody Lie algebra case is similar to that in the finite-dimensional case mentioned above. The algebraic part has already appeared in [K3] and this paper is devoted to the topological part. The proof is again similar to the finite-dimensional case except two points.

160

KASHIWARA

AND TANISAKI

The first point is that we use the theory of mixed Hodge modules of M.

Saito [S] instead of the Weil sheaves. Note that mixed Hodge modules and Weil sheaves are already employed by several authors in order to relate the Hecke-Iwahori algebra of the Weyl group with the geometry of Schubert

varieties ([LV], [Sp], [T]). The second point is that we interpret the inverse Kazhdan-Lusztig polynomials as the coefficients of certain elements of the dual of the HeckeIwahori algebra. The appearance of the dual of the Hecke-Iwahori algebra is natural because the open Schubert cell corresponds to the identity element of the Weyl group, contrary to the finite-dimensional case in which the open Schubert cell corresponds to the longest element. 0.3. We shall state our results more precisely. Let g be a symmetrizable Kac-Moody Lie algebra, b the Cartan subalgebra and W the Weyl group

(see [K’]). For A € b* let M(A) (resp. L(A)) be the Verma module (resp. irreducible module) with highest weight À. For w € W we define a new action of W on b* by wo = w(1\+p)—p, where p is an element of b* such that (p,h;) = 1 for any simple coroot h; € b. For w,z € W let P,,,(q) be the Kazhdan-Lusztig polynomial and Qw,2(q) the inverse Kazhdan-Lusztig

polynomial ([KL1], [KL2]). They are defined through a combinatorics in the Hecke-Iwahori algebra of the Weyl group, and are related by

(0.3.1)

DM Go AE Pont

tea rye:

wEW

Our main result is the following. THEOREM.

For a dominant integral weight À € h* we have

ch L(wod) = ÿ(-1))-4%)0,,(1)ch M(z0 À), 2€W

or equivalently

ch M(wod) = Ÿ P,:(1)chL(zo )). ZEW

Here ch denotes the character and {(w) is the length of w.

0.4.

Let X be the flag variety of g constructed in [K2] and let X, be

the Scubert cell corresponding to w € W. Note that X, is a finitecodimensional locally closed subvariety of the infinite-dimensional variety

Xx.

KAZHDAN-LUSZTIG

CONJECTURE

161

By the algebraic part [K3] g-modules correspond to holonomic Dxmodules. Hence by taking the solutions of holonomic Dx-modules, we obtain a correspondence between g-modules and perverse sheaves on X.

Since M(wo À) and its dual M*(w o À) have the same characters and since the perverse sheaf corresponding to the highest weight module L(w o À)

(resp. M*(wo À)) is "Cx, [—€(w)] (resp. Cx,[—£(w)]), the proof of the theorem is reduced to

(0.4.1)

Cx. Lo) = S7 (-1)

Qu, (Cx, HAI]

z2€EW

(in the Grothendieck group of perverse sheaves). We shall prove it for any (not necessarily symmetrizable) Kac-Moody Lie algebra in §6 by using Hodge modules. 0.5. We finally remark that the Kazhdan-Lusztig conjecture for symmetrizable Kac-Moody Lie algebras is explicitly stated in Deodhar-Gabber-Kac

[DGK]. We also note that we have received the following short note announcing the similar result: L. Cassian, Formule de multiplicité de Kazhdan-Lusztig dans le cas de Kac-Moody, preprint.

1. Infinite-dimensional schemes 1.0. In this section we shall briefly discuss infinite-dimensional schemes.

1.1. A scheme X is called coherent if the structure ring Ox is coherent. A scheme X over C is said to be of countable type if the C-algebra Ox (U) is generated by a countable number of elements for any affine open subset

U of X (cf. [K2]). smooth if xy

A morphism f: X — Y of schemes is called weakly

is a flat Ox-module, where yy

is the sheaf of relative

differentials.

1.2. We say that a C-scheme X satisfies (S) if X ~ lim PA Sn for some projective system {S;}r1en of C-schemes satisfying the following conditions:

(1.2.1) S, is quasi-compact and smooth over C for any n. (1.2.2) The morphism pam: Sm — Sn is smooth and affine for m 2 n.

In particular, X is quasi-compact.

Remark that by [EGA IV, Proposition (8.13.1)], the pro-object “lim” Sy is uniquely determined in the category of C-schemes of finite type. More precisely, we have

1122)

lim Hom(Sh, Y) > Hom(X,Y) n

AND TANISAKI

KASHIWARA

162

for any C-scheme Y locally of finite type. Note that the projection pn: X — Sp is flat and and we have

(1.2.4)

OQ ~ lim(pn)*Q5,, n

where Q) = Qc: Thus we obtain (1.2.5) Q4 is locally a direct sum of locally free Ox-modules of finite rank.

We see from the following lemma that, if X is separated, we may assume that S, is also separated for any n.

LEMMA 1.2.1. Let X be an affine (resp. separated) scheme such that X =lim Sn, where {Sn}nen is a projective system of schemes satisfying the following conditions: (1.2.6) S, is quasi-compact and quasi-separated for any n. (1.2.7) Pam: Sm — Sn is affine for m 2 n. Then S, is also affine (resp. separated) for n > 0.

Proor:

Let p,: À — Sy be the projection.

(1) Assume that X is affine. We see from the assumptions that there exist an affine open covering So = UjeU; and f; € T(X;Ox) (i € I) such

that py (U;) > Xs, and X = UjerXy,, where I is a finite index set and Xs, = X\Supp(Ox/Oxf;). Setting A = [(X; Ox) and A, =T(S,;Os,), we have A = lim | An by [EGA IV, Theorem(8.5.2)], and hence there exists some n satisfying f; € An (i € I). Thus we may assume that f; € Ao from

the beginning. It is easily seen from the assumptions that (S,);, C po, Ui and An = Der Anfi for n > 0. Then (S,);, is affine, and hence S, —

Spec(A,) is an affine morphism. (2) Assume that X is separated. In order to prove that S, is separated for n > 0, it is enough to show that, for any affine open subsets U and V

of So, Pon (UNV) — por (U) x pot (V) is a closed embedding for n > 0. Since pp (U NV) is affine, pyl(U NV) is affine for n > 0 by (1), and hence we may

assume

from the beginning that U N V is affine.

Since

Os, (UNV) is of finite type over Os,(U), Os,(UNV) is generated by finitely many elements a; over Os,(U). Since pp (U NV) — po'(U) x pp! (V) is a closed embedding, (po)*a; is contained in the image of Ox(pp'(U)) ® Ox(p5 (V)) = Ox (ps (UNV)). Thus (Pon )*ai is contained in the image

of Os, (Pin (U)) ® Os, (pod (V)) — Os, (ps (U NV)) for n > 0. Therefore Os, (Pon (U)) ® Os, (por (V)) — Os. (po, (U NV)) is surjective. D

KAZHDAN-LUSZTIG

CONJECTURE

163

1.3. Let (L) (resp. (LA)) denote the category of quasi-compact smooth C-schemes and smooth (resp. smooth affine) morphisms. PROPOSITION

1.3.1. Let X be a C-scheme satisfying (S). Then “lim” Sp

as a pro-object in (LA) does not depend on the choice of the projective system {Sy }nen as in §1.2.

PROOF: It is enough to show that, for any quasi-compact smooth C-scheme Y, the natural map

(1.3.1) lim Hom(r)(Sn,¥)+{ f € Hom(X, Y) ;(f*Qy)(z) — Q(z) is injective for any x € X } is bijective. Here, for an Ox-module F and x € X, F(z) denotes F,/m,F,, where m, is the maximal ideal of Ox.. In fact, by Lemma 1.2.1, we then have

(1.3.2) lim Hom(La)(Sn,Ÿ) >{ f € Hom(X,Y); f is affine and (f*Qÿ)(x) — A} (x) is injective for any x € X }. The injectivity of (1.3.1) follows from (1.2.3). Let f: X — Y be a Cmorphism such that (f*Q},)(z) — QL(x) is injective for any x € X. Then f splits into the composition of pn: X — S, and f: Sn — Y for some n.

Since (f*QL )(pa(x)) — (f*QL)(x) is injective for any x € X, (f*QL)(s) > 2%, (s) is also injective for any s € p,(X).

Hence there exists an open

neighborhood Q of p,(X) such that (f*21)(s) — QE (s) is injective for any s € 2. Now [EGA IV, Proposition (1.9.2)] guarantees that there exists

m > n such that pz},(Q) = Sm, and hence ((f0Pam)*24)(s) — 24 (s) is injective for any s € S,,. This means that fo Pam is smooth.

LEMMA

O

1.3.2. Let f: X — Y be a morphism of C-schemes satisfying (S).

Then the following conditions are equivalent.

(i) f is weakly smooth (i.e. Qy,y is flat). (ii) For any z € X, (f*QY,)(x) Q(z) is injective. (iii) There exist projective systems {Xn}, {Yn} satisfying (1.2.1), (1.2.2)

and a morphism {fn}: {Xn} — {Yn} of projective systems such that X ~ lim

—— 1

X,,Y ~lim

ri

Y,, f =lim

—n

fh, and f, is smooth for any n.

164

KASHIWARA

PROOF:

AND TANISAKI

(i)=(ii) is evident. (iii)>(i) follows from the fact that Q}x/y is the

inductive limit of the flat Ox-modules (p,)*x

/Y.) where ph: X — Xn is

the projection. Assume (ii). By (1.2.3), there exist {Xn}, {Yn} and {fn} such that X = lim DETTE

lim Yat

= lim fn. Then we see from the

3.1) that, for any ‘n, there state some m 2 n such that the bijectivity of (1.3. composition Xm — Xn — Y\ is smooth. This implies (iii). Oo 1.4.

A C-scheme X is called pro-smooth if it is covered by open subsets

satisfying (S). LEMMA

1.4.1 (cF.

[K2]). A pro-smooth C-scheme is coherent and of

countable type. LEMMA

1.4.2.

Let f: X — Y be a smooth morphism of C-schemes.

IfY

satisfies (S), so does X. Proor: Let Y ~ lim Sp, where {S,} is as in §1.2. By [EGA IV, Then orem (8.8.2)] there exist some n and a morphism f,: X, — Sy satisfying f = fn Xs, Y. Then, by [EGA IV, Proposition (17.7.8)], fn Xs, Sm is smooth for

m > 0. O

COROLLARY 1.4.3. A C-scheme smooth over a pro-smooth also pro-smooth.

C-scheme is

1.5. We give several examples of pro-smooth C-schemes.

(a) A® = Spec(C[X, ; n = 1,2,...]) (cf. [K2]). Denoting by p, :A® — A” the projection given by (X1,...,X,), we have A® ~ lim

A”.

=n

(b) P®© (cf. [K2]): (c) Let E be a countable subset of C, and let A be the C-subalgebra of the rational function field C(x) generated by x and {(r—a)~!; a € E}. Then X = Spec(A) is a pro-smooth C-scheme and we have X(C) ~ C-E. (d) Let A be the C-algebra which is generated by the elements en (n € Z) satisfying the fundamental

Z) and € be the points of X

relations e,em

= bnmen.

Let zy (n €

= Spec(A) given by the prime ideals

A(1—en) (n € Z) and Deg ee respectively. Then X is a pro-smooth C-scheme consisting of x, (n € Z) and £. The underlying topological

space is homeomorphic to the one-point compactification of Z with discrete topology, and the structure sheaf Ox is isomorphic to the sheaf of locally constant C-valued functions.

KAZHDAN-LUSZTIG

CONJECTURE

165

1.6. A C-scheme X is called essentially smooth if it is covered by open subsets U, each of which is either smooth over C or isomorphic to W x A for a smooth C-scheme W. An essentially smooth C-scheme is obviously pro-smooth. PROPOSITION 1.6.1. IfW is a C-scheme offinite type such that W x A® is pro-smooth, then W is smooth.

PROOF:

A®

We may assume that W x A® satisfies (S). Hence we have W x

= limS, for some {S,} satisfying (1.2.1) and (1.2.2).

Then there

exist n and m such that the morphism po, :Sm — So splits into Sy — W x A”

Therefore

—

So.

Hence

W x A”

—

Spo is smooth

at the image of Sj.

W x A” is smooth and hence so is W. O

PROPOSITION 1.6.2. Let X and Y be C-schemes and let f: Y — X bea morphism of finite presentation. Assume X = W x A® for a C-scheme W offinite type. Then there exist some n and a C-morphism f': U — Wx A”

offinite type satisfying f = f' x A PROOF:

Since

(Note that we have A® = A” x AT).

X = lim W x A”, there exist some n and a C-scheme U of

finite presentation over ir

x A” such that

Y = X xy,an U by [EGA IV,

Theorem (8.8.2)]. Then f’: U — W x A” satisfies the desired condition. O COROLLARY 1.6.3. A C-scheme smooth scheme is also essentially smooth.

over an essentially smooth

C-

LEMMA 1.6.4. Any essentially smooth C-scheme is a disjoint union of open irreducible subsets. Proor: Let X be an essentially smooth C-scheme. Since X is covered by open irreducible subsets, it is enough to show that, if U is an open irreducible subset of X, then U is also an open subset of X. Let x € U and let W be an pedueible open subset of X containing x. Since WNU #90, we have W = WNU =U. This shows that U is a neighborhood of x. 0

1.7. We shall recall the definition of Dx and admissible Dx-modules for a pro-smooth C-scheme X.

AND TANISAKI

KASHIWARA

166

For a morphism f: X — Y of pro-smooth C-schemes we set (1.7.1) PE (Dxay)

=0

(n < 0),

(12722) Fh(Dx-y)

={ Pe

Homc(f~'Oy,Ox);

[P,a] € Fn-1(Dx_-y) for any

a€ Oy}

(n2 0),

(1.7.3)

F(Dx-y) = Un Fr(Dxy) C Home(f~* Oy, Ox). We have

(1.7.4)

Fo(Dxy) ~Ox,

(19725)

F\(Dx-y)

where Oxy

~Ox

:= Homy-10,(f7'QY,

80 x_y,

0x) is the sheaf of derivations.

Let Ia, denote the defining ideal of the diagonal Ay in Y x Y and set Oay(n) = OP IUA) Then Oa,(n) is locally a direct sum of locally free Oy-modules

of finite rank with respect to the Oy-module

structure induced by the first projection. Then we have

Fa Dx sy) 2H om pe.gy (F TOR, Ges Ox and hence F,(Dx_,y) has a structure of a sheaf of linear topological spaces

induced from the pseudo-discrete topology of Oay(ny (cf. [EGA 0, 3.8]). More concretely, for an affine open subset U of X and an affine open subset

VeotY such the

(M),

{PE PF,(Dx sr)

PU:)= 0G ED

form a neighborhood system of 0 in T(U; F,(Dx—y)), where {fi}hier ranges

over finite subsets of Oy(V). If g: Y — Z is also a morphism of pro-smooth C-schemes, we can define the composition

(1.7.6)

Dxay @f-'Dyiz

— Dxz.

In particular, Dx := D, id y is asheaf of rings and Dx_,y isa (Dx, f~'Dy )bimodule.

KAZHDAN-LUSZTIG

CONJECTURE

167

If Y is a smooth C-scheme, we have (18777)

Dxiy

= Ox ®;-10, fUDy:

Definition 1.7.1. Let X be a pro-smooth C-scheme.

A Dx-module M

is called admissible if it satisfies the following conditions: (1.7.8) For any affine open subset U of X and any s € T(U;M), there exists a finitely generated subalgebra A of ['(U; Ox) such that Ps = 0 for any P €T(U;D x) satisfying P(A) = 0.

(1.7.9) M is quasi-coherent as an Ox-module. The condition (1.7.8) is equivalent to saying that Dx acts continuously on IN with the pseudo-discrete topology. 1.8. Let f: X — Y be a morphism of pro-smooth C-schemes. Then for any admissible Dy-module MN, f*MN = Ox ®;-10, f719 has a structure

of Dx-module. Moreover f*® is admissible (cf. §1.9). 1.9.

Let X be a C-scheme satisfying (S). Let {S,}nen be a projective

system as in 81.2 and let p, : X — S, be the projection. Then we have

(1.9.1) FiDx=s,)= lim pr Fe(Ds,,+5,) SO. (1.9.2)

Fy,(Dx)

& lim Fi(Dx—s,

rs Fi(Ds.),

).

n

If M is an admissible Dx-module locally of finite type, then there exist some n and a coherent Ds, -module ® such that

(193)

* Fe M~Be (pPn)"N=Ox @-r9, Pa-1

N~Dxas, @,-1p, Pn-1 À ~S

Conversely, for a quasi-coherent Ds,-module M, the Dx-module (p,)*N is an admissible Dx-module.

1.10.

Let X be a pro-smooth C-scheme.

A Dx-module

M is called holo-

nomic (resp. regular holonomic) if it satisfies the following condition: (1.10.1) For any point x € X, there exist a morphism f: U — Y from an open neighborhood U of x to a smooth C-scheme Y and a holonomic (resp.

regular holonomic) Dy-module M such that IN|U is isomorphic to f*N. Let Xe lim Sn, where {Sn}nen is a projective system as in 81.2 and

let MN be a coherent Ds,-module. It is seen that, if (po)* is a holonomic (resp. regular holonomic) Dx-module, then (pon)*® is a holonomic (resp. regular holonomic) Ds, -module for any n. n

.

.

KASHIWARA

168

AND TANISAKI

2. The analytic structure on C-schemes 2.0. In this section, ringed spaces, schemes and their morphisms are all over C. 2.1.

Let X be an affine C-scheme.

as follows.

We define a local ringed space Xan

The underlying set of Xan is the set X(C) of the C-valued

points of X. The topology on Xan is the weakest one such that, for any

f € Ox(X), f(C): X(C) — C is continuous with respect to the Euclidian topology on C. We define the sheaf of rings Ox,, on X(C) by

(1)

Ox,, = lim f(C)~"Os,,, j

where f: X — S ranges over morphisms with schemes S of finite type as targets. Here S,, denotes the complex analytic space associated to S.

2.2.

More generally, let X be a C-scheme.

We endow with X(C) the

weakest topology such that, for any affine open subset U of X, any open subset of Uz, is open in X(C). Let Xa, denote this topological space. We define the sheaf of rings Ox,, by Ox,,|Uan = Ou,, for any affine open subset U of X. Then we can check easily the following.

LEMMA 2.2.1. (i) The ringed space (Xan, Ox,,) is well-defined. (ii) The correspondence X +> (Xan, Ox,,) is functorial.

(iii) (X x Y)an = Xan X Yan as a topological space. (iv) If X — Y is an open (resp. closed) embedding, so is Xan — Yan.

(v) Let {Xh},en be a projective system of C-schemes such that Xm — X, is affine for m 2 n. Then we have (lim >, |

Ne lim(Xn Jan as a ringed

space.

(vi) IfX is separated, then Xan is Hausdorff. (vii) If X is quasi-compact and of countable type, then Xan has a countable base of open subsets.

2.3. There exists a natural morphism of ringed spaces

(2.3.1)

C= Ux

en > oe

functorial in X, and we have a natural tx'Dx-module structure on Ox,,.

2.4. A quasi-compact stratification of a C-scheme X is a locally finite family {X,} of locally closed subsets of X such that

KAZHDAN-LUSZTIG

CONJECTURE

169

(2.4.1) X = LUX'as'a set, (2.4.2) XaNXg £0 implies X4 D Xz, (2.4.3) The inclusion X, — X is a quasi-compact morphism. 2.5. Let X be a coherent C-scheme and let k be a field.

Definition 2.5.1 A sheaf F of k-vector spaces on Xan is called weakly constructible if there exists a quasi-compact stratification X = UX, such that F|(Xq)an is locally constant. If moreover F, is finite-dimensional for any z € Xan, we call F constructible. Let D(Xan;k) be the derived category of the category of sheaves of kvector spaces on Xan. An object K of D(Xan;k) is called constructible (resp. weakly constructible) if it satisfies the following conditions.

(2.5.1) H”(K) is constructible (resp. weakly constructible) for any n. (2.5.2) For any quasi-compact open subset 2 of X, H"(K)|Qan = 0 except for finitely many n. We denote the full subcategory of D(Xan;k) consisting of constructible

(resp. weakly constructible) objects by D.(X;k) PROPOSITION Xn

=

2.5.2.

W x A”

(resp. Dw.c.(X;k)).

Let W be a smooth C-scheme.

and let pn:

X —

X,

Set X = W x A,

be the projection.

Then for any

K € Ob(D,(X;k)), there exist some n and K, € Ob(D,(X,; k)) satisfying (fy a lee PRooF:

We can take a finite coherent stratification X = UX, such that

Hi(K)|X¢q is locally constant for any j and a. Then there exist some n and a stratification X, = UX, such that X, = DA Aas Let 7: X, — X, x AT be the embedding by the origin € A®. Since X ~ X, x A® and (A™)an

is contractible to the origin, we have K ~ (ph); Kn with Kn = (ian) 71K. O PROPOSITION 2.5.3. Let X = A®° x W, where W is a smooth C-scheme, and let p: X — W be the projection. Then for a cohomologically bounded object K of D(Wan;k), we have

RHom((pan)~ 1K, kx,,) © (Pan) ‘RHom(K, kw,,). ProoFr:

We shall show that the functorial morphism

(Pan) RH om(K, kw) — RHom((pan)~'K,kx,,)

170

KASHIWARA

AND TANISAKI

is an isomorphism. In order to see this, it suffices to show that

RI‘((Pan)'V; (Pan) *RHom(K, kw,,.)) — RT((pan)” V: RHom((pan)' K, kx.) is an isomorphism for any open subset V of Wan (Observe that U x(A™)an x

V form a base of open subsets of (A® x W)an, where A

~ A” x AW, U

is an open subset of (A”)an and V is an open subset of Wan). This follows

from the following lemma. LEMMA

2.5.4.

Let X, Y and S be topological spaces and let py:

X — S

and py: Y — S be continuous maps. Let p: X x [0,1] — X be the projection, and let h: X x[0, 1] — Y be a continuous map satisfying py oh = px op. Define iy: X — X x [0,1] (v = 0,1) by i,(x) = (x,v), and set

fy =hoi,.

Let K (resp. F) be a cohomologically bounded (resp. lower

bounded) object in the derived category of the category of sheaves of kvector spaces on S and let f! be the composition of

R(py )«RHom(py'K,py’ F) >R(py )«R(fv)«RHom(f; "pyK,f, *py' F) —R(px)«RHom(px' K,px' F). Then we have fi = lic

PRoor:

Set Z = X x [0,1] and pz = py op. Then f! is obtained by

R(py )«RHom(py'K,py F) +R(py )«Rh,RHom(h~'py'K, h- py!F)

~R(pz)«RHom(pz'K,pz’ F) —R(pz)«R(i,)«RHom(i;*pz'K,i; 'pz’F) CR(px).RHom(px K, px’ F). Set K = px K and F = nee to show that the morphism

Since R(pz). = R(py)«Rp,, it is enough

il: Rp,RHom(p"'K,p"'F) +Rp,R(i,)«RHom(iz!p-!K i; 1p71F) ~RHAom(K, F) does not depend on v. Since

p*: RHom(K, F) -Rp,RHom(p7!K,p7!F) ~RHom(K, Rp,p !F) is an isomorphism and 7! o p* = id, we obtain the desired result. O For a quasi-compact separated essentially smooth C-scheme X we set (2.5.3) Dx(K) = RHom(K, kx,,)

for K € D.(X;k).

KAZHDAN-LUSZTIG COROLLARY 2.5.5. C-scheme. Then

CONJECTURE

171

Let X be a quasi-compact separated essentially smooth

(i) Dx preserves D,(X;k). (ii) Dy

2.6.

oD,

Ad:

Let X be a quasi-compact separated essentially smooth C-scheme.

Define full subcategories PDE(X; k) and PDÈUX; k) of D-(X;k) by (2.6.1) K belongs to PDÉ(X; k) if and only if codim Supp H"(K) = n for any n. (2.6.2) K belongs to PDÈ(X; k) if and only if D x(K) belongs to PDÉ(X; k). The following theorem is similarly proven as in the finite-dimensional

case (see [BBD], [KS]), and we omit the proof.

THEOREM 2.6.1. (i) (?D=°(X;k),’ D2°(X;k)) is at-structure of D,(X;k). (ii) For K, € Ob(? D=£°(X;k)) and Ky € Ob(PD2°(X;k)), we have H"(RHom(K1,K2))

—10 (n < 0).

(iii) Perv(X;k) =? D2°(X;k) MN?D2°(X;k) is a stack, ie. (a) For K,,Kz € Ob(Perv(X;k)),

U+

Hom(K1|UVan, K2|Uan) is a sheaf

on X.

(b) Let X = U;U; be an open covering. Assume that we are given objects K; of Perv(U;;k) and isomorphisms f;;: K;|\(Ui)an N (Uj )an — Fegan

M AS

such

that

IG o fix

=

Hi

ON

Coe lan N QUES

M

(Ux)an. Then there exist K € Ob(Perv(X;k)) and isomorphisms PARU — 1, such that y; 0); fp om (Up)an 1 (U; Jan: We call an object of Perv(X;k) a perverse sheaf.

When X is smooth,

this definition coincides with the one in [BBD] up to shift. PROPOSITION 2.6.2. Let X be a separated essentially smooth C-scheme such that X ~ lim Xh for some projective system {X, } satisfying (1.2.1)

and (1.2.2). Then ‘we have Perv(X;k)

~ lim Perv(Xn;k); Le. the follow-

ing two properties hold. (2.6.3) For lim

M,, Mo € Perv(Xn;k) we have

Hom((Pam

)* Mi ) (Pnm)*

M2) = Hom((pn)*

Mi, (Pn)* Mo).

(2.6.4) For anyM € Perv(X;k) there exist some n and M, € Perv(Xn;k) such that M ~ (ph)

Mn.

172

KASHIWARA

AND TANISAKI

Here, Pam : Xm — Xn and pn: X — X, are the projections.

2.7. Let X be a C-scheme satisfying (S) and let {S,},en be a projective system as in §1.2. We denote by pam: Sm — Sn (m2 n) and pn: X — Sy

the projections.

Let Be

be the sheaf of (p,q)-forms on (Sh)an with

hyperfunction coefficients. Then we have natural homomorphisms (2-721)

Qt).

(paves

gin De

sa

Por

a

BE

De ee aa Be ye

By (2.7.1) we obtain a sheaf Bee) Se linn(ngan Bes on Xan, and this (Sala does not depend on the choice of {Sn}nen by Proposition 1.3.1. the inductive limit in (2.7.2) with respect to m, we obtain

(2.7.3)

Lz Dx So

PIE Oyse

Taking

ye

Taking again the limit in (2.7.3) with respect to n, we obtain

(2.7.4)

ty Dx XBL?) — BYP);

and this defines a structure of an 1x Dx-module on Be (2.7.5)

lim(Pm an

=~Hom,-1p,

Hom,-1 pe, (Cy

=1

(x

DY ae,

Denes.

We have also

De

(0,p) 1

yD TE

2.8. More generally, let X be a pro-smooth C-scheme. We can patch the sheaves Bape for affine open subschemes U of X satisfying (S), and obtain a sheaf De

(2.8.1)

on Xan Such that

Be

EE.

We can define the derivatives

(2.8.2) (2.8.3)

2: BY _, Botha) 0 DER eu

)

KAZHDAN-LUSZTIG

CONJECTURE

173

and we have the exact sequence:

(2.8.4)

JO

te Be

RO.

of 1x Dx-modules. The Dolbeault complex:

Dr0,0) is denoted by By

9

à D (0,1) one

.

2.9. Let X be a pro-smooth C-scheme.

For a holonomic Dx-module M,

we set

(2.9.1)

Sol(M)

= Homp, (M, By, )(= Hom,-1p, CU

and regard this as an object of D(Xan;C). When X is smooth, we have Sol(N)

= RHomp, (M, Ox,,)

by [K1]. Let U be an open subset ofX satisfying (S) and let {S,} be a projective system of C-schemes satisfying (1.2.1), (1.2.2) such that U ~ limS,. Then we have IN|U = (p,)*N for some n and some holonomic Ds,,-module A, where p,,:

U — S, is the projection. By (2.7.5) we have

(2.9.2) Homp, (M, Bi”) ~Homp, (Pus, QDs, 26 BaD ~Homp,, (A, Hompy (Dus,

m0

0)

~lim Homps, (8, (Pmt Hompe, (Din Sn 89), ) ~lim(Pm)at Home, (Pam)9 Bis)... m

On the other hand we have

(2.9.3)

H"(Hompg,, ((Pnm)M B(s,,)..)) wEzth, ((Pnm)M,O(Sm)an) (Pam)

Path» (A, O(S, Jan )-

174

KASHIWARA

AND TANISAKI

Thus (2.9.4)

H4(Sol(M)|Van) + H4((Pn)an SON),

and we finally obtain

(2.9.5)

Sol(IN)|Uan X (Pn)ad Sol(M)

in D>(Uan;C).

This shows in particular LEMMA 2.9.1. Let X be a quasi-compact separated essentially smooth Cscheme. IfM is a holonomic Dx -module, then Sol(IN) is a perverse sheaf, and Sol is a contravariant exact functor from the category of holonomic

Dx-modules to Perv(X). 3. Mixed Hodge modules on essentially smooth C-schemes 3.0. We shall study mixed Hodge modules on essentially smooth Cschemes. In this section all schemes are over C and assumed to be quasicompact and separated.

3.1. In [S], M. Saito constructed mixed Hodge modules on finite-dimensional manifolds. In his formulation, the weights behave well under direct images, but not under inverse images. Since we treat infinite-dimensional manifolds, we have to modify his definition so that the weights behave well under inverse images. 3.2. Let X be a quasi-compact essentially smooth C-scheme. Let MFW(X) be the category consisting of M = (MN, F, K, W,c), where

(3.2.1) M is a regular holonomic Dx-module,

(3.2.2) F is a filtration of M by coherent Ox-submodules which is compatible with (Dx, F),

(3.2.3) W(M) is a finite filtration of M by regular holonomic Dx -modules, (3.2.4) K is an object of Perv(X;Q), (3.2.5) W(K) is a finite filtration of K in Perv(X;Q), (3.2.6) s is an isomorphism Cx @g, K = Sol(9) in D.(X; C), compatible with W; i.e. « induces a commutative diagram

Cx Sox We(K) ——> So(M/W_x-1(9))

Cx Oo

es Me ON

KAZHDAN-LUSZTIG

We define morphisms of

CONJECTURE

175

MF W(X) so that M + K € Perv(X;Q) is a

covariant functor and M + 9 is a contravariant functor.

Sometimes, ¢ in (IM, F, K,W, 1) will be omitted.

3.3. Let X be a smooth C-scheme and let MHM(X) be the category of mixed Hodge modules on X defined in Saito [S]. We define a contravariant functor

(3.3.1)

ex: MHM(X) — MFW(X)

as follows.

Let M

= (M,F,K,W)

be an object of MHM(X)

and let

Dx(M) = (9*,F,K*,W) be the dual of M (cf. [S]). Then we define pex(M)=(N, F, K, W) by

G32)

Ro

(3.3.3)

So (oe no

ED = ON) Ce MON

à

(3.3.4)

Wi (2) = Wesaim x (M") So, (QE) eat,

(3.3.5) (3.3.6)

K = K[- dim X], W,(K) = Wesaim x(K)[- dim X].

Note that Mis a left Dx-module since M and M" are right Dx-modules. Let MH M(X) be the image of yy. It is a full subcategory of MFW(X),

isomorphic to MHM(X). We define

(3.3.7)

ex: D'(MHM(X)) > D'(MHM(X))

by M’ +> px(M')[dim X]. Hence yy is compatible with

ix: D'(MHM(X)) > D.(X;Q) and ix : D'(MHM(X)) — D.(X;Q). The duality functor Dx on MHM(X)

(3.3.8) by Dx

defines the duality functor

Dx: MHM(X) — MHM(X)°P 0 vx

=

yx o Dx.

Then

we have ix oDx

Dx(K) = RHom(K, Qx,,) for K € Perv(X;Q).

=

Dx 07x,

where

176

KASHIWARA

AND TANISAKI

3.4. For a morphism f: X — Y of smooth C-schemes, we define functors

(3.4.1) (3.4.2)

f",f': D'(MHM(Y)) — D'(MHM(X)) ff: D'(MHM(X)) — D'(MHM(Y))

using those defined in [S] and the isomorphism (3.3.7). In particular, if f: X — Y is smooth and M = (M,F,K,W) then we have

(3.4.3)

is an object of MHM(Y),

f*(M) = (FM, F, f*K,W) € Ob(MHM(X)),

where

(3.4.4) (3.4.5)

FFM) = FN), Wi(FM) = FMC), Wel K) = F'WE (KR).

We extend this definition when X is essentially smooth, Y is smooth and f is weakly smooth. Hence in this case, f* is a functor from MHM(Y)

into

MF W(X) defined by (3.4.3), (3.4.4) and (3.4.5).

3.5. For an essentially smooth C-scheme X, we define a full subcategory

MHM(X) of MFW(X) as follows. An object M of MFW(X) belongs to MHM(X) if and only if X is covered by open subsets U such that there are a weakly smooth morphism f: U — Y to a smooth C-scheme Y and an

object M’ of MHM(Y) satisfying M|U = f*M'. We can easily see that MHM(xX) is a stack. In this paper we call objects of MHM(X) mized Hodge modules on X. Note that MH M(X) is an abelian category. We can define the duality functor

(3.5.1)

Dx: MHM( + MHM(X X) )

by Dx M|U = f*DyM’. Hence this extends to

(3.5.2)

It is an exact functor satisfying Dy o Dy = id.

Dx: D'(MHM(X)) > D'(MHM(X))®.

3.6. Let X be an essentially smooth C-scheme satisfying (S) and let {Sn}nen be a projective system as in §1.2. Then we have

(3.6.1)

MHM(X) = lim MHM(S,),

(3.6.2)

D'(MHM(X)) = lim D'(M H M(S)) n

KAZHDAN-LUSZTIG

CONJECTURE

177

(cf. Proposition 2.6.2). 3.7. For a morphism f: X — Y of essentially smooth C-schemes satisfying

(S), we define

(35721)

f*: D'(MHM(Y)) > D'(MHM(X))

as follows. Let X ~ lim AA and € lim Yn, where {X,} and {Y,} satisfy (1.2.1) and (1.2. 2). We may assume that there are morphisms fn: Xn — Yan € N) such that f = lim fn. Let py,: X — Xn and Py.,: Y — Yn be the projections. For a bounded complex M' of mixed Hodge modules on Y, there exist some n and a bounded complex M, of

mixed Hodge modules on Y, such that M° = (p,,)*M,. Then we define

(3.7.1) by

(3.7.2)

FM = (pxn) (A) M).

It is easy to check that this is well-defined.

3.8. Let f: X — Y be a morphism of essentially smooth C-schemes. Then, for each 7 € Z, we can define a functor

(3.8.1)

H' f*: MHM(Y) > MHM(X).

In fact, locally on X, (H'f*)(M) is defined as H'*(f*(M)), and they can be patched together. It satisfies the following properties:

(3.8.2) If f is weakly smooth, then (H° f*)(M) is given by (3.4.4) and (3.8.3) If (H* f*)(M) = 0 for i # p then we have (H'g*)(HP f*)(M) =

we have (H'f*)(M) = 0 for i £ 0, and (3.4.5). and if g: W — X is another morphism, (H'*?(f 0g)*)(M).

3.9. Let f: X — Y be a morphism of finite presentation. Assume that Y

satisfies (S), so that X also satisfies (S). Then we define

(3.9.1) (3.9.2) (3.9.3)

fx: D'(MHM(X)) > D'(MHM(Y)), fi: D'(MHM(X)) — D'(MHM(Y)), f': D'(MHM(Y)) — D'(MHM(X))

as follows. Let X = lim Aevandsy a lim Yn, where {X,} and {Y,} satisfy (1.2.1) and (1.2. 2.2), and let M° Beta a bounded complex of mixed

178

KASHIWARA

AND TANISAKI

Hodge modules on X (resp. Y). We may assume that there exists a y,Y and f = foxy,Y. Set fn = morphism fo: X4 — Yo such that X ~ Xx

fo Xy, Yn. We may further assume that there exists Loa): 6 4 xy, Yn} — {Xn} (resp. {hn}: {Xn} — {Xo xy, Yn}) such that limg,= idx (resp. lim hn = idx). Let px,: X — Xn and pyyn:Y > Yn abe the projections. There exist some n and a bounded complex M,, of mixed Hodge modules on X, (resp. Y,) such that M° = (pxn)*M, (resp. M = (pyn)*M,) .

Then we define (3.9.1), (3.9.2) (es, (3.9.3)) by

(3.9.4)

LM = (Pyn)* ¥y,(fn)*(9n)* px. (Ma);

(3.9.5)

LM = (py n)* Py, (fn) (an) ex, (Ma)

(3.9.6)

(resp. F'M° = (pxin)" 9x, (ln) (fn) ¥y, (Ma) ).

It is easy to check that they are well-defined. We have the following properties concerning them.

(3.9.7) The functor f, (resp. f') is a right adjoint functor of f* (resp. fi). (3.9.8) de o D x La Dy

o fie

(3.9.9) If f is proper, then we have f, = fi. Note that (3.9.9) follows from the fact that f, is proper for n > 0 if f is proper. 3.10. For an essentially smooth C-scheme X, we define

(32021) by

QH = (Ox, F,Qx,W,0) € Ob(MHM(X))

CT $3 cw mana 8 CD

KAZHDAN-LUSZTIG

CONJECTURE

179

(3.10.5) 1: Cx So, Qx — Sol(Ox) is induced by 1+>+ id € Homp, (Ox,Ox) C

Homp, (Ox, Bo.

Set (pt) = Spec(C) and let ax: X — (pt) be the projection. We shall identify M H M((pt)) with the category MHS of mixed Hodge structures. Then we have QË = (ax)*Q”, where QÂ is the trivial mixed Hodge structure on Q.

3.11. Let X and Y be essentially smooth C-schemes and lett jac Y e(w) Bwai.

(iii) g7 }(Bwa;) = Xv LU Xu.. (iv) g; induces an isomorphism Bwxo © Bwza; for {(ws;) < {(w). (v) q; induces an A!-bundle Bwzy — Bwa; for (ws;) > {(w). LEMMA

4.4.3.

Any B-invariant

quasi-compact open subset of X or X*

satisfies (S). PROOF: The proof being similar, we shall prove the theorem only for X. Let ( be a B-invariant quasi-compact open subset of X. Then there exists

a finite subset J of W such that

Q =U,

,;Bwzro = U,¢;wBuxo. Let © be

a subset of At such that At \ © is a finite set, (© + 0)N At C © and w-!© c At for any w € J. We denote by Vo the closed subgroup of U

corresponding to Ng = ) ace Ga; i.e. Vo = lim exp(no/(adn)*ng).

For

w € J the action of Ue on wB2p is equivalent to the action of w-'Uew(C U) on Bzo, and hence Uo acts on wBzx, freely. Thus 2/U@ exists and it is a quasi-compact smooth C-scheme. Since 2. ~ lim, Q/Uo, the assertion follows from Lemma

1.2.1. 0

4.5. Let us recall the results of [K3]. Assume that g is symmetrizable until the end of §4. For À € P, let Ox (A) be the corresponding invertible Ox-

module. Set D, = Ox(1)8o, Px 80, Ox(—A) and F(X) = Ox (So, F for an Ox-module F.

Note that, if IN is a Dx-module,

then M()) is a

D\-module. For w € W set By = Hx” (Ox), where £(w) is the length of w. Let M,, be the dual of the Dx-module B,, and let £,, be the image of

184

KASHIWARA

AND TANISAKI

the unique non-zero homomorphism MN, — By. Then £, is the minimal extension of B,,|wBzo.

4.6. For À € b* let M(A) be the Verma module with highest weight À, M*(À) the b-finite part of the dual of the Verma module with lowest weight —) and L(X) the image of the unique non-zero homomorphism M(A) —

M*(A). Then L(A) is the irreducible module with highest weight à. Set w o À = w(À + p) — p for w € W and À € b*, where p is an element of * such that (h;,p) = 1 for any 7. 4.7.

Let

\ € Py = {A € P;(h;,A) 2 0 for any i}. For a B-equivariant

D-module M we set

(4.7.1)

H"(X;M) = @yep lim(H"(2;M)),,

(4.7.2)

T(X;M) = H°(X;M),

2

where 2 ranges over B-invariant quasi-compact open subsets of X, and for a semisimple §-module M the weight space with weight y is denoted by

M,. By [K3, Theorem 5.2.1] we have

(4.7.3)

H"(X;M) = 0 for any n £ 0,

(4.7.4) (4.7.5) (4.7.6)

TX; Mu (A)) = M(wod), P(X; Bu (À)) = M*(woA), PLE A) SS (wo):

4.8. Our main theorem is the following. THEOREM 4.8.1. Let g be asymmetrizable Kac-Moody Lie algebra. Then, for \ € P, and w € W, we have:

ch L(wod) = S>(-1)8)-"Q,, ,(1)ch M(z 0 A), z2w

where Qw,z is the inverse Kazhdan-Lusztig polynomial (see [KL2] and §5.3 below). In order to prove this theorem, it is sufficient to show that, for any B-invariant quasi-compact open subset 2, we have:

(4.8.1)

[Lw|Q] = $5(-1))-*)Q,,,.(1)[BalQ] z2w

KAZHDAN-LUSZTIG

CONJECTURE

185

in the Grothendieck group of the abelian category of B-equivariant holo-

nomic Da-modules.

Note that M(w o À) and M*(wo

À) have the same

characters. Since we have

(4.8.2) (4.8.3)

Sol( Lu) = "Cx, [-L(w)], Sol(B,) = Cx, [-2(w)],

this is again reduced to:

(4.8.4)

(Cx, Eu)

= D9 (-1)- “Qu, (DICx, AIO] z2w

in the Grothendieck group of the abelian category of B-equivariant perverse sheaves on (2. The last statement will be proven for any (not necessarily symmetrizable) Kac-Moody Lie algebras in §6 by the aid of mixed Hodge modules.

5. Hecke-Iwahori Algebras 5.0. In this section W denotes a Coxeter group with canonical generator system S. The length function and the Bruhat order on W are denoted by £ and 2, respectively.

5.1. The Hecke-Iwahori algebra H(W) is the associative algebra over the

Laurent polynomial ring Z[q,q~+] which has a free Z[q, g~']-basis { Tw }}wew satisfying the following relations:

(5.1.1) (5.1.2)

(T;+1)(T; -q)=0 ag

ES == TERRE

if

for

seES,

£(w;) + £(we) =

L(w, W2).

Let ht h be the automorphism of the ring H(W) given by

(5.1.3)

fi

Dake

and define Ryw € Z[q,q~'] for y, w € W by

(5.1.4)

D

DENT: By yew

The following is easily checked by direct calculations (see [KL1]). (5.1.5) Ry w # 0 if and only if y £ w. (5.1.6) Ry,w is a ploynomial in g with degree €(w) — (y) for y £ w. (5.1.7) Row = 1.

Following [KL1] we introduce a free Z[q, q~']-basis {Cu }wew of H(W).

186

KASHIWARA AND TANISAKI

PROPOSITION 5.1.1 ([KL1]). For w € W there exists a unique element

Cu = D (-9) TOP,wTy € H(W) y£w satisfying the following conditions: (a)

vil

(b) If y< w, Py is a polynomial in q with degree £ ({(w) — £(y) — 1)/2.

(C) Cul

or OG:

We set Py » = 0 ify £ w.

5.2. Set H*(W) = Homzy-1(H(W), Z[q,q7"]). For w € W let Sy be the element of H*(W) determined by (Saal) = DT

(5.2.1)

glee

where ( , ) denotes the natural paring of H*(W) and H(W). Any element of H*(W) is uniquely written as a formal infinite sum >>, cy du Sw (dw €

Z(q,q~*}).

Define an endomorphism u +> @ of the abelian group H*(W) by

(5.2.2)

(a, h) = (u, h)

(ue H*(W),h € H(W)).

We also define a right H(W)-module structure on H*(W) by

(5.2.3)

(u- hy, he) = (u, hy he)

(u € H*(W), hy,ho € H(W)).

We can check the following lemma by direct calculations.

LEMMA 5.2.1. (i) Dowew Au Sw = wew qs (ii) For s € S we have (DE you):

Ls

Sy ((q — l)aw + dws)Sw

wEew

(ili) (u,h) = e(u-h)

ws>w

Ay Ry-1 w-1) Sw.

+

Ne Gays

(ue H*(W),h € H(W)), where e: H*(W) 4 R

is given by D _vew Guu

= Ger

5.3. For w € W we define an element D, of H*(W) by

(5.3.1)

Sw.

ws 7 of the ring R satisfying

(5.3.6)

Pele

Mai

Gehan

ete ete

Gg

Set Hr(W) = R@z,, )-1 H(W) and HR(W) = Homr(Hr(W),R). Similarly to (5.2.2) and (5.2.3), we have an involution u + @ of HE(W) and a right Hp(W)-module structure on Hp(W).

PROPOSITION

5.3.1. Let w € W. If Dy, = 3 ,>u

Qu: (Qu, € R) is

an element of H}(W) satisfying the following conditions (a), (b), (c), then

we have Di, = Dw.

(a) Quw =1. (b) Qu, € ®isez)—ew)-1% for z > w.

() Dy = es, PROOF:

We shall show Qt,,= Qu,z for z 2 w by induction on é(z) — £(w).

If £(z) — £(w) = 0, we have w = z, and the assertion is trivial. Assume that

z > w. By Lemma 5.2.1 (i) we have Re

me ge) v2w

>

Oo Hessen ey,

v>y2w

and hence (c) implies :

QUENTIN

ME OO, Re). 2>y2w

188

KASHIWARA

AND TANISAKI

By the inductive hypothesis we have

(5.3.7)

Qu

ge)

= gta) Kw)

a

Qing Nyasa

z>y2w

On the other hand (b) implies

(5.3.8) (5.3.9)

Qu © Bise(2)-ew)-1 Ris COROT

E @i2e(z)-e(w)41 Ri,

and hence the equation (5.3.7) uniquely determines Qi,,. Since Qw,z also satisfies the same equation, we have Qu 2 = 0.2

6. Hodge modules on flag varieties

6.0. In this section we shall give a proof of (4.8.4) for any (not necessarily symmetrizable) Kac-Moody Lie algebra g. For an abelian category A we

denote its Grothendieck group by K(A). 6.1. Set R= K(MHS). The abelian group R is naturally endowed with a structure of commutative ring with 1 via the tensor product. Since MHS is an Artinian category, R has a free Z-basis consisting of simple objects. For 7 € Z we denote by R; the Z-submodule of R generated by the elements corresponding to pure Hodge structures of weight 7. Since any simple object of MHS is a pure Hodge structure, we have (6.1.1)

R=

@jezh,,

RR;

G HA

In the following we regard Z[q, q~'] as asubring of R via g' = [Q¥ (—i)] € Roi, where Q# (—i) is the pure Hodge structure of weight 2i obtained by twisting the trivial Hodge structure Q”. Let r + F be the involutive automorphism of the ring R induced by the duality operation in MHS. Then we have

(6.1.2)

R = Rage

hadi

and hence the ring R satisfies the condition (5.3.6). 6.2. We have a natural R-module structure on K(MHM®#(X,,)) for w € W.

KAZHDAN-LUSZTIG

CONJECTURE

189

Lemma 6.2.1. (i) Any object M of MHM®(X,,) is isomorphic to a constant B-equivariant mixed Hodge module Q# @ L (= (ax,,)*(L)) for some LE Ob(MHS). (ii) K(MHMB(X,)) is a rank one free R-module with basis [QX.]. PROOF: This follows from Theorem 3.13.3 because the isotropy group with respect to the action of B on X, is connected. O

6.3. We say that a subset J of W is admissible if J is a finite set satisfying the condition:

(6.3.1)

weJ,

ySuayed.

We denote by C the set of admissible subsets of W. For a subset J of W set Qs; = UwesXw- By [K2] we see that Qy is a quasi-compact open subset of X if and only if J is admissible. For admissible subsets J;, J2 satisfying J; C J2, we have a natural functor and a natural homomorphism

(6.3.2) (6.3.3)

MHM®(Q,,) — MHM®(Q,,) K(MHM®(Q,,)) — K(MHMP(Q),))

by the restriction, and they give projective systems {MHM(Q))}sec and {K(MHMP(Qy))}sec.

(6.3.4)

Set

MHMP(X)=

lim MHMP(Q), JEC

(6.3.5)

K?(X) = lim K(MHM*(Q))), JEC

and let py: KB(X) — K(MHM(Q))) KB(X)

be the projection. The R-module

may be regarded as a completion of the Grothendieck group of

MHMP(X). Let is: Xy — X be the inclusion.

Let w

J, and let 5, 7: Xw — (y be the inclusion.

€ W and J EC

such that w €

We define objects (GL)QË,

and "Q# of D'(MHMP(X)) by (6.3.6)

(6.3.7) QE

(Ga) Q%, O7 = (in 7) (QX,)s

(Qs = ( the minimal extension of Q$e with respect to 1,, 7).

KASHIWARA

190

AND TANISAKI

Set [M] = yez(—1)*[H*(M)] for M € Ob(D'(MHM(Q:))).

We have

elements [(i,): Q%.] and ["Q¥] of KP(X) satisfying

(6.3.8) (6.3.9)

Pi ([(iw) OX) = [Gu) QX, [2a], ps (7 Q¥,)) = [OX [Ou].

We nextly define an R-homomorhism

(6.3.10)

Gj) K(X) ak

(MM

OG,))

as follows. For an admissible subset J such that w € J, we have an Rhomomorphism

(iy 7)": K(MHM®(Q)3)) + K(MHM*(X,)) given by

(iva) (LMD) = DOD) M)

(M € Ob(K(MAMP(Q5)))),

kez

and (6.3.10) is defined by

(6.3.11)

Cu) (mn) = (w,r) (pr(M)).

For m € K®(X) and w € W we define yy(m) € R by

(6.3.12) (see Lemma 6.2.1).

HR(W) by (6.3.13)

(tw)*(m) = pu(m)[QX,] We also define an R-homomorphism

y: KB(X)

—

p(m) = aS ATOS wEew

LEMMA 6.3.1. (i) 9([(tw):Q¥]) = Su (ii) ç is an isomorphism of R-modules.

PRooF: (i) is clear. Let us show (ii). Let J be an admissible subset of W. Since MHMP(Q;) is an Artinian category, its Grothendieck group

hasa free Z-basis consisting of the simple objects. Since any simple object

of MHM® (Qj) is isomorphic to ("QE |Q7)[-€(w)] ® L for some w € J

KAZHDAN-LUSZTIG

CONJECTURE

191

and some simple object L of MHS, we see that K(MHM(Q))) is a free R-module with basis {["Q¥ [y]; w € J}. Since we have

[QE 124] € [Céw): Q¥, 12s] + JS Ri): QF 123] for w € J, {[(iw):Q¥, [y]; w € J} is also a free basis of the R-module K(MHM®*(Qj)). Therefore the assertion follows from (i). 0 6.4. We shall define an R-homomorphism

(6.4.1)

7%: KB(X) — KB(X)

for each 71 = 1,...,£ as follows. Let C; be the set of admissible subsets J ofW such that ws; € J if w € J. For J € G let q;, ;: Q7 — qi(Qz) be the restriction of g;: X — X° and define an endomorphismr,ig Of the

R-module K(MHM(Qj)) by

(M)

= (ar) (is) M]

for

M € Ob(MHM*(Q))).

Since q;7 is a B-equivariant Pl-bundle, 7; J is well-defined.

Then we de-

fine an endomorphism 7; of K?(X)= lim | 6, K(MHM®(Qj)) by 7% = lim, Ti y-

LEMMA

6.4.1. g(ri(m)) = g(m):(Ts,

ProoFr:

By Lemma 5.2.1 (ii) and Lemma 6.3.1 it is sufficient to show

7i([(éw): QX,]) =

|

S

+ 1) form € KP(X).

us) Q¥,, J+ (iw): Q¥,]

.

; h QD )

1+[Gu s QE. Gu

(wi < w)

(us>:w).

Let J € C; such that w € J. Set X,= qi. HOM: ae pseond ie let Jw,J: X, — {y be the inclusion. Since a. qi(Xw ) is a Pl-bundle

and since X, — qi(Xw) is an isomorphism (resp. A‘-bundle) for ws; < w (resp. ws; > w), we have

(jw,3)!Q%

>) laze ry (2 us = (4:,7)" (4 1( (wt)!!) ,7 Qx,,) de

(ws; < w)

(ws>;w).

192

KASHIWARA

AND TANISAKI

On the other hand, if ws; > w, we have an exact sequence:

0

(ins, 7)QE -€(w)—1] — (tw, 3):QE [-€(w)] > Gw,7): 9%, [-&(w)] — 0

in MHM(Q,). Hence the assertion is clear. O By Lemma 6.3.1 and Lemma 6.4.1 we can define a right H(W)-module structure on KP(X) by

(6.4.2) 6.5.

m-(T,, +1)=7(m)

(me K?(X)).

We denoteby m ++ m* the endomorphisms

K®(X) and K(MHM®(X,))

of the abelian groups

induced by the duality operation of mixed

Hodge modules.

LEMMA 6.5.1. y(m*) = v(m) for me K®(X). Proor:

We have to show (y(m*),h) = (y(m),h) for me

K8(X),h €

H(W). By Lemma 5.2.1 (iii) and §§6.3, 6.4 this is equivalent to

(651)

(G)(m-T)=(()(m.T.) (me KP(X), 2€ W),

where the right action of H(W) on K®(X) is given by (6.4.2).

Let us

prove (6.5.1) by induction on é(z). The case z = e being trivial, we take

w € W satisfying s;w > w and prove (6.5.1) for z = s;w assuming (6.5.1) 102 = 10 Let J € Ci. Since q; (6.5.2)

jis a P!-bundle, we have

(4;,7)"(4:,7)!Da,(M) = (Da, (49i,7)*(4:,7)1(M4))[-2](-1)

(M € Ob MHM*(Q))), and hence we have

(6.5.3)

7(m.) = (q~‘7;(m))*

(me Ke):

Therefore we have

(Ge) (M: Ts iw) =(ie)"((ri(m*) — m*) - Ty)

=(&)"((a 7"ri(m) — m)* - Ty) =((ée)" (a rime) — m) Tu))* =((ie)"(m-Ts,w))*. O 6.6. We shall determine H*((i,,7)* ("QE |Q7)) for any admissible subset J and z,w € J. Since this does not depend on J, we re

H(i)" (QE).

We first give a weaker result.

denote it by

KAZHDAN-LUSZTIG

CONJECTURE

193

PROPOSITION 6.6.1. p(["Q#])= Dy for w € W.

ProoF: Setting Qu, = 92(["Q¥,]) € R and D, = D>, 25: € HR(W), we have v(("Q¥])

= Di,.

Hence by Proposition 5.3.1, it is

sufficient to show the following conditions:

(6.61)0/,,= 1. (6.6.2) Qu

(= Dise(z)—e(w)-1

(6.6.3) De = Ce

i for z > w.

Dk

(6.6.1) is trivial, and (6.6.3) follows from Lemma 6.5.1 and

(6.6.4)

Da, (“QX,,|2s) = (QF, |2r)[-2e(w)](—€(w)).

Let us show (6.6.2). (iz,3)*("Q2

Let z > w.

Since "Q¥ [Az is pure of weight 0,

|Qz) is of weight < 0, and hence Hi((iz,3)" (7 QE)

is of

weight < i. On the other hand we have H*((iz,7)"("Q¥ |Qy)) = 0 for i 2 &(z) — &(w) by the definition. Therefore Qi,,€ ®i 0 satisfying

(6.6.5)

RCM,

(22.2)

CO => (2824, 10272.)

SUR

and let i: {0} — Y be the inclusion. Then Hi(i*("Q#)) is a pure Hodge structure of weight j.

The proof is similar to [KL2, Lemma 4.5].

LEMMA 6.6.3. H?((iz)*(*Q¥_)) is pure of weight j. PROOF:

We may assume that z > w. Let x € X,. By [K2, Remark 4.5.14]

we can take an open neighborhood V of z in X such that there exists a commutative diagram

X,NV

(6.6.6)

|

—

X,ynv —-

|

V

|

{0} x AT ——> Y x AT ——> C7? x AW where Y is an irreducible closed subvariety of C” satisfying the assumption of Lemma 6.6.2, the horizontal arrows are the natural inclusions and the vertical arrows are isomorphisms. Hence the assertion follows from Lemma 6024 Set Qu,z = >; Cw,z,54 (Cw,z,; € Z) for z,w € W with z 2 w.

194

KASHIWARA AND TANISAKI

THEOREM

6.6.4. Let z 2 w.

(i) H77**((iz)*("QX,)) =0 for any jeZ.

(ii) For anyj € Z we have cw,z,; 2 0, and HH?) ((i,)*(*Q¥_)) is isomor-

phic to (QR, (—J))®". Proor: By Theorem 3.13.3 there exist some Ny € Ob(MHS) (k € 2) such that H*((i,)*("Q#¥))= (ax,)*(Nx). Then we see from Proposition 6.6.1 that

(6.6.7)

D cugt =) j

C1 Mel. keZ

Since [N;.] € Ry by Lemma 6.6.3, we have [N2; 41] = 0 and [No;] = Cw,z5%

and this implies that Noj41 = 0 and No; = (Q¥)®™+5.

O

It is easily seen that (4.8.4) is a consequence of Theorem 6.6.4 (or even Lemma 6.6.1). REFERENCES

[EGA] A. Grothendieck and J. Dieudonné,

Eléments de Géométrie Algé-

brique, I-IV, Publ. Math. IHES. [BB] A. Beilinson, J. Bernstein, Localisation

de g-modules,

Comptes

Rendus 292 (1981), 15-18. [BBD] A. Beilinson, J. Bernstein and P. Deligne, Faisceaux pervers, Astérisque 100 (1983). [BK] J.-L. Brylinski and M. Kashiwara, Kazhdan-Lusztig conjecture and holonomic systems , Invent. Math. 64 (1981), 387-410. [DGK] V. Deodhar, O. Gabber and V. Kac, Structure of some categories of representations of infinite-dimensional Lie algebras, Adv. Math. 45

(1982), 92-116. [K’] V. Kac, “Infinite dimensional Lie algebras,” Progress in Math. 44, Birkhauser, Boston, 1983.

[K1] M. Kashiwara,

The Riemann-Hilbert problem for holonomic sys-

tems, Publ. RIMS, Kyoto Univ. 20 (1984), 319-365. [K2] M. Kashiwara, The flag manifold of Kac-Moody Lie algebras, Amer.

J. Math. 111 (1989). [K3] M. Kashiwara, Kazhdan-Lusztig conjecture for symmetrizable KacMoody Lie algebra, to be published in the volume in honor of A. Grothendieck’s sixtieth birthday.

IS) Ms Kashiwara and P. Schapira, “ Sheaves on Manifolds,” Springererlag. [KL1] D. Kazhdan and G. Lusztig, Representations of Coxeter groups and Hecke algebras, Invent. Math. 53 (1979), 165-184.

KAZHDAN-LUSZTIG

CONJECTURE

195

[KL2] D. Kazhdan and G. Lusztig, Schubert varieties and Poincaré duality, Proc. Symp. in Pure Math. 36 (1980), 185-203. [LV] G. Lusztig and D. Vogan, Singularities of closures of K-orbits on flag manifolds, Invent. Math. 71 (1983), 365-379. [M] ©. Mathieu, Formules de caractères pour les algèbres de Kac-Moody

générales, Astérisque 159-160 (1988). [S] M. Saito, Mixed Hodge Modules, Publ. RIMS, Kyoto Univ. 26 (1989). [Sp] T. A. Springer, Quelques applications de la cohomologie d’intersection, Séminaire Bourbaki, exposé 589, Astérisque 92-93 (1982), 249273. [T] T. Tanisaki, Hodge modules, equivariant K-theory and Hecke algebras, Publ. RIMS, Kyoto Univ. 23 (1987), 841-879. Received October 10, 1989

Masaki Kashiwara RI.MS., Kyoto University Kyoto 606, Japan

Toshiyuki Tanisaki College of General Education Osaka University Toyonaka 560, Japan

mel ? Seemed Ps

ca

Cotte ne

i).

|)

nm,

Litsel Stew

nd:

=

ALT DRE d'ruiee CD ie

algebras

Oe

Pie we

Visage

Beaty

iyie dr. FAR

trotautmit art th

Uretyoe

Ame ee

‘Mis

,

ty

i ™

e

®

iLe —

Standard Monomial Theory for SL, V. LAKSHMIBAI*

Dedicated to Professor Jacques Dirmier on his 65th birthday

Abstract

Let g be a Kac-Moody Lie algebra, G the associated Kac-Moody group. Let B be a Borel subgroup in G and X a Schubert variety in G/B. For g = $l(n,C), we construct a characteristic-free basis for H°(X,L), L being line bundles on X associated:to dominant integral weights of g. Introduction

Let G be a semi-simple algebraic group and B a Borel subgroup. Let X be a Schubert variety in G/B. Let L be an ample line bundle on G/B, as well as its restriction to X. A standard monomial theory for Schubert

varieties in G/B is developed in [9], [11], [8], [10] as a generalization of the classical Hodge-Young theory (cf [3], [4]). This theory consists in the construction of a characteristic-free basis for H°(X,L). This theory is extended to Schubert varieties in the infinite dimensional flag variety SL2/B in [12]. In this paper, we extend the theory to Schubert varieties in the infinite dimensional flag variety SL,/B. Let A be a symmetrizable, generalized Cartan matrix (cf [5]). Let g (resp. G) be the associated Kac-Moody Lie algebra (resp. Kac-Moody group). Let W be the Weyl group. Let U be the universal enveloping algebra of g and Us the Z-subalgebra of U generated by X?/n!, a a simple root. Let À be a dominant, integral weight and V) the integrable, highest weight g module (over C) with highest weight À. Let us fix a generator e for the highest weight space (note that e is unique up to

scalars).

For r € W, let e, = re, Vz,, = Uje,.

In [12], we gave a

conjectural basis for Vz,,. Using this, we construct an explicit basis for *Partially supported by NSF Grant DMS-8701043

198

V. LAKSHMIBAI

Vz, for the case g = SI(n,C). We shall briefly describe below the results for sl(n,C). Let us fix a fundamental weight w;, 0 < i p(A) > 6(A). We then define a monomial pa, - PA: PAm to be

standard on X(r) if r > p(A1) > 6(A1) > p(A2) > 6(A2) > --- > (Am). Then we prove

Theorem 1. Let r € W/Wp and X(r) the associated Schubert variety in G/P. The standard monomials on X(r) of degree m form a basis of

HICX (Fr), Le), Let now X(r) be a Schubert variety in G/B and L = @L;',a; € Z*(L; being the line bundle defined as above with respect tow; (or P;)). Let F € H°(X(r),L). Further, let F = fofifo---fi, where f; = Pix: PAi2 *** Pig, We say, F is standard on X(r) of multidegree a = (ao,--- ,a;) if there exists a sequence in W

T > 901 > P01 = 002 > ++ > Poa, > 911 > + such

that

7; (83; ) =

p(Aj;), Ti (pi; ) =

> Pia

é(A;;), 1 < 7 < a;, 0 < a < l (here

m; denotes the projection 7; : G/B — W/Wp,)).

G/P; (or same as 7 :

W

—

We prove

Theorem

2. Standard monomials on X(r) of degree a form a basis of

H°(X(r),L).

The philosophy of the proof of Theorems 1 and 2 is the same as in [11], namely, given X = X(r), we fix a nice Schubert divisor Y in X. We then construct a proper birational morphism #Ÿ : Z — X such that Z is a fiber space over P’ with fiber Y. By induction, we suppose. the results to be true on Y, prove the results for Z and then make the results “go down to De As important consequences of the main theorem we obtain

(1) X(r) is normal

STANDARD MONOMIAL THEORY FOR SL,,

199

(2) Hi(X(r),L)=0,i>1 (3) A character formula for H°(X(r), L). §1. Preliminaries

Let H = SL(n,k), k being the base field which we assume to be algebraically closed. Let A = kft,t-!], A+ = k[t], A~ = k[t-]. Let D be the maximal torus in H consisting of all diagonal matrices and B the Borel subgroup consisting of all upper triangular matrices. Let N be the nor-

malizer of D in H. The projection t+ : At — k sending t to 0, induces a

map 7 : H(At) > H. Let

B=(x+)-1(B), W = N(A)/D, G = H(A).

We have a cellular decomposition

(+) G=

U BwB wEW

Let g be the Kac-Moody lie algebra corresponding to the matrix NaS) mf pl). 0 —1 2

0 | —I

-.. 7

MAO

er

den

D

0 —1 02410 0 0

Eh

9

With notation as in [5], let S = {ao,a1,...,an-1} be the set of simple roots of g. For w € W, let X(w) = U BrB (mod B) be the Schubert variety in G/B (see [6] for generalities on the infinite dimensional flag variety G/B). Let us fix a fundamental weight w;, 0 1

j21

Definition 2.5. We call a subset Z, of Z complete, if either

(1) Z1 = {A}, where A is both a head and a tail (in which case we call it trivial) or (2) Z1 = {Ao, A1,... , At} where Ao is a head, say

Ao = (41,61, 42,2,...), t= Ÿ (a; —b;) j21

and A;,..., A; are as in the proof of Lemma 2.2, with A = Ap. Remark 2.6. Given A = (aj, 61,42,62,...), following the procedure in Lemmas 2.2 and 2.3, there exists a unique complete subset Z, to which

A belongs.

Definition 2.6 (cf [2]). Let À = (A1,..., Ar) be a Young diagram with À; boxes in the jt" row. We say A is admissible if

(MeAre

Ag oS Ay

(2) Number of rows of same length is

cgh(X (0), L°71) (ax) 6 $01 > 802 > G02 > °°: > Poa, > boa, > 911 > **: > Gea, > Peay such

that 7; (9;;) =

P(Ai;),

Ti(i;)

=

ot),

PSE

DE

AMI ET

RE

À

Proceeding as in [11], we have Theorem 6.2. Standard monomials on X(r) of degree a form a basis of H°(X(r), L). Remark 6.3. In Definition 6.1, the order {Po,... , Pe} is immaterial.

REFERENCES

[1]

C.C. Chevalley, Sur les Décompositions cellulaires des espaces G/B, (manuscript non publie) (1958).

[2]

E. Date, M. Jimbo, A. Kuniba, T. Miwa and M. Okado, Paths, Maya diagrams and representations of sl(r,C), Adv. Studies in Pure Math

[3]

W.V.D. Hodge, Some enumerative results in the theory of forms, Proc. Camb. Phil. Soc. 39 (1943), 22-30. W.V.D. Hodge and C. Pedoe, Methods of Algebraic Geometry, vol.

19 (1989), 149-191.

[4]

IT, Cambridge University Press, 1952.

[5]

V.G. Kac, Infinite dimensional Lie Algebras, Progress in Math 44 (1983), Birkauser.

[6] [7]

M.Kashiwara, The flag manifold of Kac-Moody Lie Algebra, preprint. S. Kumar, Demazure character formula in arbitrary Kac-Moody setting, Inventiones Math 89 (1987), 395-423.

[8]

V. Lakshmibai, Standard Monomial Theory for G2, J. Algebra 98

[9]

V. Lakshmibai, C. Musili and C.S. Seshadri, Geometry of G/P-IV, Proc. Ind. Acad. Sci. 99A (1979), 279-362. V. Lakshmibai and K.N. Rajeswari, Towards a Standard Monomial

[10]

(1986), 281-318.

Theory for Exceptional 449-578.

Groups,

Contemporary

Mathematics

88,

STANDARD

MONOMIAL

THEORY FOR SL,

217

[11]

V. Lakshmibai and CS. Seshadri, Geometry of G/P- V, J. Algebra

[12]

V. Lakshmibai and C. S. Seshadri, Standard Monomial Theory for

100 (1986), 462-557. SL», Infinite Dimensional Lie Algebras and Groups, Advanced Series in Mathematical Physics 7, 178-234. Received March 12, 1990

Department of Mathematics Northeastern University Boston, MA 02215

Does “

s a

.

7

|

=

a

u ‘aif R Lu oy bug

4«

:

|

mad} à 5 + Tetnsiur

“.

2" re —

i 22 one e 27 me Ye:

Wi vine ablas (A, MEET. 0]

LSCTI

_

7

f

yonkae

DOTE saa)

.

tee rupees CYR (Hentai.

LC

PUS

ER

Sent Dy APE LE

tory

Py

L

Jay Pied

4

:

Equivariant Cohomology of Wonderful Compactifications

PETER LITTELMANN * CLAUDIO

PROCESI**

Dedicated to J. Dixmier on his 65th birthday

Introduction

The theory of complete symmetric varieties has evolved in several papers with the aim to give a satisfactory framework for Schubert calculus on a particularly relevant class of non compact homogeneous spaces, the symmetric spaces over adjoint groups. It is an aspect of Hilbert’s 15*” problem and it includes some classical examples like the variety of quadrics. For an automorphism o of order 2 of a semisimple algebraic group G of adjoint type we denote by H = G° the fixed point subgroup of o. Let G/H be the corresponding symmetric space. We study the geometry of the canonical G-equivariant compactification X of G/H, and more generally the geometry of the wonderful compactifications of G/H (of which we shall

recall some of the main features in sections 2,3 and 11). In the following we refer to the canonical compactification.

compactification

X as the minimal

wonderful

In previous papers (cf. [6],[{8],[9],[10],[11]) it has been carried out a detailed study of the wonderful compactifications, and it has been developed the interpretation of Schubert calculus as limit cohomology of equivariant compactifications. The computation of cohomology has been carried out

in several different ways in ((6],[8],{11]). The aim of this paper is to describe the cohomology of X extending

* Supported by C.N.R. ** Member of G.N.S.A.G.A. of C.N.R.

LITTELMANN

220

AND PROCESI

the results obtained in [8] for quadrics and at the same time to simplify

the treatment by the use of equivariant cohomology (cf. [6]).

Let T C G be a o-stable maximal torus containing a maximal o-split

torus and let B > T be a suitably chosen Borel subgroup (see 3.5). Denote by Nr the normalizer of T in G. We describe the T-fixed points X T in terms of the Satake diagram of the symmetric space (3.7, 6.7). Moreover, we construct a canonical smooth Nr-stable subvariety Mr (containing X7) with finitely many T-orbits. This variety has the following remarkable properties: a) Every B-orbit of X meets Nr in a T-orbit, and the B-orbit is a product of the T-orbit and an affine space. b) The G-equivariant cohomology of X restricts to the Nr-equivariant cohomology of Nr and its image is described by a natural combinato-

rial filtration: We group the connected components of Nr (which are torus embeddings) in Nr-orbits O; (ordered in combinatorial way). The factors of the filtration of the G-equivariant cohomology of X are then the Nr—equivariant cohomology groups of O; shifted by a suitable Euler class. We view this as a natural variation of the localization principle.

The strategy of the proof consists first in a careful geometric description of the variety Wr (which is of independent interest) and of a sequence of reductions through certain remarkable subvarieties of X associated to o-stable Levi subgroups of G.

Acknowledgements: The first author would like to thank the University of Rome for the hospitality offered to him during his visit in the academic year 88/89. 1. Equivariant

Cohomology

1.0. In this paragraph we wish to recall some results on equivariant

cohomology which will be needed later. We refer to [2], [7] and [14] for the details.

The description of the equivariant cohomology of a smooth torus em-

bedding given at the end of the paragraph can be found in [6]. 1.1. For a topological group G let Eg — Bg be a universal principal G-fibration. The space Eg is contractible and G acts freely on Eg. If X is a G-space with continuous G-action, then denote by XG the

fibre space Eg xq X over Bg. The G-equivariant cohomology HG(X, F)

EQUIVARIANT COHOMOLOGY

221

of X, with coefficients in a ring F, is by definition *

HG(X,F):= H"(Xe,F), the cohomology of the space Xg.

Note that Hi(X, F) is in a canonical

way a H{ (pt, F)-module. We will restrict ourself to rational equivariant cohomology and simply

write

HE (X) for HÉ(X, Q).

1.2. If G is a Lie group with finitely many connected components and K is a maximal compact subgroup of G, then G/K is contractible and

hence HG(X) = Hx (X). If H C G is a closed Lie subgroup, then Eg — Eg/H is a universal principal H-fibration. So if Z is a H-space, then the equivariant cohomol-

ogy rings H7,(Z) and HZ(G xx Z) are canonically isomorphic. 1.3. Let T be a compact torus. Denote by X(Tc) the character group of the complexification Tce of T. For a character À let cy € Hy. (pt) be the Chern class of the corresponding line bundle on By,. The map À — cy induces an isomorphism

Sym(X (Tc) ®z Q) — H7(pt). 1.4. For a compact group K let K° be its connected component containing the identity and denote by I the quotient group K/K°. Let T C K° be a maximal torus and let W be the Weyl group of K°. If X is a K-space,

then the morphisms

Hy (X) — Hjo(X) and Hyo(X) — Hp(X) induce

isomorphisms

Hy(X) = Hxo(X) = (Hr(X)7)" In particular, HE (pt) is a graded algebra with elements only in even degree and without zero divisors.

1.5. If X is a K-space whose cohomology vanishes in odd degrees, then the spectral sequence corresponding to the map Xx — Bx collapses.

So Hj, (X) is a free module over

Hj (pt) and

H'(X) = Hx(X)/THK(X), where Z is the ideal in Hj, (pt) generated by the elements of strictly positive degree.

LITTELMANN

222

AND PROCESI

A key result in the theory of equivariant cohomology is the following: Localization Theorem 1.6. [fT is a compact torus acting smoothly on a compact manifold X, then the kernel as well as the cokernel of the canonical map

i* : Hp(X) > H7(X") induced by the inclusion i: XT — X are torsion modules over H7(pt). In

particular, if H7.(X) is a free module over H7(pt), then i* is injective. 1.7. In our case we will study the action of a reductive algebraic group G on asmooth projective variety X such that XT is finite for a maximal

torus T of G.

By the theory of Bialynicki-Birula (see [3]-[5]), the odd

cohomology of X vanishes, and hence by 1.5 the G- as well as the T— equivariant cohomology is a free module over H#(pt) respectively H7(pt). Let W be the Weyl group of G. The Localization Theorem implies: Proposition

1.8.

The odd G-equivariant cohomology ofX vanishes

and H*(X) = H¢(X)/THZ(X).

Furthermore, the inclusion i: XT > X

induces an injective map

FASO)

ONDES

1.9. We want now to recall the description of the ring H7(X), where T is an algebraic torus and X is a smooth T-embedding, i.e. X is a smooth T-variety with an open orbit isomorphic to T. For details we refer to [6], 64. Denote by X(T’) the character group of T. Recall, the smooth torus embedding X of

T'is completely determined by its associated rational par-

tial polyhedral decomposition (r.p.p.d.) of Homz(X(T), R). Let {vq} be the primitive integral vectors generating the one-dimensional cones in the decomposition. Consider the polynomial ring Q[z] in the variables x, corresponding to the vectors vg. We define a morphism

Q[za] — H}(X) by sending each variable x, to the equivariant Chern class of the T-stable divisor D, corresponding to vq.

Denote by I the ideal in Q[r,] generated by the monomials I

:

:

{I]j=12a,; |Vois,Va, do not generate a cone in the given r.p.p.d.}.

Then J is contained in the kernel of the map defined above, so we have a

well defined map from the “Reisner-Stanley” algebra Rx := Q[za]/I to

Hy(X).

EQUIVARIANT COHOMOLOGY

223

Theorem 1.10. ([6], Theorem 8) The morphism Rx — H*(X) is an isomorphism.

Remark. The class c, in H#(X) associated to a character y € H?(pt)

under the map H7(pt) — H7(X) is cy := 57, X(va)Ta2. The minimal wonderful compactification of the group 2.0. As an example of our approach to compute the equivariant cohomology we consider in this paragraph the minimal wonderful compactification G of a semisimple algebraic group G of adjoint type.

2.1. Consider the group G x G and the involution 0(91, 92) := (g2,g1) of G x G. Then its fixed group (G x G)® is the image of the diagonal embedding of G, and the symmetric space G x G/G is as G x G-variety isomorphic to G. Denote by G the Lie algebra of G. For n = dimG, consider the Grassmann variety

Grass, (GG) C P(À(G @6)).

Let A" G be the point in Grass,(G @ G) corresponding to the subspace Lie (G x G)° of G&G. The orbit (GxG)./\”" G is isomorphic to GxG/G and can be identified with G (as G x G-variety) by the map g + (g,id). A" G. We define the minimal wonderful compactification G of G as the closure

of the orbit (G x G). \"G in Grass, (G ® G). The variety G has the following “wonderful” properties (see [9], §3): G is a smooth variety with finitely many G x G-orbits and only one closed orbit. The complement of the open orbit G C G is the union of rk (G) smooth divisors intersecting transversely, and the closure of an orbit is ob-

tained as the intersection of the codimension one orbit closures containing 16e 2.2. Consider the closure T of a maximal torus T C G in G. This is a T x T-stable variety. The Weyl group W of G acts on T via the diagonal embedding of Nr in G x G, where Nr is the normalizer of T in G. The

open orbit (T x T').id in T is isomorphic to the torus S = T/H, where H is the subgroup of T of elements of order 2.

The structure of this torus embedding is well understood ([10], §5): T is a smooth S-embedding. The r.p.p.d of Homz( X(T), R) corresponding to T is the decomposition into the Weyl chambers of the root system of G.

224

LITTELMANN

AND PROCESI

Furthermore, every G x G-orbit in G intersects T in a union of T x Torbits, which are permuted transitively by W.

A precise description of the stabilizer of a point in G is given in [9], §5. We will need here only the fact that the unique closed orbit is isomorphic to G/B x G/B for a Borel subgroup B of G, and all T x T-fixed points in

G are contained in the closed orbit. Theorem

2.3. The inclusion

T — G induces an isomorphism

Héxq(G) Hp xr(T)™ © (Hz(pt) @ H5(T))™. Remark. Let K be a maximal compact subgroup of G such that T, = KNT is a maximal torus in K. A consequence of the theorem is by §1 (see

[11)):

Ho (G) A (CCR) TT)

PI

Proof. Consider the following inclusions: GATE

_ 1

(GS

Bok

iy

L

sas Wy

The T x T-fixed points in G are in one W x W-orbit, and the intersection of this orbit with T is one W-orbit. So by the Localization Theorem the inclusions induce the following commutative diagram:

Hye ((TY*7)%

>

Ay (T)™

Hpxr(G PT À Hoyg(G) which proves the morphism H#,G(G) — Ht,.7(T)™ is injective. Recall, H¢,g(G) is a free Hp (pt)”*” -module. And H#,r(T)is a

free H7,r(pt)-module. It is well known, one can choose a graded W x Wsubmodule R of H7,.7(pt), isomorphic to the regular representation of W x W, such that Ary 7 (pt) Yu

as graded Hp, 7(pt)”*"—module. a free Ht, (pt) *W —module.

@ Hryp(pt)”

*™.

So H5,r(T)"*W is in a natural way

EQUIVARIANT COHOMOLOGY

225

Let Z denote the ideal in Ht,.7(pt)”*™ of elements of strictly positive degree. Since the morphism we are considering is an injective, graded morphism of free H+, (pt) x —modules, to prove its surjectivity it suffices to show the induced injection

HéxG(G)/LHexG(G) — Hpyr(T)” /LHpy7(T)™ is an isomorphism. The dimension of the left hand side is by 1.8 the Euler characteristic of X, which is equal to the number of T x T-fixed points in

X and hence equal to |W x W|. To compute the dimension of the right hand side, note first we can

find a graded W-stable submodule U of H%,.7(T) such that the morphism U ® Hr, (pt) —

Hr y7(T)

is a W-equivariant isomorphism of graded H7,r(pt)-modules. And the dimension of U is by 1.8 equal to the Euler characteristic of T', and hence equal to |W|, the number of T x T-fixed points in T. So Hpxr

D)" /LHty7(T)”

is isomorphic to (U @ R)™ as W-representation. Since R decomposes into the direct sum of |W|-copies of the regular representation of W, the

following simple lemma shows dim (U @ R)" = |W x W|. And hence dim Hoy

(T)” /THrxr(T)"

which proves

= |W x W|= dim Héy¢(G)/THéx6(G),

Ht, ¢(G) — H},7(T)™ is an isomorphism.

Lemma. I[f N is a finite group, and U and V are two finite dimensional representations ofN such that V is the sum of copies of the regular representation of N, then

dim (V @U)N =

dim V - dim U

Ny]

Since the image of the diagonal embedding T — T x T on 7’, the isomorphism

acts trivially

H7y7(T) = Hp (pt) ® H3(T) follows by the exact sequence

1— T—TxT—S—1

Q.E.D.

LITTELMANN

226

AND PROCESI

3. Minimal wonderful compactifications 3.0. This paragraph is entirely devoted to introduce some notation and recall the construction of the minimal wonderful compactification of a symmetric space and its orbit structure. 3.1. Let G be a semisimple group of adjoint type and denote by G its Lie algebra. Consider an involution o of G and denote its fixed group G? by H, and the Lie algebra of H by H. If M C G is a o-stable torus, then M admitts a decomposition M = M°.M!, where M! is the connected component containing the identity of

the subgroup {m € M|o(m) = m~*}, the so called o-split part of M. We fix a maximal o-stable torus T of G such that T1 is a maximal osplit torus in G. The dimension ! of T1 is called the rank of the symmetric

space G/H. We fix further a o-stable maximal compact subgroup K of G such that T; := TN K is a maximal torus in K.

3.2. To construct the minimal wonderful compactification X of G/H, consider first the compactification G of G defined in 2.1. Via the “twisted embedding” GoGxG, g'(g,0(g9))

we obtain an action of G on Grass,(G @G) such that the stabilizer in G of the point A” G is H. So the orbit map g + g./\”" G induces an embedding G/H — G. The action ofG on the open orbitG C G is given by g.g' := gg’a(g)7! and the embedding of G/H into G C Gis given by the map gH +> ga(g)7. We define the minimal wonderful compactification X of G/H as the closure of the orbit G. A” G in G.

Remark. For the definition of the minimal wonderful compactification X we refer to [9], §2 and §4. In [9], §6, it is proved that X is isomorphic to the closure of the orbit G. A” H in Grassm(G), m = dim H. The proof that X is isomorphic to the closure of the orbit G. A"G we omit it.

in G is similar, so

3.3. We can consider X as a connected component of the fixed points of an automorphism of G: Let do be the Lie algebra automorphism induced by o. The linear automorphism

@:G0G—-GOG,

(91,92) + (do(g2), do(g1))

EQUIVARIANT COHOMOLOGY

induces an automorphism of order two of P(A"(G

227

G)), which we denote

by &. Since6 fixes A” G, it is easy to see G is &-stable and the restriction

of& to the open orbit G C G is the morphism g + o(g~'). Obviously we have G/H = (G)° and the &-fixed point set (G)° is G-stable (with respect to the G-action on G in 3. 2). Since dé acts on the tangent space T;qG= G at the &-fixed point id € G by gr

—do(g), we

see that

dim Tia(G)° = dim G% = dim G/H. So X is a connected component of (G)°. Since the fixed points of a finite order automorphism on a smooth variety form a smooth subvariety, we see in particular that X is smooth. Remark.

Note, g € G? if and only if int(g) o o is an involution. If we

identify G with the connected component G(c) of Aut (G) containing © via the map g + int(g)oo, then G? can be identified with the set of involutions in G(c). It is now easy to see each connected component of G? is an orbit G.r isomorphic to the symmetric space G/G’, and X, := G.t C G is the minimal wonderful compactification of the symmetric space G/G.

3.4. The variety X has the following “wonderful” properties ([9], §3): X is smooth and contains exactly 2' G-orbits. Further, the complement in X of the open orbit G/H is the union of | smooth divisors $j, ..., 5), which intersect transversely. The closure of any orbit in X is obtained as the intersection of the codimension one orbit closures containing the orbit. Finally, there exists only one closed orbit in X, namely Si N...1N Sy.

3.5. To describe the orbits and the stabilizers we have to introduce first some notation. Since T is o-stable, o operates on the root system © = ®(G,T) of G. Let ® be the set of roots fixed by o and let ®, be the set of roots moved by a. Choose a set of positive roots + such that for bf := ©; + we have o(&}) = —&} and 6; = D} U—4f. Denote by A the set of simple roots of +, and decompose A into Ag U A; such that Ao C ®o and A; C 9).

Let r: X(T) — X(T") be the restriction map from the character group of T to the character group of T1. Then r(®,) gives rise to a root system :,

called the restricted root system, with basis © := r(A:). We denote by W the Weyl group of ® and by W! the Weyl group of ,. Finally, let B be the Borel subgroup of G corresponding to the choice of ®+.

LITTELMANN

228

AND PROCESI

3.6. The following results can be found in [10], §5:

Let S be the torus

S = T!/T1N H. The characters 27, y € E, are a basis

of the character group X(S). The closure S of S in X is a smooth S— embedding with a canonical action of W1. The rational partial polyhedral decomposition (r.p.p.d) of Homz(X (T"), R) associated to S consists of the Weyl] chambers of ®:1. The negative Weyl chamber of this decomposition corresponds to a S-stable affine chart of S, which can be identified with A! in a canonical way. The embedding of S into A! is given by

Peta erent")

where

= taper fe

This affine chart is a fundamental domain for the action of W! on S, and two elements z and y in this affine chart are conjugate under G if and only if they are conjugate under S. For every subset I of 5, denote by zr the following element of A! C X:

Safiroi),

=

Del andr 0 vee Le

Then X is the disjoint union of the 2! orbits G.zp, where L runs over all possible subsets of Y. 3.7. The description of the stabilizer in G of the point zp € X we will

give now can be found in [9], §5. For F Cc & denote by Pp D B the parabolic subgroup of G associated

to the set of simple roots Ar := {a € A | r(a) = 0 or r(a) € T}, and let Ly denote the Levi factor of Pp containing T. Note that Lr is o-stable. To avoid too many indices, set P = Pr and L = Lr. Let C be the center of L and denote by C’ its connected component containing the

identity. Note that C! C T!, so C! is a o-split torus. And let P“ be the unipotent radical of P. The stabilizer in G of the point x = xr is

Gr SASCEES Note that P = NorgG¥, the normalizer ofthe unipotent radical of G,. So we can associate to any point y € X in a canonical way the parabolic

subgroup P(y) := NorgGy. Remark.

1) As an immediate consequence we see the orbit G.x has a

T-fixed point if and only if (L, L) contains a maximal torus contained in (L,L)’, i.e. the restriction of o to (L, L) is an inner automorphism. This can be easily read off the Satake diagram corresponding to a.

example the tables in [13]).

(See for

EQUIVARIANT COHOMOLOGY

229

2) For z € X let I, be the Lie algebra of the kernel of the representation of the stabilizer G, on N,, the normal space of the orbit G.z in X at the point z. Then for the embedding of X in Grass,,G (see Remark 3:2); the image of z is the subspace I, of G [6],§13. For the point x = zp is |, = Lie L° @ Lie P*. 3.8. For L := Lr the subgroups L, :=

LM K and Ci := CIN K are

maximal compact subgroups of L and C!.

The stabilizer of x = zp in K is K, = L?.C? ([6], $13). 3.9. The local structure of X around a T-fixed point z € S can be

described as follows (see [9], 2.3): Let A be the affine chart of S corresponding to z, ie. A = {y € S|z € T.y}. The stabilizer G, is a parabolic subgroup of G. Let P- be a parabolic subgroup ofG opposite to G, such that TC G,NP7~. If L is the Levi part of P~ containing T, then the semisimple part of L acts trivially on A. Let U be the unipotent radical of P~. The canonical map

UT x A(z) +X is a P~-equivariant isomorphism onto an open affine neighborhood of z.

4. Orbits with T-fixed points and T-stable subvarieties 4.0. We want now to use the construction of X as a subvariety ofG to construct a canonical set of smooth T-stable projective torus embeddings, which contains all T-fixed points in X. We use throughout the notation

g.z = gzo(g)! for the twisted action. 4.1. Let Nr be the normalizer of T' in G, which is a o-stable subgroup. The closure Nr of Nr in G is a T x T-stable variety. In fact, let us write

Nr = (J Tru, wEew

where ny is a representative of w € W. Consider Nr = Uvew [nw . Of course, Tn, is isomorphic to T, but the T x T action is twisted by w. Le. there is an isomorphism Ÿ : T — Tn, with Y((t1,t2)p) = tipty nu = ae

t;pn, w-1(t2)~!. In any case, Tn,

:

‘

—T

contains the fixed points (T

xT

Ts

the T x T-fixed points lie all in the closed orbit G/B x G/B (cf. §2) and : —T xT : : can be identified with W x W. The fixed points (T ; ) can be identified

LITTELMANN

230

AND PROCESI

with d(W), the image of the diagonal embedding of W into

W x W, and

hence W x W = Uvew €(W)(1, v). It follows that the varieties Tn,, are disjoint, otherwise the intersection of two of them should contain T x T-fixed points. We have thus proved:

Proposition. Nr is smooth and the disjoint union of |W| irreducible Tx T action). varieties isomorphic to T (with the appropriate twist for the Since Nr is o-stable, its closure Nr is 6-stable and (Nr)? is stable

under the action of Nr (acting on G via the twisted action as a subgroup

of G). Since Nr := XN

(Nr)?

is an open subset of (Nr)° by 3.3, we get Proposition 4.2. Nr stable subvarieties of X.

is a Nr-stable disjoint union of smooth T-

4.3. We will now investigate the irreducible components of Nr. Let x

denote the point zr, | C ¥, in S (3.6). For the rest of this paragraph we fix [ such that the orbit G.x contains T-fixed points.

Recall that this is

equivalent to assume that for L = Lr the derived subgroup (L, L) contains a maximal torus fixed by o (Remark 3.7.1). As before we denote by L, and C? the intersections of L and C! with the compact group K. To prove all T-fixed points in X are contained in Nr, we will choose a T-fixed point y in the orbit G.x, and describe the irreducible component

of Nr containing y. Let M be a maximal torus in L such that the intersection MN(L¢, Le) is a maximal torus in (L., L.) fixed by ©, so M C G,. If k € L. is such that &Tk-! = M, then the point y = k7!.z is a T-fixed point. And, since M = kTk=! is o-stable by the choice of M, the element

n := k~'o(k) normalizes T. In fact, n is in the image of the embedding G/H — G (see 3.2), so we can view n as a point in Np C X. Denote by T the irreducible component of Wr containing n.

Note that ¢ := int(n)oa is an involution which stabilizes T. Moreover, since K; = L?.C! we see K, = L?.C}. Concerning the choices made above, note that y,T,n and ¢ do not depend on the choice of M. And since k’/Tk’-' = M implies k~1k’ normalizes T’, the possible choices for y and T are conjugate under the Weyl group of L, and the corresponding choices for n and ¢ are conjugate under the normalizer of T'in L,.

EQUIVARIANT COHOMOLOGY

231

Fix a 1-parameter subgroup À : k* — T contained in the dominant Weyl chamber. It defines a Bialynicki-Birula cell decomposition in Nr and X. For a T-fixed point z in T denote by C, the corresponding cell in X

(i.e. C, is the set of points p in X such that lim;o A(t).p = z) and by C! the corresponding cell in T. If a is a root, then we denote by U, the T-stable unipotent subgroup of G such that Lie U, = G, is the T-eigenspace in G corresponding to a. Recall, we denote by P(z) the parabolic subgroup associated to the

point z € X (see 3.7). Proposition 4.4.

1) T is the closure of the orbit Clin = k~1.(C.id) and y = k=!.x is a T-fized point in T. 2) T is a smooth torus embedding for the torus R:= C!/C!NH R-equivariantly isomorphic to R, the closure of Rin S.

and is

3) Let z be a T-fired point in T. a) The intersection of the orbit G.z with T is a union of T-fired points permuted transitively by the normalizer ofT in K. Moreover, the intersection is transversal at the point z. b) Let A(z) be the open T-stable affine chart ofT containing z (3.9).

The intersection ofaG-orbit with A(z) is either empty or one T-orbit. c) L is a Levi part of P(z), and the stabilizer of z admitts a decomposition G, = L?.C1.P(z)"*. In particular, the compact stabilizer K, = L?.C} does not depend on the choice of z. d) Let @1,...,a, be the set of positive roots such that Gy, is not contained inG, and denote byU the product [];_; Us. The multiplication map p:Ux Cl — X gives an isomorphism of U x Ci, with the cell C,.

4) Every B-orbit in X meets Nr in one T-orbit. 5) The set of T-fired points in X is included in Nr.

Proof.

Since o(n) = o(k~!)k = n=}, the irreducible component Tn

of Nr containing n is stable under &. Using the G x G-action on G we see further that Tn = Tn = nT. But since o(n) = n_! we know G(zn) = nG(z)

for z € G. So for z € T one has zn € (Tn)? if and only if né(z)n~! = z, and for t € T one has tn € (Tn)? if and only if no(t}n=! = t7?. For ¢ = int(n)oo denote by 7% the connected component containing the identity of the ¢-split part of T. Note that ¢(t) = t-! if and only if alkika y= kt-1k-!. Sot € Thé if and only if ktk—1 is a o-split element in M. But by the choice of M this implies that ktk=1 € C’, so ktk=! =t and PC} AALS Since 7 is the irreducible component of (Tn)° containing n, the discussion above shows that dim 7 = dim T4? = dim C!. Moreover, by the

232

LITTELMANN

AND PROCESI

definition of the twisted action is t.n = n if and only ift = no(t)n—* = d(t), so Th = T*. Hence dim C!.n = dim 7, which implies T = C!.n. But k commutes with C1, so C!.n = k~!.(C!.id) = k-1.R, where R denotes the

CS =T'/T'NH. We have thus proved

torus C!1/C1N H

T=Tn=Cin=k"!Clid=Rk"!R.

To see that y = k~!.2 is contained in 7 we consider the action of C’ on the affine chart A;4 of S corresponding to the negative Weyl chamber C (3.6). Recall that A;4 can be identified with A! in a canonical way such that the

point id € S C G/H corresponds to the point p = (p1,...,pr), pi = 1 for i = 1,...,1.

Since C! is the connected component of yer Ker 27;, the

point 2 = (21,..:, 2) with s; = 1 for 7; ET and z= Dior 4,63, —T is in the closure of C!.id in Ajq. So x € R and hence y ET.

As an immediate consequence we see that all T-fixed points in X are

contained in Wr, so it remains to prove 3) and 4). Recall that TŸ operates trivially on T and k commutes with C!, so there is a canonical bijection between the C!-orbits in À and the T-orbits in 7. Hence, to prove 3) it suffices to prove the corresponding statements for the C!-orbits in R.

Denote by A,, w € W!, the affine chart of S corresponding to the Weyl chamber wC. Then, by the description of the action of T! on A, in 3.6, R contains a C!-fixed point z in A, if and only if there exists a subset

A CE such that C* is the connected component of (),,-, Ker 2w(7;). But this means C! = wC,w~', where CÀ is the connected component of the o-split part of the center of LA, identify A, again with A!, then of C!.id = C1 (1,...,1) in A! is Yi EZ-A and z; = 1 if y; € A.

Ke

wly

Chu

and hence wLAw-! = L. Moreover, if we the unique C!-fixed point in the closure the point z = (z:,..., 21), where 2; = Ouf Hence we have z = w.zq and we obtain:

ECG

= w(L7 Cp Paw

SGA

RG,

which proves c). Note further that z = wr, and C! = wChw7! implies dim T'.æ = dim T.z and hence by 3.9 also dim G.z = dim G.z.

The affine chart A(z) of R corresponding to the fixed point z can be identified in A! with the set of points p = (pi,-.-,pr) with pj = lif y; E A. By this description it is clear that the intersection of a T-orbit in A,

with A(z) is either empty or one C!-orbit. But since the intersection of a G-orbit with A, is one T-orbit, this proves b). Moreover, we see that the tangent space of A, at z admitts a decomposition T,Ay

= T,T.z @ T, A(z).

But by 3.9 we have a decomposition

EQUIVARIANT COHOMOLOGY T,X =V @T,Ayw such that T,G.z

= V

233

@T,T.z. Hence the intersection of

G.z with R is transversal at the point z. Let z’ be an element in the intersection of G.z with R. The orbit C'.z' contains a C!-fixed point z” in its closure. We have already seen dim G.z = dim G.z = dim G.z”, so z” € G.z. On the other hand is

G.2z' N:A(e") = C!.z! byib)ps0:2/iatz" Now let z be a T-fixed point in 7 and let z’ be an element of TNG.z. Since z’ is a T-fixed point there exists an element k € K such that k.z = z’ and k normalizes T (and hence k.Nr C Nr). But &T NT # implies k.T CT, so k normalizes T, which finishes the proof of a). To prove d) note first that Ci C C, and C, is B-stable by the choice of A, so the image of y is contained in C,. Now the tangent space T,X of X at z has a T-stable decomposition T,X = T,T @ T,G.z by a). Moreover, if we denote by V+ the set of elements in a representation space of T

such that lim:o A(t).v = 0, then we have the corresponding T-stable

gdecomposilon. TL A — 1) @2.Ge, and IX

= 1,C,, 1} 71,=1,C..

By the construction of YU the differential of the orbit map p : WU — G.z,

p(u) := u.z, is an isomorphism at id of T;aql/ with T;}G.z. So the differential of y is an isomorphism at the point (id, z) of the tangent space of U x C! with the tangent space of C, at z. Moreover, the action of the 1-parameter subgroup À on {{ as well as on C, and C! are linearizable (see [3]). I.e., for the one-dimensional torus Gm = k* there exist representations pj : Gm — GI(U;), i = 1,2, and Gm-equivariant isomorphisms U, = U x Ci and U2 ~ C,. Note that the Gm-equivariance implies that lim; p;(t)u; = 0 for any u; € U;, i = 1,2. Further, the morphism y induces a G;,-equivariant morphism ji : U; — U2

such that (0) = 0 and the differential dj induces an isomorphism of the tangent spaces at 0. Now Lemma 4.4 below implies that j, and hence y, is an isomorphism. To conclude the proof of Proposition 4.4 it remains to prove 4). Every B-orbit in X is contained in a Bialynicki-Birula cell, so by 3d) every Borbit meets Nr. Let now T be an irreducible component of Wr and fix

an element z € T. By 3d) there exists a T-fixed point z’ € 7 such that B.z is contained in the cell C,,. In particular, z’ € T.b.z for any b € B. So

B.zUT is contained in the affine chart A(z’) of T and is hence one T-orbit by 3b).

Q.E.D.

Lemma

4.4.

([15], p.121) Let pi : Gm — Gl(Ui), à = 1,2, be two

representations of the one-dimensional torus Gm ~ k* such that

lim pi(t)u; = 0

LITTELMANN

234

AND PROCESI

for any uj E Vi, i= 1,2. If fp: U; > U2 t8 Gym —equivariant morphism such that ji(0) = 0 and the differential dji induces an isomorphism of the tangent spaces at 0, then ji is an isomorphism.

Q.E.D.

5. The subvariety Z of X

5.0. Let I be a subset of © and denote by L = Lr the corresponding (o-stable) Levi subgroup of Pr containing T. For a Levi subgroup of this type the variety ZE

LURERX

has been investigated in [1]: Z is a smooth compactification of the variety L/L°. Every L-orbit in Z meets S in a union of S-orbits, which are permuted transitively by

W1(T), the subgroup of W! generated by the simple reflections s,, y ET. The intersection of a G-orbit with Z is a finite union of L-orbits. For z € Z the L-orbit L.z is closed if and only if the orbit G.z is closed in X. The closure of a L-orbit in Z is smooth, and it is the transversal intersection of the codimension one orbit closures containing it. Furthermore, let z € S be a T-fixed point and let A = {y € S|z € T.y} be the corresponding affine chart of S. The stabilizer L, is a parabolic subgroup of L. Let P~ be a parabolic subgroup of L opposite to L, such that T C

P~

L,, and let U

be the unipotent

radical

of P~.

The

canonical morphism

Un

A(z) 2

is a P~—-equivariant isomorphism onto an open affine neighborhood of z. 5.1. Let C be the center of L and denote by C! its connected component containing the identity. Consider the semisimple group of adjoint type L' := L/C. Since L and C are o-stable, we obtain an induced involution of L’ which we denote also by a. Let Xz be the minimal wonderful

compactification of L'/L'°.

Proposition 5.2. The canonical map 7: L/L? — L'/L'° extends to a L-equivariant morphism x : Z — X_. Proof. To define x we use the realization of X na XL as subvarieties

of Grassm(G) and Grass,/(L'), where L! := Lie L' and m’ = dim L'° (see Remark 3.2 and Remark 3.7.2).

EQUIVARIANT COHOMOLOGY We will see, Z is contained in the subset U of all subspaces

235 V €

GrassmG, such that dim (V NL’) = m’. By L-equivariance and semicontinuity, it suffices to check this for a T-fixed point z in S. Let Q be the stabilizer in G of z, which is a parabolic subgroup of G. Then Q; := QNL, the stabilizer in L of z, is a parabolic subgroup of L (since T C LN Q). Denote by /, the subspace of G corresponding to the point z. By Remark 3.7.2 is LieQ = Lie T! @1,. Since Lie C! C Lie T!, we get Lie Qr = Lie T! @(L'N1,). Furthermore, by the description of the local structure of Z in 5.0, we know T! has a dense orbit in the normal space N,

to the orbit L.z C Z at the point z, and dim N, = dim T!. This implies dim, (Lier /Lie D!) = din But = £'2 20 dim.(UU AL). dim L'?. But U can be considered in a canonical way as a fibre space with base space Grass mL’ and fibre map f : V + VNC’. Since the restriction of f to the open orbit in Z is the morphism L/L? — L'/L'°, the restriction of f to Z is the desired extension. Q.E.D.

5.3. If z € x71(x(S)), then we can find an element z’ € S such that m(z') = m(z) and l.z' = z for some | € L. But this implies | € L;ç,,. So 1=1,.t with L € L, and t EC, which implies z € S. Hence:

Lemma.

7 !(r(5))=S.

5.4. Note that the image TJ’ of T in L' is a o-stable maximal torus such that T'! is the image of T!, and T”’ is a maximal o-split torus in L’. Furthermore, 7(S) is the closure of TTS RE shih eae In [6], Theorem 27, is proved K.S = X. Applying this to the wonderful compactification Xz; and 7(S), we obtain as an immediate consequence of the lemma above: Corollary.

L..S = Z.

5.5. Throughout the rest of this paragraph we will again assume that

T C B is such that (L,L) contains a maximal torus contained in (L, L)’. We use the same notation as in §4, i.e. x denotes the point zp, R is the

image of C! in S and R denotes the closure of R in S. Consider the map 7: Z — Xr. The fibre over the point id € Xz is R. By assumption the open orbit in X; contains T-fixed points. We know by 4.3 that the T-fixed points in L’/L’° are contained in L,.id C Xz. Let k € L, be such that k.id is a T-fixed point in Xz. Since z is a C}-fixed point in the fibre À over the point id € Xz, the point y = k.z is a T-fixed point in the fibre 7 = k.R over the point k.id. In fact, T is the irreducible

236

LITTELMANN

AND PROCESI

component of Wr considered in 4.3: k.id € Xz is a T-fixed point if an only

if M = k~!Tk is a maximal torus such that the intersection MN (L-, Le) is a maximal torus in (L,,L.)”. So n = kid € Z is contained in the T— stable fibre T = k.R of x over the point k.id € Xz, which implies T is the irreducible component of Wr containing n. Denote by Z° C Z the preimage of the dense orbit in Xz. Since the stabilizer in L of the point id € Xz is Lz and in L, is Kz, we obtain:

Proposition 5.6. 1) The irreducible component of Nr containing a T-fired point y € Zs

the fibre n—!(m(y)) of x : Z — Xz over the point r(y). 2) The canonical maps Lx, RS Fane HR ea Z9 are isomorphisms. 3) The canonical maps Le XK, R— L..R and L, KK, da Dect pare isomorphisms.

5.7. As a consequence we see the map S XR R — S.R is an isomorphism onto an open subset of S. This implies the following description of

the r.p.p.d. of R (and T) as a R-embedding: Corollary. The r.p.p.d. of Homz(X(R),R) corresponding to R is obtained by intersecting the r.p.p.d. corresponding to S with the image of

Homz(X(R),R)

C Homz(X(S), R).

In particular, the intersection ofR with the open orbit G/H in X is one C}-orbit, so the intersection ofT with G/H is the open T-orbit in T.

5.8.

Consider the group NoryL, the normalizer of L in H.

Since

NoryL normalizes C!, NoryL operates on R. If z is a C!-fixed point in R

and z/ is contained in G.zN

R, then z’ = w.z for some w € W! (see proof

of Proposition 4.4). Since w can be represented by an element h € H, and h normalizes L, = L°.C', we see h normalizes L° and C!, and hence L. So h € NoryL, and

G.zNR=Nyz.z,

where Nj := Nory L/L’.

Furthermore, Nz is isomorphic to Nory: R/Ceny: R, the quotient of the

normalizer in W! of R by the centralizer in W! of R.

EQUIVARIANT COHOMOLOGY

237

Proposition. L° is a normal subgroup of NorxL of index t|G.2N RI. The quotient group Nr = Nory L/L’ is isomorphic to Norw: R/Cenw: R. Furthermore, G.zN R= Ny.z. for all R-fized points in R. 5.9 Let T be the irreducible component of Wr considered in 5.5. Two T-fixed points in 7 are in the same G-orbit if and only if they are conjugate under Nr. Since for n € Nr either RTE

we see the normalizer G.zNT for a T-fixed Nory,7/T of W, and 4.4 and conjugation it Lemma.

or

MNT

=;

Norw,.T operates transitively on the intersection point z in 7. Denote by Wz the subgroup W7 := let W, be the stabilizer in W of y. By Proposition is now easy to see:

The following quotient groups are isomorphic to Nr:

L.NoryL/L,

NorxR/K;,

NorxT/Ky,

Wr/Wy.

5.10. By abuse of notation we will denote all these quotient groups by Nz, it will be clear from the context the quotient of which groups we are considering. For example, the group L := L.NorxL operates naturally on Z°. By Proposition 5.7 (and 1.2, 1.4) we obtain the following natural isomorphisms in equivariant cohomology:

H}(Z°) = Hy(Z°)N* ~ Hy (R)N* © (Hy. (pt) ® HR(R))™* ERA

AC AMOR

ETC RME

5.11. We want now to study the relation between the T-fixed points in Z and the varieties ZA := LA.id, where A is a subset of [. Let Ca be

the center of La, denote by Ci its connected component containing the identity, and let Li be the quotient La/Ca. Denote by Zk the preimage of the open orbit in Xq under the map T4 : Za — Xa, where XA is the

minimal wonderful compactification of Li, /(L,)’.

LITTELMANN

238

AND PROCESI

Lemma. Let z € Z be a T-fired point not contained in Z°. Then there exists a subset A ofT such that (La, La) contains a mazimal torus contained in (La, La)”, and z is conjugate under the Weyl group of L to a T-fited point in Z®.

Proof. Consider the map 7: Z — Xr. Then z(z) is a T’-fixed point (where JT’ is the image of T in L’ = L/C) not contained in the open L'-orbit in X,. Note that we can consider I in a canonical way as a basis of the restricted root system of the symmetric space L’/L’’. So by Proposition 4.4 and 5.7 we know there exists a proper subset A of l such that the derived subgroup (Li, L,) of the Levi subgroup L’, of L’

contains a maximal torus contained in (L,,L,)°.

And 2(z) is conjugate

under the Weyl group of L’ to a T’-fixed point in the open subset 2% of Zy =) L/ id, where Ue is the preimage of the open orbit in X,q under the map Z, — Xa. Now if we consider the Levi subgroup La of G, then (La, La) obviously contains a maximal torus contained in (La, La)’. Since the map Z, — XA

factors into the map 7: Za — ZA and Z4 — Xj, we see 11(Zi,°) = Z0. Since 1(z) is conjugate under the Weyl group of L’ to a point in Zio it follows z is conjugate under the Weyl group of L to a point in 2

Q.E.D. 6. The Weyl group orbits in the set

of irreducible components of Wr 6.0.

Let us now study the action of the Weyl group on the set of

irreducible components of Nr. In 5.7 we have seen the intersection of an irreducible component T of Nr with the open orbit G/H in X is the open T-orbit in 7. So we can identify the set of W-orbits in the set of irreducible components of Nr

with the points in NP /N7, where N2 := Wr NG/H. 6.1. We wish to recall various different interpretations of the set Ne Denote by Tg the set

Te :={M CG|

M is a maximal torus ,o(M) = M}.

Then H acts on Tg by conjugation.

For a maximal torus

M = gTg~' is M € Tg if and only if g~!o(g) is

an element in Nr. Let us introduce the set

P:= {9 €G|g7'o(g) € Nr}.

EQUIVARIANT COHOMOLOGY

239

We have a canonical action of H on P by left multiplication and of Nr by

right multiplication. The map p : P + NP given by p(g) = g~1a(g) is the quotient under A. Its image is Np, and p is equivariant for the appropriate actions. The map v : P > Tg given by v(g) = gTg™! is a quotient under Nr

and H-equivariant. So we can identify 7G with P/Nr, and Tg¢/H with H\P/Nr and Np/Nr. 6.2.

As in Remark 3.4 let us identify G with G(c), the connected

component of Aut G containing o. We can identify the conjugacy class A,

of o with G/H, where A, := {¢ € Aut G| ¢ = int(g) 0 o int(g)~! = int(ga(g—')) oc}. Let A,(T) be the set of ¢ in A, such that ¢(T) =T. Since ¢ =int(go(g)-!)o o is in A,(T) if and only if go(g)~* € Nr, we can identify A,(T) with NS. Summarizing, we have the following identifications: Proposition 6.3.

2) Tg/H = A,(T)/Nr = H\P/Nr = N?/Nr. 6.4.

Let T be an irreducible component of Wr and let z

€ T be a

T-fixed point. If P(z) is the parabolic subgroup associated to z (see 3.7), then the Levi part L of P(z) containing T is independent of the choice of z € TT by Proposition 4.4. So if we denote by (L) the Weyl group conjugacy class of L and by (n) € N?/Nr the class corresponding to the Wey] group orbit of 7, then we can associate in a canonical way to (n) the class (L). Denote by LT the set

LT := {(Lr)|f_ CE, (Lr, Lr) contains a maximal torus contained in

(Lr, Lr)’}. By abuse of notation we will sometimes say [ € LT or L € LT if (Lr) € LT or (L) € LT. The map (n) + (L) defined above induces by Remark 3.7 and Proposition 4.4 a surjective map ¢: N?p/Nr — LT. Lemma.

The map ¢ NS /Nr — + LT is bijective.

240

LITTELMANN

AND PROCESI

To see ¢ is injective, let L = Lr be such that T € LT, and

Proof.

let M be a maximal torus in L such that the intersection

MN (L-, L.) is a

maximal torus in (L,, L,.)”. And choose k € L, such that M = kTk71. We

have seen in 4.3 that nr := k~1o(k) is an element of NP, and the possible choices of nr are all conjugate under the normalizer Nor, ,T of T in L.. So we can canonically associate to L the class (nr) of nr in W?/Nr.

The following Lemma shows that if A € LT is such that (La) = (L), then L and La are conjugate under Nr N H.

But this implies the map

which associates to L the class (nr) defined above, induces a well defined map Ÿ : LT — Np/Nr. Note that the image of (nr) in LT under ¢ is (L) by Proposition 4.4, so Ÿ is the inverse map to ¢, and hence ¢ : Np/Nr — LT is a bijective

map.

Q.E.D. Lemma

6.5. Let T, I’ € LT.

Then (Lr) = (Lr) if and only if there

exists an element w € W! such that w(I’) =T. Proof.

Set V = X(T) @z R,V! = X(T') @z R and let C and C!

denote the dominant Weyl chambers in V and Vi. Assume there exists a w € W such that wLrw-!

= Ly.

Denote by

Ar the set of roots in A such that r(a) = 0 or r(a) ET. Let F be the face of C defined by

F={v€V | a(v) > 0 for a € Ap, a(v) = 0 for a € A — Ar}. The faces F! = FOV! and Fi = wF NV! are not empty. By Lemma 2 in [1] there exists a w; € W! such that w,F1 = F!. And hence there exists

aw’ € W! such that ww; (I’) =P. Now assume there exists a w; € W! such that w1(T) =I’. Recall,

W' = {we W |w(T') =T'}/{w € W |w(t) =t Vt ET}. If we W is a representative of w1, then obviously wLpw7! = Ly.

(el HD

6.6. Denote by Or the set of all G-orbits in X containing T-fixed points. We associate to any T-fixed point z the Levi part L(z) of P(z) containing T' and introduce on Or the following relation:

Gz~Gz,

REX

==>)

116) =(1G)).

EQUIVARIANT COHOMOLOGY

241

The bijection between LT and Np/Nr implies now G.z ~ G.z’ if and only if W.z and W.z’ meet the same irreducible components of Nr. Summarizing, we obtain: Proposition 6.7. The following sets are in a bijective correspondence to each other:

1) The set of Nr-orbits in NS. 2) Fe set of Weyl group orbits in the set of irreducible components of T-

3) The set of H-orbits in the set Tg of mazimal o-stable tori in G. 4) The set of Nr-orbits in

A,(T) := {¢ € Aut G |¢ = int(g) o¢ oint(g)~', 6(T) = T}. 5) The set of H x Nr-orbits in P := {g € G|g—!a(g) € Nr}. 6) The set LT of Weyl group conjugacy classes (Lr), where T C © is such that (Lp, Lr) contains a mazimal torus contained in (Ly, Lr)’. 7) The set Or /‘~’, where Or is the set of G-orbits in X with T-fired points and ‘~’ is the relation

G.z~G.z',

2,2EXT,

(L(z))=(L(2)).

Remark. Consider again XT = NT C Nr C X. Fix a 1-parameter subgroup in the interior of the dominant Weyl chamber. It defines a Bialynicki-Birula cell decomposition in the two varieties Nr and X. So for every point p € X7 we have two cells C, in Wr and Ce in X. Then there exists a unipotent subgroup U, of the Borel subgroup such that C. =U, xCp,

furthermore the B-orbits of X meet Nr in T-orbits. This follows from [12], and it gives (with a precise analysis of the torus embeddings forming Nr) a complete description of the B-orbits in X. 7. The ring Ax 7.0. Let us first recall the graphs. For I C © denote by And let Ch be the connected center Cr of Lr. We assume that L is such

notation introduced in the preceding paraLr the Levi subgroup of Pr containing T. component containing the identity of the that the Weyl group conjugacy class (Lr)

of Lr is an element of

LT = {(LA)|A CE, (La, La) contains a maximal torus contained in

(La, La)’}.

242

LITTELMANN

AND PROCESI

Bene by Xp the minimal wonderful compactification of Lp/(Lp)”, where = Lr/Cr, and let Zp be the variety Lr.id (5.0). Consider the morphism

ReLe — Xp (5.2). The fibre over the point id € Xp is Rp, the closure of the image Rp of Ci in S. We denote by Zp C Z the preimage of the open orbit in Xp. Finally, the finite group Nory Lr/Lf is denoted by Nr. Fix a system of representatives {[,,..., I} of the set LT. To simplify the notation, we will write L;, Z;,... instead of Lr,,Zr,,...

Consider the ring A = @;_, Ai defined by

. A:=

Hi, (2})". s=1

By Proposition 4.4 and Proposition 6.7 we know every Weyl group orbit

of a T-fixed point meets some of the Z?. So by the Localization Theorem the inclusions Z? + X induce an injection HZ(X) — A. Our aim will be to describe H£(X) as a subring of A. rae Let A CT; be such that A € LT. the set LA.R; is included in Z9 and in AE

Since La C L; and R; > Ra,

If [; is W1-conjugate to A, then one can find a h € H such that h.7? = Z. Note that the corresponding isomorphism

Hi (Zp yo

ie (aye

does not depend on the choice of h € H, since we are considering only N;-invariants. Definition 7.2. A triple (7,7, À) denotes a subset A of l,; which is conjugate under W! to T';. We associate to each triple (7,7, A) the morphisms

(i, j,A) : Ai; — Hp, (La-Ri) and W(i,j,A) : Aj —

Hf, (La-Ri)

induced by the isomorphism Hy (Z})™ © Hy (ZX)%* and the inclusions

Zoi lth Ne

à

7.3. For an element € € H%(X) let (af)= (af,...,a&) be its image in

A. Since the morphisms W and © are given by peste tons we have

(i,j, A)(a) = (i, j,A)(a) for any triple (A)? This motivates the following

EQUIVARIANT COHOMOLOGY

243

Definition 7.4. Let Ax be the subring of all elements (a) = (a1,...,a,) of A such that W(2,9, A)(a;) = Bt,7,A)(a;) for any triple (i,j, A). Remark.

1) Denote by x; the point zp,. By 5.8 is Norw: R;/Cenw:R;

isomorphic to N;. And by the isomorphism Hj (Z?)%: ~ Hy (Ri): (see 5.10), we see if (1,7, A) and (5,3, A’) are triples such that A and A’ are conjugate under Norw: R;, then

Y(i,3,A)(a;) = (6,7, A)(ai) > (6,3, A)(a;) = BC, 7,A’)(ai). So instead of considering in the definition of Ax all subsets of [; which are conjugate to l';, it suffices to consider a system of representatives of the Noryw: R;-orbits in the set of W!—conjugates of I; contained in F;. 2) Since the morphisms ® are defined by restrictions, one has furthermore the following compatibility: Let A C T; be W!-conjugate to Tj and

let

TYC A be Wl-conjugate to T;. If (a) € A is an element such that

W(i,7,A)(a;) = (i, 9, A)(ai), and W(j,k, Y’)(ax) = OG, k, Y’)(a;) for any triple (j,k, Y’), then

W(i,k, T)(ax) = B(i, k, Y)(aj). 7.5. We want to describe now the morphisms ®(1,j,A) and W(i, j, A) more explicitly. Since for x; = zp, the subvariety Z? is isomorphic to Li x1,.. Ri by 5.6, we get

Ai = (Hig (Pt) © Hg RD". Furthermore, La.Ri = La X1e.c1 (Ck Xc: R;), so that

Hy, (La.Ri) = Hi (pt) ® HR.(R). The morphism ®(i,j,A) can be now described as the restriction to the N;-invariants of the tensor product of the map

Hj (pt) > Hy«(pt),

244

LITTELMANN

induced by the inclusion LY + And for

AND PROCESI

LY, with the identity map on HF, (Ri).

Aj 2 (His (pt) © Hp, (R))™ the map (i,j,

A) can be described as the restriction to the N;—invariants

of the tensor product of the isomorphism Hj j. For € € HZ(X) denote by (af)

LITTELMANN

246

AND PROCESI

Ax. By Theorem A we have a canonical graded filtration of

its image in

HG(X) D

CHAR) Cow CHE)

Ft HEY)

= EN

with FiH%(X) = {€ € HE (X)|a$ = 0 for all j < i}. Denote by Gr HE(X) the associated graded group.

4

We have seen in Proposition 5.7 the set L. ;.Riis isomorphic to L.;Te

R; and hence is a smooth, compact submanifold of Z;. We will see later that the Euler class F; of its normal bundle in Z; is a nonzero

divisor in

Hi, (Lea-Ri). Denote the degree of E; by d;. Theorem

B.

Gr HE(X)~NX @ t=1

(Hie (pt) © Hh,(Ri))™)

a+b=k—-d;

Remark. Let y; be a T-fixed point in Z? and denote by 7; the irreducible component of Wr containing y;. Denote by W; the subgroup of W of elements normalizing 7;. Another way to state Theorem B is by 5.10:

Gr H3(X) = DH (Ti. al

9. The proof of Theorem

A and B

9.0. The strategy of the proof of the theorems will be to define a filtration of Ax compatible with the inclusion H2(X) C Ax, and then to prove the associated graded groups are isomorphic.

We assume throughout the following that the enumeration of the sets P1,...,7, is such that if [; is W!-conjugate to a subset of [;, then i > 1: _ 9.1. We will first define a filtration of Ax. For i = 1,...,r denote by F'Ax the subspace F'Ax

=

{(a) € Axla;

= 0

for all

9 Dar 4h)“ /LQ@ Ar (RL) sa)

scsi

is an isomorphism. But the dimension of both sides is by Proposition 6.7 and the previous considerations equal to HXT.

Q.E.D. 10. Proof of Proposition 9.3 10.0. We use the same notation as before, only we denote by Nz the group N; and by W7 the group Wj, and we drop the index 7 else. We want to study the kernel

K = Ker{(Hj. (pt) @ HR(R))™*

+

CG)

Hi. (pt) @ HR(R)},

AELT ACT Ag£T

which is isomorphic (see 7.5) to the kernel of the map

Ve

©

A€ELT ACT AZ

Hi, (La Xxg.c4 (Ca Xo R)).

Note that the map Q — Q? is injective, for an element in the kernel would vanish at all T-fixed point in Z, which is not possible by the Localization Theorem.

Proposition 10.8.

10.9.

The map Q — Q° is an isomorphism.

This implies that the image of QN

Hence we obtain by Corollary 10.6:

is Q°N Hj(Z°)N*

= K.

252

LITTELMANN

AND PROCESI

Proposition 9.3. K = H7-4(T)"7. 10.10. The proof of Proposition 10.8 needs some preparation, for we have first to recall the results in [6] on the equivariant cohomology of regular embeddings. Let G be an affine algebraic group acting on a connected algebraic variety X. We say X is a regular embedding, if each orbit closure is smooth and it is the transversal intersection of the codimension one orbit closures containing it. We require furthermore that for any x € X the stabilizer G,

has a dense orbit in the normal space to G.z C X at x (see [6],§3). alli

Let O;,...,O; be an enumeration of the G-orbits in X such that for =1,...,r the union X; := O,; U...UO, is open in X. Then the maps a

Hg(Xi) — Ri = GDHG (Or) kal

are injective, and the restriction maps HÉ(X;) —> H%(Xi-1) are surjec-

tive. A description of H6(X) as a subalgebra of R; can be found in [6], §7. We will use this result to describe H7(Z). 10.11. Note first that the L-variety Z is a regular embedding. By 5.0, the first condition is satisfied, and we see by the description of the local structure of Z in 5.0 that L, has an open orbit in the normal space N, to the orbit L.z in Z at z. We associate to Z a simplicial complex C = (V,S), whose vertices are

the orbits of codimension one, and F C V is a simplex whenever Given Oe is not empty. We include the empty set among the vertices, so the set of simplexes is in one-to-one correspondence with the L-orbits in Z (see

[6],§3). For F € S denote by Or the corresponding orbit, which is the unique open L-orbit in (1),¢p Ov. Let x, € H7(Z), v € V, be the equivariant Chern class corresponding to the (linearized) line bundle £, associated to the divisor Oy. For F € S choose z € Or such that z = w.z, for some subset A of Z and w € W!. Denote by Lr := L, its stabilizer in L. By the choice of z the intersection Kp := Le N Le is a maximal compact subgroup of

Lr (3.8). By the description of the stabilizer K,, in 3.8 one knows Kr is o-stable and has a decomposition Kr = K%.Cpr, where Cr is a o-split compact torus, and Kp = (CenxCr)° N Le. The subgroup Kf operates trivially on the normal space N(z) of Op at z. Denote by L,(z) the fibre of the line bundle £, at z. Since Or is the transversal intersection of the O,, v € F, is

EQUIVARIANT COHOMOLOGY

N(z)~

253

OBL,(z) ver

as CF representation and dimrCr isomorphism

= dim N(z).

So there is a canonical

Hy(Or) = Hx, (pt) = He (pt) ® Q[rvher. If F’ € S is such that F’ C F (which is equivalent to Or C Or:), then we can find elements z € Or and z’ € Op, such that z = w.za, z! = w.xr,

where Ÿ D A and w € W!.

The inclusions (see 3.8) KZ C K¢%, and

Cr D Cr: induce canonical morphisms

O(F, F’) : Hj (Or) — Hye (pt) @ Qrvher/(zv,v € F’) YF, F’) : Hy (Or!) — Axz (pt) ® Qrvher/(zv,v ¢ F’). Remark.

A geometrical interpretation of these morphisms in terms of

a tubular neighborhood of the orbit Op is given in [6], §7. Definition 10.12. Let Rz be the subalgebra of R := Ores Hz (Or) of all elements (rr) res such that

®(F,F'\(rr) = V(F, F’)(rr)

forall

F,F’€S,

F'CF.

Proposition 10.13. ([6],87, §14) The image of Hj (Z) — R coincides with Rz.

10.14. Since every T-fixed point in Z is conjugate under the Weyl group of L to a T-fixed point in Ze for some A CT, A € LT, by the Localization Theorem we have an injection

Hi(Z)>

©

Hj, (42).

AGCLNELT

A connection to the inclusion H7(Z) — R is given by the following

LITTELMANN

254

AND PROCESI

Lemma. For every L-orbit Or in Z there exists a subset A ofT and a point z € Op such that z € ZS and the restriction map

H}(Or) — Hi, (La-2) is injectlive. Proof.

Choose

z € OF such that z = w.ry

for some

subset YT of Y.

Let

Krp=K2.Cp be a decomposition of the stabilizer Kr of z in Le. as in 10.11. the Levi subgroup Cenz Cr of L. By 10.11 we have (Cen, Cr)” N Le =

Let M

be a maximal

o-stable

Kp

torus in CenzCr

Consider

(2)

such that M N KEY is

a maximal torus in Kf. We can assume that the connected component containing the identity M1 of the o-split part of M is contained in T!. Note that Cr C M! by construction.

Recall that W'(T) is the subgroup of W! generated by the reflections sy, y ET. Replacing z by a W!(T)-conjugate if necessary, we can furthermore assume that Cen, M! = LA for some subset A of I. By construction, we have M° N (La, La) is a maximal torus of (La, La) , so A € LT. And (2) implies L{, is a subgroup of maximal rank of K%. Since by the choice of z the stabilizer (LA); is a maximal compact subgroup of

(La):, the inclusion

ORS

LG)

CPE

aC

induces an injection in equivariant cohomology

Hi(Or) © Hi. (pt) — Hi. (cry, (pt) © Hi (La). Consider the morphism 7, : Za — X,q, where X\ is the minimal wonder-

ful compactification of the symmetric space ln /Dis) andulc= EN Ox Since 7,(z) contains L/” in its stabilizer, we see that z is contained in the preimage of the dense L'\-orbit in X,, which implies z € VANS

Q.E.D.

EQUIVARIANT COHOMOLOGY

255

10.15. We want now to prepare the proof of Proposition 10.8. Consider a L-orbit Or in Z such that Or C Z°®. Since m(Op) is the dense L'-orbit in Y, every codimension one orbit in Z which contains Or in its closure is contained in Z°. Furthermore, by 5.6 the intersection of a L-orbit with

R is one R-orbit.

So if we still denote by x, the class in H}(R) of the

line bundle L,|z, then H}(R) is generated by the z,, O, C Z°. And the inclusion Or C Z° induces an isomorphism

Hy (Or) = Hj. (pt) @(HR(R)/(zvlv ¢ F, Ov C Z°))

(3)

Let Op, be a L-orbit such that F C F’ and Op is not contained in Z°. Let ACT, A € LT, and 2’ = w.zy € OF N ZA be such that restriction map

Hi (Or) — Hi, (La.2’) is injective (Lemma

10.14).

The injection La.R C Z9 induces by 7.5 a

morphism H7.(pt) ® HA(R) > Hj (pt) © H}(R), and hence a morphism

Hj. (pt) ® (HR(R)/(2vlv ¢ F,Ov C Z°)) *

*

(D

0

— Hj:(pt)@ HRCR)/(zlv € F,O, CZ) Since we can choose z! = w.ty

€ Op N Ra and z = w.xy

(4)

E Or, by

the isomorphism (3) and the description of Y(F"”,F) in 10.11, we see the morphism in (4) is the composition of the morphism W(F’,F) and the morphism induced by the inclusion L& C LY,

Hye, (pt) —

Hy (pt).

We have seen in 10.14 that the last map is injective. So W(F", F)(€)= 0 for any element €° in the kernel of the map H7(Z°) > H}, (La-R). For € € H}(Z) denote by €° its image in Hj (Z°) pa by ér,

FES,

its image in Hj (Or). We obtain as consequence of the discussion above: Lemma 10.16. If €€ H}(Z) is such that €° € Q then Y(F', F)(Er) = 0 for all F,F'ES, with FC F’ and Or C Z°, Or ¢ 2°.

256

LITTELMANN

AND PROCESI

10.17. Proof of Proposition 10.8. It remains to prove the surjectivity of the morphism Q — Q°. Since Z° is an open subset we can choose an enumeration O:,...,0, of the L-orbits, such that Z; = O,U...UO;, is open in Z for alli =1,...,r and Z° = Z; for some j. So by 10.10 the restriction map

Hj(Z) — Hj(2°) is surjective. For €° € Q®, let € € H7(Z) be such that its restriction to Ze

is equal to €°. Denote by £pr its image in Hf (Or) for

F € S. By Lemma

10.16, we know W(F’, F)(€r) = 0, whenever Or C Z° and Or ¢ Z°. So

the element (£/,) ofR with & = Er if Or C Z° and &, = 0 else, is in Rz. Hence by Proposition 10.13, one can assume that the restriction of € to any orbit not contained in Z° vanishes. But then the restriction of € to any T-fixed point not contained in Z° vanishes, which by Lemma 10.1 implies that £ € Q. Hence the morphism Q — Q? is surjective. Q.E.D. 11. Equivariant cohomology of wonderful compactifications 11.0. We want now to reformulate Theorem A and Theorem B in §8 for the general case of a wonderful compactification.

11.1. A wonderful compactification Y of G/H is a complete, smooth G-variety Y with an embedding G/H + Y and a G-equivariant morphism @:Y — X to the minimal wonderful compactification X of G/H (see 3.2),

which extends the identity on G/H (see [10]). 11.2. We use the notation as in §3. Let Sy be the closure of S = T!/T10H in Y. Then Sy is a smooth, complete W!-equivariant torus

embedding of S (see [10]). Let Sx be the closure of S in X.

The restriction of ¢ to Sy is a

W?—equivariant morphism of S-embeddings and $~1(Sx) = Sy (see [10]). Furthermore, Y is completely determined by Sy. This means that for every W!-equivariant, smooth, complete S-embedding S with a W1equivariant morphism 7 : S — Sx there exists (up to isomorphism) a unique wonderful compactification Y, such that Sy is isomorphic to S and the restriction of 6: Y + X to Sy is Y (see [10]).

11.3. For asubset [ C ZX let Lr be the Levi subgroup of Pr containing T, let Cr be its center and let Ch be the connected component containing the identity of Cr (§3). We assume that its Weyl group conjugacy class

EQUIVARIANT COHOMOLOGY

257

(Lr) is an element of LT = {(La)|A C EX, (La, La) contains a maximal torus contained in

(La, La)°} By Xr we denote the minimal wonderful compactification of Li/Li”, where Lp = Lr/Cr, and by Zr the variety Lr.id. Recall that Z2 is the preimage in Zp of the open orbit in Xp under the map 7: Zp — Xp (5.2).

And the fibre x! (id) is the closure Rp of Rr = CL/CLNH

in Sx. Let y

be a T-fixed point in Zp and let 7r be the irreducible component of Nr

containing y, which is by Proposition 5.6 the fibre 1~1(z(y)). We index the corresponding preimages in Y by Y, i.e. Zyr = ¢71(Zp), Zyr = o>) (70), Eye, =.¢;, (Rr) and Tyr = ¢\(Tr). We will see

later (see 11.12, 11.13) that these are smooth varieties, and the canonical morphism:

Lr Xpec1 Ry — Zyp is an isomorphism. Finally, Nr is the quotient subgroup of elements in W normalizing Tp.

Nory Lr/LF, and Wr is the

11.4. Let {T:1,...,1,} be a set of representatives of the set LT. index the corresponding groups and varieties just by 2. Consider the ring A = @j_, À; defined by

We

A= Qi (23). 1—1

Since

¢: Y — X is G-equivariant, the Wey] group orbit of every T-fixed

point meets some Z?, by Proposition 6.7. induce an injection

So the inclusions ZY air

HG(Y) — A. Posi

AC Ly A 6 LT, then LA CL;

and Ry, (ΠRyA.

So La Ry

is contained in Zy., and in Zy.;. And if À is conjugate to l';, this induces a canonical isomorphism

Hy (2Zy,5)" = Hy ,(Zy,n)”*. 11.6. By a triple (1,7,À) we mean a subset A of I’; which is conjugate

under W! toT;. To each triple (7, j, A) we associate the morphisms (i,j,

A) ; À; =>

HE, (La

Ry;) and

W(i,j,

A) 5 À; =

Hy, (La-Ry,:)

LITTELMANN

258

AND PROCESI

induced by the isomorphism AH}, (29)

— H}, (ZY A)" and the inclu-

sions

5 0 ZY; 2 La X 1201 (Ca Xc1 Ry) € Zy,nDefinition 11.7. Let Ay be the subring of all elements (a) = (a1,...,a;) of A, such that (i,j, A)(a) = Y(i,J,A)(a;) for any triple (ape)

Remark.

The comments on the definition on the ring Ax in Remark

7.4 hold as well for the definition of Ay.

Theorem A’. The inclusion HZ(Y) — A induces an isomorphism of rings

HG(Y) = Ay 11.8.

Assume that the elements l'; are enumerated such that if I’; is

W1-conjugate to a subset of l';, then i > j. For €€ H%(Y) denote by (af) its image in Ay. As in 8.3 consider the graded filtration

OS FO THCY CP Ba

CS Ger Ot

eae

with F'HE(Y)= {fé € He(Y)|ai = 0 for all j < i}. And denote by Gr H¢(Y) the associated graded group. We will see later (11.15) the set L.; Ry

is a smooth

submanifold

of Zy;. Let d; be the degree of the Euler class in Hj. (ej. Ry.;) of the corresponding normal bundle. Theorem

B’.

Gr HEY) =~ Qo t=1

@D_ (Hie (vt) @ Hh, (Ry,))™*) a+b=k-d;

DRE

(Ty)

t=)

11.9. The proof of the Theorems is essentially the same as in the case of the minimal compactification, so we omit it. It remains to prove the properties of the varieties Zy ;, ZY and Ry;; Stated above. We wish first to recall some results about the structure of the wonderful compactifications.

EQUIVARIANT COHOMOLOGY

259

11.10. Let À be the affine chart of Sx corresponding to the negative

Weyl chamber (see 3.6), and denote by V its preimage #-!(A) in Y. Then V is a smooth torus embedding, and every G-orbit in Y meets V in a unique S-orbit. Furthermore, let P be the stabilizer of the unique T-fixed point in A. Let P~ be a parabolic subgroup opposite to P such that TC PNP-, and denote by U~ its unipotent radical. The canonical map U-xV—Y

is a P~-equivariant isomorphism onto an open subset of Y. As a consequence we see that the closure of every G-orbit in Y is smooth, and it is the transversal intersection of the codimension one orbit

closures containing it (see [10]). 11.11. To describe the stabilizer in G of a point y € Y we may assume

that ¢(y) = x := zp for some subset I of & (see 3.6). Set P = Pr, and C! = Ci. The stabilizer of y is (see [6],§9)

L= Ly

GH LPC), 1112

For lcd

and f=.

Lr consider now

Z =

Evd’and.Zy

=

¢—1(Z). Since ¢-1(Sx) = Sy, it follows by Corollary 5.3: Zy = L,.Sy.

Hence every L-orbit meets Sy in a union of T-orbits, which are permuted transitively by W1(T), the subgroup of W! generated by the reflections s,, er. : Let A be the affine chart of Sx considered in 11.10, and let z be the unique T-fixed point in A. The stabilizer L, is a parabolic subgroup of L. Let P; be a parabolic subgroup ofL opposite to L,, such that TC L,NPr , and let U; be its unipotent radical. We obtain by 11.10 for V := go—1(A): The canonical map Ur

xV

—

Zy

is a P; -equivariant isomorphism onto an open subset of Zy. As a consequence we see Zy is a regular embedding in the sense of 10.10.

11.13. Consider a C1-fixed point y in #'(ær). The center C of L is contained in Ly, so the variety Yp := L.y is a smooth compactification of

LITTELMANN

260

L'/L'’, where L' = L/C. ification of L'/L'°. Since to¢:Y, — Xz, Y_z is in The image T" of T1

AND PROCESI

Denote by Xz the minimal wonderful compactthe image of y is id € XL under the morphism fact a wonderful compactification of DLE : in L’ is a maximal split torus, and

SST

(Th mule

is isomorphic to S/Sy. Since Sy Sy, the r.p.p.d. of the closure Ge of S’ in Yz, is the image of the r.p.p.d. of Sy under the morphism

Homz(X(S),R) —

Homz(X(S’), R).

This implies further we have well defined morphism Sy — oe Denote by

See the closure of S’ in Xz. Proposition

11.14.

The morphism L/L? —

L'/L'° extends to a

morphism Zy — YL.

Proof. The map L/L? — L'/L'° induces a rational L-equivariant map Zy — Y,. The set of points in Zy where this map is defined is an open, L-stable subset of Zy. Let now y be a T-fixed point in Sy, x its image in Sx, z’ its image

in sy and let y be its image in Sy. under the morphism Sy — So Note that 2’ and y are T’ = T/C-fixed points, and ¢(m(y’)) = 2’. Let

A(z) and A(z’) be the affine charts of the torus embeddings Sx and oy corresponding to the T- respectively T'-fixed point, and denote by V(y) and V(y’) the preimages in Zy and Yr. The morphism Sy — 5. induces

a well defined morphism V(z) —

V(y’).

Furthermore, let

P C L and

P' C L' be parabolic subgroups opposite to Ly respectively Lo such that TCLyNP and TC P'NL,,. Then the unipotent radicals U and U’ of P and P' are canonically isomorphic, and the restriction of the morphism

U- x V(y) 3 U™ x Vy’) to the intersection with L/L° coincides with the morphism L/L° — L'/L'°. _ Since every L-orbit meets such an open subset for some T-fixed point

in Sy, the morphism L/L? — L'/L'° extends all of Zy.

Q.E.D.

11.15. We suppose for the rest of the paragraph that LT € LT. As an immediate consequence of Proposition 11.14 we see:. 1) Set x = zp, let y be a T-fixed point in Z°, and denote bye i=

m~"(m(y)) the irreducible component of Nr containing y. Then AE #7" (29) is isomorphic to the fibre spaces

EQUIVARIANT COHOMOLOGY

Lx, By.and

xr} Tyssawhere Tr

261

om (FE)

The subset L..Ry is a smooth compact submanifold of Z. 2) Ry is a smooth R-embedding. The corresponding r.p.p.d. intersection of the image of the inclusion

Homz(X(R),R)

is the

C Homz(X(S), R)

with the r.p.p.d. of Sy. 3) For a T-fixed point y € Ty the intersection G.y N Ty is a union of T-fixed points permuted transitively by Norx7T = Norx7y. 4) Nr y := (Nr) is Nr-stable, and it is the disjoint union of smooth T-stable varieties. All T-fixed points in Y are contained in Wr y. The Weyl group orbits in the set of irreducible components of Wr y is in one-to-one correspondence with the set LT.

5) The norraalizer Wz in W of the irreducible component 7 in W is equal to the normalizer in W of Ty. Denote by Nz the quotient group L.Norx L°/L. There are canonical isomorphisms in equivariant

cohomology

Hy(Zy)™* ~ (Hie (pt) © Hp(Ry))™* ~ Hz(Ty)””. Remark. Using 5) one can show that the morphisms ®(7,7,A) and W (i,j,A) can be described as in §7 as maps induced by the inclusion La C L; and Ry; GRY:

References

[1] S. Abeasis, On a remarkable class of subvarieties of a symmetric variety, Adv. in Math. 71 (1988) , 113-129. [2] M.F. Atiyah, R. Bott, The moment map and equivariant cohomology, Topology 23 (1984) , 1-28. [3] A. Bialynicki-Birula, Some theorems on actions of algebraic groups,

Ann. of Math. 98 (1973) , 480-497. [4] A. Bialynicki-Birula, On fixed points of torus actions on projective varieties,

Bull. Acad. Polon. Sci. Sér. Sci. Math. Astronom et Phys.

22 (1974) , 1097-1101. [5] A. Bialynicki-Birula, Some properties of the decomposition of algebraic varieties determined by actions of a torus, Bull. Acad. Polon.

Sci. Sér. Sci. Math. Astronom et Phys. 24 (1976) , 667-674.

AND PROCESI

LITTELMANN

262

[6] E. Bifet, C. De Concini, C. Procesi, Cohomology of regular embeddings, Adv. in Math., to appear (1989) . [7] A. Borel et al., “Seminar on Transformation Groups,” Ann. of Math. Studies 46, Princeton, (1961) . [8] C. De Concini, M. Goresky, R. MacPherson, C. Procesi, On the geometry of quadrics and their degenerations, Comment. Math. Helvetici 63 (1988) , 337-413. [9] C. De Concini, C. Procesi, Complete symmetric varieties in,“ Invariant

Theory,”

Springer Lecture Notes in Mathematics, 996 (1983) , 1-44.

[10] C. De Concini, C. Procesi, Complete symmetric varieties II in,“ Algebraic Groups and Related Topics, Advanced studies in Pure Math.” 6 (1985) , 481-513. [11]

C. De Concini, C. Procesi, Cohomology of compactifications of alge-

braic groups, [12]

Duke Math. J. 53 (1986) , 585-594.

C. De Concini, A. Springer, Betti numbers

varieties in,“ Geometry of Today,” 107.

of complete symmetric

Birkhäuser-Boston,

(1984) , 87-

[13] S. Helgason, “Differential Geometry, Lie groupsand symmetric spaces,”

Academic Press,

(1978) .

[14]

W. Hsiang , “Cohomology Theory of Topological Transformation Groups,” Springer Verlag, (1975) .

[15]

P. Slodowy, “Simple singularities and simple algebraic groups,” Springer Lecture Notes in Mathematics, 815 (1980) . Received November 20, 1989

Peter Littelmann Istituto Guido Castelnuovo Universita di Roma “La Sapienza” Piazzale Aldo Moro, 2 00185 Roma

Claudio Procesi Istituto Guido Castelnuovo Universita di Roma “La Sapienza” Piazzale Aldo Moro, 2 00185 Roma

Italia

Italia

Nilpotent Orbital Integrals in a Real Semisimple Lie Algebra and Representations of Weyl Groups W. ROSSMANN! Dedicated to Jacques Dixmier on his 65th birthday 0. Introduction

Let gp be a semisimple real Lie algebra, Gp = Ad(gp), hp a real Cartan subalgebra. For À € hp, let u\ denote the canonical invariant measure

on the GR-orbit of À in gp. A well-known theorem of Harish-Chandra [8] says that

Jim Wf) = Ky, With

w=

II Oo; acAT

the limit is taken through regular À, and « is a constant, which is nonzero if and only if hp is fundamental. The differential operator @ transforms according to sgn under the Weyl group W of (g, h). This is the irreducible character associated to

{0} under Springer’s correspondence between nilpotent orbits in g* and irreducible characters of W. This paper deals with the problem of finding an analogous formula for arbitrary nilpotent Gp-orbits. The problem is solved in theorem 5.3 only under an additional hypothesis. The correspondence between real nilpotent orbits and certain representations of W is given in theorem 3.3, based on the theory for complex groups developed in [16], which is recalled in §2. The general framework for the study of Fourier transforms of orbital contour-integrals is given in §1.

Barbasch and Vogan [3] solved the problem for complex orbits in the classical groups and formulated

the result as a conjecture for complex

orbits in general [4]. Their conjecture was proved in [10] and in [16]. For Gp = U(p, q) the solution was given by Barbasch and Vogan in [5]. I thank Michèle Vergne for correcting a mistake in the proof theorem

4.1. lSupported by a grant from

NSERC Canada

264

W. ROSSMANN

1. Coherent families of contours and invariant eigendistributions Let g be a complex semisimple Lie algebra, gp a real form of g. Let

G:= Ad(g), GR := Ad(gp).

Fix a Cartan subalgebra h of g and let W be

the Weyl group of (g, h). We recall some definitions and results from [15], [16].

We shall be

interested in integrals of the form 1

n YO]

(—2ri)

n!

leo

po

(1)

À

where À € he is a regular element in the dual of the complex Cartan subalgebra h,

TOGO ea Coen

is an 2n-cycle with 2n = dimgQy), oy” is the n-th exterior power of the canonical holomorphic 2-form on Q), ox(z'é, yré) = E([z, y],

and

£.€

p(é) := |ie de,

GA,

2,9

ENS,

f € CR)

R

is the Fourier

transform

of a compactly supported

C™°-function

on gp.

We note that ¢ is an entire holomorphic function on g* satisfying for all N an estimate of the form

ly(é)|

AchllReéll

< —*——_,

1+ ey”

(2)

The cycles '(\) must be restricted so that (2) guarantees the convergence of (1): one considers locally finite sums of singular m-simplices on Q re Hp

which satisfy

nie

(al Say

k

(a) ||Reo, || < C for all k, >)

| c, |maz 71

1+ |lo,||*

| & (ung: (We + vu) + YN) Here

k €

UN GR, ty €

um (He +vmy=mH um (He +Y¥y)=™M,"H, GR: A. the map p,,, CS

UNM,

k-(m,* pe + Vy)

This

transformation

vy € b, + Oumar

‘ste

and

with m, € MR. Since M, fixes wp, — Hy gives Uy: (wu, + vy) = mu ‘Um Thus on is given by

maps

>

+ (m,w,

Gp-p,= GRw.u,

pe + vx)

into

itself.

The

map

W. ROSSMANN

268

MR he > M, RYH, preserves the orientations induced by the restrictions of io, (because sgn(w) = 1), hence the transformation Pao lo Pu of Gop.< preserves orientation as well.

1.3

Remark.

Q.E.D.

It may be shown that for any w €

W.R

GR He] + ++ Pup |© Pul GR’ Mel = SM where the dots indicate a Z-linear combination of contours [Gp- 4,,] with

hencuhy cr

The contours {GR:p,] fit into coherent families as follows. 1. 4 Theorem. (a) Let S be a GR-orbit on B. There ts a unique coherent family T(S, - ) so that r(5, hi) Su [Gp- He]

whenever c

€ Gand p € Bice satisfy the following conditions:

(1) he € 8p

(2)b,¢€ 8 (3) up (Hy,) > 0 if a, is an imaginary root of h, on b,. (6) The coherent families of eigendistributions 0(S, - ) corresponding to the T'(S, - ), S running over the Gp-orbits on B, form a Z-basis for

the Z-module CH(gp) (c) Let (hr, ¢ € G,

be a real Cartan subalgebra of gp.

a GR-orbit on B so that b, € S.

RA

O(S, À) = = RT,pe UMTS Cpe on

(h.)p

h,, —

and

vanishes

on

Let S be

Then

WeR

a out Cartan

ys

subalgebra

(6) (h,,)p unless

©

Proof. (a) Fix S and a pair (c, u) with the indicated properties. Let l'=T(c, p, S) € 1H», (B*) be defined by T= pop,

(GR: cp].

We show that lis independent of (c, u). pair. In view of (2) we may assume that b= Be — bo.

.

(7)

Let (cr, ur) be another such

ORBITAL INTEGRALS

269

In view of (1), h, and h,, are both conjugation-stable Cartan subalgebras in bs. It is well-known that such Cartan subalgebras are Gp-conjugate. We may therefore assume that

Thus

c= ch for some

h € Centg(h).

Observe: There is w € W{A, c-p and wc:

We may as well assume

c= cv.

R) so that

lie in the same connected component of (hs R)reg (8)

Reason: Such a connected component is a chamber cut out by the real and imaginary roots of hg. In view of (3), only reflections in real roots are needed to transform the chamber containing c-p into the chamber containing cp. Fix w € W(A, RB) so that c-y and we- pr lie in the same connected component of (hs Brey There is a continuous curve p(t),0 < ¢ < 1, in

h* so that

u(0) =p, w(1)= cf wep, c- p(t) € (hsrrey0 St A, p, [y] — [9(1)]. _

Fix a Gporbit

O=

GR:v

on

Np.

Let O be

the

Q.E.D. its closure,

00 :=

O — O its topological boundary. 3.6

Lemma.

The

natural maps

0 — H»,($(90)) > He,($(O)) — He,($(O)) — 0 form an exact sequence of W-modules.

Proof. The couple 80 C O of closed R-subvarietes of N R gives a long-exact sequence of W-modules

+ — Hy, ($(0)) + H{#(00)) > H{#(0)) > In top degree 1 = 2n this gives

— H{3(0)) > H;_,(9(00)) > +

The surjectivity of the map on the right follows from the fact that O and

00 admit compatible CW-decompositions [7] and standard facts about the homology of CW-complexes ([12], Theorem 4.1). Q.E.D. Proof of theorem 3.1. The map 4,,(#(0)) — H,,(?) is injective, as one sees from the exact sequence associated to the inclusion (0) C f. Hon(#(0)) may therefore be considered a W-submodule of H,,(#), and

H»,(£(O)) a W-subquotient of H,,(¥):

Hyn(3(0)) © Hon(3(O))/ D, Hen($(O)) ; Or< O

where Or

< Omeans Or C 00.

The filtration of Np according to the dimension of the Gp-orbits,

dimpO He,($(O)),

(13)

Hy,($(O)) % H,,(BryA¥®),

(14)

The inclusion BY S & induces a W-injection

Hp (BY)4Y S Hy (B)

(15)

({16], Lemma 3.1.) The character of Won H, (8) vi is X, = Xv1According to Borel [6], the cohomology ring of 8 can be described as follows. For À € h*, let 7) denote the U-invariant 2-form on 8 which at the base point b, is BS by

Ty([z-by, y- b,]) = A([z, y]) for x, y € u. Let It denote the ideal in the ring C[h] of polynomial function on h generated by the W-invariants without constant term. isomorphism of rings

There is a unique

(16)

C[h]/I* — H*(B) so that

DITES The dual space of C[h]/J* is the space K(h*) of W-harmonic polynomials on h*, and the transpose of the isomorphism (16) is a W-isomorphism

by

ser), 7 — ey

(17)

given by €

u AR

(-2ri)°[ra

for y € H, (8). For v € N, let 36 ,(h*) denote the space of W-harmonic polynomials on h* which are homogeneous of degree e = dima B” and transform accor-

ding to the irreducible character x,, of W.

From (13), (14), (15), and

W. ROSSMANN

278

(17), together with

He,($(O)) À Hog(3(O))/ D, Hon($(O)), O m(w) w~4 we W

operates as the projection on the space of such elements.

6(C,A) == er Sy m(w) e” =X

Then

on hp.

(26)

we W

By what has just been said, the expression on the right is non-zero.

Expand the exponentials inAG = eed HOME. k=0

S> m(w) w —1)kla. weW

Compare (27) with (25):

O(C, A)= 6, + OMAN),

a>) À 0.

One finds that wt m(w) w

1 =0

fork


> m(w) w —1X° we W

ce (A) Oy

(27)

284

W. ROSSMANN

= sgn(y) m(w) for y € W, also c (yd) = samy) c, (yA). Since m(yw) Hence c,( - ) is the (up to scalars unique) W-harmonic polynomial on h* which is homogeneous of degee e, transforms by x, under W, and Q.E.D. transforms by sgn; under Wp. Hence (p,, c,) # 0. The “multiplicity-one” hypothesis, with the help of the following criterion.

when

it holds, may

be verified

5.5 Lemma. Let G be a locally compact group, o an automorphism of G, H a o-stable, compact subgroup of G. Assume that for g in some set of H:H double-coset representatives (and hence for all g € G), Cae

—1

with h,, k, € H. Let p: H— C* be a one-dimensional representation

of

H so that p(h®) = p(h) for all h € H and p(h,)= p(k,) for al g € G. Then indy p decomposes with multiplicity one.

Proof. The algebra of continuous G-endomorphisms of ind Gp contains as a dense subalgebra the convolution algebra of compactly supported, continuous, C-valued functions y on G satisfying

y(hgk) = p(h)p(g)p(k)" for all g € G, h,k € H. algebra is abelian.

Let g7 =(g9°)~’. T is an

If suffices

to show

(28) that

this convolution

If y(g) satisfies (28), so does y7(g) := (97).

Since

anti-automorphism of G,

(pap) = px". On the other hand, a function y as in (28) satisfies

(29) y” = y:

9" (9) = p(k, }e(g)p(h,) = (9). Applied to the functions in (29), this gives p+y = yxy as required.

. 5.6

Remark.

Q.E.D.

This kind of argument goes back at least to Gelfand

[11]. (See also [9] Ch. X, Theorem 4.1.)

ORBITAL INTEGRALS

285

The following result is due to E. Neher (unpublished). 5. 7 Lemma. Assume hp is a Cartan subalgebra of compact type in 8p: Then any left (or right) coset of Wp in W has a representative s € W satisfying s° = 1. Proof.

Any element of W is a product of reflections, say

"*Sy8g°*"-

(30)

Generally

Sq5g = $8, Where

7 = 5,0.

(31)

If a is compact and B is non-compact, then s,( is non-compact.

So (31)

may be used to bring all the compact roots in (30) to the left (or to the right). The coset representative may therefore be chosen of the form (30) with non-compact roots only. It suffices to verify the following statement.

(32)

If a, B are non-orthogonal, non-compact roots, then s,( is compact.

Assuming this, any representative of the form (30) with a minimal number of reflections must consist of non-compact orthogonal reflections:

otherwise (32) and (31) could be used to reduce the numbers of reflections without changing the coset. The statement (32) concerns only the subalgebra of g generated by root-vectors for a, B and its real form obtained by intersection with gp.

This algebra has rank two and (32) may be verified by inspection of the rank-two root systems.

Q.E.D.

5. 8 Corollary. Assume hp is a Cartan subalgebra of compact type in Zp. Then ind yw,SN decomposes with multiplicity one. Proof.

5.9

One can take o = identity in lemma 5.5.

Example:

Then y, = sgn. É

Harish-Candra’s

Limit

Formula

Q.E.D.

[8]. Take

v = 0.

Let

a=

II 03 acAt

with 0,» = dp(a) as operator sgn.

on h”.

æ transforms according to x, =

Theorem 5.3 becomes lim @ ad = ENG Kin. LO

The constant « is non-zero if and only if hp is fundamental.

W. ROSSMANN

286

The last assertion is seen as follows.

[sgn | WR

sgny]

= 1, if h is fundamental = 0, otherwise.

It follows that y = sgn occurs exactly once in

Hy, (f) s&

dee >» ind

R Sgn.I

Hence y = sgn namely in the summand for which h, is fundamental. occurs in the W-module generated by 6(C, - ) if only if C lies in the

fundamental Cartan subalgebra. (“If” because of the formula for 8(C, - ) in theorem 1.4 (c).)

REFERENCES [1] D. Barbasch,

Fourier

inversion

for unipotent

invariant

integrals.

Trans. Amer. Math. Soc. 249, 51-83 (1979). . Barbasch

and D. Vogan,

Jr., The local structure

of characters.

J. Functional Analysis 37 (1980), 27-55.

œ

. Barbasch and D. Vogan, Jr., Primitive ideals and orbital integrals

in complex classical groups.

Math. Ann. 259 (1982), 153-199.

. Barbasch and D. Vogan, Jr., Primitive ideals and orbital integrals in complex exceptional groups. J. of Algebra 80 (1983), 350-382. . Barbasch and D. Vogan, Jr., Weyl group representations and Descente nilpotent orbits. In Representations of Reductive Groups, P. Trombi, ed. Birkhauser, 1983. Sur la cohomologie des espaces fibres principaux et = > > . Borel, des espaces homogenes des groupes de Lie compactes. Ann.

Math. 57 (1953), 115-207 . [7] R. Hardt, Topological properties of subanalytic sets. Trans. Amer. Math. Soc. 211 (1975), 57-70. [8] Harish-Chandra, Some results on the invariant integral on a semisimple Lie algebra. Ann. Math. 80 (1964), 551-593. [9] S. Helgason, Differential Geometry and Symmetric Spaces. Academic Press, New York and London, 1962.

[10] R. Hotta and M. Kashiwara, The invariant holonomic system on a semisimple Lie algebra. Inventiones math. 75 (1984), 327-358. [11] I.M. Gelfand, simple Lie 461-464. [12] W.S. Massey, [13] W. Rossmann,

Analogue of the Plancherel formula for real semigroups. Doklady Akad. Nauk. S.S.S.R. 92 (1953), Singular Homology Theory. Springer-Verlag, 1980.

Kirillov’s character formula for reductive Lie groups.

ORBITAL INTEGRALS

287

Inventiones math. 48 (1978), 207-220. [14] W. Rossmann, Limit orbits in reductive Lie algebras. Duke Math. ay 49 (1982), 215-229. [15] W. Rossmann,

Characters

as contour integrals.

Lecture Notes in

Math. 1077, 375-388 , Springer-Verlag, 1984. [16] W. Rossmann, Invariant eigendistributions on a complex and homology classes on the conormal variety I: formula; II: representations of Weyl groups. Preprint [17] W. Schmid, Two character identities for semisimple

Lie algebra an integral (1988).

Lie groups.

Lecture Notes in Math. 587, 129-154, Springer-Verlag, 1977.

[18] N. Spaltenstein, On the fired point set of a unipotent element on the variety of Borel subgroups. ‘Topology 16 (1977), 203-204. [19] T. A. Springer, Trigonometric sums, Green functions of finite groups

and

representations

of Weyl groups.

Inventiones Math.

36 (1976), 173-207. [20] T. A. Springer,

A construction of representations of Weyl groups.

Inventiones Math. 44 (1978), 279-293. [21]

R. Steinberg,

On the desingularization

of the unipotent

variety.

Inventiones Math. 36 (1976), 209-312 . [22]

V.S. Varadarajan, Harmonic Analysis on Real Reductive Groups. Lecture Notes in Math. 576, Springer-Verlag, 1977.

W. Rossmann Department of Mathematics University of Ottawa Ottawa, Canada

Received October 27, 1989

aw RE

RES

7

tar)

ed

.

bat

Dee TT

; —

h

LL

ves

ir

BOTS

cite

dat.

EE

asin

bolt

os

be

.

er

=

VEO! TS idan

asd) oer

=

‘tbe.

BAF

-A

he dj Rabe db AE : te

pidieiféés

(SAP oe 218$) 4

9

ve

rere

af.

ot A

on

@

pe

{

D

j

Twisted

Ideals of the Nullcone

RANEE KATHRYN

BRYLINSKI

Dedicated to Professor J. Dizmier on his 65th birthday

1. Introduction

This paper is a continuation of [B1].

Again, the goal is to generalize

Kostant’s fundamental work ([K1],[K2]) on the coordinate rings of maximal dimension adjoint orbits and the actions of the principal TDS on adjoint group representations, for a complex semisimple Lie algebra g. The generalization is to an analogous theory for twisted functions on the maximal dimension semisimple and nilpotent adjoint orbits, which then relates to the action of the TDS on all finite-dimensional representations of g. A main point is that the generalization ultimately comes from the invariant theory of the adjoint representation, in much the same spirit as Kostant’s original ideas. To do this, we study the orbits as varieties fibered

over the flag variety X ([B1]). The fibers then lie naturally in the adjoint representation- for a regular semisimple orbit Q, the fibers are translates of the nilradicals of Borel subalgebras, and for the principal nilpotent orbit N°, the fibers are open subvarieties of those nilradicals. So the geometry ultimately reduces the study of twisted functions on the orbits to the study of certain functions on these fibers. The key construction in this paper is a (non-linear) mapping 2 which “degenerates” twisted functions on Q to twisted functions on N°, by essentially picking out the “leading term”. This works because the points of the fibers of N° — X identify with direction vectors in the fibers of Q — X. Twisted functions on Q = G/T are determined by the characters of Tthese are the functions erp” on T, wheres y: is an integral weight. Twisted functions on N° = G/G* are determined just by characters y of the center

Z(G) (see §2 and §5 for more precise information).Here G is the connected

290

RANEE K. BRYLINSKI

and simply connected semisimple Lie group with Lie algebra g, T' is a maximal torus of G, and G® is the centralizer group of a principal nilpotent e. We call the restriction y, of exp“ to Z(G) the “central character of p”. Our map Q is fully compatible with twisting in the following way: 2 degenertes exp4-twisted functions on Q to y,-twisted functions on N°. As our mapping 2 is defined geometrically, it rather automatically has a lot of nice properties, like G-equivariance. To describe its effect, we first need to identify the twisted functions. Once we fix the weight —y, with central character y = y-,, we can build homogeneous line bundles L~* on Q and E., on N°. Then the spaces

F(Q/L=F)'and

R,(N°):=T(N°,E,)

of global sections are the spaces of twisted functions. If the weight y lies in the root lattice, then 7 is trivial, and R1(N°) is simply the coordinate ring

R(N°) = R(N), where N is the cone of all nilpotents in g (see [K2] for the fact that all regular functions on N° extend to the closure N). (Note that N itself supports no non-trivial homogeneous line bundle.) Our map Q degenerates the whole space [(Q, LZ “) to a certain subspace

I_, of R,(N°). We now summarize our main results. Let —p be an integral weight, with central character y. We construct a graded R(N)-submodule I_, of R,(N°); if p lies in the root lattice, then I_, is just an ideal of the ring R(N). I_, is the image of a (non-linear)

map Q:T(Q,L*) — R,(N°). The multiplicity in I_,, of each irreducible finite-dimensional representation Vx ofg is the weight multiplicity dim(V,’). (I.e., as a G-representation, I_, coincides with the “induced representation” T(Q,L~").) Moreover, under a certain vanishing hypothesis (satisfied, for instance, when p is regular or g ts classical), the graded multiplicity > p>0 (Vie IP )g? is equal to Lusztig’s polynomial m\(q), the q-analog of weight multiplicity. With respect to the fiber degree grading introduced in [B1], we get a natural G-linear graded isomorphism gr T(Q,L7*)—1., Now I_, contains a unique copy of V (since dim(V it) = 1); call this subspace K_,. Then K_, is the haghes! degree copy of Vi occurring in

R;(N°); this copy was first studied by Kostant ([K2), [K3)). Moreover, K_, 1s the pullback to N° of the space of global exp“ -twisted functions on the flag variety of G.

At u = 0, we have Ip = R(N), and we recover Kostant’s picture. It turns out that the J_ » have interesting properties.

For instance, it is

well known that the basic lattice relation among dominant integral weights

TWISTED IDEALS OF THE NULLCONE

291

is reflected by a dimension inequality of weight spaces. L.e., take u,v dominant integral, so that y < v yp occurs as a weight of V,. Then y < v implies that dim(V”) < dim(V#) for every representation V. We prove something much stronger, in the presence of the vanishing hypothesis: The basic lattice relation among dominant integral weights is reflected by an inclusion of our modules: if u,v are dominant integral with y < v, then

I_, € I_y (Proposition 7.2). Moreover, we also obtain (Corollary 7.5) a term-by-term coefficient inequality for the q-analogs of weight multiplicities. In the case y: lies in the root lattice, the most natural invariant of the ideal I_, is its zero-locus, the locus of points of N where all f € J_, vanish. (We will consider only the set-theoretic zero-locus, but cf. Conjecture 13.3.) It turns out that the zero-locus depends only very coarsely on y- it depends only on the list of simple components of g relative to which p has zero projection. See Theorem 9.1 for the precise statement. In particular, we prove: For g simple and p # 0, the zero-locus of I_, is precisely the boundary of N° in N, 1.e., the closure of the subregular nilpotent orbit. Theorem 9.1 was motivated by an unpublished result of Kostant, and I am grateful to him for explaining it to me and letting me reproduce it here as Theorem 8.1.. We conjecture (see §13) that, for y dominant, J_, is generated as an R,(N°)-module by its unique copy K_, of Vi. At u = 0 this is true: R(N) is generated over itself by the constants. Some evidence is given in §13. The connection of the J_, with the principal TDS action comes (by Frobenius reciprocity, following [K2]) in §6, see in particular Corollary 6.5. In later sections, statements about the J_, are most often translated into statements about representations.

Let us explain the relation of this paper to the previous one. In [B1], we degenerated twisted functions on Q to twisted function on the cotangent bundle Ty of the flag variety X = G/B. In this paper we complete the process to reach twisted functions on N° by just restricting functions from T to N°. It turns out that this process is somewhat subtle, as most of

the twisting (in fact, all save the central twisting) is lost in the restriction. We describe this in §4, though the main work, analyzing the degree shift that occurs, is done in §3. There should be some strong connection between our twisted ideals and

the work of Gelfand and Kirillov ({Ge-Ki]) on the modules R, of regular differential operators of weight p on the basic affine space. The multiplicities are the same, and one can see similarities in some of the constructions and calculations. The connection should become clearer in light of [J-S,

292

RANEE

K. BRYLINSKI

Proposition 5.5]. Kirillov and Joseph have, respectively, pointed out these references to me, but I have so far been unable to find the connection. However, it also seems clear that our Proposition 9.2 should be compared

with Shapovalov’s earlier divisibility result [S, Theorem 3(3)]. Ginsburg has recently informed me that he proved in [Gi] (by completely different means) [B1, Theorem 6.4], but without the assumption on vanishing of cohomology. Thus, the vanishing hypothesis can be removed from at least some of my results here. Acknowledgements. I thank Jens Jantzen for suggesting that I compute

the Hilbert series of the ideals Z_, (§10) to check my ideas on their zero-loci. The proofs in §9 were in fact found during the week of Colloque Dixmier, and I thank the organizers for arranging the conference and inviting me. I thank Bert Kostant for teaching me all about the invariant theory of the adjoint representation, and for countless inspiring conversations. 2. Twisted Functions

on N?

We set notations for the paper. g is a complex semisimple Lie algebra of rank £. G is a connected and simply connected complex semisimple algebraic group with Lie algebra g and adjoint group G*?. We write ad, for the adjoint action of g € G on g. Fix a pair T C B of a maximal torus inside a Borel subgroup, and let

t and b be the Lie algebras.

The pair (t,b) determines the sets +

A of positive and simple roots, and the cone P++

of dominant

and

integral

weights in the lattice P of integral weights. Let Q be the root lattice, and

set ON

OM PT

Set Pht SNe Prey

(ory

Z(G); here

à is the restriction to the center Z(G) of exp”. Let p be the half-sum of the positive roots; its “dual” is the vector h, € t on which all simple roots take unit value. View the flag variety X = G/B as the variety of Borel subalgebras of g; we write zp, when considering the subalgebra b; as a point of X. Take Zo := Lp as the base point of X. A nilpotent e € g is principal if dim g° = £. Let N° be the variety of principal nilpotents in g. N° forms a single dense adjoint orbit in the cone N of all nilpotents. We fix as the base point of N° a principal nilpotent eo in b satisfying [hp, eo] = €0; SO eo is any linear combination ees CaXa, With all cy non-

zero, of the simple root vectors X, (see [B1, 2.3]). Let

7: N° — X be the

G-projection (i.e., G-equivariant projection) sending eo to 24; then 7 sends each principal nilpotent e to the unique Borel subalgebra 6°) containing

TWISTED IDEALS OF THE NULLCONE

293

it. The choice of eo identifies, once and for all, N° with G/A, where A := G°9. Aconn, the connected component of A through the identity, is a sub-

group of the unipotent radical U of B ([K2]). Form the covering variety N° := G/Aconn with natural G-projection o:N — N°. A semisimple h € g is regularifdim g* = £. Let (Q, 7) be a fixed regular semisimple adjoint orbit over X; i.e., Q is a regular semisimple adjoint orbit and 7 : Q — X is a G-projection. The corresponding base point of (Q,7) is ho := 77! (xe)Nt. Let Q, be the adjoint orbit through hp, With Tp : Q, — X the G-projection sending h, to zy. Then h, is the base point

of (Qp, To).

The purpose of this section is to set the stage for §3. We will look at “twisted functions” on N°, 1.e., sections of G-homogeneous line bundles on N°. Twisted functions on V° may also be regarded as functions on N°.

We will define an “internal” C*-action on N compatible (see Lemma 2.4) with the usual dilation C*-action on N° given by scalar multiplication on the vector space g. Homogeneity with respect to this internal C*-action will then give us a grading of twisted functions on N° generalizing the usual

grading of untwisted functions on N°.

Lemma

Proof:

2.1. For ange € N°, we have G° = G%,,,, x Z(G).

The projection

G —+ G/Z(G)—-+G" carries G%,,,, onto (G24)°,

as both are connected ([K2, Proposition 14, pg.

362]) groups with Lie

algebra g°. So G° = Z(G)G£,,n. Now Z(G)NGÉnn = {1} since G£,,,, lies in the unipotent radical of B ([K2]), and Z(G) is central, so the product is direct.

Q.E.D.

G-homogeneous line bundles on N° correspond exactly to the characters of A. Since unipotent groups have no non-trivial characters, a character of A is just a character y of Z(G), extended trivially over Aconn. For such 7, we build over N° the G-homogeneous locally trivial algebraic vector bundle EV :=Gx4C,

with base fiber C7’. Let £7 be the sheaf of algebraic sections of E” over N°. The space

Ry(N°) = (N°, £7)

of global sections is naturally a module over the ring R(N°) = R(N) of global functions.

294

RANEE K. BRYLINSKI

Lemma 2.1 implies that each line bundle E” trivializes upon pullback to N°, and we have

Lemma 2.2. o,0n~ = ® xa)? hence, R(N as R(N°)-modules and G-representations.

) = ® ema) (N°),

R,(N°) is the sum inside R(N_ ) of all G-irreducibles on which the center acts by the character 7; Ri(N°) = R(N°) is the subspace of Z(G)invariants in R(N ~). Clearly, every G-irreducible occurs in R(N

Vy occurs in R,(N°)

), so

@ 1e Pit.

We can G-equivariantly lift the square of the dilation C*-action on N° to a C*-action on N as follows. First define a one-parameter subgroup c:C* —T by

Cy := exp(—2(log t)h,); so “cy = t~24e”, Although log is only determined modulo 277, c; is weildefined, on account of

Lemma 2.3. For any h € Q,, exp(2rih) is a central element of G of order 1 or 2, and tts order is independent of h. Proof:

Decomposing

g into h-eigenspaces, we find that a := exp(27ih)

acts trivially on g, hence lies in Z(G). But also a lies in the exponential of a principal TDS in g, hence in a subgroup ofG isomorphic to SL: or PGL2. Then a has order 2 in the former case (by direct calculation), and order 1 in the latter. As h varies, the subgroups in question are all G-conjugate, so isomorphic. Q.E.D.

As all multiples of h, preserve the line in g through eo, all c; normalize A. So we have the action of con N through the right multiplication: Ct *@Aconn

ae

=

-1

GC,

Ann:

Call this the internal C*-action on N°. A simple calculation gives Lemma 2.4. The internal C*-action on N C*-action on N°.

lifts the square of the dilation

Remark 2.5. © is completely determined, modulo Agonn, by the choice of eo; © does not depend on the choices of b, t, and h,. To see this, consider the elements ©, := exp(—2(log t)h), for h € g satisfying [h,e] = e, and t € C*. For each t, the fact ad.,,e =t~7e implies right away that, at least

TWISTED IDEALS OF THE NULLCONE

295

modulo G*, c:,, is independent of h. But h varies in a linear coset of go so the continuity of exp insures that, as h varies (and t is held fixed), the

Ct,n all lie in a connected component of a coset of G°. We will say a regular function f on a C*-stable open set U of N° is homogeneous of internal degree p if

(2.6)

F (ce *z) = t?? f(z),

for all z € U, t Ge.

Thus we get a notion of internal degree for any

section of €, y € Z(G), over a dilation stable open set of N°. Equivalently, one can define the internal degree of a section s of £7 directly by using the “internal C*-action on E” (again, by the right multi-

plication action) and the following variant of (2.6): (2.7)

ec, x s(t?z) = t??s(z).

3. N° fibered over the flag variety In this section, we compare, in Proposition 3.3, two notions of degree for a twisted function f on N°: (1) the internal degree, measuring how f transforms under the C*-action constructed in the last section, and (2) the fiber degree, measuring the homogeneous degree of f upon restriction to

the fibers of 7. As the fibers of on are dilation stable in N°, the construction of “internal degree” in §2 produces on the pushdown sheaf 7,0,0N- a internal grading over the half-integers by subsheaves of Ox-modules;

7.0,Oy~

=

Dopez(Mx7xOn~ )?. Thus we get the internal gradings:

MET = Baper(ME)?,

Ry(N°) = Oropez RE(N°),

YE Z(G): When 7 is trivial, we recover the familiar grading 7 One = @pez(™One)? coming from the dilation action. As the global regular functions on N° all extend to N ([K2]), the global functions on N° occur only in non-negative degrees. Lemma

3.1.

Let y be any character of Z(G).

Then

(1) m ET is Z—graded or (Z+4)-graded, according to the sign of y(c-1). (2) The space of global sections of E” is graded in non-negative degrees.

296

RANEE K. BRYLINSKI

Proof: (1) According to Lemma 2.2, the group element c_ is central. So c_1 acts on E” by dilating each fiber of n by a factor of y(c_1). So if s is

a non-zero local section of £7 and is homogeneous of degree p, then (2.6) forces that y(c_1) = (—1)”?. Thus the integer 2p is even if y(c_1) = 1, and odd if y(c-1) = —1. (2) As Z(G) is finite, some finite power of a global section of E” is a global function on N°, and hence has non-negative degree. Q.E.D. The G-line bundles over N° all arise through pullback from X, but not uniquely. Given an integral weight y of t, form the homogeneous line bundle F* :=Gx® cerP(#) over X, with base fiber C°“?() and sheaf of sections F#. (As usual, we extend exp(y) trivially over the unipotent radical of B to get a character

of B.) Then 7" F"“ and E™ are equivalent as algebraic G-line bundles on N°, since 7, is the character of Z(G) obtained by restricting exp(y). For n* FY = N° xx EM = G/A xe;p (G xB CoP)

= G xA Cr

= BE:

the map is given by (gA, f,v) + (g,(g=!f) : v), where g,f € G satisfy gB = fB and v is a vector in the B-representation C°?(4). Hence, we obtain a G-equivariant isomorphism of ONn--modules OY : ntFUE.

What is happening here is that most of the twisted structure of 7*F# collapses G-equivariantly in this isomorphism. This collapsing is effected by the action of G. When y lies in the root lattice, the G-action Gequivariantly trivializes n*F". (To see this happen for general pz, we need to pullback F“ all the way to N°.) Our goal right now is to understand the collapsing map

OM :T(N°, n°FH) — Ry,(N°) on global sections. The target Ry, (N°) consists of those functions f on N° satisfying f(grAconn) = Yu(r)~'f(gAconn), for all g € G, r € Z(G). Recall that our very construction of F4 equips the base fiber Fe,» and hence the

base fiber (n*F").,, with an identification to C.

TWISTED IDEALS OF THE NULLCONE Lemma

3.2. Let pe P.

O4(s) on N°

Then, fors

€ T(N°,n*F*),

297

the function f =

is given by f(gAconn)

= Une ; s(g tseo) = Oe

dene =C.

We can now describe the degree shift between the induced grading Nn" N* FY = Dopez FF @ (NxoxOn~ )? and the internal grading on 7.€™. The former is the fiber degree grading (in the sense of [B1,§4§5]) coming from the fibering of N° over X. Proposition 3.3.

Let € P. Then O

FH @ (mao On

Ÿ

gives, for each p, an isomorphism

(me E™ )P— (He),

In particular, ifs ET(N°,n*F*), and f = O#(s) € R,,(N°), then s is of homogeneous fiber degree p ifff is of homogeneous internal degree p—(p1, p). Proof:

Set F = FF and y = y,. Suppose U C X and s is a section over

n—\(U) of n*F, and f = O"(s). For x € X and e € 7 !(x), we have the fiber projection 7. : (n*F)e — F,. Now s is of homogeneous fiber degree d iff for all x € U, the equality

n«(s(te)) = tn.(s(e)) holds in F, for alleen !(r) and t € C*. On the other hand, f is of homogeneous internal degree n iff for all e €n~1(U), the equality

cl x s(t?e) = t?"s(e) holds in (n*F)., for all t € C*. Hence, f is of homogeneous internal degree n iff for all x € U, the equality

nu(cy * 8(t7e)) = 17? .(s(e)) holds in F, for alle € n~'(x) and t € C*. Now

putes x 5(t2e)) = of + e(s(t2¢)) = trim (5(t2e)), since the action of c; ‘x on the entire bundle F is just multiplication by the

scalar exp"(c7!) = t7?#(*») = 424),

So f is of homogeneous internal

degree n iff

na(s(t?e)) = 17442", (s(e)).

298

RANEE K. BRYLINSKI

Comparing with the condition for fiber degree, we find the notions of hoQ.E.D. mogeneity coincide, save for the degree shift d = n + (1, p).

Kostant proved ([K2]) that, for y € Q*t*, VX occurs exactly once in degree (u,p) in R(N), and this is the highest degree copy. The same argument works for R(N ) and all y € PF. Evaluation at eo gives a linear isomorphism eve, : Home(V,r, R,(N°)) > vs”, + = Y-u. Under €ve,, the grading on R,(N°) translates into the grading on VS by the eigenspaces of h,. The highest weight space VF is the highest graded component, with degree (y, p). So we make Definition

3.4. Let K_, be the unique highest degree copy of V/ in

Ry_, (N°), for p € P++. Corollary

3.5.

TARE

through" to: FUN

Suppose

p € P++

and y = y-y.

Proof: 3.3 says in particular that if s has section of F#, then f = O#(s) has internal also one knows (Borel-Weil) that T(X, F7) ole So its pullback is a copy of VX in Remark

Then the pullback of

as" AE. fiber degree 0, i.e., if s is a degree equal to —(y,p). But carries the G-representation RM“) Ne), hence it is K_,.

3.6. We can calculate in a more direct way the degree of f =

6-*(s) © (N°) for s € TX, F ). -Supposetet= > aa. te =) nea CeP (t)Xa. S0 X—a(t€).— exp (t) Thus

Liner

f(t-e) =(t7! - f)(e) = exp -4(t-1)f(e) = exp"(t) f(e).

Now ezp*(t) is a polynomial of total degree >.¢,Pa = (H,p) in the exp%(t), where p = 5 ea Pa®.

4. N° as a subvariety of the cotangent bundle

Let

1: N° — Ty be the natural G-equivariant inclusion over X of N° =

G x m° as an open dense subset of the cotangent bundle TY = Gx? m

of X.

As T’y identifies canonically with the variety of pairs (e1,b1) of

a nilpotent vector e1 inside a Borel subalgebra 6;, we write Ée,,6, for the corresponding cotangent vector, or simply €., when ei € N°. Let 7 : Ty > X be the natural projection, and let Tx be the tangent sheaf of X.

TWISTED IDEALS OF THE NULLCONE

299

So we have the picture

Re

oN,

N°

Tx

D 7e x

The diagram is equivariant for not only the action of G, but also for the internal C*-action, which we continue to denote by “x”.

Restriction of functions gives a natural inclusion &* : S(Tx) = TsOrs > «One, of sheaves of Ox-modules. Twisting, with respect to some p € P, and composing with the collapsing map of $3, we get an Ox-module inclusion Le

O4

jt : FY @ S(Tx) = TT FY + mntFY —

n, Er.

On global sections, this gives the inclusion

3x : T(X,F* @ S(Tx)) = T(Tx, °F") > Ry, (N°). It is well known isomorphism.

that the restriction map 5% : R(T)

—

R(N°) is an

The grading of F# @ S(Tx) induced by the grading of the symmetric algebra is again a fiber degree grading in the sense of [B1, §4,§5]. So v% is graded of degree 0. Now 3.3 gives Proposition 4.1. Let y € P. Then j* and j% are G-linear, graded injections of degree —(pt,p). j4 is a module homomorphism over the sheaf of

algebras S(Tx )—>* (S(Tx)); 7% is a module homomorphism over the ring

R(Tx) = R(N°). Corollary 4.2.

Let d be any positive integer and suppose a1,...,aq are d

(not necessarily distinct) simple roots. Then NS

Fa

es

Q S(Tx))

=

C.

Furthermore, if u € PTT and v € P with p—v = a, +---+ a4, then T(X,F7" @ S4(Tx)) contains a unique copy of VX. Proof:

Let

6 = a; +-:-+ aq.

We construct

a G-equivariant

d:F4 es F-” @S(Tx) of Ox-modules as follows.

inclusion

300

RANEE K. BRYLINSKI

For any simple root a, form the parabolic subalgebra p, = 6+ CX_. of g. The inclusion p,/b + g/b is b-linear, and hence induces a inclusion

F-% — Tx. Tensoring these maps, we build the inclusion F-° S4(Tx) of Ox-modules; tensoring next with F® we get ¢. Thus T'(X,F-"@S4(Tx)) = Vj. contains a copy of '(X,F~*) On the other hand, jy” injects r(x, F-" @S4(Tx)) into R44") (N°) = R\?)(N?), which contains just one copy of V*. Moreover at y = 0, simply R°(N°)

=

C.

Q.E.D.

We will make a more precise statement about j = jx” in Proposition 9.2. 5. Degeneration from Q to N° of twisted functions Recall (Q,7) is a regular semisimple adjoint orbit over X.

For each

integral weight y, form the pullback bundle L* = x*F" on Q, with sheaf of sections L#. In [B1, §4 and §5], we constructed the G-equivariant symbol map ANA, Le

FTF

= Fe OS TR)

of sheaves in the following way. Let U be an open set in X, and let s be a

section of L’ over m~1(U). Take x = zp, € U, and let F4 be the fiber of F* at x. Consider the map f(s,x) : -!(x) — F# defined by evaluating s on 7—1(z) and then projecting these values down to F“. The fiber 1~!(z) is an affine space, in fact a translate h1 + m1 of the nilpotent radical m,

of b;. So f(s,x) has a well-defined affine degree. The fiber degree of s is the maximum of the affine degrees of the maps f(s,z), as x varies in U.

Now, the value of the symbol st = Af(s) € T(r-!(U),r*F#) at any point = eee of r~!(U) is given by, for p the fiber degree of s,

(5.1)

st(€) = lim r'r,5(h1 + te1)/t?

Here, 7, gives the fiber projection Li, tte, — £2, while r* gives the fiber

lifting FS— (r*F")¢.

Note the limit may be computed in either F# or

CRUE In this setting, we proved

Theorem

5.2. ([B1, Theorem 5.5]) Let (Q,7) be a regular semisimple

adjoint orbit overX. Let FF be the sheaf of exp(u)-twisted functions on X, forp E P. Then the sheaf x, L" is filtered by the G-stable Ox BSeibodulds

TWISTED IDEALS OF THE NULLCONE (7.£4)S?, the sheaves of sections offiber degree at most p, forp>0. symbol map induces a graded G-linear isomorphism

301 The

gr AF : gr 1,LY*—+F* @ S(Tx) of modules over the Ox-algebra gr r,00—S(Tx).

Set, for p > 0, PS?(Q, L*) := T'(X, (4L")S?), TOI)

= DÉC,

£0) /T shit (X yan £2)

and gr I'(Q, £L") = @p>ol?(Q, L*). We call the composition of the symbol map A# with j# the special symbol map QP : 7 LE — FY @ S(Tx) — me E™.

Theorem 5.2 together with Proposition 4.1 gives at once: Theorem 5.3. Keep the notations of 5.2. Then the special symbol map induces a graded, of degree (y,p), G-linear injection

gr QY : gr mi LF — NEM

of modules over the Ox-algebra gr ™40g—15(S(Tx)).

In particular, the

induced map on global sections is a graded, of degree (u,p), G-linear injection

gr 2% : gr T(Q,L*) — Ry, (N°) of modules over gr R(Q)—R(N°). Remark 5.4. Let H be any graded G-stable complement in S(g*) to the ideal of the nullcone N; so H is also a complement to the ideal of Q ([K2]). Then the untwisted special symbol map 2% on global functions can be characterized as the unique map such that, for all p > 0, the triangle below, with the vertical maps given by restriction, is commutative.

HP

EN 2

RSP(Q) —+ RP(N?) Commutativity is easy to check. For, given e € N°, take any h € m—}(age)). Then, for any non-zero F € HP, f = Flo has fiber degree

p (see proof of [B1, 5.4]), so that

O% (f)(e) = lim f(h + te)/t? = lim F(h/t +e) = F(e).

302

RANEE K. BRYLINSKI

Following [B1, §2], define the eo-order of a vector v in a G-representation V to be the least non-negative integer p such that eft -v = 0. Construct

a (non-linear) map 6, : V — V° by _1)P 64(0) = ore -v, where p is the eg-order of v.

Proposition 5.5. Suppose y is a weight of Vy, À € Ptt, and y = y-y. Then the following diagram, where the vertical maps are the linear isomorphisms given by evaluation at h, and eo, ts commutative. 4 2x

Homg(Vx,T(Qp,£7")) ——> (5.6)

en, | Bu

Vy Proof:

Homa(V*,R,(N°)) |oss

eo

g°0

ai.

YY

The commutativity of the diagram follows at once from the proof

of [B1, Theorem 5.8].

Q.E.D.

Remark 5.7. This gives another proof of the fact [B1, Proposition 2.6] that be,(V%) is annihilated by g°°. 6. The twisted ideals J_, and 1_,

Define, for any weight p € P,

Jap = dx" (U(Lx, °F") C Ry, (N°),

Lu = OG!(gr T(Q,£-*)) € Ry,(N?). So Jo = 10 = R(N°). Note that I_, (a priori) depends on our fixed pair (Q, 7); as usual, we suppress this dependence in the notation. The homogeneous components of J_, are the images, for p > 0, of the sequences of G-linear inclusions, via A = Ay” and j = jx":

(6.1)

x

;

TP(Q.L-)ST(X, FA @ SP(Tx)) & REHM) (2),

Set mi := |Vy'|, À € Ptt,

wp€ P.

We call a polynomial f(q) with

non-negative integral coefficients a q-analog of its particular value f(1). The results of the last two sections imply

TWISTED IDEALS OF THE NULLCONE Proposition 6.2.

Let

E P. Then Je

303

Dp>od?, andIs = Bp>ol® ,

are graded G — stable R(N°)-submodules of R,_,(N°), with Ty € J_,. We have (V*,1_,) =m}, for allX E Pt*. The polynomial

(=a

NEUVE TPO, 0-*)\G— SV TE Na? p20

p20

is a q-analog of m\.

Lemma 6.3. Let y € P*t. Then IP, = J?, =0 for 0 < p < (p,p), and pie = juni heu secrs 1). Proof: Immediate from Corollary 3.5 and (6.1), since, for all vy € P, EENO LS poten): Q.E.D. Example 6.4. Suppose G = SL2, and y is the highest or lowest weight of SP(C?). Then R(N_) contains each irreducible G-representation exactly once. As a representation, I_, = @;>,S'(C?). If p is even, then J_, is the ideal of R(N°) generated by the copy of SP(C?). If p is odd, then

I_, is the R(N°)-submodule of R_,(N°) generated by the copy of SP(C?). J_, = 1, if y is dominant, while J_, = R,_,(N°) if y is anti-dominant. Through Proposition 5.5, we can explicitly relate computing the 1_, to doing certain calculations on representations. Recall from [B1, §2] that the eo-limit lim.,(V") of V4 is the subspace generated by 6.,(V#).

Corollary 6.5. In diagram (5.6), eve, carries HomG(V*,1_,) phically onto lim.,(V,').

isomor-

In particular, Lemma 6.3 is equivalent to the obvious fact: lime, (VE) =

VES ps Vanishing of the higher sheaf cohomology of F~4 @ S(Tx) simplifies the situation in various ways. The two relevant conditions are Hypotheses 6.6. for u E P.

We have the following two conditions on the pair (u, G),

(1) FVH (First Vanishing Hypothesis): H'(X,F—" @ S(Tx)) = 0. (2) HVH (Higher Vanishing Hypothesis): H'(X, FF @S(Tx)) = 0, for dia i

Lemma6.%

Leip

OH

(XF

“@S:7x)) =0 fori =0,....p—1,

then yeaa ed = SLAM dlp In particular, if (u,G) satisfies the FVH, then I_, = J_,, and hence I_, is independent of the choice of (Q, 7). Proof:

The hypothesized vanishing of cohomology implies that the first

map in (6.1) is surjective. (Cf. [B1, Proof of Corollary 5.6].)

Q.E.D.

304

RANEE K. BRYLINSKI

Lusztig ([Lu]) introduced, for y € P,A € P+*, the polynomials

(6.8)

mh(g):= D sgn(w)pq(w(r +p) —#— p), wEew

where W = N(T)/T is the Weyl group, and g, is the usual g-analog to

Kostant’s partition function. Then mX(1) = m\, so that m 0; let x; = Xrp, etc. Then P, = x; — Xr-29, and (10.2) says (cf

Example 6.4):

Xr" + Xr4297 +

= 4" (Xr — Xr-29)(X0 + X29 +---)/(1 + a).

Let P, be the image of P, under the natural Z[q]-linear projection dim : Ag] — Z{g] obtained by replacing each irreducible character by its dimension. Theorem 9.1 and Proposition 10.1 give

310

RANEE

K. BRYLINSKI

Corollary 10.4. Keep the hypotheses of 10.1, and assume Then the polynomial

also p uO:

fu(q) = to(q) — qt. (q) Pu

vanishes to order exactly 2 at q = 1. Proof:

10.1 gives us a Hilbert series formula:

HS(R(N)/Ly) = HS(R(N)) — HS(_y) = (1— qt, (q)to(q)~*Pu)dim(ch,(R(N))) Clearly dim(ch{(R(N)))

= HS(R(N)).

So we may apply the fact:

for

any graded ideal J of the coordinate ring R of an irreducible affine variety A with C*-action, the order of the zero at q = 1 in the rational function

HS(R/1)/HS(R)

is equal to the codimension of the zero-locus in A of

I. Now from Theorem 9.1, we know the zero-locus of J_,, # 0, has codimension exactly 2 in N. So we conclude that the rational function

(1 — q!*)t,(q)to(q)~P,) must vanish at q = 1 to order exactly 2. As to(1) = |W| # 0, nothing is changed by multiplying by to(q). Q.E.D. Examples 10.5. Suppose g = 51,. Let w1,...,w,_1 be the fundamental dominant weights, in the usual order, and let J be the highest root. Then

fo(q) = DE) anti

—qnr-l

— (n? = 1) gn. 7 —

Other cases are:

fow,(q) = 1 + q + 2q° + 9° — 1994 + 159° —q’,n=4 fau tws(q) =1 + 2q + 4q7 + 5q° + 6g° + 5q° + 4q° — 1279” + 100q° Lipp Deeg NE NME

few, (4)= 1+9+:..+ 0" — 330q!* + 630q!5 — 378q1° + 70q!7,n = 8 Remark 10.6. It might be interesting to understand how to compute these polynomials f,(q), for even say sl,, and to make a combinatorial proof of 10.4. More generally, it might be interesting to look at the polynomials

fu,r(q) = a?) ty (q)P,—q'"")t,P,. Since Py\q=1 = |W/W?|, it is obvious that f,,1 vanishes at q = 1. However, no other vanishing (even when À = 0) seems apparent from the algebra.

TWISTED IDEALS OF THE NULLCONE 11. Some more

consequences

311

of §9

Proposition 11.1. Let p € Q*t. Then I_,, J_,, and the ideal generated by K_,, all have the same restriction to m, namely the principal ideal Aj of S(m—) generated by k_,.

Proof:

Theorem 8.1(2) implies that ~(K_,R(N)) = A_,, while Lemma

6.3 and Proposition 9.2 imply the ~(J_,) = ~(J_,) = Ap.

Q.E.D.

Remarks 11.2. (1) Theorem 8.1(2) implies that K_,R(N) is the smallest ideal of R(N) with restriction to m equal to A_,. (2) In general, restriction to m is not injective on ideals. For example, suppose g = 5l3, and let a and 8 be the simple roots. Let J be the ideal of R(N) generated by the the lowest degree copy L of Voca+p) (the copy with lowest weight vector X?._ g)- Let I be the ideal generated by L together with the top degree copy (so the degree 2 copy) of the adjoint

representation V,48. Then

J CI, J # I, but ÿ(J) = Y(1). The latter

holds since X_gX_g € ¥(L); in fact, (J) is the ideal of S(m7 ) generated by Piao,

ye

2, Se, Gar are

Negi

a- sp, and

D Rp

Let Ÿ be the highest root of g. Proposition 11.3. Let g = sl,. Then I_3 = J_3 is the defining prime ideal ofN — N°, the closure of the nilpotent subregular orbit, and is generated by K_y. Proof:

It is known that K_» generates the defining ideal of the closure of

the nilpotent subregular orbit S. But J_»y and J_» vanish on S (Theorem 9.1) and both these ideals contain K_» (Lemma 6.3). So J_y and J_y both

equal the defining ideal.

Q.E.D.

Corollary 11.4. Let g = sl,. Form the Lie subalgebra | := t@ CXy © CX_» (so lis isomorphic to the Levi factor of a supraminimal parabolic

subalgebra). Then for all\ € PF+, |V2| = |VP|+ [Vi]. Proof:

Multiplicities are preserved in the coordinate rings of adjoint orbits

through elements in the “subregular sheet” ([Bo-Kr]). So |Vy| — |V,|, the multiplicity of V* in the nilpotent subregular orbit, is equal to |V,|, the multiplicity of V* in the semi-simple subregular orbit.

Q.E.D.

Remarks 11.5. (1) The equality in 11.4 is easy to check in the case that V)

is a “first layer representation” (i.e., a component in (C”)®", see [B2, 8.1]). For then, one finds that |V,| is the number of standard Young tableaux of

shape À, while |V,’| counts those tableaux in which “2” occurs in its first

312

RANEE K. BRYLINSKI

row, and |V,'| counts those tableaux in which “2” occurs in its first column. It would be interesting to have a combinatorial proof of the general fact.

(2) It should be possible to prove a stronger assertion than 11.4: VE ° = lime Vy ® lime Vy.

12. Multiplication of twisted functions. Probably the simplest result on multiplication in our setting is

Lemma 12.1. Letu EP.

Then T(Q, LT“) is generated over R(Q) by any

non-zero G-stable subspace.

Proof:

£7# is a locally free sheaf of rank 1 on Q. So any set {s;}ie7 of

global sections generates £~4, as long as for each point h € Q, some 5; is non-vanishing at h. Thus, if s is a non-zero global section, then the set

{g-s|g9 € G} generates.

Q.E.D.

We can describe the generation of [(Q, £~“) more precisely by comput-

ing the multiplication map M @ R(Q) seth T(Q,£7#) on each G-isotypic component.

For any two vector spaces U and V and u€ U, let uŸ : Hom(U,V) — V be evaluation at u. If x is a character of an algebraic group H, i.e., y € H, let VX be the space of y-semi-invariants in an H-representation V.

Lemma 12.2. H = G.

Let 7,’ € Ptt. Let O be an adjoint orbit, with p € O, and

Let Li be the sheaf of sections of the homogeneous

Gx#Cxi" (x; € H) onO,i=1,2. with F = evpor € VEX:

and M = r(V*).

Then, for a the natural map

induced by r, the following diagram is commutative, linear isomorphisms.

Home(Vx,M @T(O,L?)) (12.3)

Homa(V*

Q VLC

line bundle

SupposeO#r€ Hom,(V;,T(O,L')),

—~+

and all vertical maps

Home(Vz,T(0,L ® £?))

LE)

ev

EVp

=V

(Ve © V,)#x2

Proof:

mate

Vpbxaxa

Here a is the composition of natural maps

Homg(Vx,M QU) — Home(Vx ,Vz QU) — Homc(Vr @ Vx ,U),

TWISTED IDEALS OF THE NULLCONE

313

where U = T(O, L?), and the first map is induced by r-1. Consider the “larger” diagram obtained by dropping, for each of the 5 spaces, the invariance and semi-invariance conditions. (I.e., we replace

Homa(V*,M

@T(O, L?)) by Hom(V*, M @T(O, L?)), etc.) All the maps

still make sense. We will show that this “larger” diagram is commutative.

We compute the two images in VX of F € Hom(V*,M @ T(O, L?)). Suppose F = v@s@ f, for v € V,,s € M, f € T(O,L?). Then (evp o X)(F) = s(p)f(p)v. Following the diagram in the other direction, we find Fvu@r-!(s)@ fr f(p)v@r—'(s) & f(p)(r-1(s), Fv. Clearly, (re) (8), 7) = (D):

Q.E.D.

Remark 12.4. We can apply Lemma 12.2 in the case where O = Q and L=L*, pe P, to get another proofofLemma 12.1. For then 0 # r € V#, —V

and the bottom map in (12.3) is (V*@V;)° Le Vy". The latter is obviously surjective, which means that r(V*) generates the V*-isotypic component

PQ

ESF):

Set e := eo The following is an “e-analog” to the fact: f € Homi(Vr, Vy)

=> f(VF) € VE. Lemma 12.5. Letr,}E P**, LE P. Suppose f € Homy-(Vr, Va). Then f(lim. V#) C lim. Vy. Proof: Set 7 = 7x. Suppose r € Homc(V},1-,). Then F = ev. or € lim.V,# (Corollary 6.5); lim.V# is spanned by such vectors. Applying Lemma 12.2 when O = N°, we find the following diagram, with maps as

in (12.3), is commutative: Homa(Vr ® Vx, R(N)) ——

(12.6)

Homa(V*,1-,)

[ew

eve | (Ve)

r

=

limeV#

So rV(f) = f(7) lies in lim. Vy’.

Q.E.D.

Remark 12.7. One can also prove 12.5 more directly by using the fact

ites = Jim exp(te) : VF. Proposition 12.8. Suppose that for some m; € PTT, i = 1,...,m, we have r; € Homg(Vz.,1_,). Set 7; = eve or; € lime VK. Then: the spaces ri(Vz), i= 1,...,m, together generate I_, over R(N) if and only if for each AE (se the map

FY @-:-@Tm" : (Vi, @...@ VX.) @ Va) — lim.VE

314

RANEE K. BRYLINSKI

is surjective.

Proof:

Q.E.D.

Immediate from diagram (12.6).

is finitely generated, we derive immediately a fact about rep-

Since J_, resentations:

Corollary 12.9. Let y E P. Then there exists finitely many pairs (Vx;, vi), i=1,...,m, with v; € lim, VE, such that, for all A € Fa as the map

vy @--- Ow, : (Vi @-- OVE)ON)® slim. Vy is surjective.

13. On the generation of the I_,

Let us now filter [(Q, £~“), for y € P**, by the spaces

C£P(Q,L"*) = RSP(Q) -TS(Q,£74), p > 0; call this the induced filtration of T(Q, £~*).

Proposition 13.1. Let y € Ptt, Then the following are equivalent:

with v a highest weight vector of V,.

(1) I_p is generated over R(N) by K_,. (2) For allp> 0, RS?(Q) -TS°(Q, £-#) = TS?(Q, L-*); i.e., the fiber degree filtration coincides with the induced filtration.

(3) For all} € Ptt, the map (V@Vj)® — lim, VY given by evaluation at v is surjective.

Proof: While I_, is the associated graded object of [(Q, £7“) with respect to its fiber degree filtration, K_, - R(N) is the graded object

@p>00S?(Q, L-*)/(CS?(Q, LT) NTS?“ (Q, £-*). So (2) ==> (1)

(1). Since also CS?(Q, LH) C TS?(Q, L-*), all p > 0, also

= > (2).

On the other hand, Proposition

Q = Q,, (3)

12.8 implies that, when

(2). But clearly, (1) holds for a particular pair (Q,7)

< (1) holds for every pair (Q, 7).

Q.E.D.

Lemma 13.2. Let p € Ptt. Then the following are equivalent: (1) J_, is generated over R(N) by K_,.

(2) T(Tx,Tr*F #4) is generated over R(x) by T(X,F-"). Proof:

Immediate from Proposition 4.1 and Corollary 3.5.

Q.E.D.

TWISTED

Conjecture 13.3. 13.1 and Lemma

IDEALS OF THE NULLCONE

Let u € Pt+.

315

Then the 5 statements in Proposition

13.2 are all true. In particular, I, = J_,.

Note that 13.2(1) implies J_, = J_, and 13.1(1). Our conjecture is true if G = SL», and of course Theorem 9.1 (in view of Theorem 8.1) strongly supports the conjecture. Moreover, the next result is a piece of quantitative evidence. We have seen above that K_, @ R(N) contains J_, as a representation; i.e., each irreducible V) occurs at least many times in K_, @ R(N) as it

does in J_,. This means that, as a representation, K_, @ R(N) is “large enough” to G-linearly project onto J_,. The graded version of this fact is Proposition

13.4.

Suppose p € Pt*

and (u,G) satisfies the HVH (cf.

(6.10)). Form the tensor grading:

(K_, @ R(N))? = K_, @ R(N)P~ WP), p>0.

Proof:

Then, as graded representations, K_, @ R(N) contains I_,.

We need to show for all A € Lu

the polynomial

fau(g) = Vo (VX, (K-u @ R(N)))? p20

satisfies the inequality f,,,(q) > r\(q). Now

Prulq) =a) p>0 D (VE @ Vu, RINP)G =gl rEP++ mI(q)mz(q) oD, tr(q) x

T

T

t

q

The second equality is exactly [G2, Corollary 2.4]. The “first term” in the

last summation occurs at 7 = y; the term is just q!”?)m(q)to(q)/tu(q)Since to(q)/t,(q) has constant term 1, we have shown that f;,(q) > q\“?) mk (q). The vanishing assumption insures that gmk (q) = rX(q) (Theorem 6.9). Q.E.D. REFERENCES [Bo-Kr] W. Borho and H. Kraft, Uber Bahnen und deren Deformationen

bet linearen

Aktionen reduktiver Gruppen, Comment. Math. Helv. 54 (1979), 61-104. [Bi] R.K. Brylinski,

Limits

of weight

spaces,

Lusztig’s

q-analogs,

and fiberings

of

adjoint orbits, J. Amer. Math. Soc. 2 (1989). [B2]____, The stable calculus of the mized tensor character I, in Séminaire d’Algébre Paul Dubreil et Marie-Paule Malliavin”, M.-P. Malliavin ed., p 35-94,

Lecture Notes in Mathematics 1404, Springer-Verlag.. [Ge-Ki] I. M. Gelfand and A.A. Kirillov, The structure of the Lie field connected with a split semisimple Lie algebra, Funct. Anal 3 (1969), 7-26.

316

RANEE

K. BRYLINSKI

[Gi] V. Ginsburg, Ilyuxkx Ha rpynne nerent

xv /IsoncrsenHocTE

Jlenrnenjca

preprint. [G1] R.K. Gupta (now Brylinski), Generalized exponents via Hall-Littlewood symmet-

ric functions, Bull. A.M.S. 16 (1987), 287-291. [G2] =, Characters and the qg-analog of weight multiplicity, Jour.

London Math. Soc. (2) 36 (1987), 68-76. [Hs] W. Hesselink, Characters of the nullcone, Math Ann. 252 (1980), 179-182. [J-S] A. Joseph and J.T. Stafford, Modules of t-finite vectors over semi-simple Lie algebras, Proc. Lond. Math. Soc. 49 (1984), 361-384. [Ka] S. Kato, Spherical functions and a q-analog of Kostant’s weight multiplicity formula, Inv. Math. 66 (1982), 461-468. [K1] B. Kostant, The principal three-dimensional subgroup and the Betti numbers of

a complex semi-simple Lie group, Amer. Jour. Math. 81 (1959), 973-1032. [K2] =, Lie group representations on polynomial rings, Amer. J. Math. 85

(1963), 327-404.

[K3] personal communication. [La-S] A. Lascoux and M.P. Schutzenberger,

Croissance

des polynomes

de Foulkes-

Green, C.R. Acad. Sci. Paris 288, 95-98. [Lu] G. Lusztig, Singularities, character formulas, and a q-analog of weight multiplicities, in ” Analyse et Toplogie sur les Espaces Singuliers (II-III)”, Asterisque 101-102

(1983), 208-227.

[S] N.N. Shapovalov,

Structure

of the algebra

of regular differential operators

on a

basic affine space, Funct. Anal 8 (1974), 43-54.

Received September 29, 1989 Department of Mathematics McAllister Building 312 The Pennsylvania State University University Park, PA 16802 E-mail: RKB1@ PSUVM.BITNET

Representations with Maximal Primitive Ideal DAN BARBASCH! Dedicated to Prof. Dixmier on his 65** birthday

1. Introduction

Let G be a connected Lie group. An irreducible representation is a strongly continuous homomorphism

a: G—

Aut(H),

where # is a Hilbert space, such that there is no non-trivial closed invariant subspace. For a reductive group this is studied via (g, K)—modules. These are representations of the enveloping algebra,

Too : U(g) —

End(H®)

with a certain natural compatibility with respect to the action of the maximal compact subgroup K. There are many aspects of the study of these modules. In particular, I mention the following.

(1) Classify all irreducible representations, (2) Find a character theory, (3) Classify all irreducible unitary modules. In his book, ” Enveloping algebras”, Dixmier suggests in the introduction that for some of these problems it should be easier to study the corresponding problems for the primitive ideals. In this note I want to illustrate this for the problem of computing the characters of the irreducible spherical representations in the case of complex groups. This is a subproblem of (2). 1Supported by NSF grant DMS-8803500

318

DAN BARBASCH

These repesentations are intimately connected with the maximal primitive

ideals. Most cases were done in [BV1] and [B]. In particular all the classical groups were done in [B], so only the exceptional groups will be treated here. The main difficulty is that the representations of the Weyl group on the left cells do not decompose with multiplicity 1. I will rely heavily on

the results in [BV1] and [B]. In the case of left cells which decompose with multiplicity one, Theorem

3.5 in this note is equivalent to Theorem 3.8 in [J1], where a different approach is used. In the case of left cells which do not decompose with multiplicity 1, Theorem 3.14 gives character formulas for a certain set of irreducible representations including the spherical representation. (The corresponding result in [J1] is Theorem 4.6 which gives some character identities for the same set of representations). The character formulas allow one to compute K-type multiplicities which are necessary in dealing with questions of unitarity. 2. Notation

Let G be a connected simply connected complex Lie group. We denote its Lie algebra viewed as a real algebra by gr. Denote by K a fixed maximal compact subgroup and by Bt a Borel subgroup. Let tx and bg be their

respective Lie algebras. Then T = K N Bt is a maximal torus in K. If tx is its Lie algebra, then ay = /—It, is a maximally split component. Let A = exp ag and H = T : À be a Cartan subgroup. Its Lie algebra is denoted by bg. Fix an automorphism © that preserves tx and such that

o|b» = —Id. Define

X =-X, Then ~ and formulas

UB Craw ax,

for X € gp.

(2.1)

* extend uniquely to antiautomorphisms of U(gr) by the

(Mae

NES di

EEREES

Cap ee

RE

22 Ge)

Let g = (gr). Write j for the action of /—1 coming from the complex-

ification and à for the action of /—1 coming from the original complex structure on gg. Then g can be written as the direct sum of two ideals each isomorphic to gx,

g’ = {1/2(X + jiX)|X € gx},

g® = {1/2(X — jiX)IX € ax}.

oe

REPRESENTATIONS

WITH MAXIMAL

PRIMITIVE IDEAL

319

They identify with gy via the maps

PH) = 11/2(XH: EX):

EUX) = 1/2(—-'X+ji),

ee)

for X € gæ(the bar denotes complex conjugation with respect to the real form fy of gg viewed as a complex algebra this time). Then

U(g) =U(g") @U(g*).

(2.5)

The complexification of x can be identified as the image of gg under the map

¢” : gx — 9,

$°(X) = o4(-!X) + 67(X).

(2.6)

Thus we can identify & = (Ex), t = (te)., a = (ax). with the following objects in gr ® gx. t= {(-'X, X)|X = gr },

t= {(H,—H)|H € 5a}, a= {(H, H)| € bx}.

(2.7)

Furthermore j(X,Y) = (— ‘Y,X) so that gg identifies with

glx = {(— 'X,X)|X € gx}.

(2.8)

We now describe the classification of irreducible (g, K) modules. Denote by A the roots and by At the roots of the fixed Borel subgroup Bt = H-Nt. Let À, X € 5} be such that A—’ = y is the weight of a finite dimensional holomorphic representation of G. Let v = À+ X’. Define a character C(},)

of H via

Coair = Cu Cala = Co.

(2.9)

Car) can be extended to a character of Bt by making it trivial on Nt. Let X(A, ’) =

[Ind$+

Ca x) ÏK-finite,

X(A, A’) = the unique irreducible subquotient of X(A, À") containing the K-type with extremal weight y.

(2.10)

320

DAN BARBASCH

THEOREM. (Parthasarathy-Rao-Varadarajan, Zhelobenko) Fix (à, ’) and (A1,A/1) as before. The following are equivalent.

(1) X(A, X‘) and X(A1, À'1) have the same composition factors with multiplicities. _ (2) X(A, A’) = X (A1, À). (3) There is w € W such that wA = À; and wX' = 4‘). Any irreducible (g, K) module is isomorphic to an X(X, \').

Proof: See [D1].

Q.E.D.

In particular given a primitive ideal Z, we can form the (g, K)-module

U(g)/Z with the action (ax b)-v=

‘a-v-b.

(2.11)

This is spherical and has an infinitesimal character. We want to study the particular case when Z is maximal so that in fact this module is irreducible. 3. Spherical Modules 3.1. For each infinitesimal character x there is a unique maximal primitive

ideal Z(x) C U(g). Then the unique irreducible spherical module satisfies

X(A,A) & U(g)/Z(x).

(3.1.1)

There are two general results that are useful in reducing the problem to a smaller number of cases.

Translation Principle.

(Jantzen-Zuckerman)

For character formulas it is enough to find such formulas for a representative of À € (h*/A)*, where A is the weight lattice.

Reduction to Integral Infinitesimal Character. conjecture for non-integral infinitesimal character)

(Kazhdan-Lusztig

To each À one can attach the integral root system,

A(A) = {al(à,À) € Z}. Then it is enough to prove such formulas in the case A(X) = A. We can describe the representations in terms of data involving the nilpotent orbits as follows.

REPRESENTATIONS

WITH MAXIMAL

PRIMITIVE IDEAL

321

3.3. Given a representation (7, V) one can attach to it an Ad G invariant set in the nilpotent cone of the Lie algebra called WF(x). In particular, this is the closure of one nilpotent orbit by the work of [BV2], [BV3], [J2],

[BB1] and [BB2]. Let

O(A) = WF(X(A,A)),

X(A) = {X(A, wA)|WF(X(A, wd)) C O(—)},

(3.3.1) and G(A) be the Grothendieck group of virtual characters generated by the

elements in #(À). THEOREM.

(1) There is a left cell V(A) of Wey! group representations in the sense of Joseph-Kazhdan-Lusztig such that

card X(A) = dim G(À) = dim Homw[V(À) : V(A)].

(2) G(A) is generated by

{Row(X) = D tr o(2)X(A, 2wd)} crew

fora E V(A), we W.

This is essentially proved in sections 5 and 6 in [BV]]. 3.4. Assume (as we may) that the infinitesimal character is integral. Then the spherical representations are parametrized by standard parabolic subalgebras p = m+n) b. Each such parabolic subalgebra determines a unique À which is dominant for the fixed positive system At and has inner product 0 with each simple root in m, 1 with each simple root not in m.

The nilpotent orbit O(A) satisfies

LO(à) = ind, §[Trivl,

(3.4.1)

where Lg is the dual algebra and LO the dual nilpotent as in [BV1]. Finally,

the cell V(A) satisfies

VO) © Jn [Sgn] ® Sgn.

(3.4.2)

where J is truncated induction as defined in [L]. Thus, to describe the character theory of the spherical representations,

we can try to find a basis formed of R,(A) and then express the irreducible representations in terms of that. This works very well when V(,) ' decomposes with multiplicity 1.

322

DAN BARBASCH

THEOREM. Assume that V(\) has multiplicity 1. Then there is 3.5. a finite group M(Q) and correspondences

[z] € [M(O)] + 02 € VOX) x € M(O)rXX € XA) such that at the level of characters,

X, = oar Heal\).=

de

D tr x(x) Ro. (à) (emo) tr x(z)Xy.

xeM(O) Here

1

Re (A) = MO

> tro,(w)X(A, wd).

For the classical groups this is proved in [B]. The group M(O) x A(O) is defined in [L] chapter 4. Except for the case of special unipotent, the correspondence

Le] € [M(O)] + of € VO)

(3.5.1)

is different (see [B] section 5). As mentioned in the introduction, this is equivalent to Theorem 3.8 in

[J1]. Note first that in this case, {R,(A)} form a basis of G(À). Then all the reductions in sections 7 and 8 in [BV1] apply. Thus we can reduce to the

case when neither (0,) nor (/O,À) are smoothly induced (hypotheses 7A, 7B, 7C, and hypotheses 8A, 8B in [B]). Furthermore, it is enough to consider only one representative of “m of linked Levi components as in

definition 7.3 in [BV1]. Then we are reduced to the following cases.

(1a)

g

O

Lm

V(A)

Go

Ad

Aj

V, V',€

Al

6 V',e«

Fa

F(a3)

A! A; A; A}

121,92,61,43, 16; 12:,93,61,44, 16,

(1b)

(2a) (2b)

(38) (3b)

mg thew Da (a,

2454 AA:

SD COM) 80,,60,,90,

REPRESENTATIONS

Gay

tees

2A,

WITH MAXIMAL

A4A3

PRIMITIVE IDEAL

323

4480,,3150,,4200,,420, 7168, ,1344, ,2016

Cases (la), (2a), (3a) and (4a) are done in [BV1] and are known as special unipotent. Cases (1b), (2b) and (3b) are entirely similar. The correspondence

alluded to earlier is as in the following table.

In cases

(a), the correspondence is as in chapter 4 of [L]. When more than one representation corresponds to one conjugacy class, the second one refers to

case (b).

G2.

14V; 93 €2,61; g2 + V'.

F4.

1612);

gs + 92,93; ga + 43,44; 93 © 61; go + 161.

E6. 1+ 80, g2 + 60, g3 + 90,. In cases (1b) and (2b) the argument is the same as in the case of special unipotents with the long and short roots reversed. The proof will be given in a series of Lemmas. We follow [BV1] section 9 closely. The details will be useful in the case when V(A) does not decompose with multiplicity 1.

3.6. LEMMA. There is a parabolic subalgebra p(A) = m(A)+ n(A) > b satisfying the following. Let P(A) = {wAlw € W’, wX|n dominant regular}. Then

(1) PO) = [M(O). (2) There is a unique

|

À! such that for any y € P(A) y — À! is a sum of negative roots and (à, À!) = 1 for all a € At(m) simple. (3) Suppose p and y’ are in P(A). If p' = wp for some w € W(A') then /

(4) The elements in P(X) can be ordered so that

ERA Proof: G2.

E ni):

This is done by direct calculation. A(m) =

ay (1,0, —1) (0, 1, -1) (—1,1,0)

{short }-

x — À! (0,0, 0) (1,—1,0) (—2,1,1)

Dim 1 2 2

Length 0 2 6

324

DAN BARBASCH

F4.

A(m) =A — {short}. ees:

x (2,1,0,—1) (1,1,0,—2) (1,0, -—1, —2) (0, 1, -1, —2)

Dim 1 3 2 3

(0,0,0,0) (1,0,0,1) (as Pale D (27044; 1)

(241, Opp Sa) ue Bele

E6.

bet

Length 0 2 4 6

12

Let the labelling of the roots be as in [Bou]. Then we take A(m) =

{a1, 04,05,

a6}.

À

Dim

RE 2202? RE oad

CE

A

TP

333)

1

acidPEUR

2

Length

0 2

4

Q.E.D.

3.7. For y € P(X) let F7 be the finite dimensional representation of m of highest weight | — p(m). Set E, = FE @

Fyi,

I, = Indÿ[E,l], X, = unique irreducible subquotient of 1,

(377)

containing the K-type y — À

LEMMA.

(1) All I, and X, have infinitesimal character (A, À). (2) The X, are distinct and exhaust X(À). (3) There is a constant c(A) depending only on À such that

(lu) = c(A)dim Fi. (4) c(X,) is an integral multiple of c(A). Proof: 3.8

This is Lemma 9.10 in [BV1].

PROPOSITION.

Q.E.D.

1, is irreducible.

The proof is the same as for Proposition 9.11 in [BV1]. It follows from the following Lemmas.

REPRESENTATIONS 3.9.

WITH MAXIMAL

Let

PRIMITIVE IDEAL

325

Bs M, = U(g) @ le, ® Co(n)h

L, =

t unique irreducible quotient of M,

(3.9.1)

Write M; for M;:. LEMMA.

Assume g is of exceptional type.

(1) L, exhaust

the irreducible

representations

in category O of

Bernstein-Gelfand-Gelfand with infinitesimal character À and GKdimension dimen.

(2) The multiplicity of M, is dim FF. (3) M; = L; + Dr ai;L; on the level of characters.

Proof: This is 9.15 in [BV]1].

Q.E.D.

LEMMA. Suppose i < j and L; has a nonsplit extension with L;. Then L; is a composition factor of M;.

Proof: 3.10

Proof:

This is 9.16 in [BV1]. LEMMA.

Q.E.D.

M; is irreducible.

This is Lemma 9.17 in [BV1].

3.11 Proof of Proposition 3.8. vector À — Àl, and TX)

Q.E.D.

Let F; = F;@C, where F; has extremal

= Pa,ay(X ® Fi)

(3.11:1)

This is a functor in the category of Harish-Chandra modules. Because of multiplicity 1, J): is irreducible. Then

TI) = Di OS

(3.11.2)

where S comes from the corresponding formula for the M; from 3.9 and 3.10. If I): is not irreducible, then one of Home (1), X 5),

Homc(X

5, Li)

(3.11.3)

is not 0 for some j > 7. Say the first one is nonzero. Then Homa(l:

DEL

Axi

0.

(3.11.4)

@ Xi)

# 0,

(31156)

Then Homa(lu,F*

DAN BARBASCH

326

so since I): is spherical irreducible,

Homg(Triv, F7 ® Xyi) & Homx(F;, X5) # 0.

(3.11.6)

This is a contradiction. The proof follows because by 3.2,

|i = at P< [Ar A}

(S117)

for i ww rte

re

Pat

fi

LA

a;

(20,0, 0) Cope eal ‘newts thetrival av

=

CTU

Tee

,

En A

ve =

pr NT

yi

|

7

2 cette) '

te

in

eprrosnitsiriins

Pal, nd

pa

ue St amp

tbe sagt

ee

a sat “ale

à tp! rit

foie ES oh rh

ae

a4

MATE

‘ A

i|

A

ag

i

Dixmier Algebras, Sheets, and Representation Theory DAVID A. VOGAN, JR.* Dedicated to Professor Jacques Dixmier on his sixty-fifth birthday

1. Introduction.

One of the grand unifying principles of representation theory is the method of coadjoint orbits. After impressive successes in the context of nilpotent and solvable Lie groups, however, the method encountered serious

obstacles in the semisimple case. Known examples (like SL(2,R)) suggested a strong connection between the structure of the unitary dual and the geometry of the orbits, but it proved very difficult to formulate any precise general conjectures that were entirely consistent with these examples. In the late 1960’s, Dixmier suggested a way to avoid some of these problems. Motivated in part by the theory of C*-algebras, he suggested that one should temporarily set aside a direct study of unitary representations and concentrate instead on their annihilators in the universal enveloping algebra. Classification of the annihilators would be a kind of approximation to the classification of the unitary representations themselves. The hope was that this approximation would be crude enough to be tractable, and yet precise enough to provide useful insight into the unitary representations themselves. This hope has been abundantly fulfilled: the two classification problems are now inextricably intertwined, and they continue constantly to shed new light on each other. To be more precise, suppose Gg is a connected real Lie group with Lie algebra gg. The set of irreducible unitary representations of GR is written Unit Gr. Write g for the complexification of gg , and U(g) for its universal enveloping algebra. If (7,7) is a unitary representation of Gg , then U(g) acts on the dense subspace H® of smooth vectors in H. We define Ann(7)

to be the annihilator in U(g) of H°. Then Ann(7) is a two-sided ideal in * Supported in part by NSF grant DMS-8805665.

334

DAVID A. VOGAN

U(g). (Our ideals will always be two-sided unless the contrary is explicitly

stated.) An ideal IJ in any ring R with unit is called (left) primitive if it is the annihilator of a simple (left) R-module. (This says exactly that I is the largest two-sided ideal contained in some maximal left ideal.) A maximal ideal is always primitive, but a primitive ideal need not be maximal. The ideal I is called prime if whenever J and J’ are ideals with JJ’ C I, then either J C I or J‘ C I. A primitive ideal is necessarily prime, but a prime ideal need not be primitive. We say that I is completely prime if the quotient ring R/I has no zero divisors. A completely prime ideal is prime, but a prime ideal need not be completely prime. We write Spec R = set of prime ideals in R Prim R = set of primitive ideals in R Spec,R = set of completely prime ideals in R

(1.1)

Prim,R = set of completely prime primitive ideals in R .

Suppose now that (7, #) is an irreducible representation of G. Because the irreducibility is topological rather than algebraic, the space of smooth

vectors H® will not be a simple module for U(g) (unless the representation is finite-dimensional). Nevertheless, Dixmier proved

Theorem 1.2 ([7]). Suppose x is an irreducible unitary representation of a connected Lie group Gy . Then Ann(r) is a primitive ideal in U(g). To explain the connection is an orbit of GR on gg. Let Lie algebra g , and let Oc = coadjoint orbits seeks to attach

with the orbit method, suppose that Og G be a complex connected Lie group with G- Og. Roughly speaking, the method of to Og an irreducible unitary representation

m(Og) of Ge . (Actually one needs to restrict attention to certain orbits, called “admissible,” and one needs some additional data beyond the orbit

itself.) We will call a correspondence from orbits to representations an orbit correspondence, and denote it KK (for Kirillov-Kostant). Under favorable circumstances (for example if Gg is an algebraic group) it appears that almost all interesting unitary representations should appear in the image of an orbit correspondence. The problem of constructing an orbit correspondence is therefore one of the most important unsolved problems in representation theory. We consider next what Dixmier’s ideas can tell us about this problem. One of Dixmier’s insights was that one should attach to the complexified orbit a primitive ideal in U(g). This may be formulated as

SHEETS, ALGEBRAS

AND REPRESENTATION

THEORY

335

Conjecture 1.3 (Dixmier). Suppose G is a complex connected Lie group with Lie algebra g, and Oc is an orbit of G on g*. Then there is attached

to Oc a completely prime primitive ideal J(O¢) in U(g). In fact the ideal should depend only on the Zariski closure of Oc in g*. A correspondence of the form in the conjecture is called a Dizmier map, and often written Dix. In its strongest original form (never formulated by Dixmier) the “Dixmier conjecture” asks that Dix should be a bijection from Zariski closures of orbits to completely prime primitive ideals. (Work of Borho and others has shown that this is not possible if the Dixmier map is to have other reasonable properties.) If one had both an orbit correspondence and a Dixmier map, there would be (roughly speaking) a diagram Ore — 7

|

|

Oc

—

(1.4)(a)

I

of maps among the sets

Can

SenUnitCy

| g*/G

| Ann Prim U(g)

—

(1.4)(b)

A natural requirement to impose on KK and Dix is that this diagram ought to commute.

Unfortunately this is not a reasonable condition. The Dixmier

map is supposed to take values in Prim, U(g) (the completely prime primitive ideals); but the annihilator of a unitary representation need not be completely prime. (The simplest example is the defining representation

of SU(2). The corresponding quotient U(g)/Ann(7) is isomorphic to the algebra of 2 x 2 matrices, and therefore has zero divisors.) Since the diagram (1.4) cannot commute, we look for slightly weaker requirements. The simplest one consistent with examples like SU(2) is Ann(r(OR)) C [(Oc)

(1.5)

m(Og) is a U(g)/I(Oc)-module.

(1.5)'

Equivalently,

Properly understood, (1.5) casts representation theory in an entirely new light. We want to interpret it as a program for constructing an orbit correspondence. The first step is to construct a Dixmier map: more precisely, to

336

DAVID A. VOGAN

construct and understand the algebra U(g)/I(Oc). A unitary representation (Og) attached to Og —that is, the image of the orbit correspondence — should then be constructed and understood as a module for this algebra. From now on we will confine our attention almost exclusively to reductive groups. To a large extent the point of view just described is the one

adopted by Beilinson and Bernstein in their fundamental paper [1]. (It is not difficult to find similar ideas in much earlier work — for instance in the theory of C*-algebras, or in the work of Gelfand and Kirillov on quotient rings of enveloping algebras. What is unique to Beilinson and Bernstein is the successful application of a general structure theorem for quotients of U(g) to representation theory.) They showed that if J is any minimal prim-

itive ideal in U(g), then U(g)/J is isomorphic to an algebra of differential operators on a flag variety. Consequently any irreducible g-module may be regarded as a module for a differential operator algebra. This perspective has proven to be tremendously illuminating in a wide range of contexts: for the classification of representations, for the construction of intertwining operators, for analysis on symmetric spaces, and for primitive ideal theory, for example. Nevertheless, the Beilinson-Bernstein approach has some limitations.

In the context of (1.5), it amounts to looking for (Og) as a module not for the natural algebra U(g)/I(Og), but rather for some much larger algebra (of which U(g)/I(Og) is a quotient). The modules we want are certainly present in the Beilinson-Bernstein picture, but so are many extraneous ones. Putting more precise constraints on the primitive ideal should put

more precise constraints on the modules, and so (one hopes) help to suggest the definition of the orbit correspondence.

For this reason, it is still

worthwhile to pursue the program described after (1.5). There is already a tremendous amount of information available about the construction of a Dixmier map. Most of it is based on the notion of

“parabolic induction” in one form or another. Roughly speaking, the idea is that most coadjoint orbits for G can be constructed in a simple way from coadjoint orbits for a Levi subgroup L. Parabolic induction also provides a way to construct primitive ideals for G from those for L. If we already know something about a Dixmier map on L, then we can hope to specify part of a Dixmier map for G by requiring that induction of orbits should correspond to induction of primitive ideals under the Dixmier maps for L and G. (One can make exactly parallel remarks about representations and

orbit correspondences.) In the case of SL(n), Borho in P used exactly this idea to define a Dixmier map completely. For reductive groups not of type À, Borho discovered a fundamental obstruction to this approach. It can happen that the same coadjoint orbit Oc for G arises in two different ways by induction, and that the corres-

SHEETS, ALGEBRAS AND REPRESENTATION

THEORY

337

ponding induced primitive ideals I, and 12 are different; apparently both ought to be attached to Oc. One of the goals of the theory of “Dixmier

algebras” initiated in [20] and [15] is to circumvent this problem. Roughly speaking, the idea is this. In Conjecture 1.3, the orbit Oc is replaced by an “orbit datum,” consisting of some additional algebro-geometric structure

on an orbit (Definition 2.2). The primitive quotient U(g)/I is replaced by a “Dixmier algebra” (Definition 2.1), which is an extension ring of a quotient of U(g). Conjecture 1.3 is replaced by a conjectural map from orbit data to Dixmier algebras (Conjecture 2.3). (Such a map would automatically descend to a multi-valued Dixmier map in the sense of Conjecture 1.3.) The primary purpose of this paper is to extend to orbit data and Dixmier algebras the notions of parabolic induction discussed above. This is accomplished in Proposition 3.15 and Corollary 4.17 respectively. These results suggest a way to define a Dixmier map on induced orbit data (in terms of Dixmier maps on Levi subgroups). In order to justify this definition, one would have to show that if an orbit datum is induced in two different ways, then the corresponding induced Dixmier algebras must coincide. This we have not been able to prove; it would follow from Conjectures 3.24 and 4.18. A more complete discussion of the status of Conjecture 2.3 may be found in section 5.

These ideas of course focus attention on the orbit data that are not induced. (In fact the primary motivation for this paper was not so much to say something about induced orbits (or primitive ideals, or Dixmier algebras) as to understand by a process of elimination those that are not.) We call these non-induced orbit data rigid, for reasons that will be clearer

in section 3 (cf. Definition 3.22 and Proposition 3.23). An interesting point is that rigidity is a property of the full orbit datum, and not just of the underlying orbit; it may be possible to deform the orbit but not the orbit datum. In section 5 we recall from [21] a conjectural construction of Dixmier algebras attached to rigid orbit data. The program described after (1.5) suggests that one should turn next to the description of modules for induced Dixmier algebras, seeking among these candidates for representations attached by the orbit correspondence to real forms of Oc. In this direction we do only a little. Induced Dixmier algebras are generalizations of Beilinson and Bernstein’s twisted differential operator algebras. It should therefore be possible to analyze their modules by the kind of geometric “localization” familiar in the differential operator case. We prove here only a few of the basic facts about such a localization

theory (notably Corollary 6.16 and Theorem 7.9). Here is a more detailed outline of the contents of this paper. Section 2

recalls from [20] and [15] the definition of Dixmier algebras and orbit data, and a corresponding refinement of Conjecture 1.3. (One of McGovern’s re-

338

DAVID A. VOGAN

sults in [15] is that the main conjecture in [20] is false; and McGovern has since found counterexamples for a revision circulated in an earlier version of this paper. Conjecture 2.3 appears to be consistent with all of his work to date.) Section 3 outlines the extension to orbit data of some of the basic structure theory for coadjoint orbits: Jordan decomposition, parabolic induction, and sheets. In the theory of sheets we find some strong (conjectural!) geometric evidence for the correctness of the general approach to the Dixmier conjecture in [20]: Conjecture 3.24 says that distinct sheets of “orbit data” should be disjoint. The failure of the corresponding fact for sheets of orbits is at the heart of the non-uniqueness problems discovered by Borho and discussed above. Section 4 presents the construction of Dixmier algebras by parabolic induction. Section 5 outlines how these ingredients should fit together to define a Dixmier map for G. The rest of the paper is devoted to related technical results. In section 6 (following [5]), we relate induction of Dixmier algebras to ordinary induction of Harish-Chandra bimodules. (Recall that Harish-Chandra bimodules are closely related to infinite-dimensional representations of G regarded as a real Lie group. By “ordinary induction” we mean the bimodule construction corresponding to parabolic induction of group representations (in the sense of Mackey and Gelfand-Naimark).) Perhaps the most important consequence is a cohomology vanishing theorem (Corollary 6.16). This generalizes the fact that the higher cohomology of G/Q with coefficients in the sheaf of differential operators is zero. Section 7 considers the translation principle for induced Dixmier algebras and their modules. A key tool in all the induction constructions (both for orbit data and for Dixmier algebras) is the notion of equivariant bundles on homogeneous spaces (in the algebraic category). These help to formalize the idea that G-equivariant constructions on G/H are equivalent to H-equivariant constructions at a point. A few of the basic definitions and results are summarized in an appendix for the convenience of the reader.

2. Dixmier algebras. Suppose for the balance of this paper that G is a connected complex reductive algebraic group with Lie algebra g.

Definition 2.1 (cf. [15]). A Dixmier algebra for G is a pair (A,¢) satisfying the following conditions. i) A is an algebra over C, equipped with a locally finite algebraic action (called Ad) of G on A by algebra automorphisms. li) The map ¢ is an algebra homomorphism of U (g) into A , respecting the two adjoint actions of G. The differential of the action Ad of G on

SHEETS, ALGEBRAS

AND REPRESENTATION

THEORY

339

A is the difference of the left and right actions of g defined by 9.

ii) A is a finitely generated U(g)-module. iv) Each irreducible G-module occurs at most finitely often in the adjoint action of G on A.

The simplest example of a Dixmier algebra is any quotient U(g)/I of U(g) by a primitive ideal. Definition 2.2. An orbit datum for G is a pair (R,W) satisfying the following conditions. i) R is an algebra over C, equipped with a locally finite algebraic action (called Ad) of G on R by algebra automorphisms. ii) The map Ÿ is an algebra homomorphism of S(g) into the center of R, respecting the two adjoint actions of G. iii) Ris a finitely generated S(g)-module. iv) Each irreducible G-module occurs at most finitely often in the adjoint action of G on R. The support & of the orbit datum is the algebraic variety V(ker ~) C g*. (Thus © is the image of the moment map y* : Spec R — g*.) The orbit datum is called completely prime ifR is completely prime, and commutative if R is commutative. It is called geometric if R is commutative, completely prime, and normal. It is called pre-unipotent if G is semisimple, and © is contained in the nilpotent cone. Finally, it is called unipotent if it is pre-unipotent and geometric.

The simplest example of an orbit datum is the quotient S(g)/J of S(g) by the ideal of functions vanishing on an orbit. In this case the support © is the closure of the orbit. (An example not of this kind appears as Example

3.21 below.) For general orbit data, condition (iv) implies that © is a finite union of orbit closures. (To see this, consider the algebra Z = S(g)S of invariants in the symmetric algebra. The maximal ideals of Z parametrize the semisimple orbits of G on g*: if m is such a maximal ideal, then the associated variety V(m) consists of all elements of g* for which the semisimple part of the Jordan decomposition belongs to the corresponding

orbit. Consequently each V(m) is a finite union of coadjoint orbits; in fact Kostant’s theorem on the principal nilpotent element implies that it is the closure of a single coadjoint orbit. On the other hand, the image 7(Z) is contained in the G-invariants of R, which form a finite-dimensional algebra

by (2.2)(iv). It follows that the kernel of Ÿ contains an ideal Z of finite codimension in Z. Consequently © is contained in the (finite) union of the

various V(m), with m a maximal ideal in Z containing Z.) If the orbit datum

is completely prime, then © is necessarily the closure of a single

340

DAVID A. VOGAN

coadjoint orbit.

A geometric orbit datum (R, 4) is the same thing as a

normal irreducible affine algebraic variety X (namely Spec R) equipped with a G action and an equivariant finite morphism Ÿ* from X to an orbit closure in g*. A unipotent orbit datum is therefore exactly a unipotent Poisson variety in the sense of [21]. Before formulating the Dixmier conjecture, we should say a little bit about filtrations. Suppose A is an algebra filtered by iN. This means that we are given an increasing family of subspaces

Ap C A1C AC: so that

DAS

A,

ApAg C Aptg-

Then the associated graded space grA is a graded algebra.

(Our basic

example is the standard filtration U,(g); in this case gr U(g) = S(g).) An increasing filtration on an A-module M is called compatible if

U;Mj;=M,

ApMy C Mpaq.

In this case the associated graded space gr M is in a natural way a graded module for gr A. A compatible filtration of M is called good if grM is finitely generated as a module for gr A. Here is a version of the Dixmier conjecture for reductive groups. It is

taken from [20], but modified in accordance with the requirements of [15]. Conjecture 2.3. Suppose G is a complex connected reductive algebraic group. Then there is a natural injection Dix from the set of geometric orbit data for G (Definition 2.2) into the set of completely prime Dixmier algebras for G. This correspondence should have the following properties. Fix an orbit datum (R, 7), and write (A,¢) for the corresponding Dixmier algebra.

i) The Gelfand-Kirillov dimensions of A and R are equal. ii) A and R are isomorphic as G-modules. iii) A and R admit filtrations indexed by 3N, with the following properties. a) The filtration of A is good (and therefore by definition compatible) for A regarded as a U(g)-module. In particular, 6(Un(g)) C An.

b) The filtration of R is good for R regarded as an S(g)-module. In particular, ~(S"(g)) C Rn. c) The associated graded algebras gr A and gr R are completely prime. d) There is a G-equivariant isomorphism € : gr A > gr R carrying gr ¢ to gr w.

SHEETS, ALGEBRAS

AND REPRESENTATION

THEORY

341

In (a), Un(g) is the nth level of the standard filtration of U(g); and in (b), S"(g) is the nth level of the standard gradation. (Of course we could equally well use the standard filtration of S(g) in (b).) Conditions (i) and (ii) are included only for expository purposes; they

are consequences of (iii). That the filtration in (iii) ought to be indexed by $N (rather than some iN) is suggested by [16]. The map Dix should extend to a bijection from some larger set of completely prime orbit data onto all completely prime Dixmier algebras. McGovern has pointed out that it cannot be defined on all completely prime orbit data, however. Orbit data are close enough to coadjoint orbits to admit Jordan decompositions, which we now describe. Suppose (R, #) is a completely prime orbit datum. Fix À in g* so that kerŸ is the ideal of functions vanishing on O=G-X. Write MATRA, (2.4)(a) for the

Jordan decomposition of A, and

L={g€G|Ad*(g)A;,

= 4; }

(2.4)(b)

(a Levi subgroup of G). L is again a connected reductive algebraic group; we want to relate (R,#) to a completely prime orbit datum (Az, pz) for L. Define AL =)

Ir;

(2.4)(c)

we want the support of (Rz,~z) to be the closure Ly of Or = L- Ar. Let s be the unique Ad(ZL)-invariant complement for [ in g. We identify (* with the linear functionals on g vanishing on s. By Proposition A.2(b), the natural inclusion of Xz; in Z induces a G-equivariant morphism

Gee

(2.4)(d)

Every point or € Uz has semisimple part À,. By the Jordan decomposition, the stabilizer of oz in G is contained in L. By (A.3), the map (2.4)(d) is one-to-one. À slightly more careful analysis, which we omit, shows that (2.4)(d) is actually an isomorphism of varieties. By Proposition A.5, the category of finitely generated modules with G-action on © is equivalent (by passage to geometric fibers) to the category of finitely generated modules with L-action on 27. A little more explicitly, let J, C S(g) be the defining ideal for Zr. Define

Ry = R/RyY(JL).

(2.5)(a)

Obviously RL is an algebra equipped with an action of L and a map

br : S(1) — RL.

(2.5)(b)

342

DAVID A. VOGAN

(The map comes from # by restriction on the domain and passage to the quotient on the range.) The preceding discussion implies that (Rr, ŸL) is a completely prime orbit datum for L, and that

FC /CCENRE

(2.5)(c)

(cf. (A.4)). By (A.4)(c), this last formula says that R may be identified with the space of algebraic maps p from G to Rz, subject to the condition

p(gl) = Ad(I-")p(g)

(2.5)(d)

for g in G and / in L. The algebra structure on R is just pointwise multiplication. Finally, one can check easily that the adjoint action of Z(L)o on Ry must be trivial. Consequently (Rx, pz) gives rise to a pre-unipotent orbit datum (Ru, Yu) for L/Z(L)o. The algebra R, is just Rr, and the map py is the restriction of yz to [[, ], composed with the natural isomorphism

Lie(L/Z(L)o) = [U 4.

(2.5)(e)

Conversely, one can recover wz, from y, by the requirement

br(A)=A(A)

(A €a(1)).

(2.5)(f)

The following theorem summarizes this discussion.

Theorem 2.6 (Jordan decomposition for orbit data). There is a natural bijection between the set of completely prime orbit data (R,%) for G (Definition 2.2) and the set of G-conjugacy classes of triples(L, X,,(Ru, Yu)). Here t) L ts a Levi subgroup of G. it) As: 1—+C is a G-regular Lie algebra homomorphism. 1) (Ru, Pu) is a completely prime pre-unipotent orbit datum for L/Z(L)o. The correspondence is specified by (2.4)-(2.5). In this bijection, R is commutative (respectively normal or geometric) if and only if Ry is.

In (ii), the “G-regular” hypothesis is just the condition (2.4)(b) above. In light of the classification of unipotent orbit data in [21], Theorem 5.3, we get Corollary 2.7. correspondence:

The following three sets are in natural one-to-one

SHEETS, ALGEBRAS

AND REPRESENTATION

THEORY

343

1) geometric orbit data (R, %Ÿ) for G; ü) G-conjugacy classes of pairs (A,S), with À in g* and S a subgroup of the “G-equivariant fundamental group” G/G2 of G : À; wi) G-conjugacy classes of triples (L,\5,(Ru,Wy)), with L a Levi subgroup ofG, As a G-regular character of \, and (Ru, Ÿu) unipotent orbit data

for L/Z(L)o. IfG is simply connected, we can add iv) finite coverings of coadjoint orbits. Unfortunately, no analogous theorems are known for completely prime primitive Dixmier algebras. There is, however, a construction (by parabolic induction) of a Dixmier algebra associated to an orbit datum, assuming that such an algebra associated to (Ru, %y) is already available. It will be described in section 4, after some geometric preliminaries in section 3. We conclude this section with some useful formal ideas; for more back-

ground, see [17] and [8]. Definition 2.8. Suppose (A, ¢) is a Dixmier algebra for G. The opposite

Dizmier algebra is the pair (A®,9%®) defined as follows. The algebra A°? is the opposite algebra to A, with the same underlying vector space and multiplication

(a) ‘op (b) = ba.

The action of G on A

by

is the same as on A. The map $” is characterized

PP(X)=G(-X)

(XE 9).

A transpose antiautomorphism of (A, ¢) is an isomorphism (usually written at ta) of A with its opposite Dixmier algebra.

Definition 2.9. Suppose (R, ) is an orbit datum for G with support Y. The opposite orbit datum is the pair (R, p??) defined in obvious analogy with Definition 2.8; it is an orbit datum with support —. define a transpose antiautomorphism.

We can also

The Dixmier correspondence of Conjecture 2.3 should respect passage to opposite algebras in the sense of these definitions. Now a geometric unipotent orbit datum is easily seen to be isomorphic to its opposite. The corresponding Dixmier algebras should therefore admit transpose antiautomorphisms. Such automorphisms will play a role in the theory of unipotent

representations (cf. [23], Theorem 8.7(ii)).

344

DAVID A. VOGAN 3. Induction, sheets, and rigid orbit data.

In this section we consider parabolic induction for orbit data. Suppose L is a Levi subgroup of G. It is well known that there is a codimensionpreserving map from coadjoint orbits for L to coadjoint orbits for G. This

map sends a G-regular semisimple orbit L - À to G- A, and sends {0} to a Richardson nilpotent orbit. The purpose of this section is to extend this map to orbit data. Because the construction is not a very easy one to grasp, we will begin by recalling the theory on the level of orbits.

Definition 3.1 (cf. [14]). Suppose L is a Levi factor in G, and OZ is a nilpotent coadjoint orbit in “. We will construct from OZ a nilpotent coadjoint orbit for G. Although this orbit turns out to depend only on L, its construction requires the choice of a parabolic subgroup Q with Levi factor L. Write U for the unipotent radical of Q; then L ~ Q/U. This quotient map gives rise to an injection

ig: "oq"

(3.1)(a)

identifying linear functionals on [ with linear functionals on q that vanish on u. Define

Og = iq(Oi) € q°.

(3.1)(b)

(Thus Og is isomorphic to Or.) Next, consider the restriction map

TGJQ : 9° #4".

(3.1)(c)

This exhibits g* as an affine bundle over q*, with vector space (g/q)*. (That is, each fiber of xGyQ is a principal homogeneous space for the vector space (g/q)* — a copy of the vector space with the origin forgotten.) Define

Bag = ™Gig(Oe) C8",

(3.1)(d)

an affine bundle over Og. In particular, we have

dim Og/g = dimO, + dimG/Q.

(3.1)(e)

The action of Q on g* evidently preserves Ogya- By Proposition A.2(b), we get a G-equivariant morphism

G xq Oajq — G Ogi C 9.

(3-1)(f)

The induced bundle on the left has dimension equal to

dim G/Q + dim Og/q = dimO; + 2: dimG/Q = dimO, + dimG/L. (3.1)(9)

SHEETS, ALGEBRAS AND REPRESENTATION

THEORY

345

It follows that any G-orbit in the induced bundle — and a fortiori any G-orbit in G- Ocya — has dimension at most equal to dim Oz + dimG/L. The induced orbit

Og = Indors(L t G)(Oz) is the unique nilpotent coadjoint orbit in g* satisfying any of the following equivalent conditions. i) The orbit Og is contained in G- Og/a; and the dimension of Og is

dimO; + dimG/L. ii) There is a À in Og such that the restriction of À to q is trivial on u and belongs to Oz on [; and the codimension of Og in the codimension of Oz in [*. ili) Fix a Cartan subalgebra § of [, and write Wr, and W groups of L and G. Then the Springer representation harmonic polynomials on 4) for Og is generated by the resentation for Or.

g” is equal to for the Weyl of W (on WSpringer rep-

Several remarks are in order here. First, the equivalence of (i) and (ii) is immediate from the definitions. In (iii), the Springer correspondence must be normalized to take the principal orbit to the trivial representation.

We have included (iii) only because it shows that the induced orbit is independent of the choice of Q. The proof of its equivalence with (i) and

(ii) ([14], Theorem 3.5) would require a substantial digression, so we omit it: Second, there is (given Q) at most one orbit Og satisfying (i) and (ii). To see this, notice that OZ (as a homogeneous space for a connected group) is an irreducible algebraic variety. As an affine bundle over Oz, the variety OcJa is irreducible as well. Since G is connected, it follows that G-Oggq is irreducible. We have already observed that the dimension of this set is

at most dimO, + dimG/L. It can therefore contain at most one G-orbit of the desired dimension. For the proof that one exists, we refer to [14]. Third, we should check that Og is nilpotent. For this it suffices to show that Og Je consists of nilpotent elements. Now it is an elementary exercise to show that a linear functional À on a reductive Lie algebra is nilpotent if and only if there is a Borel subalgebra 6 such that A |,= 0. So suppose A € Og/a- Then À |y= 0, and À |= A, € Or. By hypothesis Az is nilpotent, so there is a Borel subalgebra 6; of [ on which Az is zero. Since Q is parabolic, b = by + u is a Borel subalgebra of g; and clearly A6="0

A nilpotent.orbit is called rigid if it is not induced from a proper Levi subgroup, and induced otherwise. It is called Richardson if it is induced

346

DAVID A. VOGAN

from the zero orbit on a Levi subgroup. (Thus the zero orbit is called Richardson but not induced.) To get a little feeling for the notion of induced orbit, we consider some examples in the classical groups. If G is GL(n) or SL(n), then every nilpotent orbit except 0 is induced. (To describe this explicitly, one can replace 2 by 1 everywhere in the discussion of other classical groups below.) If G, is any other classical group of n by n matrices, then any nilpotent coadjoint orbit © gives rise (via Jordan blocks) to a partition R= (Pi,

Pr)

(3.2)

of n. The sequence of non-negative integers p; is (weakly) decreasing, and the sum of the p; is n. (It is often convenient to allow some of the p; to be zero; we identify partitions when their non-zero parts agree.) Suppose that there is a jump of 2 in this sequence; say (3.3)(a)

Pm — 2 > Pm+1.

Then © is induced, in the following way. G has a Levi factor

LS Gilet

(3.3)(b)

Glin)!

with the first factor a classical group of n — 2m by n — 2m matrices. There is a nilpotent orbit O’ for G,_2m corresponding to the partition

PUS

D SD PAs

PE)

(3.3)(c)

(In the case of SO(2k) the orbit O’ may not be uniquely determined by m’; one must make an appropriate choice of O’.) Define Or to be the orbit

(O’,0) in L. Then O = Indo,s(L ft G)(Oz).

(3.3)(d)

There are other ways for a nilpotent orbit to be induced in the classical groups, but the preceding special case captures most of the general flavor. The first example of a non-zero rigid orbit is the minimal (4-dimensional) nilpotent in Sp(4); it cannot be induced because its dimension is not of the form dimO, + dimG/L for any orbit in proper Levi subgroup L. The corresponding partition is (2,1,1), which of course has no jumps of 2.

(Viewed within the locally isomorphic group SO(5), this orbit gives rise

to the partition (2, 2,1), which still has no jumps.) We now extend Definition 3.1 to include induction from non-nilpotent orbits.

SHEETS, ALGEBRAS

AND REPRESENTATION

THEORY

347

Definition 3.4. Suppose L is a Levi subgroup of G, and Oz C (* is a coadjoint orbit for L. The induced orbit Og

=

Ind,,5(L

Î G)(O_)

is defined precisely as in Definition 3.1(i) or (ii), as the unique dense orbit inG.- (™GjQ(ie(Oz))). It follows easily from the construction that the codimension of Og in g* is equal to the codimension of Oz in [*. (Again the construction depends on a choice of parabolic, but the orbit constructed does not. We will not stop to prove this independence; it follows from the special case of Definition 3.1 by Proposition 3.6.)

We have already seen one instance of this more general induction in

(2.4). Here is the connection. Lemma 3.5. Suppose À = À, + Ay is the Jordan decomposition of a linear functional in g*. Write L for the centralizer in G of À,, a Levi subgroup ofG, and Ay = (AL)s + (AL)u for the restriction of À to L. Set Op, Sh

Oy, = (Az)s + L: (due

Then the orbit for G induced by OZ is

Ind (ist GO) = Gan.

Proof.

Choose

a parabolic Q =

LU

as in Definition

Og € q* and Ogg C g* as in Definition 3.1. the definitions that A € Og/a-

the correct dimension.

3.4.

Define

It is immediate from

It remains only to check that G- À has

But this follows from the fact (part of the Jordan

decomposition) that the centralizer in G of À is just the centralizer in L of Àz- Q.E.D. In general, the induction operation of Definition 3.4 can be expressed in two steps: the first an induction of nilpotent orbits in the sense of Definition 3.1, and the second the “Jordan decomposition” case considered in Lemma 3.5. Here is a precise statement. Proposition 3.6. Suppose L is a Levi subgroup ofG, and Oz Cl a coadjoint orbit for L; write

Og = Indors(L t G)(Oz).

ts

348

DAVID A. VOGAN

Then the Jordan decomposition of an element of Og may be computed as follows. Fiz an element A, of Or, and write its Jordan decomposition as

At = À: + (Ai): Let L' be the centralizer in G of \,, and put M = L'NL. Then dz 1s zero on the natural complement of m in |, and so may be identified with an element AL =

AM

=

Xs ae (Am)u-

of m*. Similarly, 4, may be regarded as a semisimple element of(V)* of g*. Write

or

(Om)u =M-(Am)u (Oz')u = Indors(M 1 L’)((Om)u).

Fix a representative (Ar')u € (Oz')u. Then A=A,+

(AL )u

is the Jordan decomposition of an element of Og.

We have

Og = Indoro(L' 1 G)(X + (OL'hu).

Proof. Because L’ fixes À,, the sets Om

=

As =F (Om hu,

Or

=

are coadjoint orbits for M and L’ respectively.

As SF (Ox )u

By Lemma 3.5 applied to

L, Oz

=

Inde

(M Î L)(Om).

By an easy “induction by stages” fact, we deduce

Og = Ind,,4(M 1 G)(Om). Now we induce in stages first from M to L’, and then to G. This gives

Og = Indes (L' fT GO). This is the last formula in the proposition. The description of the Jordan decomposition follows from Lemma 3.5 applied to G. Q.E.D.

SHEETS, ALGEBRAS AND REPRESENTATION

THEORY

349

We can now describe sheets of coadjoint orbits. Recall that a nilpotent coadjoint orbit is called rigid if it is not induced.

O

Definition 3.7 (cf. [3], [4]). Suppose L is a Levi subgroup of G, and is a rigid nilpotent coadjoint orbit for L. The sheet of coadjoint orbits

attached to (L,O) is the collection of orbits

{Indors(L 1 G)(As + Ou) |As € (I, 9)" }. Two sheets are identified if they contain exactly the same orbits; by Proposition 3.6, this amounts to conjugacy of the pair (L,O,) under G. A Dizmier sheet is one attached to the zero orbit in a Levi subgroup. Here are some of the most important facts about sheets.

Proposition 3.8 (cf. [3]). Jn the notation of Definition 3.7, i) Each coadjoint orbit belongs to at least one sheet. 1) Each sheet contains exactly one nilpotent orbit (namely Indo,»(L

1

G)(Ox)). ii) All of the orbits in a sheet have the same dimension (namely dim ©, +

dim G/L). Proof. By induction by stages, every nilpotent orbit is induced from a rigid nilpotent orbit. Now (i) follows from Proposition 3.6. Part(ii) is also clear from Proposition 3.6, which says that induction preserves “semisimple part.” Part (iii) follows from Definition 3.4. (In fact the original definition of a sheet is as a component of the variety of coadjoint orbits of a fixed dimension; from this point of view the description in Definition 3.7 is something to be proved.) Q.E.D.

In the case of SL(n), distinct sheets are actually disjoint. This partition of the orbits is the key to the definition of a Dixmier map in that

case (cf. [2]). Even for the other classical groups, however, distinct sheets can overlap: for Sp(4), the (Dixmier) sheets corresponding to (GL(2), {0}) and (GL(1) x Sp(2), {0}) share the same nilpotent orbit. As was explained in the introduction, this failure of disjointness is the main reason that one cannot have a nice bijection between orbits and completely prime primitive ideals for the other groups. We turn now to the problem of inducing orbit data. Each step in the geometric construction of Definition 3.1 (or Definition 3.4) has an algebraic analogue, provided by the standard dictionary between algebraic geometry and commutative algebra.

350

DAVID A. VOGAN Definition 3.9. Fix a pair (L,(Rz, px)), where L is a Levi subgroup of G, and

(Rx, Pz) is an orbit datum for L. (Definition 2.2). Notice that these hypotheses are much weaker than those of Theorem 2.6: the main difference is that Ry, (or rather its restriction

to the commutator subgroup) is not required to be pre-unipotent. We will

construct from these data (and a parabolic subgroup) an orbit datum for G. Define Zz = Supp Rz, (3.10)(a) the image of the morphism #7 from Spec RL to I*. If Ry is completely prime, then

Cr

(3.10)(b)

the closure of a single coadjoint orbit for L. If Ry is geometric, we write

Xz, = Spec RL

(3.10)(c)

for the corresponding variety (a ramified finite cover of Lz). datum for G is going to be supported on the closure of

Og = Indors(L t G)(Oz).

Our orbit

(3.10)(d)

Now fix a parabolic subgroup Q = LU of G with Levi factor L. (Recall that L is most naturally regarded as the quotient of Q by its unipotent radical, rather than as a subgroup. The reader may observe that it is this structure that we will actually use.) We will use the notation of Definitions

3.1 and 3.4. Define

Rou

dtp.

and let Q act on RQ by making U act trivially.

(3.11)(a) Define a map %g from

S(q) to Rg by

a(4) = (oa i :se

(3.11)(6)

The image of Vo is

Yo = tol2r) Se Dr:

(3.11)(c)

here I* is identified with (q/u)* C q*. If Rr is completely prime, then Xe is the closure of Og C q*. If Rr is geometric, then XQ = Spec Rg is a ramified finite cover of Ug as before. :

The algebraic version of (3.1)(d) is the definition

RayQ = S(9) Ss(a) Re:

(3.12) (a)

SHEETS, ALGEBRAS AND REPRESENTATION

THEORY

351

This algebra carries a natural action Ad of Q by algebraic automorphisms.

It is the tensor product of the adjoint action on S(g) with the action on Rg. There is an Ad(Q)-equivariant homomorphism

Vase : S(9) — Resa;

prp@l.

(3.12)(b)

The image of ¥% Je is

-1

Eco = TGjq(Ze)

(3.12)(c)

= {A€ g" | À |qE Zo}. This exhibits ZG/Q as an affine bundle over Lg with fiber (g/q)*. If Rr is geometric, we write Xc/Q = Spec Rg /qg; again this is an affine bundle over Xg. Suppose that Ry, is completely prime, so that Lg and therefore XG/q are irreducible. The existence of the induced orbit Og is equivalent

(by (A.3) and the definitions) to the fact that Q has a (unique) open orbit Og/a

on

Lesa:

We define a sheaf of algebras on G/Q by Re =G xq Rea.

(3.13)(a)

If V is an open set in G, then the space of sections of Rg over VQ is the space of algebraic maps p from VQ to Rg qg, satisfying

p(9q) = Ad(q~") (9) (cf. (2.5)(d)).

(3.13)(6)

(It might be slightly more natural first to identify Rg/g

with a sheaf of algebras Rgg over Ug/qg, and then to define R'g

=i (6; XQ Resa

(3.13)(a)'

as a sheaf of algebras over G xq Zcya (cf. (A.4)). Because EG,Q is affine, the two approaches are interchangeable; the one we have chosen seems slightly simpler.) The group G acts on the sheaf RG; an element g defines an algebra homomorphism from the sections over VQ to the sections over (g-!)VQ by right translation. The induced orbit datum

(Re, va) = Indors(Q 1 G)(Rz, br)

(3.14)(a)

Rg = global sections of Rg;

(3.14)(b)

is defined by

DAVID A. VOGAN

352

these are just the algebraic maps from G to Req satisfying (3.13)(b) (cf. (A.4)(c)). The algebra structure is pointwise multiplication, and the action of G is (g - p)(z) = p(g7!x). Finally,

Va : S(9) — Re,

(ba(p))(x) = (Ad(z7")p) @ 1 € Rae.

(3:14)(c)

Proposition 3.15. Suppose Q = LU is a parabolic subgroup of G, and (RL, pz) is an orbit datum for L. Then the pair

(Re, Va) = Indors(Q 1 G)(Rz, Yi) (Defintion 3.9) is an orbit datum for G. It is completely prime (respectively geometric) if (Rr, vx) is. The support of Ra is

De = Ad*(G) (Eye) = Ad*(G) - (roi (EL) Proof. Conditions (i) and (ii) in Definition 2.2 follow easily from the definitions.

We consider (iii). By hypothesis, Ry is a finitely generated

S(Q)-module supported on Dy. It follows easily from the definitions that Raya is a finitely generated S(g)-module supported on Lg. We may therefore identify it with a coherent sheaf Rg/g on Lg/g- By Proposition A.5, R'c =

XQ Req

is a coherent sheaf on G xg Ug/g. By Proposition A.2(c), the natural map G xq DT

—

G:Ecya

(3.16)

is proper. (In fact the proof shows that the morphism is projective.) By [9], Corollary II.5.20, the space of global sections of R'G is a finitely generated S(g)-module supported on G - Lg jg. By the remark at (3.13)(a)’, this space of global sections is precisely RG.

We know that © is a finite union of orbits of L (see the remarks after Definition 2.2). The theory of induced orbits recalled at the beginning of this section therefore implies that G - Xe/q is a finite union of orbits of G.

Condition (iv) in Definition 2.2 follows. That the support of RG is all of G- Zayq (rather than some proper subvariety) follows from the fact that the map (3.16) is generically finite; this in turn is a consequence of the the theory of induced orbits (cf. (3.1) and (A.3)).

SHEETS, ALGEBRAS

AND REPRESENTATION

THEORY

353

That Rg is completely prime (or commutative) whenever Rz, is follows from the definitions. Suppose that Ry is geometric; write

Xz, =Spec Rz,

X@/q = Spec Reg

(3.17)(a)

as in Definition 3.9. Then the sheaf of algebras Rg is just the structure sheaf of the bundle Ac =G

XQ XGJQ;

(3.17)(b)

over G/Q. Since Xg/q is normal, so is Yq. It follows that Rg, as the algebra of global functions on VG, is normal as well. Consequently Xg = Spec Rg is normal, so the orbit datum (Re, dg) is geometric. Q.E.D. Just as in the case of coadjoint orbits (Proposition 3.6), induction of orbit data is closely related to the Jordan decomposition.

Proposition 3.18. Suppose Q = LU is a parabolic subgroup of G, and (Rr,%z) is a completely prime orbit datum for L. Suppose that this orbit datum corresponds in the Jordan decomposition for L (Theorem 2.6) to (M,,,(Ru,VŸu)). Let L’ be the centralizer in G of 45. Then Q' =

QNL' = MU’ is a parabolic subgroup of L', and (RSS vi) = Ind,,5(Q° i!Ras

Vu)

is a completely prime pre-unipotent orbit datum for L'/Z(L')o. The induced orbit datum

(Re, Va) = Ind,5(Q T G)(Rz, vr) has Jordan decomposition (L',À,,(R,,,w)). The proof is parallel to that of Proposition 3.6, and we omit it. (In the statement, we have been a little sloppy about identifying orbit data for

(for example) L’ and L'/Z(L')o.) We know that the support of an induced orbit datum does not depend on the choice of parabolic Q; and we know (Corollary 2.7, for example) that an orbit datum is nearly determined by its support. This suggests

Conjecture 3.19. In the setting of Definition 3.9, the orbit datum Ind,,5(Q 1 G)(Rz, vz) is independent of the choice of parabolic subgroup with Levi factor L. An interesting special case (both of the construction and of the conjecture) is when Rz is € and yz is zero on I. Then RGyQ is the algebra S(g/q)

of functions on the cotangent space at eQ to G/Q, and XG is just T*(G/Q).

354

DAVID A. VOGAN

The algebra Rg is the algebra of regular functions on T*(G/Q). If Q' is another parabolic subgroup with Levi factor L, then T*(G/Q) need not

be isomorphic to T*(G/Q’). The conjecture asks for an isomorphism between the algebras of global regular functions on these (non-affine) varieties (respecting the G actions and the maps %q). Such an isomorphism does exist, but I do not know any very satisfactory proof of the fact. (Because of normality, it suffices to show that the unique dense G-orbits in the two cotangent bundles are isomorphic as homogeneous spaces. Fix elements

A € (9/4)" = Teg(G/Q), N € (9/4')" = Tig:(G/Q') representing these orbits.

Then

À and X’ are conjugate

g*, so the isotropy groups G(A) and G(X') are conjugate.

as elements

of

The identity

components of these groups are contained in Q and Q' respectively. The desired isomorphism is equivalent to the fact that G(A)NQ is conjugate to G(A‘) N Q’. This can be verified by more or less explicit computation on

a case-by-case basis.) The simplest non-trivial example is for G = GL(3) and L = GL(2) x GL(1). We can take G/Q to be the variety of lines in C3, and G/Q’ to be the variety of planes in C3. Of course these varieties are isomorphic, but the isomorphism cannot be made to respect the G actions. Nevertheless the algebras of regular functions on the respective cotangent bundles are isomorphic in a G-invariant way: they are both isomorphic to the algebra of regular functions on the corresponding (sub-

regular) nilpotent orbit in g*. Because of Proposition 3.18, a completely prime orbit datum for a semisimple group that is not pre-unipotent must be induced from a proper parabolic subgroup. It therefore makes sense to concentrate on pre-unipotent orbit data.

Definition 3.20. Suppose (R,w) is a completely prime pre-unipotent orbit datum for the semisimple group G. We say that (R, d) is rigid if it is not induced (in the sense of Definition 3.9) from a completely prime orbit datum on a proper Levi subgroup. Otherwise we say that (R, d) is induced.

This notion of rigidity is much broader than that for orbits (defined after Definition 3.1). That is, an orbit datum will be rigid if its support is rigid; but it may be rigid even if its support is not. If G is GL(n), every non-zero completely prime orbit datum is induced; butRARES for SL(2) this is not true.

Example 3.21. Suppose G is SL(2). We define a unipotent orbit datum (R, Ÿ) for G as follows. The algebra R is C[p, q], with the action of

SHEETS, ALGEBRAS

AND REPRESENTATION

THEORY

395

G induced by linear change of variables. We define Ÿ from S(g) to R on the standard basis (H, E, F) of g by

Y(H)=2pq,

Y(E)=p,

W(F)=-¢’.

Obviously R is finitely generated as an S(g) module (by 1, p, and q). It is easy to check that Ÿ respects the action of G. Every irreducible representation of G appears in R exactly once. The image of 7" is the cone H? + 4EF = 0, which is the nilpotent cone in g*. It follows that (R, Ÿ) is a unipotent orbit datum. One can show that it is rigid for example by computing all the induced orbit data; this is not difficult since there is only one proper Levi subgroup. In the parametrization of Corollary 2.7, R corresponds to the double cover of the principal nilpotent orbit. Iwan Pranata has observed that the preceding example can be modified to produce many non-geometric completely prime commutative orbit data. To do that, fix a non-negative integer k, and define R4 to be the subring of R generated by the image of Ÿ and the polynomials of degree 2k + 1.

(Pranata also found the Dixmier algebras associated to these orbit data.) Together with (C, 0), the various (Rz, d) exhaust the rigid completely prime orbit data for SL(2). Here is the analogue of Definition 3.7. Definition 3.22. Suppose Q = LU is a parabolic subgroup of G, and (Ru, Yu) is a rigid completely prime pre-unipotent coadjoint orbit datum for L/Z(L)o. To every character À, of [ we can associate a completely prime orbit datum (Rz,#Lr(À;)) for L as in (2.5)(e) and (f):

RL = Ru,

YL(A)= bu(At+a()) +As(A)

(AE).

The sheet of completely prime orbit data attached to (L, (Ru, #u)) is

{Indors(Q 1 G)(Rz, or(As)) |As € (1/0) } (Definition 3.9). It is called a Dirmier sheet if Ry = C. Two sheets are identified if they contain exactly the same orbit data; according to Conjec-

ture 3.19, this amounts to conjugacy of the pair (L, (Ru, Yu)) under G. (By considering regular A, in Proposition 3.18, one sees that this conjugacy is a necessary condition for the sheets to coincide.) We have at once an analogue of Proposition 3.8. Proposition 3.23. In the notation of Definitions 3.9 and 3.22,

356

DAVID A. VOGAN

i) Each completely prime coadjoint orbit datum belongs to at least one sheet. ii) If G is semisimple, each sheet contains exactly one preunipotent orbit datum (corresponding to À, = 0). iti) The supports of the orbit data in a single sheet are contained in a single sheet of coadjoint orbits.

This follows from Proposition 3.18. In (iii), the supports may not constitute an entire sheet. of coadjoint orbits. This happens exactly when the support of the rigid orbit datum R, is not a rigid coadjoint orbit.

Suppose that G = Sp(4), L = GL(2), and L' = GL(1) x Sp(2). Fix parabolic subgroups Q and Q’ with Levi factors L and L’. As was pointed out after Proposition 3.8, the Dixmier sheets of coadjoint orbits attached to L and L’ share the same nilpotent orbit. Let us now consider the corresponding sheets of orbit data. The unipotent orbit data for these sheets are

the rings of regular functions on T*(G/Q) and T*(G/Q’) respectively. They are not isomorphic as orbit data: the second contains the five-dimensional representation of G, and the first does not. It follows that these two sheets of orbit data are disjoint. (The non-unipotent orbit data are distinguished by their supports.) Many similar examples suggest

Conjecture 3.24. Two sheets of orbit data are disjoint or they coincide. The first possibility occurs exactly when the pairs (L, (Ry, Yu )) defining the sheets are conjugate by G. Conjecture 3.19 is more or less subsumed in this formulation. In the case of geometric orbit data, Conjecture 3.24 amounts to an assertion about the behavior of fundamental groups under induction (compare Corollary 2.7 and the example after Conjecture 3.19). It could in principle be verified by a finite calculation for each group. For the classical groups this should not be too difficult to carry out, but I have not done so. For non-geometric sheets (in particular when the normality hypothesis is dropped) the number of essentially different cases becomes infinite, and some more conceptual approach is probably necessary.

4. Parabolic induction of Dixmier algebras. The main purpose of this section is to give a construction of Dixmier algebras parallel to the construction of orbit data described in section 3.

SHEETS, ALGEBRAS AND REPRESENTATION THEORY

357

Fix a pair (Q,(Az,¢z)), where Q is a parabolic subgroup of G, and (Az, ¢z) is a Dixmier algebra for L. 4.1

We want to construct an “induced Dixmier algebra” (Ag, ¢¢) for a de course we will imitate and extend the construction of induced primitive

ideals (cf. [6]). Before we embark on the rather convoluted construction of Ag, it is worth pausing to explain why the problem is not trivial. Suppose V; is a

faithful module for Ar. Let V, = U(g) @q Vi be a parabolically induced module for g. (We are neglecting “p-shifts” — that is, the twist by the character 6 in (4.2)(b) — for this discussion.) The ideal J in U(g) induced by the annihilator of Vis by definition the annihilator of V,; so the quotient

ring U(g)/I is naturally embedded in End V,. The Dixmier algebra Ag is supposed to be some sort of nice ring extension of U(g)/J, so it is natural to look for this extension in End V,. Now it is an important theme of Joseph’s work that the algebra of G-finite endomorphisms of V, is a natural and wellbehaved ring; so it is an obvious candidate for Ag. To see that it is not the right one, consider the case L = G. In that case we had better have Az = Ag (if our notion of induction is to be related to induction of orbit data).

Our “obvious candidate” for Ag will have this property only if the Dixmier algebra Az is already the full algebra of L-finite endomorphisms of W. So this approach from the outset runs into a rather difficult technical problem: ‘is every Dixmier algebra for L the full algebra of L-finite endomorphisms of some [-module? (The answer in this generality is “no;” if Az is required to be reasonable (say prime, for example) the answer may be “yes,” but I

have no idea how to prove it.) What we actually do is this. We show that a large class of endomorphisms of V, (including all the G-finite ones) can be described essentially as matrices with entries in the endomorphism algebra of V; (cf. Definition 4.6 and (4.9)). We can then define Ag to consist of those G-finite endomorphisms of V, whose “matrix entries” belong to Az (Definition 4.7). This definition is not so difficult. What is slightly less easy is proving that Ag satisfies the finiteness requirements in the definition of Dixmier algebras, and relating the definition to induction of orbit data. To do that, we have

to give a rather different description of Ag (Corollary 4.16). (We cannot easily use Corollary 4.16 as the definition of Ag, because it is difficult to

see the algebra structure from this point of view.) Especially for the material on the symbol calculus, the reader should keep in mind the case Az = C; in that case Ag is the algebra of (holo-

morphic linear) differential operators on G/Q. (Recall that we are still neglecting p shifts!) That case has been thoroughly analyzed in [Borho-

DAVID A. VOGAN

358

Brylinski], and most of the serious ideas we use may be found there. Suppose then that

Vi is a faithful module for Az.

(4.2)(a)

(By Proposition 6.0 of [15], any prime Dixmier algebra is primitive; if Az is prime, we could therefore even choose V; to be simple as an Ay-module.

This makes no difference in the construction, however.) Let 26 denote the character of L on the top exterior power of g/q. Then 6 is a well-defined

character of [. Twisting the map ¢, by 6 we get a new Dixmier algebra (45,95): the algebra A, is equal to Ar, and

$1 (X) = o1(X)+6(X)

(XEN).

(4.2)(6)

This algebra has a faithful module

Vi = Vi @Cs.

(4.2)(c)

Make Q act on A‘, by making U act trivially; to emphasize this structure, we call the algebra Ag, and write

Ad : Q — Aut(Ag).

(4.2)(d)

pa : U(a) > Ag

(4.2)(e)

Extend # to

by sending u to 0. When we regard V/ as a module for Ag, we call it Vg. Write u for the nil radical of the parabolic subalgebra of g opposite to Q. Now define

Vy = U(g) Da Va © U(u-) ce Va.

(4.2)(f)

We are going to construct Ag as an algebra of endomorphisms of V,. We begin by constructing a larger algebra. To motivate the definition of this algebra, we will study the adjoint action of 3({) on the endomorphisms of V4. Our immediate goal is Lemma 4.5 below. Because

Ay is a Dixmier

algebra, the kernel of gr must

contain an

ideal of finite codimension in U(3(1)). To simplify the notation, we assume that this ideal is maximal (as it must be if Az is prime); the reader can easily modify the constructions that follow to cover the general case. Then 3(1) acts by a character À on Vq. If @ is any character of 3(f), write U(u- Ne

for the (finite-dimensional) subspace of U(u~) on which 3({) acts by a. Then Vj is the direct sum of its weight spaces

(Va)e = U(U )5-x ® Va.

(4.3)

SHEETS, ALGEBRAS

AND REPRESENTATION

THEORY

359

Next, recall that the algebra End V, is a bimodule for U(g). The left action of an element u on an endomorphism T is on the range of T, and the right action is on the domain. Write ad for the corresponding diagonal action of g: if T is an endomorphism, X is in g, and v is in V,, then

(ad(X)T)(v) = X - (Tv) — T(X - v).

(4.4)

An endomorphism T has weight 7 for ad(4(1)) if and only if it carries (Vg), to (V5)s47 for every 8. Using (4.3), one deduces immediately Lemma 4.5. Suppose T is an ad(3(l))-finite endomorphism of Vg. Then for every u in U(u—) there are finitely many elements u1,...,Un in U(u-), and endomorphisms E\,..., En

of Va, so that for each v in Va,

T(u ® v) = Su; ® E;v.

Here is a first approximation to the induced Dixmier algebra. Definition 4.6. Suppose we are in the setting (4.2). An endomorphism T of V, is said to be of type Ar if and only if for every u in U(u7 ) there are finitely many elements u1,...,u, in U(u~), and endomorphisms £},..., En in Ag, so that for each v in Vg,

T(u@v)= ys u; ® E;v.

(4.6)(a)

Using the defining relations for V,, one checks that it is equivalent to require this condition for every u in U(g), or to require only that u; belong to U(g). Write Ag={T€End(V,) | T is of type Ar } It is easy to check that Ag is an algebra. If w is in U(g), define ¢,(w) to be the endomorphism of Vg defined by the action of w. This is always of type Ar, so we get

og : U(g) — Ag,

(4.6)(b)

a homomorphism of algebras.

In (4.9) below we will give another description of Ag from which one can see that it depends only on the data (4.1) (and not on the choice of Vi).

360

DAVID A. VOGAN

Because of (4.4) and the existence of the map ¢g, we see that the algebra Ag is ad(g)-stable. The reason Ag is not a Dixmier algebra is that this Lie algebra adjoint action does not exponentiate to an algebraic action

of G. We remedy this in the simplest possible way. Definition 4.7. In the setting of Definition 4.6, define

Ag = Ad(G)-finite subalgebra of Ag = Ad(G)-finite endomorphisms of V, of type Az To understand

(47e)

this definition, recall that an element T of À, is called

Ad(G)-finite if it belongs to a finite-dimensional ad(g)-invariant subspace F of Ag, on which there exists an algebraic action AdF of G with differential ad. The elements in the image of 9, certainly have this property: one can

take for F the image of some level U,(g) of the standard filtration of U(g). Restricting the range of ¢, therefore defines

pa :U(g) — Aa,

(4.7)(6)

a homomorphism of algebras. It is easy to check that Ag is a subalgebra of Ag containing the identity element. Since G is connected, the actions Adp are uniquely determined, and can be assembled into an algebraic action

Ad:G— Aut(Ag).

(4.7)(c)

The action is by algebra automorphisms since ad(g) acts by derivations. We define the induced Dirmier algebra to be

(Ag, a) = Indpiz(Q 1 G)(Az, dr).

(4.7)(c)

To show that Ag is a Dixmier algebra, we only need to check the

finiteness conditions (iii) and (iv) of Definition 2.1. To do that, and to get a clearer picture of the structure of Ag, we need first to study Ag much more carefully.

Lemma 4.8. The set Ag is an algebra of endomorphisms of V,. If AL consists of all L-finite endomorphisms of Vi, then Ag contains all the L-finite endomorphisms of Vg. This is straightforward. Here is another description of Ag.

Fix characters @ and + of 3(D).

Define (End V,)s, to consist of those endomorphisms transforming by the

SHEETS, ALGEBRAS

AND REPRESENTATION

THEORY

361

character 8 by the left action of 3(l), and by + under the right action. These are precisely the endomorphisms vanishing on all the weight spaces of V, except the one for y, and whose image is contained in the 3 weight space:

(End V,)a7 = Hom((Vg),, (Vg)a)

= Hom(U(u-)y-a @ Va,U(U")a-» ® Va) (4.9)(a)

= Hom(U(u7),~-,, U(u)g_y) @ End Vy. The corresponding double weight space for Ag is

(Ag)ay = Hom(U(ué )y-.,U(u- )p-a) @ Ag.

(4.9)(b)

The algebra À, is built from these pieces in a slightly subtle way:

A=II (us) : À

(4.9)(c)

B

The reason we insist on a direct product outside is to ensure that the identity operator belongs to Ag. The direct sum inside ensures that the resulting algebra acts on V,.

In the description (4.9) of Ag, the algebra structure is the obvious one (induced by the algebra structure on Ag and the “composition maps”

Hom(U(u7),, U(u-)g) x Hom(U(u-)s5, U(u-),) — Hom(U(u-)5, U(ue )g). The structure that is subtle in this picture is the homomorphism

¢, of

(4.7)(b). At any rate, (4.9) shows that Ag depends only on Az (and Q), not on the choice of module Vy. We need two more descriptions of Ag. For each of them, it is convenient to introduce an auxiliary space. Definition 4.10. In the setting of (4.2), define

Agjq = U(G) Sa Aa:

(4.10)(a)

This space is clearly analogous to the algebra Rg g of (3.12), but I do not know any very simple motivation for its introduction. Clearly Ag;g carries a left action of U(g) and a commuting right action of U(q). There is also an algebraic action Ad:

Q ze End(Ag/q);

(4.10)(b)

which is the tensor product of the adjoint actions on U(g) and Ag. The differential of Ad is the difference of the left and right actions of q.

362

DAVID A. VOGAN The first of our final descriptions of Ag is Lemma

4.11. In the setting of Definition 4.10, define Aj = Homa (right,right)(U(9), Ag/Q)-

(7)

(Here q acts on the first U(g) and on Ag/g on the right to define the Hom,.) Then there is an isomorphism

a! : Ag — Aj.

(ii)

In this isomorphism, the left action of U(g) on Ag corresponds to the left action on Agjq; and the right action of U(g) on Ag corresponds to the left action on the domain U(g) in the Hom. Explicitly,

al (2Ty)(u) = x (a'(T)(yu))

(x,y,uEU(g), TE Ag.

(iii)

For x € U(g), the element $4(x) of (4.6)(b) satisfies

a'(d9(z))(u) = zu @ 1.

(iv)

We use the cumbersome subscript in (i) to distinguish this realization from that of Lemma 4.14 below. Proof. Fix T € Ag; we must define a'(T) as a map from U(g) to Ag/q = U(g) ® Ag. So fix u in U(g), and write

T(u ® v) = de ui © Eiv

(4.12)(a)

(with u; € U(g) and Ei € Ag) as in Definition 4.6. Then put

al (T}(u)= Sou; @ Ei.

(4.12)(b)

We leave to the reader the straightforward verification that a!(T) is a welldefined element of A. and that a! is an isomorphism.

The assertion (ii)

follows from (4.12). Ber (iv), one can use (iii) and the fact that ¢,(z) = z-1l4 (the left action of x on the identity element of Ag). Q.E.D. Using (4.12), my can compute the algebra structure on Aj induced by the isomorphism a!. Suppose S’ and S are in A

To dite

D

Tata

element u of U(g), re write

S(u) = du SE.

(4.13)(a)

SHEETS, ALGEBRAS AND REPRESENTATION

THEORY

363

Next, write

S'(ui) = du; @ By.

(4.13)(b)

j

Then

(S'S)(u) = oy uj ® Ey; Ej.

(4.13)(c)

We leave the straightforward verification of this to the reader. The last description of Ag is the least transparent of all, but it will be crucial to the symbol calculus. Lemma

4.14. Jn the setting of Lemma 4.11, define

Ae = Homa(test,aa)(U(9), Ac/a)

(7)

to be the space of maps & from U(g) to Ag/g with the property that

E(Xu) = ad(X)z(u)

(X € q,u E U(g)).

(ii)

Then there is an isomorphism

a? : Ag — Aj.

(iii)

Under the isomorphism a?, the adjoint action of U(g) on Ag corresponds to the right action on the domain U(g) term in A1:

a?(ad(z)T)(u) = a?(T)(uz).

(iv)

The map ®,4 is computed in this picture by

a?($q(x))(u) = ad(u)z @ 1.

(v)

Before proving this lemma, we deduce the consequences we want.

Definition 4.15. quasicoherent sheaf

The complete symbol sheaf (for Ar and Q) is the AG

=

G XQ Ag/e@

on G/Q. Here we use the action Ad of Q on Agjq. It should be fairly easy to make Ag into a sheaf of algebras on G/Q, but I have not done this.

364

DAVID A. VOGAN

Corollary 4.16. The algebra formal power series sections ofAg Ag then corresponds to the global sections corresponds to the adjoint

Ag may be identified with the space of at the identity coset eQ. The subalgebra sections of Ag; the left action of G on action on Ag. In particular,

AG = Indaig(Q T G)(Acye)

= Indagig(Q t G)(U(g) Sa A). Here we use induction of algebraic representations (cf. (A.8)); Q acts on the inducing representation by Ad @ Ad.

Sketch of proof. This is a consequence of Lemma 4.14 and general facts about homogeneous vector bundles. A formal power series section Z of the “vector bundle” G xg Aya is specified by a map from U(g) to

Ag/q, sending u to the value at eQ of the derivative 0,(2). Since © must transform (infinitesimally) under Q in a certain way, this map must respect the action of q; that is, it must belong to the space A of Lemma 4.14. This gives the first assertion. For the rest, one needs to know that a G-finite formal power series section must come from a globally defined section; this is easy. Q.E.D. Corollary 4.17. In the setting of Definition 4.7, the pair

(Ac, 4) = Indpiz(Q T G)(Az, 1) is a Dixmier algebra. The kernel of dg in U(g) is the ideal induced (via q) from the kernel of dr in U(V). Proof. Since Az is a Dixmier algebra for L, the kernel of gx contains

an ideal of finite codimension in the center of U(l). Since Ag is an algebra of endomorphisms of Vg, it follows from the theory of the Harish-Chandra homomorphism that the kernel of dg contains an ideal of finite codimen-

sion in the center of U(g). By the theory of Harish-Chandra modules for G, condition (iii) in the definition of Dixmier algebra (Definition 2.1) is a consequence of condition (iv). But condition (iv) is an elementary consequence of the description of Ag as an induced module in Corollary 4.16; we leave the calculation to the reader. For the statement about induced ideals, notice first that the annihilator of Vi in U(T) is the kernel of $7 (since Vi is assumed to be faithful for Az).

By definition the induced ideal is therefore the annihilator in U(g) of Vg. Since Ag is an algebra of endomorphisms of Vg, the claim follows. Q.E.D.

SHEETS, ALGEBRAS

AND REPRESENTATION

THEORY

365

Conjecture 4.18. In the setting of Definition 4.7, the induced Dixmier algebra is independent of the choice of Q.

Of course this conjecture is analogous to Conjecture 3.19. The corres-

ponding assertion for induced ideals is true (cf. [8]). Beyond this, there is very little evidence. It may be that one has to impose some extra condition on Az, such as complete primality. Proof of Lemma 4.14. Rather than passing directly from Ag to As. we will construct an isomorphism

al? : Al — A?.

(4.19) (a)

In light of Lemma 4.11, it will suffice to show that a!? has nice properties; then the isomorphism

Got

00

(4.19)(b)

will satisfy the requirements of Lemma 4.14. Sofix S € A, and define

o'?(S)(u) = (ad(u)S)(1).

(4.19)(c)

That the map a!?(S) satisfies condition (ii) in Lemma 4.14, and that a}?(S)(uv) = a!?(ad(v)S)(u)

(4.19)(d)

(which implies (4.14)(iv)) are easy calculations. The formula (4.14)(v) follows from (4.11)(iv) and the fact that the map ¢, is ad-equivariant. The main point is therefore to show that the map a’? is a linear isomorphism. To see that, we will write an explicit inverse. Write u — ‘u for the transpose antiautomorphism of U(g); we have ‘'X = —X for X in g. Write

h: U(g) > U(g) ® U(g) for the Hopf map; this is the algebra homomorphism sending X in g to

X @1+1@X. If M is a bimodule for U(g), then the adjoint action of U(g) may be computed as follows. Fix u in U(g), and write

Then

h(u) = Sou @u.

(4.20)(a)

ad(u)m = Ÿ_(u;)m(tv;).

(4.20)(b)

We have described the left and right actions of U(g) in Aj explicitly; we deduce that for S € Aj

(ad(u)S)(z) = D u[S(uix)].

(4.20)(c)

366

DAVID A. VOGAN

Consequently

a'?(S)(u) = S> ui(S(*04)).

(4.20)(d)

Now suppose that © is an element of (Aes Define a map a?!(Z) from U(g) to Ag/@

by

a?(E)(u) = Du (El vi).

(4.21)(a)

One way to see that a?! is well-defined is to regard Aa as a subspace of

Home(U(g), Ag/q). This larger space carries commuting left and right U(g) actions, from the left actions on Ag/g and U(g) respectively. Consequently there is an adjoint action ad; and

a?" (E)(u) = (ad(u)E)(1).

(4.21)(6)

To check that a?!(X) belongs to Ar, fix u in U(g) and X in q; we must show that

a (E)(uX) = (a7!(Z)(u))X. Now h(uX) = u:X D

@ vi + uj @ 44,X), so

a (E)uX) = (ui X (Elu) + (EX vi). By (4.14)(ii), the second terms can be rewritten to give

a (E)(uX) = D (ui X (El ui)) — ui(ad(X)E (vi) = Ju X(Eu) — us(X s(*v;) — Efoi)X)). The first two terms in each summand cancel, leaving

a(E)(uX) = S>(wis(‘vs)X) = (a*"(Z)(u))X, as we wished to show. Finally, we must show that the correspondences defined by (4.20) and

(4.21) are mutual inverses. Fix S in Aj; we will show that a?! (a'2(S)) —

on

(x)

(The proof that a!?(a?!(Z)) = E is identical.) To do this, fix X1,..., Xh

in U(g); we will compute the left side of (x) at u = X1...X,.

If A is

any subset of {1,...,n}, write X4 for the corresponding product (with the

SHEETS, ALGEBRAS

AND REPRESENTATION THEORY

367

indices arranged in increasing order). The complement of A is written A°. Then the Hopf map is given by h(u)

=

D

XA

® X ac.

AC{1,..,n}

Therefore a?1(a1?(S))(u)

=

5

Xa(a"?(S)('Xa-))

A

= ee |+ “X95(%-0)) A

BCA°

DDR TETE

TT)

A BCA°

This last expression may be regarded as a sum over partitions of {1,...,n} into three disjoint subsets A, B, C; it is

D Xa'XBS(Xc) A,B,C

=) (ad(Xe-) - 1)$(Xe). C

Of course ad(Xc-) acts by zero on 1 unless C° is empty. The only non-zero term is therefore

(ad(1) : 1)S(X{1,.n3); which is eu as we wished to show. Q.E.D. We conclude this section with a symbol calculus for the induced Dixmier algebra. Definition 4.22. Suppose we are in the setting of Definition 4.10. The

standard filtration of U(g) induces an Ad(Q)-stable filtration

Agja,n = image of Un(g) ® Ag

(4.22)(a)

of Agjg- The left action of U(g) is compatible with this filtration in the usual sense (see the discussion preceding Conjecture 2.3). Define Sg/g to be the associated graded object:

SG/q = S"(9/4) & Ag:

(4.22)(b)

368

DAVID A. VOGAN

This is a graded algebra with a graded algebraic action (still called Ad) of Q by automorphisms. (In fact it is the pushforward to a point of a constant sheaf of O-algebras on (g/q)*.) The principal symbol sheaf is the graded

sheaf of algebras on G/Q SG = G XQ Se/a

(4.22)(c)

(cf. (3.13)). (This is the pushforward to G/Q of a sheaf of O-algebras on T*(G/Q).) The space of global sections of Sg is a graded algebra Sg that we call the principal symbol algebra; explicitly,

Sa = Indaig(Q Tt G)(S(g/4) ® Ag) = functions on G with values in S(g/q) ® Ag,

(4.22)(d)

transforming by Ad under Q. An immediate consequence is that if Ag (or, equivalently, Ar) is completely prime, then Sg is as well. We write Ad for the action of G on Sg by left translation of sections. Now define the weak filtration on Ag by Agora ={Se A, |S(Un(g)) € (Aesq)ptn}-

(4.23)(a)

(Here and below we will use the isomorphisms a! and a? to transfer filtrations among Ag, A}, and A?.) This makes Aj a filtered algebra (as one checks using (4.13(c)), but not every element of Aj belongs to some Aj ». It follows from (4.11)(iv) that Pa(Un(9)) € (Ag)nwe-

(4.23)(b)

Using (4.20) and (4.21), one can describe the weak filtration directly on A? as well: it is

Apur = {2 € AG |E(Un(9)) C (Aayq)ptn}-

(4.23)(c)

The difficulty with the weak filtration is that it is not ad(g)-invariant. Given any filtered algebra A and a family D of derivations of A, we can define a smaller filtration of A by Anan

{ae A [|Di... Dy(a) € An, all {D;} CRDI

This gives A a new structure of filtered algebra, this one preserved by D. In our case, we set

Agp = {T € Ag |ad(u)T € Ag pwe, all u € U(g) }.

(4.24)(a)

SHEETS, ALGEBRAS

AND REPRESENTATION

This is not very easy to understand

THEORY

369

directly; but Lemma

4.14(iv) and

Age = {2 € A7 |E(Un(8)U(9)) € (Aaya)p+ns alln}.

—(4.24)(a)

(4.23)(c) show immediately that the corresponding filtration of A? is

Clearly this is

(4.24)(b)

Aj» = {Z| E(U(9)) C (Aaja) }-

Using Corollary 4.16, we deduce filtration to Ag:

a description of the restriction of this

AG,p = Indaig(Q 1 G)(Acyop).

(4.24)(c)

Regarded as an algebraic map from G to Ag Je, every element of Ag has its image contained in some finite-dimensional subspace. Consequently

Ae» = Ac.

(4.24)(d)

P

On the level of sheaves, Definition 4.22 gives rise to an exact sequence 0 —

AG,p-1

Fr: AG p >. Se —

0.

(4.25)(a)

(Here we have used and extended slightly the notation of Definition 4.15 and (4.22)(c).) Taking global sections gives an exact sequence

DAS | The last map

Gre oe

(4.25) (6)

in this sequence is called a principal symbol map,

and is

denoted 7). Because of (4.24)(c), these maps can be assembled to a graded injection

x: g Ac — Sg.

(4.25)(c)

Clearly 7 respects the action Ad of G. What is less obvious is

Proposition 4.26. The principal symbol map x of (4.25) ts an algebra homomorphism. In particular, if Az is completely prime, then the induced Dizmier algebra Ag is as well.

If H'(G/Q, Sag) = 0, then x is an isomorphism of graded algebras. Proof.

Suppose S € AT

Write © = a/?(S) for the corresponding

element of AZ ,. If u belongs to U,,(g), then (4.21) shows that . S(u) = ud(1)

(mod AO

By the definition of a!?, this is the same as

nn)

DAVID A. VOGAN

370

(mod Ag/a,ntp-1)-

S(u) = uS(1)

(4.27)(a)

Write

S(1) =) a SE;

(4.27)(b)

with u; in U,(g). It is immediate from the definition that the value at the identity of the principal symbol is given by

m)(S)(e)= > pi @ Ei;

(4.27)(c)

here p; is the image of u; in S?(g/q). Suppose S’ belongs to Ag. Write

Sn

VR CIES,

(4.27)(d)

with v; in U,(g). If q; is the image of v; in S?(g/q), then

ms NES

a;@ ES.

(4.27) (e)

By (4.23) applied to S’, S'(u)=

>uv; SE;

(mod Agjap4q-1)-

(4.27)(f)

j

By Lemma 4.11(iv),

(S’S)(1) = Suiv; @ EVE;

(mod Agya,p4q-1)- (4.27)(9)

i,j

Consequently

T™+q(S'S)(e) = DP

@ E;E;

= ,(S')(e)m(S)(e).

(4.27)(h)

The proposition follows from (4.27)(h) and the Ad(G)-equivariance of 7. Q.E.D. There is a variant of this symbol calculus that is of interest for the Dixmier conjecture. (Because we will never need both at the same time, we will not use a different notation for the filtrations in the variant.) Suppose that Az is endowed with a good filtration indexed by 1/2N. Recall from

SHEETS, ALGEBRAS

AND REPRESENTATION

THEORY

371

the discussion before Conjecture 2.3 that this amounts to the following requirements: AL pAL,q C AL,p+9;

Arp is Ad(L)-invariant;

Cre

4

(4.28)

P

L(Un(l))

(E (Ar )n;

(Rx, vx) = (gr Az,gr gr) is a graded orbit datum. We want to construct

a good filtration of the induced Dixmier algebra

(Ac, ¢a). To begin, filter Ag/g by

Aaja = >, Un(9) ® Aop-n-

(4.29)(a)

(Of course the underlying algebra of Ag is just Az, so Ag inherits the

filtration of Ar.) This filtration is Ad(Q)-stable, and Un(g) -AG/Q,p Le AGJQn+p-

(4.29)(b)

The associated graded object therefore carries an action Ad of Q and a graded S(g)-module structure. A little thought shows that it is the graded tensor product

gr Aya = S(g) s(a) gr Aag-

(4.29)(c)

This is exactly the algebra Rgjg attached to Ry in the construction of

induced orbit data (cf. (3.12)). We call iE

= G XQ Rese

(4.29)(d)

a good symbol sheaf. The space of global sections is a graded algebra RG that we call a good symbol algebra. By (3.13) and (3.14), RG

=

Indoro(Q

Î G)(Rz).

(4.29)(e)

Of course Rg is an orbit datum for G by Proposition 3.15. Now we can filter A by

Al, = {2 € AG |E(U(8)) C Ag/ap }-

(4.30)(a)

372

DAVID A. VOGAN

Exactly as in (4.23) and (4.24), one shows that this defines a filtered algebra structure on A?. The induced filtration on Ag is

and

Ag. = Indaig(Q 1 GXAc/Q»),

(4.30)(6)

Ae» = Ac.

(4.30)(c)

P

Again we get an exact sequence of sheaves on G/Q

0 — Ag p-1 > Aap — RE — 0.

(4.31)(a)

This gives rise to an exact sequence

0 — AG,p-1,94 — AG,p,gd > (Re)?-

(4.31)(6)

The last map in this sequence is called a good symbol map, and is denoted Yp. We therefore have a graded G-equivariant injection

y: gr Ag — Re.

(4.31)(c)

Exactly as in Proposition 4.26, one shows that y is a homomorphism of algebras. We have proved Proposition 4.32. Suppose the Dirmier algebra (Az, 1) is endowed with a good filtration indered by 1/2N (cf. (4.28) with associated graded orbit datum (Rr,%r).

Write (Re, va) for the induced orbit datum for G.

Then the induced Dixmier algebra (Ag, ¢G) has a natural good filtration, and the associated graded orbit datum injects into (Rg, vq). If the cohomology H!(G/Q,RG) vanishes, then this injection is an isomorphism. Evidently this result has some relationship to Conjecture 2.3. We will discuss this in detail in section 5.

5. Sheets of Dixmier algebras. We can now say something about what a “sheet of Dixmier algebras” ought to look like. Of course such a sheet ought to consist of the Dixmier algebras corresponding via Conjecture 2.3 to a sheet of geometric orbit data. Our goal, however, is to make a little progress toward proving that conjecture. We fix therefore a sheet of geometric orbit data attached to

SHEETS, ALGEBRAS

AND REPRESENTATION

THEORY

373

a Levi subgroup L of G and a rigid unipotent orbit datum (Ry, vu) for

L/Z(L)o (Definition 3.20). Thus

Xy = Spec À, is a unipotent Poisson variety for L/Z(L)o

(5.1)(a)

(cf. [21]). One consequence is that the action of C* on the support Dy must lift to X,, at least after passing to a finite cover of C*. This means that R, is graded by k~1N for some positive integer k. As was already mentioned in section 2, Moeglin has observed that k is at most 2. (This is a consequence of the Jacobson-Morozov theorem.) Consequently

Ry is graded by 1/2N;

(5.1)(b)

this gradation is compatible with the standard one on S(g) and the map

Vu. The first serious step is to attach to (Ru, ~u) a Dixmier algebra for

L/Z(L)o. There is a conjecture for how to do this in section 5 of [21]; we reproduce a strengthened version here.

Definition/Conjecture 5.2. Suppose G is a semisimple group, and (R, Ÿ) is a rigid geometric (and therefore unipotent) orbit datum (Definition 2.2 and Definition 3.20). Recall the natural grading of R by 1/2N, and the natural Poisson structure {,} on R. A (rigid) unipotent Dirmier algebra associated to (R,%Ÿ) is a Dixmier algebra (A,¢) (Definition 2.1) having an Ad(G)-invariant filtration by 1/2N, subject to the following conditions.

i) $(Un(g)) C An.

ii) The pair (gr A,gr ¢) is isomorphic as an algebra with G-action to

(R, #). These two properties conjecturally determine exactly one Dixmier algebra. An immediate consequence is

iii) A is completely prime.

As a conjectural consequence of (i) and (ii), one should have four additional properties.

iv) The infinitesimal character ofA corresponds to a weight in the rational span of the roots.

v) The kernel of ¢ is a weakly unipotent primitive ideal in U(g). vi) Suppose a and b in A, and A, have images r and s in RP and R?. Then the Poisson bracket {r,s} is the image of ab — ba. vii) A admits a transpose antiautomorphism (a — ‘a) of order 1, 2, or

4 (Definition 2.8). This preserves the filtration, and the associated graded antiautomorphism acts by exp(ip7) on RP.

374

DAVID A. VOGAN

(The notion of weakly unipotent primitive ideal may be found in [19], section 8. The condition is that the infinitesimal character cannot be made shorter by tensoring with a finite-dimensional representation. The prototypical example is the augmentation ideal.) This conjecture is very easy if R = C (corresponding to the zero nilpo-

tent orbit). If R corresponds to a minimal non-zero nilpotent orbit (which is rigid except in type A) then the conjecture is true because of Joseph’s work on the Joseph ideal (as supplemented by Garfinkle). The complete conjecture is known only in a handful of other cases, although there is some evidence for it in many other infinite families of examples. Some of

the most powerful results in the direction of (ii) are those in [16]. Now assume that we are in the setting (5.1), and that we have a unipotent Dixmier algebra

(Au, Du)

(5.3)(a)

for L/Z(L)o attached to (Ry, #4) in the sense of Conjecture 5.2. To every character À of 3([) we can attach an orbit datum

(Rx(A), Yz())

(5.3)(6)

for L; here Rz(A) is isomorphic to Ry, and wz(A) is given as in (2.5)(e) and (f). Similarly, we can construct a Dixmier algebra

(AL(A), #1 (à))

(5.3)(c)

for L. Here Az(A) is just Ay as an algebra with L action, and

PLOA(X) = A(X)

érQ(Y)=¢.(Y)

|

Definition 5.4.

(X € 30),

(els).

(5.3)(d)

Suppose we are in the setting (5.1), and that the

Dixmier algebra (Au,¢u) for L/Z(L)o exists.

Fix a parabolic subgroup

Q = LU with Levi factor L. Form a sheet of orbit data

(Ra(à), Pa(A)) = Indors(Q 1 G)(Rz(A) br(A)) , (Definition 3.22).

(5.4)(a)

(Recall that, according to Conjecture 3.19, this sheet

is independent of the choice of Q.) The sheet of Dirmier algebras for G attached to (Q, (Au, ¢u)) is

(Ag(A), da(A)) = Indpie(Q 1 G)(Ax(A), 61(À)

SHEETS, ALGEBRAS

AND REPRESENTATION

THEORY

375

(Definition 4.7). According to Conjecture 4.18, these algebras do not depend on the choice of Q.

By Proposition 4.32 (and hypothesis (ii) in Definition/Conjecture 5.2) each Ag(A) carries a natural good filtration indexed by 1/2N. These filtrations have the property that there is natural injection of graded orbit data

(gr Ac()),gr¢a(A) > (Re (0), da(0)).

(5.5)(a)

An elementary argument also provides natural good filtrations of the orbit data in the sheet, with the property that

(gr Re(A),erVa(A)) = (Re(0), da(0)).

(5.5)(b)

Both of these injections ought to be isomorphisms. Because of Proposition

4.32 (and an easier analogue for orbit data) this would be a consequence of Conjecture 5.6. In the setting of Definition 5.4, form the sheaf R¢(0) on G/Q as in (3.13). Then the higher cohomology of G/Q with coefficients in Rg(0) vanishes. If R, = €, then the sheaf RG(0) is the sheaf of functions on the cotangent bundle of G/Q. In that case Conjecture 5.6 is true, by a wellknown result of Elkik. The correspondence taking (Rg(A), ve(A)) to (Ag(A), dg(A)) is our candidate for a Dixmier correspondence on geometric orbit data (cf. Conjecture 2.3). To define it, we need to know the existence of unipotent Dixmier algebras attached to rigid orbit data (Conjecture 5.2). For it to

be well-defined on a single sheet, we need both Rg(A) and Ag(A) to be independent of Q (Conjectures 3.19 and 4.18). For it to be well-defined on all geometric orbit data, we need the geometric sheets to be disjoint (Conjecture 3.24). For it to satisfy Conjecture 2.3, we need a cohomology vanishing result (Conjecture 5.6). 6. Relation with bimodule

induction.

In this section we will resume our study of induced Dixmier algebras, focusing on special conditions on the inducing algebra that permit a further analysis of their structure. The main tool is the idea of induction of HarishChandra bimodules, which we now recall.

Definition 6.1. Suppose Q = LU is a parabolic subgroup of G, and Br is a Harish-Chandra bimodule for L. Write Q® = LU°? for the opposite

DAVID A. VOGAN

376 parabolic subgroup.

Define a new bimodule By by subtracting from the

left and right actions of { the character 6 defined after (4.2)(a). Make By into a left q-module by making u act by zero, and into a right q°?-module by making u?? act by zero; we denote this object Bg qor. Consider Bi

= By = Homg,qor(U(g) © U(g), Ba,qer)-

Here q acts on the left on the first U(g) factor and q® on the right on the second to define the Hom. Bg has the structure of a g-bimodule: the left action of g comes from the right action on the first U(g), and the right action from the left action of g on the second U(g). Put

BGvi = Ba = Ad(G)-finite part of Bg; this is the Harish-Chandra bimodule for G induced by Br. We write Be

=

Indy; (Q T G)(Bz).

Induction of Harish-Chandra bimodules is a very simple and wellunderstood process. It is an exact functor, and (as algebraic representations under Ad) we have Be & Indaig(L

Î G)(Br)

(6.2)

(notation (A.8)). Because this is induction from a reductive subgroup, it is much better behaved than the induction appearing in (say) Corollary 4.16. The reader may wonder why we did not use it in section 4 to construct induced Dixmier algebras. The reason is that bimodule induction does not in general take algebras to algebras. Nevertheless, there is much to be gained (computability, for example) whenever we can relate induction of Dixmier algebras (which are, among

other things, Harish-Chandra bimodules) to Indj;.

Examples for SL(2)

show that the relationship cannot be quite trivial. To understand it, we will first interpret bimodule induction in terms of endomorphisms between modules (Corollary 6.5). Once that is done, it is convenient to extend the induction construction of Definition 4.7 from Dixmier algebras to arbitrary

Harish-Chandra bimodules (Definition 6.7). Once the two kinds of induction are described in parallel, it becomes clear that they are related by

something like the Shapovalov form on a Verma module (Definition 6.11). Establishing an isomorphism between them therefore comes down to proving some irreducibility results for (appropriately generalized) Verma mod-

ules (Theorem 6.12). The idea of this argument comes from [5]; much more

SHEETS, ALGEBRAS

AND REPRESENTATION

THEORY

377

sophisticated incarnations of it appear in Joseph’s work relating primitive

ideals to highest weight modules (cf. [11] and [12], section 1.3). Suppose Vi and W;, are modules for {. Assume that

Br C Hom(\, Wi)

(6.3)(a)

is a Harish-Chandra bimodule of maps. Here the left action of { comes from the action on W,, etc. (It is easy to see that any Harish-Chandra bimodule arises in this way; we do not require Vi and WA to be particularly nice.) Write V/ for Vi with the action of [ twisted by —6, and then V or for the extension to q?? on which u® acts by zero. Similarly define W,. Then Ba, qer

G

Hom(Vgor,

W,).

(6.3)(b)

Define Viind

Vaor

(6.3)(c)

Wa pro = Hom,(U(g), Wa) = Home(U(u?, Wa).

(6.3)(d)

Lemma

=

U(g) © gor Vacr ve, U(u) Qc

6.4. In the setting (6.3), there is a natural identification of

U(g)-bimodules Home (Vg ing, Wa,pro) = Homa, qer(U(g) © U(g), Hom(Vqer, Wa)) = Home(U(u??) @ U(u), Hom(Vaor, Wa)).

We will not give the (easy and well-known) formal proof, but it is helpful to recall the form of the isomorphism.

If ¢ on the left corresponds

to ® on the right, then for u and wu’ in U(g) and v in Vy?,

[é(u ® v)] (u’) = (®(u' @ u))v. Here u @ v is an element of Vg ing. The term in square brackets on the left

is therefore an element of Wg p,0; that is, it is a map from U(g) to Wg. We specify it by specifying its value at w’. Definition 6.5. Suppose we are in the setting of (6.3). A map p : Vg,ind 4

Wg pro

is said to be of type By, if the corresponding map & : U(g) @U(g) — Hom(Vqep, Wa))

DAVID A. VOGAN

378

(Lemma 6.4) takes values in Bg,qer. By Definition 6.1, Basi © {9 : Vasina + Wapro| ¢ is of type Br }. Definitions 6.1 and 6.5 combine (like all good definitions) to give a result.

Proposition 6.6. Suppose we are in the setting (6.3). Then the induced Harish-Chandra bimodule Bg = Inds;(Q t G)(Bz) consists precisely of the Ad(G)-finite maps from Vg ina to Wapro that are of type Br. We now outline the extension to bimodules of the construction of section 4.

Definition 6.7. In the setting of (6.3), extend W{ to a module Wgor for q® by making u® act by zero. Define

Wg,ina = U(g) @qor Waor.

(6.7)(a)

In analogy with (4.2), identify Br with a U(q°?)-bimodule

Baer C Hom(Vaep, Wer).

(6.7)(b)

As in Definition 4.6, we say that a map T from Vg ing to Wg ina is of type

B_, if for every u in U(g) there are elements u; in U(g) and FE; in Be» such that for every v in Vqop,

T(u@v) =)

uj @ Ev.

(6.7)(c)

The collection of all such maps is a U(g)-bimodule

Bg Diz C HOM(V, ina, Wo,ina):

(6.7)(d)

Define Be,pir = Ad(G)-finite maps from Vg ing to Wgina of type Br.

(6.7)(e)

The proof of Corollary 4.17 shows that Bg piz is a Harish-Chandra bimod-

ule for G depending only on Q and By (and not on the choices of Vi and

Wi). We call it the Dirmier induced Harish-Chandra bimodule, and write

BG,pir = Indpiz(Q” 1 G)(Bz).

(6.7)(f)

SHEETS, ALGEBRAS

AND REPRESENTATION

THEORY

379

As in Corollary 4.16, one gets an isomorphism of G-modules for Ad

Be, pie © Indaig(Q” 1 G)(U(g) @qer Baer).

(6.7)(9)

Here q°? acts on Bger on the left to define the tensor product, and Q% acts by Ad @ Ad on the inducing representation. Recall that what we want is to compare BG wi and BG pic. We will do this by comparing the larger bimodules B, 4; and Bg piz. To do that, we

determine their bimodule structure under the center 3({) of [. As in section 4, it is harmless and convenient to assume that 3({) acts by characters À

and p on W{ and V/ respectively. We assume also that the Harish-Chandra bimodule By has finite length. Fix characters 8 and y of 3(1). Then the subspace of Bg piz transforming on the left by @ and on the right by y is (cf. (4.9))

(Bg, vie)py = Hom(U(u)y—p,U(u)p-r) ® Boer.

(6.8)(a)

The whole bimodule is assembled from these pieces by the prescription

By, Diz © I] (517.05) : ay

(6.8)(b)

B

Similarly, Lemma 6.4 gives (Bavi)s

a, Hom(U(u?)_

45 ® DUR,

Ba,qer.

(6.8)(c)

Now the indicated weight spaces (for ad(3(1) in U(u) and U(u®) are finitedimensional. As an -bimodule, Bg qe» is equal to Bocr. We may therefore rewrite this last equation as

(Bg,i)ay © Hom(U(u)+—p,(U(u?)-g+x)") ® Baer.

(6.8)(d)

By Definition 6.5,

Bgvi © [[(] [(Bovi)sr)-

(6.8)(e)

Wo

Lemma 6.9. Suppose By is a Harish-Chandra bimodule of finite length for L. Then as algebraic representations of Ad(L), the bimodule weight spaces (Bg si)ay and (Bg nir)py for (1) contain the same represen-

tations with the same (finite) multiplicities.

380

DAVID A. VOGAN

Proof. This follows from (6.8)(a) and (d), using the isomorphism of L-modules u + (u°?)* provided by the Killing form. Q.E.D. Our next task is the construction of a bimodule map from Bg piz to

By. We need a little more notation in the setting of (6.3). We have

U(g) = U(q) ® U(g)u”;

(6.10)(a)

nm: U(g) — U(q)

(6.10)(b)

write

for the corresponding projection on the first factor. Write

Iq,qer : Wacr — Wa

(6.10)(c)

for the identity map” of the underlying [-modules. Definition 6.11. Suppose q = [+ u and q°? = [+ u® are opposite parabolic subalgebras of g. Suppose V; and Wi are \-modules; we use other

notation as in (6.3) and (6.10). Define a homomorphism of q?? modules Jqor fsW gor i

Wo pros

(jar(w))(x)

=

n(x) © (Ta ge

W).

(6.11)(a)

Here z is in U(g) and w is in Wa». It is easy to check that jqor(w) really belongs to Wg pro, and that the map is a q??-module injection. universality property of induction, jgo» induces a g-module map

J: Wo ina — We pros

(j(u © w))(x) = r(zu) - (Ia grw).

By the

(6.11)(b)

We callj the canonical intertwining operator; it is closely connected to the

Shapovalov form on a Verma module (see [6] or [10]). Composition with j induces a g-bimodule map JE

Homc(Viind,

Wg. ina) a

Home

(Vg ina, Woero)-

(6.11)(c)

We claim that this composition sends maps of type By to maps of type Br (Definitions 6.5 and 6.7) and therefore restricts to J: Bg pic — Bai.

(This will be proved in a moment.) action ad, and so restricts to

(6.11)(d)

As a bimodule map, J respects the

J :Indpiz(Q” 7 G)(Bz) — Indi(Q 1 G)(Bz);

(6.11)(e)

SHEETS, ALGEBRAS AND REPRESENTATION THEORY

381

this map we call the canonical bimodule intertwining operator. To prove that J preserves type, suppose that TE Home(Vg ina, Wg ind) is of type Br; we want to prove that J(T) = j oT is as well. Write 7 for the map from U(g) @ U(g) corresponding to J(T) (Lemma 6.4), and fix

u and u’ in U(g). Then 7(u’ @ u) is a map from Var to Wa; we have to show that it belongs to Bg ger. To compute it, fix v in Vaor. Now (6.6)(c) provides elements u; in U(g) and E; in Byer (depending only on u) so that T(u®v)=

pau; @ E;v.

Tracing through the definitions, we find

(r(u’ @ u))v = (J(T)(u ® v))(u’)

(by Lemma 6.4)

= (JD ui @ Eiv))(u’) = Jo n(u'u;)-(IqqerEiv)

(by (6.11)(b)).

Now composition with Ig qo» clearly defines an isomorphism from Bgor to Bg,qor. Write F; for the image of E; under this map. The action of m(u’u;) on F;v just corresponds to the left action of U(q) on Ba qe»; so we get

Teo)

= ie r(u'u)F.] v.

Now the term in square brackets belongs to Bg,qer, as we wished to show. Theorem 6.12. Suppose Q = LU is a parabolic subgroup of G, and By is a Harish-Chandra bimodule offinite length for L; say bare

Hom(V,, Wi).

Define Wg ina and Wgpro as in (6.3) and Definition 6.7, and the canonical intertwining operator j between them as in Definition 6.11. Ifj 1s one-toone, then the canonical bimodule intertwining operator Je Indp;:(Q°%

Î G)(Bz)

=> Ind3i(Q

is an isomorphism. Proof. Because j is one-to-one, the map

J: By viz — Bgvi

T GY(B1)

382

DAVID A. VOGAN

is obviously one-to-one. By Lemma 6.9, the restriction of J to each 3(()bimodule weight space (Bg piz)gy is an isomorphism. The map J respects

the decompositions (6.8)(b) and (e) in the obvious way. (This is easy, but it is not a formal consequence of linearity, since infinite direct products are involved. One has to recall how the decompositions on the level of endomorphisms arise from decompositions of modules. These latter decompositions are respected by the module intertwining operator j.) It follows at once that J is injective. More precisely, J carries the ad(3(l))-finite part of the domain isomorphically onto the corresponding part of the range. A fortiori

the restriction of J to the Ad(G)-finite part is an isomorphism. Q.E.D. It is worth recording the slightly stronger statement that we actually proved: under the same hypotheses as in Theorem 6.12,

(Bo,Diz)ad(3(0))-finite © (Bab )ad(3(1))-finite:

HR)

One can find in section 8 of [19] various sufficient conditions for the map j of Theorem 6.12 to be an isomorphism. Here is one of them.

Proposition 6.14 ([19], Proposition 8.17). Suppose parabolic subalgebra of g, and W

is an [-module.

Define

q = (+ u is a

Wg ina and Wg pro

as in (6.3) and (6.6), and the canonical intertwining operator between them as in (6.11). Assume that

t) The annihilator in U([l,]) ofWi ts a weakly unipotent primitive ideal. ü) The center 3(1) of (acts by a character À on Wy. ti) Ifà is a weight of 3(l) in u, then Re >0. Thenj is injective.

We can combine this with Theorem 6.12 to get

Corollary 6.15. Suppose Q = LU is a parabolic subgroup of G, and By ts a Harish-Chandra bimodule of finite length for L. Assume that t) The annihilator in U([l,Q)) of the left action on By is a weakly unipotent primitive ideal. u) The center 3(l) of{ acts on the left on Br by a character À. wi) If a is a weight of 3(1) in u, then Re > 0.

SHEETS, ALGEBRAS

AND REPRESENTATION

THEORY

383

Then the canonical bimodule intertwining operator AIS Indpiz(Q”?

T G)(BL)

>

Ind;;(Q

Î G)(Br)

ts an isomorphism.

Corollary 6.16. Suppose

Q = LU is a parabolic subgroup ofG, and

(Az, ¢z) is a Dirmier algebra for L. Assume that t) The kernel of dr in U({l,(]) ts a weakly unipotent primitive ideal. ü) The center 3(1) of |acts on Ay by a character À. ii) If w is a weight of (1) in u, then Re? >. 0:

Then the induced Dirmier algebra (Ag,¢q) is isomorphic as a bimodule to Indy;(Q f G)(Az). In particular, the induced algebra is tsomorphic

to Indaig(L | G)(AL) as an algebraic representation under Ad. The higher cohomology groups of G/Q®P with coefficients in the complete symbol sheaf

Ag (Definition 4.15) are zero. The point of the cohomology vanishing result is primarily that if it were not true, then our definition of induced Dixmier algebras would be flawed: the higher cohomology groups would have to be taken into account somehow. One expects that in interesting cases Ag has a filtration with

associated graded sheaf of the form R&G(0) (cf. (5.5)). The present vanishing theorem would in such cases be a consequence of Conjecture 5.6; so we may regard it a as a kind of evidence for that conjecture. Proof. Only the last assertion requires comment. Write [ for Zucker-

man’s functor from (g, L)-modules to G-modules (passage to the G-finite part). Write I’ for its derived functors ([18], Chapter 6). If Z is any algebraic representation of Q°?, write Z=GXqgorZ

for the corresponding sheaf on G/Q°?. proof of Corollary 4.16) that

We have already observed (in the

Hab (C IMA Z) = T'(Homgr(U(g), Z)L-finite):

It is easy to extend this fact to higher cohomology:

H(G/Q®,Z)= Té(Homger(U(g), Z)L-jinite).

384

DAVID A. VOGAN

(One reason for this is that the same u°?-cohomology spaces can be used to the higher cohomology with coefficients compute either side.) To compute in Ag, we must therefore apply I* to

Homger(U(g), U(g) @qer Ager) L-finite(We are considering only the ad action of g.) This is just the L-finite part of Ag pir. By (6.13), it is isomorphic to the L-finite part of As. Definition 6.1, this is

By

Homg,qor(U(g) @ U(g), Aa,g)L- finite.

Again we are interested only in the adjoint action of g. As a module for this action, Ag»; is isomorphic to Hom(U(g), Ar); here [ acts on U(g) on the left and on Az by ad. (This elementary fact is the basis of (6.2).) Therefore

H'*(G/Q®, Ac) = T'(Homi(U(g), Ar))L- finite.

(6.17)

The module to which I* is applied is a standard injective (g, L)-module, so the right side is zero for positive 7. Q.E.D. Geometrically, the proof shows that the sheaf Ag is the pushforward of a sheaf on G/L. Since G/L is affine, the cohomology vanishing follows.

7. The translation principle for Dixmier algebras. As mentioned in the introduction, there ought to be a theory of modules for induced Dixmier algebras entirely analogous to the Beilinson-Bernstein localization theory. I have not developed such a theory, but this section describes one of its basic results (Corollary 7.14). It is included mostly as evidence that the definition of induced Dixmier algebras is reasonable and interesting. The experts will find no surprises here; but such readers may wish to examine carefully the hypotheses in Theorem 7.9, which are substantially weaker than in some formulations of the translation principle. We begin by recalling from section 3 of [22] the notion of translation

functors for Dixmier algebras. Write 3(g) for the center of U(g). Suppose @ is a character of 3(g) and I, is the associated maximal ideal. If M is a 3(g)-finite (left) g-module, write aM={meEM

|

for some n, (I,)”-m=0}.

(7.1)(a)

This is an exact functor on the category of 3(g)-finite g-modules, and

M =) aM,

(7.1)(b)

SHEETS, ALGEBRAS

AND REPRESENTATION THEORY

385

the sum running over all characters of 3(g). We use analogous notation for right modules. | Suppose (A, ¢) is a Dixmier algebra. Then A is a 3(g)-finite bimodule, so there is a finite direct sum decomposition of A

ESO

(7.2)(a)

a,p

This decomposition is preserved by the U(g)-bimodule structure and (as a consequence) by Ad(G). The multiplication satisfies

eh

are

In particular, each , 4, is a subalgebra. module,

(7.2)(b)

More generally, if M is an A-

enna ea eee bres

(7.2)(c)

As a formal consequence of these facts, we get Lemma

A-module.

7.3.

Suppose A is a Dirmier algebra and M

is a simple

With the notation (7.1) and (7.2), each non-zero 4M is an

irreducible , A-module. Conversely, suppose N is an irreducible gAqg-module. M

= AG

A

Define

N

Then ,M' = N. Consequently M' has a unique mazimal proper submodule S. The quotient M = M'/S is a simple A-module, and ,M = N. These constructions establish a bijection between the set of simple A-modules M with , M non-zero, and the set of simple , Ag-modules. Next, suppose (7, F) is a finite-dimensional representation of g. If M

is a 3(g)-finite g-module, then so is F @ M (cf. [13]). To make the corresponding construction for Dixmier algebras, fix a Dixmier algebra (A, ¢a) and form the algebra

B= End(F)@

A.

(7.4)(a)

We can define an algebra homomorphism ¢g of U(g) into B by p(X) = 7(X)@1418

ga(X).

(7.4)(b)

386

DAVID A. VOGAN

To make a Dixmier algebra, we need an action Ad of G. This will exist

whenever the adjoint action of g on End(F) exponentiates to G (as it always does if F is irreducible, for example). In that case we can define

Ad(g) : (T ® a) = (Ad(g) - T) @ (Ad(g) - a).

(7.4)(c)

These definitions make B into a Dixmier algebra. Lemma 7.5. Suppose A is a Dirmier algebra and F is a finitedimensional representation of g. Assume that

the adjoint action ofg on End(F) exponentiates to G.

Form the Dirmier algebra

B = End(F) @ A as in (7.4).

(2)

Then the map

M — F@M is an equivalence of categories from A-modules to B-modules. The inverse functor is N — Homgnar)(F, N).

This well-known result is an elementary exercise.

(The Dixmier algebra

structure is purely decorative; B is really just the ring of n x n matrices

with entries in the ring A.) Fix now a character a of 3(g) and a finite-dimensional representation F of g. The elementary translation functor for modules attached to a and F is the functor

TM =,(F ®@ M)

(7.6)(a)

on 3(g)-finite g-modules. Assume in addition that F satisfies condition (i) of Lemma 7.5. The elementary translation functor for Dizmier algebras attached to a and F is

TA = «(End(F) @ A).

(7.6)(b)

Proposition 7.7 (Translation Principle for Dixmier algebras). Jn the setting of (7.6), suppose A is a Dixmier algebra. Then the translation functor T takes modules for A to modules for TA. This functor has the following additional properties. a) T is exact. b) If M is an irreducible A-module, then TM is irreducible or zero as a T A-module. c) IfN is an irreducible T A-module, then there is a unique irreducible A-module M such that N=TM. This proposition is an immediate consequence of Lemmas 7.3 and 7.5.

SHEETS, ALGEBRAS

AND REPRESENTATION

THEORY

387

The expert reader may be wondering what has been hidden, since results of this form about the translation principle usually require more hypotheses and more proof. The difficulty arises if we want to speak about

modules for U(g) instead of for some Dixmier algebra. If A = U(g)/I, then an A-module is just a U(g)-module annihilated by J. There will always be a natural inclusion

¢:U(g)/J — TA.

(7.8)

If ¢ is surjective, then a T A-module is just a U(g)-module annihilated by J, and Proposition 7.7 becomes a result about g-modules. It is exactly such surjectivity results that require additional hypotheses and more difficult proofs. Since we have cast off the shackles of g-modules, we take a slightly different view of what is required to make Proposition 7.7 interesting. We will begin with a known Dixmier algebra A, and try to understand the translated algebra 7 A. Here is such a result. Theorem

7.9.

Suppose Q = LU

is a parabolic subgroup of G, and

(Au, Qu) ts a Dirmier algebra for L/Z(L)o. For each character£ of 3(() define a Dirmier algebra (Azr(E), or(€)) by (5.3)(d). Define

(Ac(), da(€)) = Indpie(Q 1 G)(AL(E), 61(6). Fix a one-dimensional character y of \, and assume that yp occurs in the restriction to | of a finite-dimensional representation of g. Let F be the unique irreducible finite-dimensional representation of g containing yu as an extremal weight. (This means that F has a q'-invariant line of weight u, for some parabolic subalgebra q’ having Levi factor t.) We sometimes identify p with its restriction to 3(I). Fiz a character À of 3(0). Assume that

i) ker(¢u) is a weakly unipotent primitive ideal in U([l, 1). ti) If B is any weight of 3(1) in g/\ and (B,u) > 0, then

Re (B,) > 0. In particular, (i) implies that Ag(A) has an infinitesimal character, which we denote by a. Let T be the elementary translation functor associated to a and F*. Then

T(Ag(A + #)) = Ag(A).

Proof. We argue as in [19], section 8. Fix a faithful module V,, for Ay. For every character € of 3(1), Vu becomes a faithful module V\(€) for

388

DAVID A. VOGAN

the Dixmier algebra Az(£). (The point is that the underlying algebra of Ax(€) is just Au; only the map ¢z(€) is changing.) Just as in (4.3) we can construct V,(€) and Va(€) = U(9) Da Val(E)(7.10) By Lemma 7.5, F* @V,(A+) is a faithful module for End(F*)®Ac(A+). Consequently

T(Vg(A + #)) = alF* © Vg(A + #)]

(7.11)(a)

is a faithful module for T(Ag(A +yu)). We will show that

T(Vq(A + 1) = VO).

(7.11)(8)

This will show that T(AG(À + u)) and Ag(A) are both represented as endomorphism algebras of V,(A). It is then very easy to check from the definitions that they are exactly the same endomorphisms; we leave this to the reader.

It therefore remains to check (7.11)(b). By (7.11)(a) and the algebraic version of Mackey’s tensor product theorem, the right side of (7.11)(b) is

a [U(8) Sa ((F™ la) @ Va(A + #))]-

(7.11)(¢)

Now F* has a q-stable filtration whose subquotients are irreducible repre-

sentations of [. We get a filtration of (7.11)(c) whose subquotients are

a [U(g) Sa (E” ® Va(A + p))],

(7.11)(d)

where E can be any irreducible constituent of F |x.

If E is the u weight space of F, then (7.11)(d) is Vg(A).

We must

therefore show that the other terms are all zero. Explicitly, this means the following. Suppose E is a representation of [ occurring in F', other than the weight 4. Then we must show that the infinitesimal character a does not occur in

U(9) @q (E* @ Va(A + u)).

(7.12)(a)

To prove this, fix a Cartan subalgebra t of [I, 1]. Then h = t + 3(1) is a Cartan subalgebra of g. Fix a weight ag € t* corresponding to the infinitesimal character of A, in the Harish-Chandra correspondence. Then a corresponds to

(ao,À) € €* + 3(D* = b*.

(7.12)(b)

Let 1 be the weight of 3(1) on E, and a; an infinitesimal character for [1] in E* @ Vy. Then a typical infinitesimal character in (7.12)(a) is

(ar, À + y — y).

(7.12) (0)

SHEETS, ALGEBRAS AND REPRESENTATION THEORY

389

What we are trying to show is that this cannot be equal to a; that is,

that the weights (7.12)(b) and (7.12)(c) are not conjugate under the Weyl group. To do that, it is obviously sufficient to show that

Re ((@0, À), (&o, À)) < Re((a1,A +p — 1), (a1,

A+ p@—p1)).

(7.13)(a)

To prove (7.13)(a), we first use the hypothesis (i) of the Theorem. This implies that the infinitesimal characters occurring in E* @ VW, are all longer than apg: (ao, ao) < (a1, a1).

(7.13)(b)

Next, any weight of 3(1) in F must be of the form

m=n-

np.

Here the sum is over weights B of 3(1) on g/l having positive inner product with y, and ng is a non-negative integer. If yz; is different from y, then the sum is non-empty. By hypothesis (ii),

Re(_ ngB,A) > 0. Consequently

Re (A + p— pi, A+

— 1) = Re(A+ D> g8,A+ 9 npf) = Re (A, A) + 2-Re(>_ nf, d)

(7.13)(c)

+ (So neh, Soe ng) > Re (À, À). Adding (7.13)(b) and (7.13)(c) gives (7.13)(a) and completes the proof. Q.E.D. It is very easy to refine the argument so that the positivity hypothesis

(ii) is needed only for those weights 8 that are restrictions of roots integral on the infinitesimal character. Corollary 7.14. Under the hypotheses of Theorem 7.9, every 1rreducible module N for Ag(A) is of the form TM, with M a unique irreducible module for Ag(A+p). Conversely, if M is an irreducible Ag(A+p)-module,

then TM 1s an irreducible Ag(A)-module or zero. The point of this corollary is that it allows one to study problems of irreducibility at “very regular” parameters, where they are typically much easier.

DAVID A. VOGAN

390

Appendix.

Induced bundles.

We assemble here some basic definitions used throughout the paper. Suppose throughout this appendix that G is an (affine) algebraic group, and H is a closed (and therefore affine algebraic) subgroup. Recall that the homogeneous space G/H is a quasiprojective algebraic variety. As a point set, G/H is just the coset space. Its topology is the quotient topology from G: a subset U of G/H is open if and only if its preimage V in G is open. A regular function on such an open set U is by definition a (right) H-invariant regular function on V. The main point in the construction of G/H is that for small enough V there are many such functions. Suppose now that Zy is any algebraic variety on which H acts. The induced bundle G xq Zu is a bundle over G/H whose fiber at the identity coset eH is Zy: GXxH

ZH

—

G/H

(A.1)(a)

This bundle is constructed from the product G x Zy in the same way that G/H is constructed from G. That is, we define an equivalence relation ~

on (closed points of) G x Zy by

(z,z) ~ (zh7!,h-z)

(cE€G,z€Zy, he #).

(A.1)(b)

As a point set, G X q Zy is the set of equivalence classes for this relation. That is, it is the set of orbits of an action of H on G x Zy (called the right action) defined by

h-R(z,z)=(æzh"l h.2)

(A.1)(c)

The subscript R is included to distinguish this action from the (left) action of G defined by

g:(2,z)=(92,2z)

(eE€G,z€ Zy,9 €G).

(A.1)(d)

This action of G commutes with the right action of H, so it is inherited by G XH

ZH.

A subset U of Gx x Zy is defined to be open if and only if its preimage V in G x Zy is open. The regular functions on U are defined to be the regular functions on V that are invariant under the right action of H:

OU)={fEOV) |fh-nv)=f(v)

(VEVhEH)}

(A.1)(e)

This makes sense because the preimage in G x Zy of any subset of Gx y Zy is by definition a union of orbits of the right action of H. That it makes G XH Zy into an algebraic variety with a G action is proved just as for the

case of G/H itself.

SHEETS, ALGEBRAS AND REPRESENTATION

THEORY

391

Here are some properties of the induced bundle construction; all follow fairly easily from the definitions.

Proposition A.2. Suppose G is an algebraic group, H is a closed subgroup, and Zy is an algebraic variety on which H acts. a) The induced bundle G xy Zn is an algebraic fiber bundle over G/H. The fiber over eH is naturally identified with Zy, and the isotropy action of H on the fiber is the original action ofH on Zu. b) Suppose Y is an algebraic variety on which G acts, and f : Zy — Y is a morphism respecting the actions of H. Then there is a natural morphism F =G xy f from Gx Zn to Y, respecting the actions of G. On the level of equivalence classes,

Farm): c) In the setting of (b), suppose in addition that f is proper, G/H is a projective variety. Then the map F is proper. d) Suppose Zy ~ H/K is a homogeneous space for H. GXH

LH

and that

Then

~G/K.

In the setting of (b) in the proposition, one can easily describe the fibers (over closed points) of the map F. Of course the image of F is G - f(Zx), so the fibers outside that set are empty. By G-invariance, it suffices to understand the fiber over a point y € f(Zx). Write K C G for the isotropy group of the G-action at y. Then

FT (y) = K xanax F7 (y).

(A3)

In particular, F is injective if and only if the following two conditions are satisfied: f is injective; and the stabilizer of every point in f(Zx) is contained in H. Suppose now that My is a quasicoherent sheaf on Zy with an action

of H (compatible with the action on Zx). sheafMg = G xx My quasicoherent sheaf

on G x Zy.

We can define the induced

on Zg = G XH Zn as follows.

Consider first the

(This can be thought of informally as “functions on G with

values in My; that description is accurate on any open set that is a product of an open set in G with one in Zy.) The sheaf N carries a right action of H compatible with the right action of H on G x Zy, and a

commuting (left) action of G. With notation as in (A.1)(e), we define

Mg(U) ={meEN(V)|h-nm=m (he #).

(A.4)(b)

392

DAVID A. VOGAN

The space Mg of global sections of Mg has a simple description. Write My for the space of global sections of My. This is a vector space (generally infinite-dimensional) carrying an algebraic action of H. We may speak of the algebraic functions on G with values in My; such a function is required to take values in a finite-dimensional subspace, and to be algebraic (in the obvious sense) as a map to that subspace. That is, it should belong to

(Og(G)) ® Mu. Now it is clear from the definitions that Mg ={f:G—>My|

f is algebraic, and f(zh) = h~*f(z)}.

(A.4)(c)

Proposition A.5. Suppose G is an algebraic group, H is a closed subgroup, and Zy is an algebraic variety with G-action. Write Zg = GxH Zn. Then the category of quasicoherent sheaves on Zy with H-action ts equivalent to the category of quasicoherent sheaves on Zg with G-action, by the induction construction My + GxH My of (A.4). This equivalence identifies the subcategories of coherent sheaves. We omit the proof. It is worthwhile to describe the inverse functor, however. Let Zy be the sheaf of ideals defining the subvariety Zx of Zg = G xy Zn. Then O7z,, may be identified as a sheaf of Oz,-modules with

Oz,/Tu.

If Ma is any quasicoherent sheaf on Zg, then the “geometric

fiber” MinaOz.

Q0z, Me

~ Me/ToMe

(A.6)

is a quasicoherent sheaf on Zy. If Mg carries an action of G, then this fiber

inherits an action of H (the largest subgroup of G preserving the ideal Ty). As the notation indicates, the geometric fiber provides a natural inverse for

the induced sheaf construction of (A.4). Again we omit the proof. We conclude by considering the relationship between the induced sheaf

construction of (A.4) and the notion of induced representation for algebraic groups. Recall first of all that an algebraic representation of G is a pair (x,V) with V a vector space and

7: G— GL(V)

(A.7)(a)

a homomorphism. We require in addition that m be algebraic, in the following sense. For every v € V, there should be a finite-dimensional subspace E C V, containing v, with the property that

m(G)E CE,

(A.7)(b)

and the resulting homomorphism

TE : G— GL(E)

(A.7)(c)

SHEETS, ALGEBRAS AND REPRESENTATION THEORY

393

is a morphism of algebraic groups. Suppose now that (7, Vi) is an algebraic representation of H. The algebraically induced representation of G is the algebraic representation

Indaig(H 1 G)(tH, Vir) = (rc, Va)

(A.8)(a)

Va ={f :G— Va |fis algebraic, and f(zh) = ty(h-')f(z)(zx €G,h € H)}

(A.8)(b)

(ra(g)f)(x) = f(g~*z).

(A.8)(c)

defined by

(We will drop the maps z from the notation when no confusion can result.) The analogy with (A.4) is clear, and in fact the connection is very close. Proposition A.9. Suppose G is an algebraic group, H is a closed subgroup, and (xx, Vx) is an algebraic representation of H.

a) The induced representation Vg = Indgig(H {| G)(V#) is an algebraic representation of G. b) Identify Vi with a quasicoherent sheaf My with an H-action on a point. Then Vg may be identified with the space of global sections of GxH Mu (cf. (A.4)). c) Suppose that Vy is the space of global sections of a quasicoherent sheaf My on some Zn as in (A.4). Then Vg may be identified with the space of global sections of the induced sheafG xy My on G xy 2H. d) (Frobenius reciprocity.) Suppose W is any algebraic representation of G. Then there is a natural isomorphism Homg(W,

e) Suppose dimVy

Va) =

Homy(W

lx, Va).

< oo, so that G xx Vy (defined as in (A.1)) is a

vector bundle over G/H. Then Vg may be identified with the space of sections of this vector bundle.

This is very well-known, and we omit the straightforward proof.

REFERENCES [1] A. Beilinson and J. Bernstein,

“Localisation de g-modules,”

C. R.

Acad. Sci. Paris 292 (1981), 15-18. [2] W. Borho, “Definition einer Dixmier-Abbildung für s{(n,C),” Inventiones math. 40(1977), 143-169.

394

DAVID A. VOGAN

[3] W. Borho, “Uber Schichten halbeinfacher Lie-Algebren,” Invent. math. | ‘ 65 (1981), 283-317. bei Deformationen deren und Bahnen “Uber Kraft, H. [4] W. Borho and linearen Aktionen reduktiver Gruppen,”

Comment.

Math.

Helvetici

54 (1979), 61-104. [5] N. Conze-Berline and M. Duflo, “Sur les représentations induites des groupes semi-simples complexes,” Compositio math.

34 (1977), 307-

336. [6] J. Dixmier, Algébres Enveloppantes. Gauthier-Villars, Paris-BrusselsMontreal 1974. [7] J. Dixmier, “Ideaux primitifs dans les algebres enveloppantes,” J. Al-

gebra 48 (1978) 96-112. [8] M. Duflo, “Sur les idéaux induits dans les algebres enveloppantes,” Inventiones math. 67 (1982), 385-393. [9] R. Hartshorne, Algebraic Geometry. Springer-Verlag, New York, Heidelberg, Berlin, 1977.

[10]

J.C. Jantzen, Moduln mit einem Hochsten Gewicht, Lecture Notes in Mathematics 750. Springer-Verlag, Berlin-Heidelberg-New York, 1979. [11] A. Joseph, “Dixmier’s problem for Verma and principal series modules,” J. London Math. Soc.(2) 20 (1979), 193-204. [12] A. Joseph, “The primitive spectrum of an enveloping algebra,” 1353 in Orbites Unipotentes et Représentations III. Orbites et Faisceaux

pervers, Astérisque 173-174 (1989). [13] B. Kostant, “On the tensor product of a finite and an infinite dimensional representation,” J. Func. Anal. 20 (1975), 257-285.

[14] G. Lusztig and N. Spaltenstein, “Induced unipotent classes,” J. London Math. Soc.(2) 19 (1979), 41-52. [15] W. McGovern,

“Unipotent

representations

and Dixmier

algebras,”

Compositio math. 69 (1989), 241-276. [16] C. Moeglin, “Modèles de Whittaker et idéaux primitifs complètement premiers dans les algèbres de lie semi-simples complexes II,” Math. Scand. 63 (1988), 5-35. [17] R. Rentschler, “Comportement de l’application de Dixmier par rapport à l’anti-automorphism principal pour les algèbres de Lie résolubles,” C. R. Acad. Sci. Paris 282 (1976), 555-557. [18] D. Vogan, Representations of Real Reductive Lie Groups. Birkhauser, Boston-Basel-Stuttgart, 1981. [19] D. Vogan, “Unitarizability of certain series of representations,” Ann. of Math. 120 (1984), 141-187.

[20] D. Vogan, “The orbit method and primitive ideals for semisimple Lie algebras,” in Lie Algebras and Related Topics, CMS Conference Pro-

SHEETS, ALGEBRAS AND REPRESENTATION

THEORY

395

ceedings, volume 5, D. Britten, F. Lemire, and R. Moody, eds. American Mathematical Society for CMS, Providence, Rhode Island, 1986. [21] D. Vogan, “Noncommutative algebras and unitary representations,” in The Mathematical Heritage of Hermann Weyl, R. O. Wells, Jr., ed. Proceedings of Symposia in Pure Mathematics, volume 48. American Mathematical Society, Providence, Rhode Island, 1988. [22] D. Vogan, “Irreducibility of discrete series representations for semisimple symmetric spaces,” 191-221 in Representations of Lie groups, Kyoto, Hiroshima, 1986, K. Okamoto and T. Oshima, editors. Advanced Studies in Pure Mathematics, volume 14. Kinokuniya Company, Ltd., Tokyo, 1988. [23] D. Vogan, “Associated varieties and unipotent representations,” preprint. Received October 12, 1989

David A. Vogan, Jr. Department of Mathematics Massachusetts Institute of Technology Cambridge, MA 02139

el Gas)

wre

Ve ain me

SSE

ORNE

4 Las,

f

,

.

AT,

ieodaors

EPH PRE Ss

greg

of rv

i!’ Nopwteshtabons ,

ul mn 4 4à ñ

pR8 (IME, iad sum

je

oe

Mdchitse ail

tn

|

oF

à,

(17

-

on

i

y :

Slee

Cote

od al

OAD)

4

à

celia

Ane)

à lite

an

30 (With

A

ol, hp pride

Ss LETTRE

TL

ètre i

‘i 3

faye)

FE

. a

‘wet ‘

11

slt bi 6

U

¥

i

Jp Pons *.

art:

2

|

in clame"

d

}

7

1

uv eral Mails

Dixmier Algebras and the Orbit Method

WILLIAM M. McGOVERN Dedicated to Professor Jacques Dizmier on his 65th birthday

Abstract

Let G be a complex Lie group with Lie algebra g, enveloping algebra

U(g), and symmetric algebra S(g). The Nullstellensatz provides a tight link between ideal theory in S(g) and geometry in g*.

One naturally

wants to generalize it to the slightly non-commutative setting of U(g). The fundamental work of Dixmier and others does this for solvable g . Here we give an historical survey of the various attempts to generalize Dixmier’s results to the semisimple case, showing how these attempts

have led to the notion of what we call a Dixmier algebra. We give an exposition of some recent work on Dixmier algebras. We conclude by giving a number of interesting examples of Dixmier algebras, some of them geometrically nice while others are not. 1. Introduction

Let g be a complex Lie algebra with enveloping algebra U(g) and symmetric algebra S(g). Let G be a simply connected group with Lie G = g. The Nullstellensatz provides a tight link between ideal theory in S(g) and geometry in g*; more precisely, it yields a bijection between prime quotients of S(g) and irreducible affine varieties in g*. A basic goal in any reasonable theory of non-commutative algebraic geometry is to extend this bijection to a (slightly) non-commutative setting; a natural one to consider is that of U(g). For solvable g, Dixmier extended the bijection to U(g) as follows: he showed that primitive ideals of U(g) correspond 1 - 1 to G-orbits in g*. We call this the orbit correspondence. It reduces the classification of the primitive spectrum of U(g) to a purely geometric question which in many cases admits a complete solution. In view of the power and generality of the orbit correspondence in the solvable case,

398

WILLIAM M. McGOVERN

it was natural to believe that it should extend to the semisimple case without too much difficulty. Twenty years of effort have unfortunately shown that there are fundamental obstacles to its implementation for semisimple g. In this paper, we will describe these obstacles in some detail and show how the attempt to overcome them has led to the notion of a Dixmier algebra, roughly a finite algebra extension of a primitive quotient of U(g). We will then survey the work done so far on Dixmier algebras, briefly reviewing the methods currently available for constructing them. We will conclude the paper by giving a number of interesting examples of Dixmier algebras. Most of these algebras are completely prime and have nice geometric properties, but a few of them are not. The latter algebras are, however, closely related to certain algebras which we call unipotent, so we call them mock unipotent. Mock unipotent Dixmier algebras help to illustrate a basic principle of any orbit correspondence: only completely prime Dixmier algebras can hope to have a nice geometric theory.

2. Dixmier algebras Henceforth, assume that g is semisimple.

One of the most important

differences between primitive ideals of U(g) and primitive ideals in the solvable case, is that the former need not be completely prime. As we stated in the introduction, a basic tenet of any orbit correspondence is that it should involve only completely prime ideals of U(g). For exam-

ple, U(g) has irreducible finite-dimensional modules of arbitrarily large dimension. The image of U(g) in the endomorphism ring E of any such module V is always E itself, whence it has zero divisors if V is nontrivial.

Since any such image of U(g) is finite-dimensional, it must correspond to a 0-dimensional subvariety of g* under any reasonable implementation of the orbit correspondence. But the only G-stable such variety is the single

point 0. We can attach this point to the augmentation ideal of U(g) but then it is clear that we cannot attach any G-orbit to the other primitive ideals of U(g) of finite codimension. We can also see the necessity of restricting to completely prime primitive quotients of U(g) in a less expected way. Take g = s/(n), the algebra of traceless complex matrices, and consider primitive ideals of U(g). of a fixed regular infinitesimal character. It is well known that these are parametrized by the set of all standard Young tableaux with n boxes [15]. But there is no reasonable injective map from standard Young tableaux to coadjoint orbits; the best we can do is to attach to a Young diagram the nilpotent orbit whose Jordan form is given by the corresponding partition. Thus we need a way to cut down from the set of Young tableaux to the set of Young diagrams. Moeglin’s beautiful characterization [30] of the completely prime primitive ideals as induced

DIXMIER ALGEBRAS AND THE ORBIT METHOD

399

from codimension-one ideals in a parabolic subalgebra in effect does this. More precisely, her main result implies that the Dixmier map [6] yields a

bijection between the set g*/G of coadjoint orbits and Prim’(U(g)), the sets of completely prime primitive ideals of U(g) . Resuming to general semisimple g, one is thus readily led to make Conjecture 2.1. There is a natural bijection between Prim'(U(g)) and g*/G. This bijection sends an ideal of Gelfand-Kirillov dimension d to an orbit of the same dimension.

(For generalities on Gelfand-Kirillov dimension, see [13], Chapter 8.) The first thing to say about this conjecture is that it is false in general for any reasonable interpretation of the word “natural”. It is true and easily verified for the smallest example g = sl(2), by a direct calculation. For g = sl(n) we have already mentioned the Dixmier map from g*/G

to Prim'U(g)) which is a bijection by [8] and [31]. The smallest g not isomorphic to a product of sl(n)’s is 50(5), the algebra of 5 x 5 complex skew-symmetric matrices.

In this case, Conjecture 2.1 was first studied

by Borho [5]. He parametrized the sets Prim’(U(g)) and g*/G explicitly. It is clear from his parametrization that there is a bijection between these sets, but it is not clear how to implement this bijection canonically. Any straightforward attempt to generalize the Dixmier map leads to a correspondence that is not well defined. Thus Conjecture 2.1 already stood on shaky ground shortly after it was first formulated and studied. It was

given a death blow by Joseph, who studied Prim’(U(g)) for g of type Gy. He showed that there are two elements U(g)/J,, U(g)/J2 of this set of Gelfand-Kirillov dimension 8, but only one orbit in g*/G of this

dimension [17]. Vogan was the first to produce a geometric interpretation of these first failures of the orbit correspondence. More specifically, he showed that two geometric phenomena in g* have algebraic consequences for U(g) : nilpotent orbits can admit nontrivial covers, and they can have nonnormal closures. The first of these phenomena accounts for the trouble in type Bz, while the second causes the problems in type G2. For more details, see [34]; the bad orbits in question are the 6-dimensional subregular nilpotent orbit in 50(5) and the 8-dimensional nilpotent orbit in Go.

It follows that objects slightly larger than orbits can affect ideal theory in U(g) . To introduce this in the orbit correspondence, one may attach to each orbit not just the ring of regular functions on its closure but also certain extensions of this ring. Thus we are led to attach to each prim-

itive ideal I certain extensions of the primitive quotient U(g)/Z.

More

generally, we are led to the notion of a Dixmier algebra and to Vogan’s reformulation of Conjecture 2.1.

Definition 2.2. (cf. [24]) A Dirmier algebra A over (G and) U(g) is an

400

WILLIAM M. McGOVERN

algebra of finite type over a quotient U of U(g) equipped with an algebraic G-action extending the adjoint action of G on U and realizing A as an admissible G-module. Definition 2.3. A ramified orbit cover X for G is an irreducible affine variety equipped with an algebraic G-action and a G-equivariant finite morphism X — g* whose image is the closure of a single G-orbit.

Conjecture 2.4. [34] There is a natural bijection between completely prime Dixmier algebras and ramified orbit covers. This formulation disposes of the difficulties mentioned above in types Bz and G2. We will see later that there is a partial implementation of the orbit method correspondence using rings of twisted differential operators. Vogan has generalized this implementation using the theory of endomorphism rings of induced modules [37]. If we supplement Vogan’s generalized implementation by attaching the minimal 4-dimensional nilpotent orbit in type Bz to U(g) modulo the Joseph ideal, then we get a bijection between g*/G and completely prime quotients of U(g) , which may be fairly easily extended to a bijection from the set of orbit covers to a certain set of completely prime Dixmier algebras. (This extension uses Vogan’s theory of induced Dixmier algebras in [37]. It does not unfortunately prove Conjecture 2.4 for g = 50(5), because neither ramified orbit covers nor completely prime Dixmier algebras have been classified in this case.) In type Gz, one can attach the closure Og of the 8-dimensional nilpotent

orbit to one of the U(g)/I;, say U(g)/Ii, and the normalization N(Og) of this closure to U(g)/I2. (By an old result of Borho and Kraft [9], the normalization N(Q) of any orbit closure © may be identified with the orbit O.) Unfortunately, Conjecture 2.4 is still false, due to an algebraic phenomenon with geometric consequences: it is possible for distinct Dixmier

algebras to be isomorphic as U(g) -bimodules. Ramified orbit covers are not subtle enough to distinguish two such algebras. The simplest example has g = 5/(2) x sl(2). The second Weyl algebra A», generated by two commuting copies of the first Weyl algebra Aj, is a Dixmier algebra

over U(g). This can be seen by embedding sl(2) into A; in the standard way as operators of total degree 2. Now form the algebra A (= “twisted Ay”), generated by two copies of A, with anticommuting generators. It is easy to see that 4°, like Az, is a completely prime Dixmier algebra. We may attach the universal (fourfold) cover of the principal nilpotent orbit in g* to Ag, but then there is no ramified orbit cover which can

reasonably be attached to Aj. This example first appears in [24]. There we study Dixmier algebras A with the kernel of the map U(g) — A a spe-

cial unipotent primitive ideal (in the sense of [3]). It is shown that under

certain circumstances completely prime Dixmier algebras having a fixed

DIXMIER ALGEBRAS AND THE ORBIT METHOD

401

bimodule structure are parametrized by H?(F,C%), the Schur multiplier

of a suitable finite group F. For the bimodule structure of Az and A}, we

have F = Z2 x Z, so that H?(F,C*) & Zo.

It is fairly clear why A‘ causes problems in the orbit correspondence: it is a sort of super version of A» and so ought to correspond on the geometric side to a super-commutative, rather than a commutative, algebra. More precisely, 4, should correspond to the polynomial ring C[x1, 22, yi, yo] in four variables 21, 22,41,y2 such that the z’s commute with each other and anticommute with the y’s, and the y’s commute with each other. In attempting to repair Conjecture 2.4, Vogan was led by this example and many others to the following definition.

Definition 2.5. [37] An orbit datum (R,Ÿ) for G is a pair satisfying the following conditions.

(a) Ris a C-algebra on which G acts algebraically by automorphisms. (b) Ÿ is a homomorphism from S(g) into the center of R which respects both G-actions. (c) Ris finitely generated as a module over S(g).

(d) R is G-admissible. Note that any orbit cover gives rise to a unique orbit datum via the ring

R(X) of regular functions on X and the comorphism S(g) — R(X). We identify the former with the latter. Vogan then reformulated Conjecture 2.4 as Conjecture 2.6. There is a natural bijection between completely prime Dixmier algebras and completely prime orbit data (orbit data (R,%#) with

R completely prime).

The known examples (including the ones in [24]) confirm this latest version. The most interesting orbit data are (identified with) covers of nilpotent coadjoint orbits and are called unipotent (in [36], they are called unipotent Poisson varieties). The Dixmier algebras attached to them by

Conjecture 2.6 are also called unipotent. In [27] and [28] we call the process of attaching a Dixmier algebra A to a unipotent orbit datum X via Conjecture 2.6 quantization and we say that A quantizes X. Given X, we give criteria that the A quantizing it should satisfy (which should specify A uniquely). The most important of these state that A and R(X), the regular functions on X, should both admit G-stable good filtrations mak-

ing gr A & gr R(X) as Poisson algebras, S(g)-modules, and G-modules. For X corresponding to a trivial or universal cover, we construct an A which conjecturally satisfies these criteria. Using the techniques of [37], we make substantial progress towards proving this conjecture. There is only one algebra g for which the complete list of completely prime Dixmier algebras over U(g) is known, namely s1(2). This list is

402

WILLIAM M. McGOVERN

given in [33], and it satisfies Conjecture 2.6. The precise situation is as follows. Every coadjoint orbit is either semisimple or nilpotent. The nonzero semisimple orbits are normal, closed, and simply connected. They

are attached to quotients U(g)/I with I a minimal primitive ideal (such

quotients attached admits a [19]). O

are automatically completely prime). The 0 orbit is of course to U(g)/gU(g). The only other orbit O is principal nilpotent. It double cover ©, but has normal closure (by a result of Kostant is attached to the Weyl algebra Aj, but © is not attached to

the subalgebra of even operators in A; (the image of U(g) ). This fact can be deduced from the differential operator construction in the next section, as we shall see; we will discuss its implications later. Instead, O is attached to U(g) modulo the minimal primitive of infinitesimal character half the positive root. (The infinitesimal character of A;, on the other hand, is one-fourth of the positive root.) There is also a very interesting family of orbit data lying between O and © in the following sense: the underlying algebras contain R(O) and are commutative and nonnormal, with normalizations all isomorphic to R(Ô). This family is indexed by odd integers > 3; the n-th orbit datum has as its underlying algebra the subalgebra of the polynomial ring C[x, y] generated by the homogeneous polynomials of degrees 2 and n. The corresponding Dixmier algebra may be realized as the ring of differential operators on the singular variety y? = x”; note that for n = 1 this variety is smooth and has A, as its ring of differential operators. There are no other completely prime Dixmier

algebras or orbit data for U(g) . Just as one tries to understand quotients U(g)/I of U(g) by realizing them in a different way, so one wants nice realizations of Dixmier algebras as well. Most of the existing realizations of Dixmier algebras are actually constructions of the latter, since it is rather difficult to give abstract existence proofs for Dixmier algebras. These constructions are of two closely related types. One either looks at the ring of L(M, M) of G-

finite endomorphisms of some one-sided U(g) -module M of finite length (the case where M lies in category O being of particular interest), or one looks at rings of differential operators on a (possibly singular) G-variety. In the former case, Joseph and Stafford [18] have powerful results showing that C(M, M) is to a large extent determined by its ring of fractions Fract C(M, M), which is in turn severely restricted by Goldie’s Theorem and the Gelfand-Kirillov conjecture. If M is simple, then under certain hypotheses we can even assert that C(M,M) © L(L, L) for some simple highest weight module L having the same annihilator as M. Most of the techniques used and results obtained in studying the ring L(M, M) carry over to the module C(M,N). The algebras £(M, M) are particularly important because of an embedding theorem of Joseph and Stafford: any prime Dixmier algebra A with an infinitesimal character embeds in

DIXMIER ALGEBRAS AND THE ORBIT METHOD

403

L£(M, M) for some M in category O (but M need not be simple). Moreover À is a direct summand of L(M, M) as a bimodule [32]. We have already observed that certain Dixmier algebras over U(sI(2)) can be realized as rings of differential operators on a singular variety. Such rings were studied in [22] and [23]; these papers point out a very striking connection between the singularity of certain G-varieties and the

finite generation of their rings of differential operators as U(g) -modules. In [22], it is shown that if g is of type B,, C,, Dn, Es, or Ez, then U(g) modulo the Joseph ideal may be realized as differential operators on an appropriate irreducible component of OM n, where © is the minimal nilpotent adjoint orbit and n is the nilradical of some Borel subalgebra. In [23], differential operators are studied over certain classical rings of invariants studied by Weyl. These are shown to take the form U(g)/J with J a completely prime maximal ideal; some of the ideals J arising in this way are special unipotent in the sense of [3]. Once again the varieties in question (i.e., the spectra of the rings of invariants) are singular (and consist of matrices satisfying a rank condition). In the embedding result A = L(M, M) it may not be possible to choose M € ObO simple. If A is a primitive ring then by definition it admits a simple faithful module N. One can ask if N can be chosen simple over

U(g) . The work of Moeglin ([29], [30]) strongly suggests that one can even choose N to be a generalized Whittaker module. Let J be a primitive ideal with associated variety

O = G-e C g*, O anilpotent orbit. Then we

have the comorphism 7 : S(g) — R(O) of the natural map O — O — g*, with © the universal cover of ©. Suppose that U(g)/I admits a filtration Filt such that x factors through the associated graded algebra Gr U(g)/I and the map Gr U(g)/I — R(O) is injective. Then there is a faithful simple Whittaker module M for U(g)/I and a filtration on M such that the associated graded module gr M has an algebra structure and injects as an algebra into R(N - e) for some unipotent subgroup N of G. The converse of this statement is also true. Moreover, in this situation, any

Dixmier algebra Filt on the copy embeds in L(M, the requirements

A over U(g)/J equipped with a filtration agreeing with of U(g)/I inside A and an embedding Gr A — R(O) M). The hypothesis imposed on A is a weaker version of given above for quantization (from [27] and [28]).

3. Rings of twisted differential operators and related Dixmier algebras The remainder of this paper is devoted to discussing various interesting examples of Dixmier algebras. In this section, we give the general constructions which account for most of these examples (one of them arises in an ad hoc way from a very special fact about G2). We begin by

404

WILLIAM M. McGOVERN

recalling the definition and basic properties of the Beilinson-Bernstein rings of twisted differential operators. We also discuss their role in the

orbit method correspondence. For more details see [4], [7], and [34]. Let

g be reductive with G as above. Let § be a Cartan subalgebra of g. Let P > exp § be a parabolic subgroup of G with commutator subgroup Po;

then T = P/Pp is a torus. Choose At(g,h), a set of positive roots of 5 in g, in such a way that all a € At(g,h) occur in Lie P. Write Po = MoN,

m=

a Levi decomposition,

Lie Mo,

tale

At(m,t)=

7.

induced choice of positive roots of t in m from A*(g,5), 1

PO

te

DE

Qa,

ae At(m,t)

Ve

L9

ys

a= Po + pi,

a€ A+(g,b) Z= GLP,

Y = G/P, wei,7; TRao

Then, using standard identifications, we may £ € t*, attach a Dixmier algebra A; to € as Diff G/ Po of algebraic differential operators action and a commuting right T-action on

actions, we get a map U(g)

@U(t) —

identify t* with u/n. Given follows. Start with the ring on G/Pp. There is a left GG/Po. Differentiating these

Diff G/Po. Set

A = centralizer of T in Diff G/P5;

then we get a map

Now bring € into the picture by letting Ig be the ideal of A generated by

all H + (€ — p1)(H) for H € t. These elements are central in A, since p; is trivial on Lie Po. Finally, we set Ag LT A/Ie

=>

DiffeG/P,

the ring of €-twisted differential operators on G/P [4]. The map ® induces a map ®e : U(g) — A. The basic properties of the algebras Ag are given

by

DIXMIER ALGEBRAS AND THE ORBIT METHOD

405

Theorem 3.1. [34] (a) Ag is finitely generated as a U(g) -module (b) Ag has infinitesimal character £ + po, so that it is indeed a Dixmier algebra. (c) Ag is primitive and completely prime. (d) If € + po is nonnegative (or, more generally, never a negative rational) on any positive root, then ®¢ is a surjection.

Parts (a) and (b) are rather easy. The complete primality of (c) follows from the existence of a filtration on Ag with gr Ag completely prime (we

will see this in a moment). Then the primitivity in (c) follows from primality and finite length of Ag. Part (d) is the key to the localization theory of Beilinson and Bernstein; they study quotients of U(g) and their modules by realizing these quotients as rings of differential operators. We are, however, primarily interested in the opposite situation, where Ag is strictly larger than the image of U(g) inside it. From the point of view of the orbit correspondence, the algebras A¢ are interesting chiefly because to each of them there corresponds a ramified orbit cover X¢, which we now define. Set. n+€={y€u:y

maps to € in t*},

an affine space, and de =Gxp

(n + £)

a homogeneous affine bundle over G/P. (If€ is 0, this is just the cotangent

bundle.) Now set Xe =

affinization of Yg =

Spec R(Y¢),

where R( ) as usual denotes the regular functions. The basic properties of X¢ and Y¢ are given by Proposition 3.2. [34] (a) Ye is equipped with a proper moment

map 7e : Ye —

g” sending

(9,y) tog-y.

(b) The image of z¢ is the closure O ofasingle G-orbit Of in g*. Over Ok, Te is a finite covering, so that X¢ is a ramified orbit cover. (c) Og is nilpotent if and only if €= 0. In this case it is a Richardson orbit. (d) Og is semisimple if and only if (a,€) # 0 for any root of § in n. In this case 7e is an isomorphism onto Og = O¢ and Og is simply connected

Here part (a)-follows from the projectivity and simple connectedness of G/P. The remaining parts are well known and easy to verify. The

fundamental link between Ag and X¢ is given by

WILLIAM M. McGOVERN

406

Theorem 3.3. [34] 4e and R(Ye) admit natural G-stable filtrations making gr Ag © gr R(Y¢) as algebras and G-modules. The filtrations come from the symbol calculus; the isomorphism follows from the pair of isomorphisms

gr Ac = R(Yo) = gr R(Y¢) which in turn follow from a vanishing theorem in sheaf cohomology due

to Elkik (see [34]). Thus we get a partial implementation of the orbit correspondence by attaching X¢ to Ag. It turns out, however, that this implementation has some rather surprising properties. We will discuss these in the next section. There is another construction of Dixmier algebras which can produce algebras that are not completely prime. Let p be any parabolic subalgebra of g and b C p a Borel subalgebra. Choose h C 6 a Cartan subalgebra. For every À € b* such that there is a finite dimensional irreducible representation L(À) of p with highest weight À, form the general-

ized Verma module M,(À) = U(g) Sup) L(A). Then we have the algebra Ap(A) = L(M,(A), Mp(A)) of all G-finite endomorphisms of M,(A). Its basic properties are given by Theorem

3.4.

(a) Ap(A) is prime with infinitesimal character À + p, p the half-sum of the roots in 6. (Thus Ap(A) is a primitive Dixmier algebra.) (b) The multiplicity of any irreducible G-module in A,(À) is bounded below by its multiplicity in the principal series representation

L(M,(À), M,(A)4), d the standard contravariant duality functor _ in category O.

(c) The Gelfand-Kirillov dimension of Ap() is 2 dimc(g/p). (d) The Goldie rank of Ap(A) is dime L(A).

Here parts (a) and (c) follow from [16], part (b) from [10], and part (d) from [13]. It should be noted that parts (a) and (d) can fail for the image of U(g) inside A,(A). The algebras Ap(X) seem at first glance rather different from the algebras Ag; for example, the former algebras do not have obvious ramified orbit covers attached to them.

Nevertheless, it is not difficult to show

that every Ag is isomorphic to some A,(A) with L(A) one-dimensional. This observation was made by a number of people, including Vogan, who has also, as we mentioned above, generalized the twisted differential operator construction to produce a theory of induced Dixmier algebras. He has also shown how to induce orbit data from a parabolic subgroup P to the group G [37]. One expects the correspondence of Conjecture 2.6 to respect induction of Dixmier algebras and of orbit data, but limitations

DIXMIER ALGEBRAS AND THE ORBIT METHOD

407

on current technology of cohomology vanishing theorems make verifying this difficult. 4. Examples

We begin by looking at the simplest example of an algebra Ag. Set G = SL(2), P = B, the Borel subgroup of upper triangular matrices. Put € = 0. It is easy to see that A¢ is isomorphic to U(g) modulo the minimal primitive J of infinitesimal character p, while Y¢ identifies with the principal nilpotent orbit O. Thus O is attached to U(g)/I. We see that the Dixmier algebra quantizing O does not embed in the Wey] algebra A; quantizing its double cover O. Thus we cannot expect the bijection of Conjecture 2.6 to have good functorial properties. We now give some further examples, which will show how this lack of functoriality leads to a rather loose connection between the algebraic map ®¢ of Theorem 3.1 and the geometric map z¢ of Proposition 3.2. Let G be of type Go. Let Pshort be the parabolic subgroup of G with the short simple root in its Levi factor, Piong the other proper non-minimal parabolic subgroup. First set P = Pshort, € = 0. One may calculate that O¢ is Oio, the 10-

dimensional subregular nilpotent orbit in g*, and that 7¢ restricts to an isomorphism over this orbit. Coordinatize the root system and choose the positive root system of g in the standard way, so that the short simple root

is (1,—1,0), the long simple root is (—2,1,1) and p = (—1,—2,3) [12]. Then the infinitesimal character of Ag is 3(1, —1,0) and ¢ is surjective with kernel a maximal ideal. (The surjectivity obtains because the image of Se admits no bimodule other than itself, as is easily seen.) Thus Oo gets attached to Ao. Next set P = Piong, € = 0. Once again, O¢ is Oro, but this time 7e restricts to a triple cover over Og. The infinitesimal

character of Ag is $(2 — 1,—1), and once again we deduce that 9, is a surjection. The algebra quantizing Oj is thus not a subalgebra of the one quantizing the triple cover of O19. We also see that it is possible for ®¢ to be a surjection while 7¢ is not an isomorphism. The converse phenomenon can also occur. Let P = Prong, € = $(0, 1,—1). Now 7e is an isomorphism onto a semisimple orbit but ®¢ is not surjective. To see the last assertion, note first that the infinitesimal character of Ag is (1,—1,0) and that ® has maximal kernel. This kernel is special unipotent, and we may apply the character formulas of [3, Theorem III; see also Section 9] to it. These formulas yield explicit K-type decompositions of the irreducible bimodules over the image of ®¢. Then we may combine Theorem 3.3, the well-known formula for the ring of regular functions on a semisimple orbit as a G-module [19], and these decompositions, to deduce that A¢ splits into two pieces as a bimodule.

It turns out that there is an extension of this last algebra Ag which gets

WILLIAM M. McGOVERN

408

attached to the universal (sixfold) cover Où of O0. To construct it, let

J be the Joseph ideal of U(s0(8)). Put Ajos = U(s0(8))/J.

Theorem 4.1. Take g simple of type Gz. Ayos is a Dixmier algebra over U(g) which admits an action of the symmetric group S3 on three letters by U(g) -bimodule and algebra automorphisms. As a (U(g), S3)-bimodule AJos takes the form

@ (W @ Ex) rESÂ

where E, is the irreducible S3-module corresponding to x (on which U(g)

acts trivially) and V, is an irreducible U(g) -bimodule (on which S3 acts trivially). The map 7 — V, is a bijection from S to the set of irreducible Harish-Chandra U(g) -bimodules having the same annihilator as A Jos.

Proof. There is an action of S3 on $0(8) by automorphisms of the Dynkin diagram; the fixed subalgebra is well known to be isomorphic to g. The

extended action of U(s0(8)) preserves J as it is well known to be the unique completely prime primitive with Gelfand-Kirillov dimension 10 [14]. It is also well known that the K-types of Ajo, as an 50(8)-module are just the Cartan powers of the adjoint representation, each occurring

once. Decomposing these first over 50(7) by a classical branching law and then over G2 by the branching law in [26], we see that Ayo, contains the

trivial K-type only once as a g-module. It follows at once that the kernel K of the map

U(g) —

Ay,, has an infinitesimal character.

Using the

realization of J as the annihilator of a simple highest weight module in

[11], we can calculate that the g-infinitesimal character of A is (1, —1,0). Using the complete primality of A, we highest weight considerations can be character formulas of [3] take over; we to S$ occurs in the decomposition of

see that K is prime and then by shown to be maximal. Now the easily see that every 7 belonging Aj,, over S3, and we know what

all the irreducible U(g)/K-bimodules look like. We can compute that the m-isotypic component of Ajo, takes the form V,; @ E, with V, irreducible,

and all assertions in the theorem drop out. (See also [26]). Thus

Ay,,

has a reductive dual pair decomposition

QED (as do all the

Dixmier algebras arising in [27] and [28]). To see the role of Aj,, in the orbit method correspondence, note first that it is known by [11] that if Ajos is given the filtration induced by the standard one on U(s0(8)), then the associated graded algebra gr Ajo; is isomorphic to R(O{,), the

regular functions on the minimal nilpotent orbit Oj, in $0(8)*. The moment map 50(8)* — g* yields a universal covering map O4, — O19 when restricted to O’5; the fundamental group of O1 is isomorphic to S3 [1]. We therefore conjecture that O{) = Ons should be attached to Ajo, in the orbit correspondence for U(g) , even though Aj,,, when regarded as

DIXMIER ALGEBRAS AND THE ORBIT METHOD

409

a Dixmier algebra over U(g), is not an induced Dixmier algebra (and cannot be realized as L(M, M )for any simple M € ObO). We observe that the algebra Ag with € = $(0,1,—1) mentioned earlier is a subalgebra of Aj,, (but corresponds es a PR

Pat

orbit). Setting P = Prong,

€ = 5(0,—3, 3), we obtain an algebra A¢ isomorphic to the image of U(g) in Ajo, (which corresponds to yet another semisimple orbit). It turns out that there is another proper subalgebra of Ay,, which is not realizable as a ring of twisted differential operators; it has length two as a U(g)-bimodule and presumably corresponds to the double cover of O19 [26]. We conclude this section by constructing the mock unipotent Dixmier algebras mentioned in the introduction, together with certain auxiliary unipotent Dixmier algebras which we will need in the next section. First let G be of type C2. Define Prong andPshort as for Gz and denote their Lie algebras by Piong and Pshort- Coordinatize the root system and choose

the positive root system in the usual way, so that the short simple root is (1,—1), the long simple root is (0,2), and p = (2,1). Set Amc =

a Sern (had ey 4Pee Pyros

Abjong(—2; 0). Here m,u,C stand for “mock,”

“unipotent,” and “type C2,” respectively. Amc is a mock unipotent Dixmier algebra; we see from Theorem 3.4 that it has infinitesimal char-

acter (1,1), Gelfand-Kirillov dimension 6, and Goldie rank 2. It is also easy to see that Ker(U(g)) — Amc) is maximal. Ayc is a unipotent Dixmier algebra; it is a ring of twisted differential operators and corresponds to the double cover Og of the 6-dimensional nilpotent orbit of O6 Mi 0. Next let G be of type Gz and define P,hort, Piong as in the last para-

graph... Set. AniGge= An ois 1,—4), Anag = Apat(calis ae), Au,G = Ajsos. Both m’s stand for “mock,” u stands for “unipotent,” and G stands for “type G2.” The kernel of both maps U(g) — Am1i,a, U(g) — Am2,¢ is maximal of infinitesimal character (—2,1,1). The dimension of Am1,g, Am2,6, and Aug is 10, while their respective Goldie ranks are 3, 2, and 1. We saw above that A, gq is a unipotent Dixmier

algebra. Finally let G be of type F4. Once again, coordinatize and choose a positive root system in the standard way simple roots are (0,1, —1,0), (0,0,1,—1), (0,0,0,1), and Let p, be the parabolic subalgebra with Levi factor p2 the maximal parabolic with Levi factor containing

the root system [12], so that the $(1, —1,-1,-1). of type B2 and the long simple

roots and of type A2 x A1. Put Amr = Ap, $(- 11,—3,—5,—5), Aur = Ap,(—5, —2, 0; 0). Then the kernels of U(s) — Amr, U(g) > 1s are maximal of infinitesimal character (2,1, 1,0), $(3, 1,1,1,), respectively. Am,r

and Ay,r have dimension

40 and Goldie ranks 6,1, respectively.

Au,F is a ring of twisted differential operators corresponding to a sixfold

410

WILLIAM M. McGOVERN

cover of O40, the 40-dimensional nilpotent orbit in g*. This orbit has fundamental group S4, the symmetric group on four letters [1].

5. Mock unipotent Dixmier algebras We conclude this paper by studying the algebras Amc, Am1,G, Am2,G; and A,r in more detail and giving a precise account of their relationship to the unipotent algebras Ay c, Au,g, and Ayr. For this we need to recall

some of the constructions of [3]. The central notion of that paper is that of a special unipotent representation, defined to be an irreducible Harish-

Chandra bimodule whose (left and right) annihilator is maximal with infinitesimal character belonging to a certain specified list. This list is given as follows. Given a semisimple g, let L g denote its dual Lie algebra. Let LO be an even nilpotent orbit in “g with representative e. By the Jacobson-Morozov theorem, e embeds in a standard triple {h,e, f} C” g

satisfying the usual bracket relations in s1(2)([he] = 2e, etc.). Then h belongs to some Cartan subalgebra of g, which may be naturally identified with a Cartan subalgebra dual of g. Set Ao = sh; we may regard Ao as an infinitesimal character of g. As an infinitesimal character, Ao is

well defined thanks to the conjugacy of all standard triples containing e. The evenness of /O implies, by definition, that Ao is integral. Then the list of infinitesimal characters in the definition of special unipotent representation consists precisely of the \o for each even £O. Given such a Aw, Barbasch and Vogan give character formulas for the special unipotent representations of infinitesimal character Ao. They prove these formulas by an elaborate induction argument on “O (and its “dual orbit” O C g*, which they define). The orbits “© serving as base cases in the induction are called triangular in [2]. The character formulas for the corresponding special unipotent representations are proved by realizing these representations as induced from finite-dimensional representations on a parabolic. This realization is in turn obtained from the following combinatorial lemma (see the proof of Proposition 9.11 in [3]).

Lemma

5.1. ([3], Lemma

9.7) Let \o be the infinitesimal character

attached to a triangular orbit LO. Choose a 7 be the unique dominant weight conjugate of simple roots on which y is nonzero. Then conjugates of y that are strictly positive on way that

Ga")

set of positive roots, and let to Am. Let S denote the set we may list the Weyl group S as (y},... ,7/) in such a

2(7',a)/(a,a) =1 for aE S,

(b) no two distinct y‘ are conjugate by a Weyl group element fixing

1

T5

(c) y' — 7" is a sum ofpositive roots for k < j and is strictly shorter than y! — y*+} fork -fixed vectors on o;. Recall that primitive ideals of infinitesimal character À are parametrized by the basis of nr T; obtained via Joseph’s Goldie rank polynomials [13, Chapter 14]. Given one such primitive ideal J,, we may obtain J, from a primitive ideal I): of some regular integral infinitesimal character \’ by the Jantzen-Zuckerman translation principle. Then there is a unique left cell C of W corresponding to I, (by definition). As usual, regard C as a W-module and list the irreducible submodules of it according to multiplicity as 21,... ,%. Let v{ denote the subspace of W>-fixed vectors in v;. Then the set of irreducible Harish-Chandra bimodules with left and right annihilator J)

is parametrized by a basis of )~;_, v{. (This may be deduced from [13, Kapital 14], or from Joseph’s work, but to the best of my knowledge it is stated explicitly only in an unpublished preprint of Barbasch and Vogan.) Now let LO be an even nilpotent orbit; then it is automatically special. Recall that the Weyl group of “g identifies with that of g, so that left cells

for g and “g coincide. Barbasch and Vogan show in [3, Proposition 5.28] that Lusztig’s cell Cz,, is the relevant cell to use in counting irreducible Harish-Chandra bimodules of infinitesimal character Ào and maximal annihilator. Moreover, they show that each W-representation occurring in Cie has a unique W0-fixed vector. Hence, there is a bijection between

A(£Q)* or A(Q)4 and special unipotent representations of infinitesimal character Ag. Denote it by r + V,. The trivial representation of A(O) corresponds to the spherical special unipotent representation, a quotient

of U(g) ([3], Corollary 5.30). It is natural to ask for the relationship between the left cells Co, Cio

attached to Barbasch-Vogan dual orbits 0,“0. As usual, regard these left cells as W-modules. From Lusztig’s algorithms for computing Co in the classical cases and his tables giving Co in the exceptional cases [21, Chapter 4], one can compute that CLo is just Co tensored with the sign representation, with three exceptions. These exceptions occur exactly for the orbits

O = O6, O10, O40 of the last section. Each of these orbits is self-

dual, and yet the corresponding left cell is not stable under tensoring with sign. Thus we obtain three new left cells Co; = Co; ®sgn for O; = O6, Or0,

and O49. Each of the infinitesimal characters Ag, M5, and X{, of the mock

unipotent Dixmier algebras Amc, Am1,c, and Am,F differs from that of

the corresponding unipotent Dixmier algebra, as we have seen. One may

DIXMIER ALGEBRAS AND THE ORBIT METHOD

413

verify directly for each À! that the relevant left cell to use in counting irreducible Harish-Chandra bimodules with infinitesimal character À; and maximal annihilators is Co; Moreover, each W- representation occurring

in Co, has a unique Wi-fixed vector. Thus the bijection between A(O;)* and the set of special unipotent representations attached to O;(= 6, 10, 40) induces one between (A(Q;)* and what we may call the set of mock unipotent representations attached to O;. Denote it by r + W,. Each A(Q;), by the way, is isomorphic to 7,(0;), which is in turn isomorphic to So, S3, Sq for i = 6,10, 40 [21]. Now it can be seen why we restricted attention to Og, O0, and O4o in

defining mock unipotent Dixmier algebras. The infinitesimal characters

Ag,

Aigo, Ago Were obtained from the Barbasch-Vogan infinitesimal char-

acters Ag, A190, À40 as follows.

For each self-dual semisimple Lie algebra

g, there is an involution z on the set of simple roots of g induced by the isomorphism g € “g. Thus z interchanges short and long roots. It induces

an involution 7 on b* via the requirement (7(H),a¥) = (H,(ta)”) for a and À € b* (but the map 7 does not extend 2). For i = 6,10,40, we have À! = (À). (We make 7 act on infinitesimal characters by restricting it to the dominant Wey] chamber.) We conclude this section by giving the bimodule decompositions of the unipotent and mock unipotent Dixmier algebras of the preceding section. Denote the representations of S2 by {1, sgn}; those of S3 by {1, sgn, re fl} with refl of dimension two; and those of S4 by {1,sgn,two, refl,refl @ sgn}, where two is two-dimensional and refl is realized on a Cartan sub-

algebra dual of s[(4) by the action of its Weyl group. We have seen that to each of these representations r there corresponds a special! unipotent representation V, and a mock unipotent representation W,. One can verify in the case of S3 that this notation V, is consistent with that of Theorem

4.1.

If we combine Theorems

3.3 and 4.1 with the character formulas

of [3], we can compute that the respective bimodule decompositions of Au,c, Aug, and Aypr are " B a functor. Suppose that

(a) The functor F has a left adjoint functor E : B —> A and a right adjoint functor

G: B —

A.

(b) We have fixed a morphism of functors v : G —

E.

Then for all objects X,Y € A we define morphism

tr : Homp(F(X), F(Y)) — Homa(X,Y) by tr(a) = iy o E(a) ovp(x) 0 jx : X — GF(X) — EF(X) — EF(Y) — Y,

where jx : X — GF(X) and iy : EF(Y) — Y are adjunction morphisms, vp(x) : GF(X) — EF(X) v-morphism, corresponding to the oba: F(X) — F(Y) any morphism in B and E(a) : EF(X) > ject F(X), EF(Y) the corresponding morphisms in À.

420

JOSEPH BERNSTEIN

In particular, for any object X € A we get a morphism tr : End(F(X)) > End(X). It is easy to see that this morphism has natural functorial properties. In particular, it defines a morphism

tr : End(F) —

End(Jda).

The case discussed in section 2 corresponds to

A = B = Mg),F =

Py, f= G = fy

4. An application. Description of the endomorphism algebra of the big projective module Fix a maximal nilpotent subalgebra n normalized by § and denote by O the corresponding category of highest weight modules. For every weight À € L = b* we denote by M, the corresponding Verma module of highest weight À — p, by Ly its irreducible quotient, and by P, the projective

cover of L in the category O (see[BGG1)). Fix a regular integral antidominant weight A. Then M) is irreducible and P; is what we call the “big” projective module. We want to describe

the algebra End(P;) of its endomorphisms in category O.

Theorem. phism.

The natural morphism n : Z(g) —

End(P;) is an epimor-

Its kernel coincides with the ideal J), described below.

In par-

ticular, the algebra End(P;) is isomorphic to Z(g)/J\ which in turn is isomorphic to the cohomology algebra of the flag variety X of algebra g.

Let us describe the ideal J\. We identify Z(g) with the algebra F(L)” and consider a linear functional v = v, : F(L)Ÿ — k, given by

v(f) = [D> T(wd)(AF))/AN(0) = [D uw) - w(TA)(AF))/A](0). Here T(y) is a translation operator on F(L), T(u)(h)(x) = h(x + p), e(w) is the sign of element w € W. In order to see that these two expressions coincide, we use that w(f) = f, w(A) = e(w)A and T(wd) = wT (A)w-?. Remark. In order to compute v(f) we use the fact that f is a polynomial, i.e. we computed v(f) using some kind of limit. In terms of the functional v the ideal J) is described as

Jn = {f € F(L)™ |(f - F(L)”) = 0}. In order to prove the theorem it is enough to check the following three lemmas.

TRACE IN CATEGORIES

421

Lemma

1. dim(End(P;)) < #(W).

Lemma

2. Set J = Annz(,)(Pa), i.e. J is the kernel of morphism 7 :

F(L)Ÿ —

End(P;). Then v(f) = 0 for allf € J. In particular, J C Jy.

Lemma 3. The algebra F(L)/J\ has dimension equal to #(W) and is isomorphic to the cohomology algebra of the flag variety X. Remark. Lemmas 1 and 3 are more or less straightforward exercises on category O and cohomology algebra of flag variety respectively. From the point of view of this note the key statement is lemma 2. Proofs. 1. Let V be an irreducible finite dimensional g-module with lowest weight À, 9 the corresponding character of Z(g). Consider Verma module Mo, corresponding to weight 0. Then P) is the direct summand of

F\(Mo), corresponding to the character 6 of Z(g). This implies that P, has a composition series whose factors are isomorphic to Verma modules

M, with y of the form wA for w € W; moreover [P; : Mwa] < [Fv (Mo) : My] = 1. Thus dim Hom(P,, Py) < >> dim Hom(P,, w

Myx) = [Mu D

:

w

Ly) < #(W). 2. Let P(V), be the set of extremal weights in P(V). Clearly P(V), = {wr|w € W}. Choose a W-invariant polynomial p on L with the following properties: a) The polynomial p vanishes up to order

> #(W) at all non extremal

points of P(V). b) The polynomial 1

—p vanishes up to order > #(W) at all extremal

points of P(V). Clearly, the corresponding element z(p) € Z(g), acting on the module Fy (Mo), gives a projection onto the submodule P,. This shows, that a

function f € F(L)™ lies in the ideal J = Ann(P;) iff z(f) - z(p) = 0 on Fy (Mo). In this case clearly try(z(f) - z(p)) = 0 on the module Mo. We claim that the action of the operator try (z(f)-z(p)) on Mo is given by multiplication by v(f), which implies that v(f) = 0 for f € J.

Using formula (+) from section 2 we see that the operator try (z(f)-z(p)) acts on Mo as a scalar [5 (Afp)(x + »)/A(x)](0). m

Using properties of p we can rewrite this sum as [S-(Af)(x+u)/A(x)](0), mn

where the sum is over extremal weights y. Since Af is skew-symmetric under the action of W, and extremal weights are of the form wA, this sum equals to v(f). 3. Given a commutative

k-algebra B and a linear map v : B —

k

we denote by J(B,v) the ideal J(B,v) = {b € B|v(bB) = 0} and by

422

JOSEPH BERNSTEIN

Q(B, v) the quotient algebra B/J(B,v). By definition J, = J(F(L)” ,v). Our aim is to compute the algebra Q = Q(F(L)” , v). Set A = Fale aes Clearly v vanishes on some power of ideal Jg C A corresponding to the character 6. Hence the algebra Q will not be changed if we replace A by its completion À at 86. Since À is regular point of L, the algebra A is naturally isomorphic to

the completion Fy of F(L) at point A. Translation operator T(A), T(A)f(x)= f(x + À), identifies Fy with the algebra Fo-completion of F(L) at 0. Let us identify A with Fo using

T(A). Then the functional v on A corresponds to the following functional v' on Fo

v'(f) = u(Tx*(F)) = [DC e(w)w(TO)A - f))/A(0). In other words, if we define a linear map 7 : Fy — k by

r(h) = [Alt(h)/A](0)) = [D e(w) - w(h))/A](0) then v/(f) = 7(T)(A) - f). Since the function T,(A) is invertible in Fo, we have Q(A,v) = Q(A,v) = O( Fo) = Ob

=O (DL);1);

In order to describe this last algebra let us consider an ideal J} in F(Z), generated by W-invariant polynomials of positive degree, and denote by

H the quotient algebra F(L)/J}. It is easy to see that r(J;) = 0, ie. 7 can be considered as a functional on H, and Q(F(L),7) = Q(H,7r). By well known result of A. Borel (see [BGG2] or [D]) H is isomorphic to cohomology algebra of flag variety X and functional 7 on H is given by evaluation on fundamental class of X. This implies, that the bilinear form < h,f >= r(hf) on H is non degenerate and hence Q(H,r) = H (direct algebraic proof of the fact that this form is non degenerate see in

[D], Prop. 4). This proves lemma 3. Remark. Slightly modifying above arguments one can prove the following more general result Theorem. Let À € L be any antidominant weight, L) an irreducible module with highest weight À — p and P) its projective cover in cate-

gory O. Then the natural morphism n : Z(g) — End(P,) is an epimorphism. Its image is isomorphic to F(L)“)/J(W(A/R)), where W(A)= {w € WlwA= A}, W(A/R)= {w € W|wA-X € Root lattice R}, F(L)W® is the algebra of W(A)-invariant polynomial functions on L and J(W(A/R)) is an ideal, generated by W(A/R)-invariant polynomials of positive degree. This finite-dimensional algebra can be realized as cohomology algebra of some partial flag variety.

TRACE IN CATEGORIES

423

REFERENCES

(BGG1] [BGG2]

J.N.Bernstein,

I.M. Gelfand, S.I. Gelfand,

Category of g-modules,

Funct. Anal. Appl 10 (1976), 87-92.

D]

J.N.Bernstein, I.M. Gelfand, S.I. Gelfand, Schubert cells and cohomology of the spaces G/P, Russian Math. Surveys 28 3 (1973), 1-26. M. Demazure, /nvariants symetriques des groupes de Weyl et torsion,

[S]

Invent. Math. 21 (1973), 287-301. W. Soergel, Kategorie O, perverse

Garben

und Moduln

uber den

Koinvarianten zur Weylgruppe, preprint. Received April 30, 1990

Department of Mathematics Harvard University Cambridge, MA 02138

i

% ALT,

a din te

:

L

AW

peed cH

Lis

dete:

7G ty) 06 1000 et SAR =

A doi, a, and ; LA: « we aa 116) +4; 184 -

.

s

Ore

TOC

A) =,

A oe VENT

' ate Lan clins) wen {tua

pee

te

ei à ike

ie’

LL

wd

haute ALL ET 2 Tonà Cale

no

‘

Ê

iat

his

Un

al)

prave

À;

"4 de a)

| ‘a

bre f

|

=

4

inal,

Gq au.

en

thle

fective meee te cate = nd FIN Ga efitbort WA)

lu wi

CA

aii

LION

whhed LUE

& À à Root D resi OLA func

iver

4 he

ter où A

anys prlyndaale

CD

lige alrates

”

=

Filtered

Auslander—Gorenstein

Rings

J.-E. BJORK AND E.K. EKSTROM Dedicated to Jacques Dixmier on his 65th birthday

Introduction

| In this paper we construct a class Gy of good filtrations on a pure _ module M over a filtered noetherian ring whose associated Rees ring is _ Auslander-Gorenstein such that the graded modules grg(M) are pure for every ( in the class Gy. We refer to Gy as the class of Gabber filtrations.

Our work has its origin in lectures by O. Gabber at the Université Paris VI in 1981 where he used homological methods to prove in particular that the associated variety of U(g)/p is equi-dimensional when p is a primitive ideal of an enveloping algebra. The main idea in Gabber’s work was to extend results about pure modules over regular commutative noetherian rings to filtered non-commutative noetherian rings. For non-commutative _ noetherian rings it turns out that finiteness of the global injective dimension is not sufficient for a nice duality theory. Therefore certain conditions were introduced by M. Auslander in the late sixties. These conditions will play an important role and are recalled in Section 1 where we also define Auslander-Gorenstein rings. In Section 2 we define and study pure _ modules over an Auslander-Gorenstein ring. Using a construction due to R. Fossum we find an affirmative answer to a question raised in [Bj, p. 144]. To be precise, the purity of certain Ext-modules in Proposition 2.11 was only known for Auslander-regular rings in [Bj]. So called tame pure extensions are studied in §3 and will later on be used in §4. In order to make this paper reasonably self-contained we have given a fairly detailed presentation of filtered noetherian rings in §4. The systematic use of Rees modules should be noticed. The material about the €-functor in §4 has been inspired from lectures by V. Oystaeyen at a conference in Antwerpen in June 1988. At the end of §4 we give a recent example due to G.M. Bergman which shows that it is essential to use Rees

426

J.-E. BJORK AND E.K. EKSTROM

modules in order to define good filtrations over non-positively filtered noetherian rings. The main results of this paper occur in §5 where Theorem 5.23 gives the conclusive result about Gabber filtrations on a pure module. 1. Auslander’s

Condition

By a noetherian ring we mean a (not necessarily commutative) ring with a multiplicative unit which is both left and right noetherian. Let A be a noetherian ring. The category of finitely generated left, respectively right

A-modules is denoted by Mod,;(A), resp. Mod;(A°).

If M € Mod;(A),

then right multiplication in the ring A gives a right A-module structure on Ext%,(M, A) for every v > 0. Moreover, using projective resolutions we

see that Ext (M, A) belongs to Mod;(A°). Similarly, if N € Mod;(A°), then Ext4(N,A) € Mod;(A).

Keeping the ring A fixed we simplify the

notations for Ext-groups having A as a second factor and write E°(M) = Ext (M, A). More generally, if v and k is a pair of integers then E”-*(M)

denotes Ext’, (Ext*,(M, A), A). 1.1 Definition. A finitely generated left or right A-module M satisfies Auslander’s condition if the following hold: For every v > 0 and every

submodule N of E*(M), it follows that E*(N) = 0 for every i < v.

1.2 Remark. The condition in Definition 1.1 occurs in [F-G-R] and [A-B]. These works also contain various equivalent definitions based upon flatness and syzygies. We shall not discuss this, but only rely on Definition jh

1.3 Injective dimensions.

A noetherian ring A has a finite injective

dimension if there exists an integer w such that E’(M) = 0 for every left or right A-module M and v > w. We refer to [Z] for the proof that the injective left dimension of A is equal to the injective right dimension when both are finite. Suppose now that A is a noetherian ring with a finite injective dimension. If M is a finitely generated left or right A-module we construct

the derived dual RHom,(M, A) whose cohomology groups are E’(M). We have the equality M = RHom4(RHom4(M, A), A), called the biduality formula. We refer to [Le 2] for the proof, based upon the use of the Ischebeck compler. The biduality formula implies that if M isa non-zero finitely generated A-module, then there exists at least one integer v > 0 such that

E"(M) # 0. This gives

FILTERED AUSLANDER-GORENSTEIN

RINGS

427

1.4 Definition. Let M be a non-zero and finitely generated A-module.

The unique smallest integer k such that E*(M) # 0 is denoted by j4(M) and called the grade number of M. 1.5. Definition. A noetherian ring A with a finite injective dimension is called an Auslander—Gorenstein ring if every finitely generated A-module satisfies Auslander’s condition. 1.6 Auslander regular rings. A noetherian ring A with a finite global

homological dimension is called an Auslander regular ring if Auslander’s condition holds for every finitely generated A-module. Since finiteness of gl.dim(A) implies that A has a finite injective dimension, it follows that Auslander regular rings is a subfamily of Auslander-Gorenstein rings. So all the subsequent results which are announced for Auslander-Gorenstein rings also hold for Auslander regular rings. 1.7 Remark. In [Re: Example 2.4.6] occurs an example of a noetherian ring A with gl.dim(A) = 2 but Auslander’s condition does not hold for every finitely generated A-module.

So A is not Auslander regular. See also [Bj:

p. 138] for Reiten’s example. 1.8 The case when A is commutative. Let A be a commutative noetherian ring with a finite injective dimension. Then the work by H. Bass in

[Ba] shows that Auslander’s condition holds and hence A is an AuslanderGorenstein ring. 2. Pure

Modules

Throughout this section A is an Auslander-Gorenstein ring. We shall perform a construction due to R. Fossum in [Fo]. Let M be a non-zero finitely generated A-module and put n = j4(M). We stop a projective resolution of M after n + 1 steps. Thus, we consider a complex

(2.1)

Cee

ee Py

M D

where P5,..., Pn+1 are finitely generated projective A-modules. The map Pn+1 — Pn is in general not injective. Now we consider the complex Pe defined by (2.2)

Hom

2.3 Definition.

(Po,

À) SO

Homa(Pn4i,

A).

We set H"+!(P’) = §(M) and refer to §(M) as a

Fossum module associated with M.

J.-E. BJORK AND E.K. EKSTROM

428

2.4 Remark. Let P, and Q, be two projective resolutions of M which both stop after n + 1 steps. By a well known result in homological algebra it follows that the projective equivalence classes of H’(P*) and H°(Q*) are the same for every integer v. In particular this holds if v = n+ 1. We conclude that the projective equivalence class of a Fossum module associated with M is unique. So

E’(§(M)) depend only upon M if v > 1. 2.5 Proposition. Let M be a non-zero finitely generated A-module. Then there exists an exact sequence

0— E"**($(M)) — M — E™"(M) — E"**($(M)) — 0 and ifv>n, then E"(M) = E**?(§(M)). Proof. Consider the derived Hom-complex RHom,(P*, A), where P, is a projective resolution of M from 2.1. We notice that H°(RHom,(P*,

A)) = M while H~"+)(RHom,(P*, A)) = Ker(Pa41 — PA). Next, we have a spectral sequence with

EB," = E?(H7*(P")) which abuts to RHom,(P*, A). Moreover, the definition of the grade num-

ber j4(M) shows that H*(P*) = 0 fori and H"t!(Pp*)

=

$(M).

gives the exact sequence

4 n, n+ 1; H"(P*) = E"(M)

It follows that the spectral sequence in Proposition

2.5 when

we identify M

above with

H°(RHoma(P", A)). 2.6 Remark. The exact sequence in Proposition 2.5 is due to R. Fossum

in [Fo: Proposition 6]. 2.7 Definition. A finitely generated A-module M is pure if j4(M’) = ja(M) for every non-zero submodule M’. In Theorem 2.12 we establish equivalent conditions in order that an A-module is pure. But first we need some preliminary results.

2.8 Lemma. Let M be a non-zero finitely generated A-module. Then, ifn = ja(M), it follows that E"(M) is non-zero and its grade number is equal to n. |

Proof. Auslander’s condition gives j4(E"t!(¥(M)) > n+1. Then Proposition 2.5 shows that E™"(M) # 0. It follows that E"(M) # 0 and

FILTERED AUSLANDER-GORENSTEIN

RINGS

429

ja(E"(M)) < n. Finally, Auslander’s condition also gives j4(E"(M)) > n and Lemma 2.8 is proved. 2.9 Lemma. Let 0 — M — M' — M" — 0 be an exact sequence of finitely generated A-modules. Then

inf{ja(M), ja(M")} = ja( M”). Proof. We have the long exact sequence E*(M")

Li E*(M')

ne E’(M)

=

EM").

Now Lemma 2.8 and Auslander’s condition easily give Lemma 2.9. 2.10 Lemma.

Let M

be a finitely generated A-module

JA(E"(M)) > v for every v > j4(M).

such that

Then M is pure.

Proof. Let 0 # M' C M and suppose that j4(M') = v > ja(M). The long exact sequence from Lemma 2.9 applied to 0 — M' — M —

M/M‘ — 0 gives j4A(E"(M")) > v. This contradicts Lemma 2.8 and hence M is pure.

2.11 Proposition.

Let M

be a finitely generated A-module.

Then,

with n = j4(M) it follows that E”(M) is pure. Proof. Set M’ = E"(M). Lemma 2.8 gives j4(M’)

= n. If v > n we

have E(M') = E°+?($(M)) by Proposition 2.5. This gives j4(E”(M’)) > v +2 > v and hence M’ is pure by Lemma 2.10. 2.12 Theorem. Let M be a finitely generated A-module. n = jA(M) the following are equivalent:

Then, with

(1): M is pure, (2): E"T(3(M)) = 0, (3): E°"(M) = 0 for every 2

Pe

Proof. Lemma 2.10 gives (3) => (1). Next, if (1) holds then the exact sequence in Proposition 2.5 shows that E"+1(3(M)) is a submodule of the pure A-module M. Auslander’s condition gives j4(E"+'($(M)) > n +1

and hence the purity of M implies that E"+1(3(M)) = 0. This proves that (1) > (2). There remains to show (2) = (3). Put M' = E"(M) and then (2) and Proposition 2.5 give the exact sequence 0 — M — E”(M’) —

E"+2(3(M)) — 0. Moreover, v > n gives E"(M') = E"+?($(M)) and hence j4(E’(M’)) > v +2. Then, using the long exact sequence from the

J.-E. BJORK AND E.K. EKSTROM

430

proof of Lemma 2.9 we get j4(E"(M)) > v for every v > n. This gives E**(M) = 0 when v > n and hence (2) => (3). Let A be a commutative 2.13 The case when A is commutative. Auslander-Gorenstein ring. If M is a finitely generated A-module then

the radical of the annihilating ideal (0 : M) = {a € A:aM = 0} is denoted by J(M) and called the characteristic ideal of M. The radical ideal J(M) has a unique prime decomposition, i.e. we have J(M) = piN---M ps, where P1,...,p, are the minimal prime divisors of J(M). Next, for every prime

ideal p of the ring A we denote by Ay the localization of A with respect to p. Recall that À, is a local ring whose maximal ideal is pAp. 2.14 Theorem. Let À be a commutative Auslander-Gorenstein Then, for every finitely generated A-module M we have

ring.

ja(M) = inf{inj. dim(4;) with inf taken over the family of minimal prime divisors of J(M).

Proof.

The work by H. Bass in [Ba] gives the following for every

prime ideal p of the ring A:

depth(A,) = inj. dim(A,) = ht(p). Moreover, the study of grade numbers in [Ba] gives the equality jA(M)= Garonne inf ja(M)

à

Then we easily get Theorem 2.14. 2.15 Remark. Let p be a prime ideal in the ring À and consider the cyclic A-module M = A/p. Then J(M) = p and Theorem 2.14 gives ja(M) = inj.dim(Ap). Using this formula and the well known fact that every non-zero submodule N of M contains a submodule which is isomor-

phic with A/p, it follows that j4(N) = j4(M) and hence the A-module M is pure. More generally, if g is a primary ideal and M = A/g, then J(M) is the prime radical ,/g. By a similar reasoning as above we find that M is a pure A-module. Next, using Theorem 2.14 one easily obtains the following: . 246 Proposition. Let À be a commutative Auslander-Gorenstein ring. Then, if M is a pure A-module, it follows that inj.dim(A;) = ja(M) for every minimal prime divisor of J(M).

FILTERED AUSLANDER-GORENSTEIN 2.17 Remark. radical ideal J(M)

RINGS

431

Proposition 2.16 may be expressed by saying that the if a pure A-module M is equi-dimensional in the sense

that inj.dim(A,) are equal for all minimal prime divisors p of J(M). 2.18 A final remark about purity. Using Theorem 2.14 we find a necessary and sufficient condition in order that a cyclic A-module A/I is pure. Namely, let J be an ideal of the commutative Auslander-Gorenstein ring. Then A/I is a pure A-module if and only if J has a primary decomposition without embedded primary components and VJ is equi-dimensional. In other words, A/I is pure if and only if J = g;N--:Ng,, where V8:,--., V8, are the minimal prime divisors of the radical ideal VI and

moreover, inj.dim(A y.) = j4(A/T) holds for every 1 ja(M) + 2. Remark.

Let (M',a) be a tame pure extension.

Lemma 2.9 gives

ja(M") = ja(M). Example.

$M.

Let M be a pure A-module and construct a Fossum module

Set n = ja(M}).

Theorem 2.12 gives E”+1(¥M) = 0 and then

Proposition 2.5 gives the exact sequence

0— M — E""(M) — E"t?(§M) — 0. Lemma 2.8 and Proposition 2.11 imply that E""(M) is pure.

Next,

Auslander’s condition gives j4(E"+?(§M)) > n+ 2. Hence the injective map M — E”"(M) gives a tame pure extension of M. In Theorem 3.6 we are going to show that if M is a pure A-module with n = j(M), then E""(M) is a marimal tame pure extension of M. But first we need some preliminary results.

3.3 Definition.

A finitely generated

A-module

M

which satisfies

j(E"(M)) > v + 2 for every v > j(M) is called a strongly pure A-module. Remark.

A strongly pure module is pure by Lemma 2.10.

432

J.-E. BJORK AND E.K. EKSTROM Let M be a strongly pure A-module.

3.4 Proposition.

Then, for any

tame pure extension (M',a) it follows that a(M) = M' Proof. We have the exact sequence

0 —

M = M’ —

M'/M —0.

Suppose that M'/M is non-zero and put v = j(M'/M).

The long exact

sequence contains

(i)

EM) — BM" /M) — EM) —

Since M is strongly pure and v—1 > j(M)+2-—1> j(M), it follows that j(E"’-1(M)) > v+1. Also, since M’ is pure we have j(E"(M')) > v. Now

(i) and Lemma 2.8 imply that 3(E°(M'/M)) > v. But this contradicts Lemma 2.9 and we conclude that M’/M is zero and hence a is surjective. 3.5 Proposition. Let M Ei(M)(M) is strongly pure.

be a finitely generated A-module.

Then

Proof. Put n = j(M). If v > n we have E*(E"(M)) = E*+?($M) by Proposition 2.6. This gives j(E°(E"(M)) > v+2 and hence E"(M) is strongly pure.

3.6 Theorem.

Let M be a pure A-module with n = j(M).

Then,

if (M',a) is a tame pure extension there exists an injective A-linear map a’:M' —+ E™” such that the diagram below commutes

nee E™"( (M)

Before we prove Theorem 3.6 we insert some remarks about Fossum

modules. Consider an injective A-linear map a: M —>+ M' where j(M) =

j(M") = n.

We can construct a commutative diagram Pasi

—

ce

—

PR

—

M

—

0

Qnt1

—

+--+ —

Qo

—

M!

—

0

where P,, respectively Q, are projective resolutions as in 2.1.

Next, the

map a: M — M' induces a map 8: E™"(M) — E™"(M’) using functoriality. See A.7 Corollary in [L-S] for details. Moreover, since E”+1($(M)) =

FILTERED AUSLANDER-GORENSTEIN

RINGS

433

E"+1($(M’)) = 0 by (2) in Theorem 2.12, it follows from the remarks made above and Proposition 2.5 that we obtain a commutative diagram

0 —

M — a |

D

M!

3.7 Lemma.

A

À

E""(M) Bl

—

£&"+2(3(M))

—

E""(M")

FD

E"*?($(M'))

aaa)

0

The map B is an isomorphism.

Proof. Identify M with a submodule of E™%"(M). Then, the injectivity of a implies that M N Ker(8) = 0. So the A-module Ker(B) is a submodule of E””"(M) and is also isomorphic with a submodule of E™"(M)/M. Since j4(E™"(M)/M) > n+ 2 we get j4(Ker(B)) > n +2. Then the purity of E”'"(M) shows that Ker(B) = 0 so B is injective. To prove that @ is surjective we notice that Im(f) contains Im(a). Since ja(E™"(M"')/M"') and j4(M'/a(M)) both are > n + 2, it follows from Lemma 2.9 that j4(E”"(M’)/Im(@)) > n+ 2. Finally, apply Proposition 3.5 to the A-module E”(M’). It follows that E””"(M’) is strongly pure and then Im(B) = E""{(M') by Proposition 3.4. Proof of Theorem 3.6. Denote by y the injective map from M’ into

E”"(M’). 3.6.

Then we can use the composed map 8!

as a’ in Theorem

Remark. Theorem 3.6 is proved when the ring A is Auslander regular in [L-S] and of course the proof above was inspired by the material in the

appendix of [L-S]. 3.8 Corollary. Let M be a non-zero and finitely generated A-module with n = j4(M). Then the following are equivalent:

(1): M is strongly pure; (2): M = E""(M);

(3): E"t(3(M))

=

E£"+1($(M)) = 0. 4. Filtered Rings Let A be a ring with a unit element 14. The ring A is said to be filtered if we have an increasing sequence of additive subgroups --- Ay_-1 C Ay C Ay41 C -::, indexed by integers such that: WA

pe 4

ANA; = 05

14 E A0;

Ay Apc

Auth:

Notice that we do not require that {A,} is a strictly increasing sequence. If A_; = 0 we say that the filtration is positive.

J.-E. BJORK AND E.K. EKSTROM

434

4.1 The Rees ring. Let A be a filtered ring. Then we construct a graded ring R4 as follows: Consider first the ring of finite Laurent series

in one variable over A. Thus, we take the ring S = A[T,T~*] and define a graded ring structure on S such that T is homogeneous of degree 1 while A is the subring of S-homogeneous elements of degree zero. Hence the vth homogeneous component S(v) is equal to AT”. Now Rg is the graded subring of S such that R4(v) = A,T” for every integer v. If v is an integer we denote by jy the bijective map from A, into Ra(v), defined by j,(xz) = x T° for every x in A,. We notice that if x € À,

and y € A, for some pair of integers, then the product j,(r)jz(y) in the ring RA is equal to jy4z(rzy). The ring Ra contains the element T, where T = j,(1,4). It is obvious that T is a central element in R4 and since the filtration on the ring A is exhaustive, it follows that the localization of Ra with respect to the multiplicative set {1,7,T?,...} is equal to S. In other words, we have

APR

Ve AL

|

Next, denote by (1 — T) the two-sided ideal in the ring Ra generated by the central element 1 — T. Then we have

4.3 Proposition. R/(1—T) = A. We refer to [Bj: Proposition 2.13] for the easy proof. Next, let (T') be the two-sided ideal in R4 generated by T. Let us also construct the associated graded ring of A, defined by GA = @A,/A,_-1. Since TRa(v — 1) = A,-,T” holds for every integer v we easily obtain 4.4 Proposition.

The graded rings R4/(T) and GA are isomorphic.

4.5 Filtered A-modules. Let A be a filtered ring. A filtration on a left A-module M consists of an increasing sequence of additive subgroups {T,} such that UF,

= M

and AëT,

C lx4, for all pairs v,k.

The filtration

{T,} is called separated if NT, = 0. When {T,} is a filtration on M we refer to the pair (M,T) as a filtered A-module.

filtered right A-modules.

Similarly we construct

Let (M,T) be a filtered left A-module.

we construct a graded Ra-module

Then

Mr such that the v:th homogeneous

component Mr(v) = T,T*. We refer to Mr as the Rees module associated

with (M,T).

Concerning the R4-module

Mr we notice that the Ra-element T is

injective on Mr since the map Mr(v) ER Mr(v + 1) corresponds to the

FILTERED AUSLANDER-GORENSTEIN

RINGS

435

injective inclusion F, — T',4 in the given A-module M. Next, denote by GM the associated graded GA-module of (M,T). Thus, GM = @T,/T,-1. Identifying R4/(T) with GA, it follows easily

that the graded GA-modules GM and Mr/TMr are the same. 4.7 The E-functor. Denote by G(Ra) the category of graded left Ramodules. A graded R1-module M is T-torsion free if T is injective on M. If we instead assume that every element in a graded R-module M is annhilated by some power of T, then we say that M has T-torsion. If M € G(Ra) we see that M contains a unique largest graded submodule with T-torsion.

4.8 Lemma.

It is denoted by M, and then M/M, is T-torsion free.

Let M € G(Ra).

Then M; = (1-T)M,

and 1-T

is

injective on the T-torsion free module M/M;. We leave out the easy proof. Let us now consider a graded R4-module

M. Set M = @®M(v). Identifying R4/(1 — T) with À, it follows that M/(1—T)M is an A-module, denoted by €(M). We obtain a filtration on the A-module €(M) such that

(4.9)

ECM), = [M(v) + (1-T)M]/(1-T)M. We see that M — E(M) gives a functor from the category of graded

left Ra-modules into the category of filtered left A-modules.

4.10 Proposition. Let M be a graded T-torsion free left Ra-module. Then the Rees module of E(M) is equal to M in the category G(Ra).

Proof. By 4.9 and Noether’s Isomorphism we see that €(M), is equal

to M(v)/[M(v)N(1—T)M]. Since M is T-torsion free it is easily seen that (1-—T)MNM(v) = 0 for every integer v. Then we get Proposition 4.10 by the construction of the Rees module @€(M),T”. Next, let (M,T) be a filtered A-module. Apply the €-functor to the Rees module Mr. Using Noether’s isomorphism as in the proof of Proposition 4.10 we obtain

E(Mp)y = Mr(v) = TT? = Ty. We conclude that €(Mr) is equal to (M,T) in the category of filtered A-modules. 4.11 Noetherian filtered rings.

Let A be a filtered ring.

If R4 is a

left and a right noetherian ring then we say that A is a noetherian filtered

J.-E. BJORK AND E.K. EKSTROM

436

ring. From now on we assume that A is a noetherian filtered ring. Since

A = R,/(1—T) and GA = R4/(T) it follows that the rings A and GA

are noetherian. Denote by Mod,(A) the category of finitely generated left A-modules.

4.12 Definition.

Let M € Mod,(A). A filtration

T on M such that

Mr is a finitely generated R4-module is called a good filtration.

4.13 Remark.

Let M € Mod,(A).

If m1,...,m,

generators of the A-module M and k:,...,k, some then we construct a filtration T on M such that r, = Ay-k,™1

tee

is a finite set of

s-tuple of integers,

+ Ay—k,™Ms.

Now the R4a-module Mr is generated by jx,(7m1),..-,Je,(™Ms) and hence [ is a good filtration. Conversely, let [ be a good filtration on M. We can find a finite set of homogeneous elements jx,(71),.--, jk, (™s) in Mr which generate Mr as a left R4-module. It follows that Mr(v)

=

Mol

=

ERa(v

=

ki) jx, (mi).

Then we see that [, = UA,-z%,;m; holds in M, i.e. T is of the form above. So in this way we have described the class of good filtrations on M. In particular this description of good filtrations shows that if l and Q are two good filtrations on M, then there exists an integer w such that (4.14)

| NBs oy G

Ge

(€ Dotw

hold for every v. Thus, up to shifts the two good filtrations increase with the same rate.

4.15 The use of Rees modules. Let M € Mod,(A) and choose some good filtration T on M. Now Mr is a T-torsion free R4-module and hence Mr is a graded R4-submodule of the localization Mr[T-1]. Recall that Ra[T—'] is equal to A[T,T-!]. It follows easily that we have

Mle | All |] Oy i= Meee Hence Mr[T~"] does not depend on the chosen good filtration. Next, consider M[T,T~*] which is a graded R4-module. Let M be a graded Ra- submodule of M[T,T~*] such that the localization M[T~!] is equal TEE]; ton

FILTERED AUSLANDER-GORENSTEIN

RINGS

437

Applying the €-functor we obtain the filtered A-module E(M). More-

over, since M[T~']/M has T-torsion, and since the A-module M[T,T-!]/ (1 —T)M[T,T~"] obviously is equal to M, it follows from the surjectivity on every T-torsion R4-module that the A-module £(M) is equal of 1—T to M. Thus, €(M) is a filtered A-module of the form (M,Q), where Q is some filtration on M. By Proposition 4.10 we have Mn = M and hence Q is a good filtration if and only if M is a finitely generated R4-module. Conversely, let 2 be some good filtration on M which in general is

not equal to l. Now Mg[T~!] = MIT,T-!] and we conclude that Mg is a graded and finitely generated R4-submodule of M[T,T~']. we have proved the following:

4.16 Proposition.

Let M € Mod;(A).

Summing up,

Then there exists a 1-1 cor-

respondence between the family of good filtrations on M and the family of finitely generated and graded Ra-submodules M of MIT, T-!] whose local-

izations MIT-!] = M[T,T—']. 4.17 Remark. Let M € Mod,(A) and consider a pair of good filtrations F and 9. We say that Q increases faster than T if fy C Q, for every v. Then we write L < 2. We notice that [ < Q holds if and only if Mr is a submodule of Ma. Next, if I < Q holds we know that Mr[T-1] =

MalT=!]

= M[T,T~'].

Hence the R4-module Mn/Mr

has T-torsion.

Since it is finitely generated we find a positive integer w such that T” annihilates Mg/Mr. This means that (,_, reflects the comparison relation from 4.14.

C Ty, for every v and this

4.18 Induced good filtrations. Let T be a good filtration on some finitely

generated A-module M. NNT,.

Let N be some A-submodule of M.

Put Q, =

Then Q is a filtration on N such that Na is a graded R4-submodule

of Mr. It follows from the noetherianness of R4 that Na € Mod;(Ra). Hence 22 is a good filtration on N. So the good filtration on M induces a good filtration on every submodule. Similarly we see that a good filtration on M induces a good filtration on every quotient module. 4.19 Associated graded modules.

Let T be a good filtration on some

M in Mod;(A). Identifying Ra/(T) with GA, it follows that Mr/TMr is a graded GA-module which we denote by grr(M). So we have grr(M) = @T,/Tv-1 and refer to grp(M) as the associated graded GA-module of the filtered A-module (M,T). Since Mr € Mod}(R4), it follows that grr(M) € Mod;(GA). Since we have not assumed that filtrations are separated it may occur that grr(M) is the zero module. Of course, we have grr(M) = 0 if and only if T, = M for every integer v. For a given noethe-

J.-E. BJORK AND E.K. EKSTROM

438

rian filtered ring A we obtain a subcategory of Mody(A) whose objects are such that grr(M) = 0 for every good filtration I’. Denote this subcategory by Mod,;(A). Notice that if M € Mod/(4) is such that grp(M) = 0 for some good filtration I’, then 4.9 implies that gra(M) = 0 for any other good filtration ( and hence M is an object in Mod;(A). 4.20 Example.

An important example of a filtered noetherian ring A

such that Mod,(A) contains non-zero modules is the following: Let A be the Weyl algebra A;(K) in one variable over a commutative field K with its two K-algebra generators { and à satisfying Ot — t0 = 1. We construct a filtration {A,} such that t € A_, and 0 € A1. This is the so-called V-filtration. Now t?0 belongs to A_; and the A-element 1 — {? is not

invertible in the ring A. So the cyclic module M = A/A(1 — t?@) is nonzero. We consider the good filtration on M defined by I, = [A, + A(1 — t29)]/A(1 — t?0). It is easily seen that grr(M) = 0 and hence M belongs to Mod,;(A). 4.21 Zariskian filtered rings. A noetherian filtered ring À is called a Zariskian filtered ring if good filtrations on finitely generated left or right A-modules always are separated. 4.22 Example. Let A be a positively filtered ring. Then, if GA is a left and right noetherian ring it is well known that Ry is a left and a right noetherian ring. Hence a positively filtered ring A is filtered noetherian under the sole assumption that GA is noetherian. It is also obvious that if A is positively filtered with GA noetherian then A is zariskian. In fact, if

lis a good filtration on some M in Mod,;(A), then Remark 4.13 and the hypothesis that A_; = 0 imply that there exists an integer vo such that v < vo gives l'y = 0. Moreover one has the following

4.23 Proposition. Let A be a positively filtered ring such that GA is noetherian. Then a filtration T on some M in Mod,(A) is good if and only fl, = 0 and grr(M) € Mod;(GA).

We leave out the easy proof. Let us now consider a zariskian filtered

ring A where we assume that A, # 0 for every v < 0. Let M € Mod;(A)

and suppose that I is a separated filtration on M such that gr-(M) is a finitely generated GA-module. Then we may ask if Proposition 4.23 still holds, i.e. we may ask if the conditions above imply that I is a good filtration. The example below shows that T is not good in general. Namely, we are going to construct a zariskian filtered ring A and a separated filtration

I on the free left A-module A such that grr(A) € Mods(GA) and yet I is

FILTERED AUSLANDER-GORENSTEIN

RINGS

439

not a good filtration. Let us also remark that the example below answers a question raised on page 158 of [Bj]. 4.24 An example.

The constructed example below is due to G.M.

Bergman. Let Z be the ring of integers and consider the ring A = {p(t)/q(t) : p(t), q(t) € Z[t] and q(0) = 1}. We notice that A is a subring of the formal power series ring Z{[f]] which we denote by A. Now we have 4.25 Lemma.

There exists an element y in A of the form GB

e

Eit+eot? +... such that each €; is 0 or 1 and 7 does not divide any element of Ain A.

Proof. The ring A has countably many elements and A is a unique factorization domain, hence up to associates there are only countably many elements in A dividing elements of A. But there are uncountably many elements and hence a sequence £1,€2,... exists so that no non-zero element f in A belongs to the principal ideal a A generated by +. Having found y from Lemma 4.25 we consider the localization of the ring A with respect to 1,t,t?,.... Denote this ring by B. Since t belongs to the Jacobson radical of the ring A we see that B has a zariskian filtration given by B, = t* A for every integer v. Then, if M is some A-submodule of B such that A C M and N° M = 0, we get a separated filtration on the free

B-module B by T, = t’M. We notice that gr-(B) is a finitely generated GB-module if M/tM is a finitely generated Z-module. Moreover, lis a good filtration if and only if M is a finitely generated A-module. Now we construct M so that Nt”M = 0 and M/tM is a finitely generated Z-module, but the A-module M is not finitely generated. To obtain M we use the

localization of A with respect to 1,t,t?,....

So then A[t~1] is the ring

Z((t)). Put

M = BN (yA[t71] + À). We see that M

A C M C B and M is an A-module.

contains 27" + €,t7"+! 4...+ ¢,t7!.

If n > 1 we see that

It follows that the A-module

M is not finitely generated. Next, if m € M we write m = o(t)y + 6(t) with y € Àft-!] and 6 € A. Now 6(0) € Z and we have m — 6(0) = t(t-!p(t)y + t6:(t)), so m — 6(0) € tM and hence M/tM = Z. There remains only to show that NM = 0. Suppose this intersection contains a non-zero element f. Multiplying f with a t-power we may assume that

f EA. Ifn > 1 we get t-"f in yAlt-1]+A and write t-" f = pa(t)y+6n(t). We see that t” pn(t) belongs to A and get f = tnt) + t”dn(t). Now t"y,(t) converges in A to some element w and we obtain contradicts the choice of y. So f must be zero.

f = wy.

This

J.-E. BJORK AND E.K. EKSTROM

440

5. Filtered Auslander—Gorenstein

5.3].

Rings

First we recall the following result which was proved in [Ek: Theorem

5.1 Theorem. Let A be a filtered noetherian ring such that A and GA are Auslander-Gorenstein rings. Then Ra is an Auslander-Gorenstein ring.

5.2 Remark.

In the special case when

À is a zariskian ring, then

Theorem 3.9 in [Bj] shows that if GA is Auslander-Gorenstein so is A and hence also R4 by Theorem 5.1. We also refer to [H-O] where it was proved that if A is a zariskian ring such that GA is Auslander regular then A and Ra are so.

From now on we consider a filtered noetherian ring A such that A, GA and Ry, are Auslander-Gorenstein. We shall begin to study graded Ra-modules. Keeping the ring A fixed we denote the Rees ring by R to simplify the notations.

Graded free resolutions. If k is an integer, then R[k] denotes the free R-module of rank one graded by R[k](v) = R(v + k). If M is a graded and finitely generated R-module we can construct a resolution — F, — Fo — M — 0, where F6,F1,... are graded free R-modules of finite rank and the differentials are R-linear maps which are homogeneous of degree zero. We refer to F, as a graded free resolution of M. Now Homr(F%, R) is a complex of graded right R-modules whose cohomology groups are given

by ER(M). R-module.

Thus, every E%(M) is a graded and finitely generated right

Let us now apply the €-functor to F,. Since the R-element 1 — T is injective on every graded R-module, it follows that €(F,) is a free resolution

of the A-module €(M). Moreover, for every graded free R-module F of finite rank it is obvious that €(Homr(F, R)) = Homa(E(F), A). Since the cohomology groups of Hom,(€(F), A) are given by E4(E(M)) we conclude 5.2 Proposition.

Let M

be a finitely generated and graded left R-

module. Then E(ER(M)) = EX (E(M)) for every v > 0. Above we assumed that M is a left R-module. Of course, we get a similar result when M is a graded right R-module. Let us now consider a pair (M,T), where I is a good filtration on a finitely generated left Amodule M. Now Mr is a graded and finitely generated R-module and we

apply Proposition 5.2. Since €(Mr) = M, it follows that €(E%(Mr)) =

FILTERED AUSLANDER-GORENSTEIN

RINGS

441

E%,(M) holds for every integer v. Denote by W, the T-torsion part of ER(Mr). Then the material about the £-functor in Section 4 gives 5.3 Proposition.

E(ER(Mr)).

Denote by '(v) the good filtration on the A-module

Then the Rees module E%(M)ryy) is equal to EX (Mr)/Wy.

5.4 Remark.

Starting from a good filtration T on the left A-module

M we notice that the good filtration l'(v) on the right A-module E4(M) is obtained in a unique way since E? (Mr) is unique in the category of graded R-modules. We refer to l'(v) as the T-induced good filtration on E’,(M). 5.5. Filt-free resolutions. If k is an integer then A[k] denotes the free A-module of rank one, filtered by A[k], = Ax4,. If T is a good filtration on some finitely generated left A-module M we construct a resolution — F1 — Fo — M — 0, where every F, is a filt-free A-module, i.e. a finite direct sum

of filtered A-modules of the form A[k]. Moreover, the differentials in the complex are strictly filter preserving, i.e. we have d((F;)y) = d(Fi)N(Fi-1)v

for i > 1 and d((Fo)y) = T, for every integer v. If F, is a graded free resolution of Mp we notice that £(F,) is a filt-free resolution of (M,T). Let us then consider a filt-free resolution F, of (M,T) such that F, = E(P,), where P, is a graded free resolution of Mp. Now Hom,(F,,A) is a complex of filtered A-modules and its cohomology groups are the filtered right A-modules given by (£4(M),T(v)). Next, the filtered complex Hom,(F,,A) gives a spectral sequence whose Eo-term is the associated graded complex given by a complex of graded right GA-modules. If F is a filt-free A-module we notice that G(Homa,(F,A)) = Homga(GF,GA). We conclude that the Eo-term is the complex Homg,4(GF,, GA). Next, since the differentials in the filt-free resolution are strictly filter preserving, it follows that GF, is a free resolu-

tion of the GA-module gr-(M). So the cohomology groups of the Eo-term of the spectral sequence are given by EZ ,(grp(M)). Using the comparison condition from 4.14, applied to finitely generated right A-modules one can easily show that the spectral sequence converges. The convergence implies that for every integer v, it follows that grrç,)(E4(M)) is a subquotient of

the GA-module E%,4(grp(M)). In the case when M belongs to Mod,(A) we conclude that £4 (M) belongs to Mod;(A) for every v. We shall now study the case when M does not belong to Mod;(A). So then grp(M) is non-zero and this GAmodule has some grade number k. If v < k we have Ej A(grr(M)) = 0 and since this is the term E? of the spectral sequence, it follows that no coboundaries occur in degree k during the passage to higher terms in the spectral sequence. This gives the following

J.-E. BJORK AND E.K. EKSTROM

442

Put k = jga(grp(M)).

5.6 Proposition. sequence

Then there exists an exact

he grræ)(E*(M)) rw EG a(grr(M)) — WwW,

0

where jaa( We) > jcA(Eki"(gr(M)))> k+1, unless Wy = 0.

not

5.7 Proposition. Let M be a finitely generated A-module which does belong to Mod;(A). Then, for every good filtration we have

jcA(grr(M)) > ja(M). Proof. If k = jca(grr(M)) < ja(M) then the exact sequence in Proposition 5.6 gives jga(EE,(grp(M))) > k+1. This contradicts Lemma 2.9 applied to the Auslander-Gorenstein ring GA. 5.8 Corollary.

Mod;(A).

Let M

be a pure A-module

which does not belong to

Then jaa(grr(M)) = ja(M) for every good filtration T.

Proof. Suppose that k = jGa(grr(M)) > j4(M). The purity of M gives j4(E*(M)) > k and then jaA(grra)(E*(M)) > k by Proposition 5.7. Now Proposition 5.6 and Lemma 2.8 imply that jga(E%,(grp(M)) > k. But this contradicts Lemma 2.9 and hence k < j4(M). follows from Proposition 5.7.

Now k = ja(M)

Remark. If A is a zariskian filtered ring then the class Mod;(A) is empty. So in the zariskian case we see that the equality j4(M) = jaaA(grr(M)) holds for every finitely generated A-module M. 5.9 A base change formula. Recall that GA = R/(T). So every Rmodule M such that TM = 0 has a GA-module structure and we denote the resulting GA-module by g(M). Since T is a central non-zero divisor in the ring À, a well known result in homological algebra gives:

5.10 Proposition. g(E,*'(M)) = E%,4(g(M)) holds for every finitely generated R-module M annthilated by T and every v.

Let us again consider a pair (M,T), where [ is aà good filtration on some finitely generated left or right A-module M.

9.11 Lemma. torsion free.

Put k = jr(Mr).

Then the R-module Ek(Mp) is T-

FILTERED AUSLANDER-GORENSTEIN

RINGS

443

Proof. Consider the exact sequence

DM

MR

Mp TM ps 0!

Then we conclude that the T-kernel of E&(Mr) is a quotient of EË(Mr/TMr). Next, by Proposition 5.10 we have g(EË(Mr/TMr)) =

EGa (grr(M)). So Lemma 5.11 follows if EËz'(grr(M)) = 0. To prove

this we use that jg4(grp(M)) > ja(M) by Proposition 5.7. Next, Proposition 5.2 implies that j4(M) > jr(Mr) = k. So jga(grp(M)) > k and

hence EX7)(grp(M)) = 0. 5.12 Proposition. j4(M) = jr(Mp) holds for every good filtration I. Proof.

Proposition 5.2 gives j4(M) > jr(Mr). Next, if k = j,(Mr),

then E%(Mr) is T-torsion free and therefore £(E*(Mr.)) is non-zero. Since this A-module is EX (M) we conclude that k > j4(M) and Proposition 5.12 follows. Let us now

consider a Rees module

Mr and put n = jr(Mr).

We

can construct a Fossum module F(Mr) using a graded free resolution Fn41 — ++: — Fo — Mr. Then we notice that E(Fn41) — --- — E(Fo) — E(Mr) — 0 is of the form in 2.1. Since E(Mp) = M we conclude that a similar reasoning as in Proposition 5.2 gives 5.13 Proposition. E(F(Mr)).

The A-module M has a Fossum module of the form

5.14 Corollary. The A-module M 1s pure if and only if Mr is a pure R-module for every good filtration T.

Proof.

Put n = ja(M)

= jr(Mr).

Proposition 2.5 shows that

Et (F(Mr)) is a submodule of Mr and hence T-torsion free. Since the €-functor is faithful on T-torsion free graded R-modules, it follows that

E},*'(F(Mr)) = 0 if and only if its €-image is zero. Next, using Proposition 5.13 and 5.2, we see that the £-image is equal to Et (F(M)). Then the purity criterion in (2) of Theorem 2.12 gives Corollary 5.14. We shall now discuss good filtrations on a pure A-module M. Keeping M fixed we put n = j4(M) and let T be some good filtration on M. Then the equality j4(M) = jr(Mr) and Lemma 5.11 imply that E%(Mr) is T-torsion free. Hence Proposition 5.3 gives

(5.15)

ER(Mr) = E4(M)rqn)-

444

J.-E. BJORK AND E.K. EKSTROM

Next, starting from the filtered A-module (E(M),T(n)) whose grade number is n, we construct the induced good I'(n)-filtration on E%”(M) which we denote by I'(n,n). Repeating the previous arguments we obtain

(5.16) Now

ER” (Mr) = Ex" (M)r(nn)we

consider

the

pure

R-module

Mr.

The

purity

gives

E},*'(F(Mr)) = 0 so Proposition 2.5 gives the exact sequence

(5.17)

0 — Mr —

EX" (Mr) + ERt?(F(Mrp)) — 0.

Let W be the T-torsion submodule of Eñ*?(F(Mr)) and put M =

g"!(W). Then M is a graded R-submodule of Ep’”(Mr). Repeated use of Lemma 5.11 shows that E%"(Mr) is T-torsion free and hence So then M

M is.

is a Rees module, say Ng. Next, by construction M/Mr

has

T-torsion and hence €(M) is equal to €(Mr). We conclude that N = M, i.e. M = Ma where 22 is a good filtration on the A-module M. Since Mr is a submodule of Mg, it follows that Q increases faster than [. The good filtration ( has some special properties.

5.18 Lemma. gra(M) is a GA-submodule of gtp(nny(Ea’”(M)). Proof. The equality in 5.16 and the construction of Mg shows that the R-module E%”"(M)r(nn)y/Ma is T-torsion free. Then we divide by T and Lemma 5.18 follows.

5.19 Lemma.

Proof.

jr(Ma/Mr) > n +2.

The R-module Mn/Mr

is a submodule of E%*?(F(Mr))

whose grade number is > n + 2. Now we analyze when GA-modules are pure and begin with 9.20 Lemma. Let T be a good filtration on some A-module M. Then, if the R-module Mr is strongly pure, it follows that the GA-module grr(M) 1s pure.

Proof. Put K = Mr/TMfr so that g(K) = grr(M). Repeated use of Proposition 5.10 shows that if v > 0, then jg4(E%,4(gtp(M))) is equal to jr(Ext'(K)) — 1. Next, by a long exact sequence and Lemma 2.8, it follows that

jr(ER*(K)) > inf {jr(ER(Mr)), jr(ERt!(Mr))}.

FILTERED AUSLANDER-GORENSTEIN

RINGS

445

Put n = jr(Mr) which by 5.12 and 5.8 is equal to jg4(grp(M). Then,

if v > n the inequality above and Definition 3.3 show that jp(E%*1(K)) > v +2 and hence jga(H@4(grr(M))) > v+1. Then grp(M) is pure by (3) in Theorem 2.12. 5.21 Corollary.

Let ( be the good filtration on M

constructed after

5.17. Then gra(M) is a pure GA-module. Proof.

By Lemma

5.18 it suffices to show that the GA-module

Elr(n,n)(EZ4""(M)) is pure.

But this follows since its Rees module is the

strongly pure R-module E”"(Mp). Let us again consider a pure A-module M which does not belong to Mod;(A). Denote by A the family of all good filtrations on M. If T and 2 belong to A, then I + Q is the good filtration whose Rees module is Mr + Mg, while FAQ is the good filtration whose Rees module is

MpN Ma.

5.22 Definition. Let M be a pure A-module and set n = j4(M). pair [ and Q in A are tamely close if jr(Mria/Mrna) > n + 2.

A

It is easily seen that Definition 5.22 gives an equivalence relation on

A. 5.23 Theorem.

Every equivalence class in A contains a unique good

filtration Q such that gra(M) is a pure GA-module. unique fastest filtration in the equivalence class, 1.e. is tamely close with Q.

Moreover, Q is the Q>T for anyT which

Proof. Let Ag denote an equivalence class. Take some F in Ao. Corollary 5.21 and Lemma 5.19 gives some 2 in Ao such that grg(M) is pure and 2 >IT. There remains to show that Q is unique and that Q > I’ for any I’ in Ag. This follows from the following

Sublemma. pure GA-module. close to (1. Proof.

Let ( be a good filtration on M such that gra(M) is a Then 2 >T

Set M

prove this we put

=

Mrin.

W = M/Ma.

for any good filtration T which ts tamely

It suffices to show that M

=

Ma.

To

Now we have jr(W) > n+ 2, where

n = ja(M) = jaa(gra(M)). We notice that the T-kernel of W is a GAsubmodule of grg(M). By the base change we have jga(g(Kerr(W))) = jr(Kerr(W)) —1 > jr(W)-1>n+1. So the purity of gra(M) gives

Kerr(W) = 0. Since €(M) = E(Mo) it follows also that W has T-torsion.

446

J.-E. BJORK AND E.K. EKSTROM

Hence W = 0 and the Sublemma is proved.

5.24 Remark. The unique fastest good filtration in an equivalence class of A is called a Gabber filtration on the pure A-module M. 5.25 The case when M is strongly pure. Let M be strongly pure and set n = ja(M). The constructions made after 5.17 show that the good filtration Q in Corollary 5.21 gives Mn = E”™"(Mpg) and hence Ma is a strongly pure R-module. Then Theorem 5.23 shows that Mo is strongly pure for every Gabber filtration Q. The question arises if gro(M) is a strongly pure GA-module for every Gabber filtration. This is not true as the example below shows. 5.26 Example. Suppose that there exists a graded and pure GA-module M which is not strongly pure and has jg4(M) = 0. Let us remark that M exists if A is the Weyl algebra A1(C) endowed with the positive Bernstein filtration for then GA is a polynomial ring in two variables and we can choose a torsion free GA-module which is not reflexive. With M as above we regard M as a graded R-module annihilated by T. Let a:F — M bea surjective map where F is a graded free R-module and a homogeneous of

degree zero. Let K be the kernel of a. We notice that E?(K) = Ex*'(M) holds for every v > 1. Using Proposition 5.10 and the GA-purity of M we see that the R-module K is strongly pure. Next, since K is T-torsion free we have K = Na for some A-module N with a good filtration 2. The strong purity of K and Lemma 5.20 imply that gra(N) is pure and hence {2 is a Gabber filtration. But grg(N) is not strongly pure. Namely, since M is pure but not strongly pure there exists some v > 1 such that

jaA(E*(M)) = v+1. We also have an exact sequence 0 — M — gro(N) — W — 0 and conclude that 5G4(E*(gra(N))) = v +1 so then gra(N) is not strongly pure.

5.27 An open problem. Let M be a strongly pure module over a filtered Auslander-Gorenstein ring. Does there exist some Gabber filtration 2 on

M such that gro(M) is a strongly pure GA-module. À final remark. The existence of Gabber filtrations is non-trivial even in the case when À is commutative. For example, let À be a local regular commutative ring with its maximal ideal m. The m-adic filtration on À is Zariskian, i.e. we use the filtration defined by A_, = m”: v > 1, while

À = Ap = A; =.... If M € Mod;(A) we get the good filtration defined

by {m’M}. Now the associated graded module @m’M/m’t!M is not pure in general even if M is a pure A-module from the start. In this case

FILTERED AUSLANDER-GORENSTEIN

RINGS

447

we construct the Gabber filtration Q which is tamely close to the m-adic filtration. Thus, we have M = Q5 > (1 D... and Q_, > m°M for

every v > 1. Moreover, NQy = 0 and gra(M) is a pure gr, (A)-module. It would be interesting to see concrete non-trivial examples and if possible find algorithms to compute Gabber’s filtration associated with the m-adic filtration on suitable pure A-modules such as A/p where p is a prime ideal in A.

REFERENCES

[A-B] M. Auslander and M. Bridger, Stable module theory, Mem. Amer. Math. Soc. 94 (1969). [Ba] H. Bass, On the ubiquity of Gorenstein rings, Math. Zeit. 82 (1963), 8-28. [Bj] J.-E. Bjork, The Auslander condition on noetherian rings, Sem. Dubreil-Malliavin. (1989), 137-173.

Lecture

Notes

in Math.

1404,

Springer-Verlag

[Ek] E.K. Ekstrom, The Auslander condition on graded and filtered noetherian rings, Ibid. 220-245.

[Fo] R. Fossum, Duality over Gorenstein rings, Math. Scand. 26 (1970), 165-176. [F-G-R] R. Fossum, P. Griffith, and I. Reiten, Trivial extensions of Abelian categories, Lecture Notes in Math. 456, Springer-Verlag (1975). [H] Li Huishi, On pure module theory over zariskian filtered rings, University of Antwerpen, preprint (1989). [H-O] Li Huishi and F. Oystaeyen, Global dimension and Auslander regularity of Rees rings. [Le 1] T. Levasseur, Grade des modules sur certains anneaux filtrés, Comm.

in Algebra 9 (1981), 1519-1532. [Le 2] T. Levasseur, Complere bidualisant en algèbre non commutative, Sem. Dubreil-Malliavin. Lecture Notes in Math. 1146, Springer-Verlag (1983-84), 270-287. [L-S] T. Levasseur and J.T. Stafford, Rings of differential operators on classical rings of invariants, Mem. Amer. Math. Soc. 81, no. 412 (1989).

448

J.-E. BJORK AND E.K. EKSTROM

[Re] I. Reiten, Trivial extensions and Gorenstein rings, Thesis, Chicago

Univ. (1971). [Z] A. Zaks, Injective dimensions of semi-primary rings, J. of Algebra 13 (1969), 73-86. Received February 21, 1990

Department of Mathematics University of Stockholm Box 6701 113 85 Stockholm Sweden

Relévements d’opérateurs différentiels sur les anneaux d’invariants THIERRY LEVASSEUR A J. Dirmier pour son 65ème anniversaire

O. Introduction et Notations Le corps de base est celui des nombres complexes noté C.

0.1 Si X est une variété algébrique irréductible affine on note : O(X) l’anneau des fonctions régulières sur X, D(X) l’anneau des opérateurs différentiels (o.d. en abrégé) sur X, Der™(X) le O(X)-module des o.d. d’ordre < m sans terme constant, D'(X) = O(X) @ Der™(X), cf. 1.1 et 1.2. Soit G — GL(V) une représentation linéaire de dimension finie d’un groupe réductif G, on désigne par V//G la variété affine telle que O(V//G) = O(V)S. Le groupe G opère naturellement sur D(V) en laissant stable chaque Der™(V). On dit que (V, G) a la propriété de relèvement à l’ordre m si le morphisme de restriction y : Der™(V)° — Der™(V//G) est surjectif; si cette propriété est vérifiée pour tout m on dira que (V,G) a la propriété de relèvement, ce qui implique alors la surjectivité du morphisme d’algèbres

yp: D(V)S — D(V//G). 0.2 Sim = 1Der!(V//G), resp. Der'(V)®, n’est autre que l’espace des champs de vecteurs sur V//G, resp. des champs de vecteurs invariants

sur V.

Le relèvement à l’ordre 1 a été étudié par G. Schwarz dans [S].

Rappelons que V//G possède une stratification finie, paramétrée par les classes de conjugaison des groupes d’isotropie des points d’orbite fermée; il existe une strate ouverte que nous appellerons strate principale et noterons

Q(V//G). On désigne par A(V//G) l'idéal de O(V)S définissant le complémentaire de Q(V//G). Il est en particulier démontré dans [S] que si G >

GL(V) est orthogonalisable et si la hauteur de A(V//G) est > 2 alors (V,G) a la propriété de relèvement à l’ordre 1. Si m est quelconque des conditions nécessaires et suffisantes pour qu’il y ait relèvement à l’ordre m

THIERRY LEVASSEUR

450

ont été données, lorsque G est fini dans [K], lorsque G est un tore dans

[Mu]. 0.3 Nous proposons

au §3 une condition suffisante pour que (V, G)

possède la propriété de relèvement.

Soit gry : grD(V)S — grD(V//G) le

morphisme gradué associé au morphisme filtré y, posons

K= Ker(gry),

R=

lta ee: notons que R est une C-algèbre commutative intègre, de type fini,

graduée, contenant O(V) en degré 0. On montre alors, (3.6, théorème) :

Théorème.

Supposons que R vérifie la condition (S2) et que A(V//G)

engendre un idéal de hauteur > 2 dans R, alors (V,G) a la propriété de relèvement. Ce résultat, de démonstration élémentaire, a l’inconvénient de nécessiter une bonne connaissance de R. Néanmoins il permet de traiter les cas

G fini et G = C* (l’anneau R étant connu dans le cas des tores grace a [Mu]). 0.4 Une autre application de 0.3 est donnée au §5, elle concerne les o.d. sur les anneaux classiques d’invariants. On étudie les trois cas :

(Mpx(C) x Meq(C), GL(k,C))

(V,G) = 4 (Min(C), O(k, C))

(Moen(C), Sp(2k, C))

où GL,O,Sp désignent respectivement les groupes linéaire, orthogonal et symplectique opérant “naturellement” sur les espaces de matrices de taille appropriée, (cf 5.3). On retrouve un des résultats de [Le-St] :

Théoréme.

Soit (V,G) comme

ci-dessus ; si V//G n’est pas lisse

alors (V,G) a la propriété de relèvement. Utilisant les résultats de R. Howe sur les paires réductives duales, on

sait qu’il existe un morphisme surjectif w : U(g’) + D(V)® où g’ est une algèbre de Lie semi-simple d’algèbre enveloppante U(g’), le noyau de yw étant un idéal primitif J. La preuve donnée dans [Le-St] repose en partie

sur le fait que J est maximal (lorsque V//G n’est pas lisse). Le théorème de 0.3 permet de s’affranchir de cette vérification (qui utilise la classification des idéaux primitifs) et fournit de plus l’information /gr1 = grI , où gr I est l’idéal gradué associé à I pour la filtration naturelle de U(g’). Malheureusement cette nouvelle démonstration ne donne pas la maximalité de I.

RELEVEMENTS D’OPERATEURS DIFFERENTIELS

451

0.5 Les notations qui suivent seront librement utilisées. Si A est une C-algèbre commutative de type fini on désigne par Spec A l’ensemble de ses idéaux premiers et on pose

Reg A = {P € Spec A/Ap régulier}, Sing A= SpecA\Reg A. Si A est intègre on dit que : A est normale lorsque A est intégralement clos et A vérifie la condition (S2) si A = Narp=1Ap où At est la hauteur d’un idéal.

Soit V une variété algébrique définie sur C, Oy son faisceau

structural et O(V) la C-algébre des fonctions régulières sur V. Si V est

affine on identifie (V, Ov) à Spec O(V). Soit p un point (fermé) de V on note M,

l’idéal maximal

Reg V = {p E V/Ov,

de l’anneau

local Ov, de V en p.

On définit

régulier}, Sing V = V\ RegV ; si Reg V = V on

dit que V est lisse. Supposons de plus la variété V affine ; si Z est un idéal de O(V) on pose

V(Z) = {P € SpecO(V)/P DT} et inversement si F est fermé dans V, Z(F) F ; V est dite normale si O(V) l’est.

est l’idéal radiciel définissant

Si t,,...,t, sont des éléments d’un anneau et a = (a1,...,a,;) € N°

on pose t® = t{?t5?---t%, jal = a1 +agt+---+as.

1. Généralités 1.1 Etant donnée une C-algebre commutative intègre A, nous désignerons par D(A) l’anneau des opérateurs différentiels sur A. Rappelons, cf

[E.G.A.] ou [M.R.], que D(A) = Un>oD™(A) où D™(A) est le A-module des opérateurs différentiels (o.d. en abrégé) d’ordre < m, défini par : D™(A) = {D € EndcA/V ao,...,4m € À

[ao[ai[- -- [am D]] ---] = 0}.

On pose D™(A) = (0) si m < 0 ; notons que D°(A) = A. La sousalgèbre D(A) de EndcA est filtrée par les D™(A) d’anneau gradué associé gr D(A}

=

Opty

note grm : D(A) — P € D™(A)\D™~1(A) le symbole principal provenant de celle de Ky

, qui est une C-algèbre commutative

intègre.

On

grmP(A) := = At la projection canonique. Si on dit que P est d’ordre m et on pose ord P= m, de P étant alors grP. L’action de D(A) sur À, EndçA, sera notée a > P*a, a € A, PE D(A). Er?

.

.

.

.

452

THIERRY LEVASSEUR

Le sous-A-module de D™(A) formé des o.d.

sans terme constant

d’ordre < m est donc

Der™(A) = {D € D™(A)/D *1 = 0}.

Ainsi D™(A) = A @ Der™(A) et Der! (A) est le A-module des dérivations C-linéaires de A. Rappelons que Der!(A) ~ Homa(Q4,A) où (4 est le A-module des différentielles d’ordre 1. On pose Der(A) = Um>oDer™(A). Supposons désormais que A soit de plus une C-algèbre de type fini.

Alors, les Der™(A) sont des A-modules de type fini et si S C A\(0) est multiplicativement stable, S est une partie de Ore dans D(A) et on a les identifications :

D(S-14) = S-1D(4) = D(A)S-!, Der™(S~1A) = S-1A @a Der” (A). En outre, puisque groD(A)

= A, les éléments de S s’identifient à leurs

images par gro et l’algèbre localisée

S~!gr D(A) s’identifie canoniquement

à gr D(S-1 A). Lorsque S = A\P, P € SpecA, on notera D'(Ap), Der (Ap)etc...

à

la place de S-1D'(4),S-1Der (A) etc.... Signalons enfin que si S~!A est un anneau régulier, l’algèbre D(S-!A) est engendrée sur C par S~1A et Der!(S~1(A)), qui est un S~! A-module projectif de type fini. L’anneau grD(S—1A) s’identifie alors à l’algèbre symétrique de ce module et D™(S—1A) est égal à l’ensemble des combinaisons linéaires sur S~1.A des produits d’au plus m dérivations.

1.2 Soit V une variété irréductible affine ; ce qui a été rappelé en 1.1 s’applique à A = O(V). Dans ce cas nous substituerons les notations

D'(V), Der (V) aD (A), Der (A) etc... Si p est un point fermé de V,D(V), désignera D(Ov,). Nous utiliserons parfois la notation Oy, à la place de Der!(V), pour désigner le module des dérivations. I] découle de 1.1 que si V est de plus supposée lisse, grD(V) est une Calgèbre de type fini régulière, et pour p € V l’espace tangent à V en p, noté Tp et à l’espace vectoriel RER. des dérivations ponctuelles sur Oy p.

2. Opérateurs différentiels sur les espaces d’orbites

2.1 On fixe une variété algébrique affine irréductible V et un groupe algébrique réductif G opérant rationnellement sur V. On note V//G la

RELEVEMENTS D’OPERATEURS DIFFERENTIELS

453

variété affine telle que O(V//G) = O(V)$ (anneau des invariants). Le comorphisme de l’inclusion O(V)% + O(V) sera désigné par my (ou +), donc ty : V + V//G, c’est un morphisme surjectif. Le groupe G opére naturellement sur D™(V) pour tout m par la régle: g:-D*f=g-(D*g"-f),g€G,

DED™(V)f

EO(V).

Il n’est pas difficile de vérifier que D™(V) est un G-module rationnel. On peut donc considérer la C-algèbre D(V)S des o.d. invariants ; elle est filtrée par les D”(V)© qui sont des O(V)%-modules de type fini, cf ([B-H], Theorem 3.1). On dispose alors de morphismes naturels de O(V)-modules

g:D™(V)° > D™(V//G) g étant l’application de restriction: p(P)+*f = Pxf, PE D™(V)%, f € O(V)%. Ceci donne un morphisme de C-algébres filtrées y : D(V)® — D(V//G). On notera J le noyau de ¢ et U l’algèbre quotient Be ; elle est filtrée par les U™ = BE

~ Re.

On pose grU = DE.

Remarquons que G opère dans grmD(V) pour tout m et que, G étant

réductif, on peut identifier les O(V)°-modules [grmD(V)]° et grm[D(V)°]= FE et les algèbres graduées [gr D(V)]S et gr[D(V)®]. Le morphisme ¢ induit un morphisme (injectif) filtré que nous noterons

encore y, de U dans D(V//G), et des morphismes gradués gry: grD(V)° — G grD(V//G) et grp: grU — grD(V//G). Notons que gr U = wee ou grJ est l’idéal gradué associé, et que K = Ker(gry) 2 gr J. On fera l’abus de notation consistant à encore noter K à la place de As7. Insistons sur la

définition de K : grmP € Ker(gry) = K équivaut à ÿ(P) € D! (V//G). Signalons aussi que : JN O(V)° = K N groD(V)S = (0), p et gry sont Videntité sur O(V)%, U™ et JND™(V)® sont des O(V)°-modules. 2.2 On suppose ici que V est une variété algébrique affine lisse et que G est un groupe réductif opérant sur V (rationnellement). Dans ce cas

gr D(V)© est une C-algèbre de type fini, cf (1.2. et 2.1.), donc D(V)S et U le sont également. Définition. On dit que (V,G) a la propriété de relèvement à l’ordre m si

p : D(V)S — D™(V//G) est surjective. Si (V, G) satisfait la propriété de relèvement à tous les ordres on dira que (V,G) a la propriété de relèvement. Notons, qu’avec les notations de 2.1., on peut évidemment remplacer D"(V)S par U™ dans cette définition et que la propriété de relèvement

équivaut à y : U — D(V//G) est un isomorphisme filtré.

454

THIERRY LEVASSEUR

(a) Si Q € D(V//G) et s’il existe P € D™(V) tel que

Remarques.

P = Q sur O(V)® alors il existe D € D™(V)® tel que p(P) = Q (on écrit D™(V) = D™(V)° @ D™(V)g où D™(V)g est un supplémentaire G-stable). (b) Soit H = {g € G/g-v = v pour tout v € V} ; alors, si = G/H,(V,G) a la propriété de relèvement (à l’ordre m) si (V,G) l’a. On peut donc se ramener au cas où l’action est fidèle, i.e. H = {1}. (c) Supposons que p : G > GL(V) soit une représentation linéaire de G. On peut écrire V = VS@Vc où VS est l’espace des points fixes et Vg un supplémentaire G-stable. Comme O(V)S est un anneau de polynômes

sur O(V&) on voit que (V,G) a la propriété de relèvement (à l’ordre m) si (Vg, G) l’a. Le cas échéant on pourra supposer V@ = (0). 2.3 On conserve les hypothèses de 2.2. On

note

G,

le groupe

d’isotropie

d’un

z€ V

{v € V/G : v fermée dans V} (points semi-simples).

et Vs, l’ensemble

Rappelons que pour

tout x € V il existe une unique orbite fermée dans G -x. Si £ € V//G on note O(£) l’unique orbite fermée dans my (€). Rappelons quelques résultats tirés de [Lu]. Si L est un sous-groupe (réductif) de G on pose

(V//G)(L) = {E € V//G/ V x € O(E)G: conjugué à L}

VO = x (V//G\1) = {v € V/G- 0 > O(8), € € (V//G)a)}Les (V//G)(z) forment une stratification finie de V//G et il existe une unique strate, que nous appellerons strate principale, (V//G)(c) vérifiant: (V//G)(c) ouvert non vide et (V//G)(r) # @ implique C conjugué à un sous-groupe de L. On notera 2Q(V//G) la strate principale et on pose :

Q(V, G) = ry Q(V//G), F(V//G) = V//G - Q(V//G),

F(V,G) = V = Q(V,G), A(V//G) = T(F(V//G)), A(V, G) = I(F(V,G)) = VA(V//G)O(V). Rappelons le résultat, ([Lu], IIL.3. Corollaire 6) :

Proposition. Soit € € V//G alors£ € Q(V//G) si et seulement si T:V — V//G est lisse aux points de O(€).

Donc s’il l’on pose Vlisse — {v € V/xlisse en v} ona: Q(V//G)

=

1(Vss nN ylissey acy, G) =

{v i V/Gv ®) Gz, re

Vi; nvlisse

RELEVEMENTS D’OPERATEURS DIFFERENTIELS Remarque.

Supposons que p : G —

455

GL(V) soit une représentation

linéaire de G, V = VS @ Vg comme en (2.2 Remarque (c)). On vérifie facilement que : A(V,G) = /A(Vg,G)O(V) , si dimVG//G = 0 alors Q(V//G) =V"%//G = VS, si dim Vg¢//G > 0 alors ht A(V, G) < dim Vg //G. 2.4 Fixons une représentation linéaire p : G — GL(V) du groupe réductif G. Le problème du relèvement à l’ordre 1, i.e. celui des champs de vecteurs sur V//G , a déja été étudié, spécialement par G. Schwarz dans

[S]. Il est résolu par le théorème suivant dans le cas orthogonalisable : Théorème lisable de G.

[S]. Soit p : G — GL(V) une représentation orthogonaL'image

de y : OG —

Ovya

est égale a l’ensemble

des

dérivations laissant stables les idéaux de O(V)S constitués des fonctions nulles sur les strates de V//G. L'exemple de ([S], p. 65), où G = C*, illustre que ce théorème est en défaut si p n’est plus orthogonalisable. Le problème du relèvement dans le cas où G est un tore a été résolu par I. Musson dans [Mu]. Dans le cas où

G est fini il a été traité par J.M. Kantor dans [K] (voir aussi [I] et [Le]). Nous reviendrons sur les cas G fini et G = C* au §4. La propriété de relèvement se caractérise facilement lorsque V//G est

lisse :

Lemme.

Soit

p:G—GL(V)

une représentation linéaire avec V//G

lisse. Alors (V,G) a la propriété de relèvement à l’ordre 1 si et seulement

y=

Preuve. Compte tenu de (2.2, remarque (c)) on suppose V = (0). Dans ce cas V,, = (0) équivaut à O(V)® = C. Supposons O(V)% = C[qi,.--,qe] avec £ > 1, q1,...,qe, polynomes sur V algébriquement indépendants, que l’on peut prendre de degré > 2 sans terme de degré < 1.

Si D € Oy alors D * q;(0) = 0 donc il ne peut exister de relèvement de ao € Ovya. La réciproque est triviale. Donnons un exemple montrant que le relèvement à l’ordre 1 ne suffit pas à assurer le relèvement aux ordres supérieurs.

Eremple. Soit (V,G) = (C°,C*),G opérant linéairement avec les poids 15 1; —2.

Si O(V)

O(V)¢-module

1 TeV//G surjective ; vlisse est un ouvert G-stable de V.

Pour simplifier posons

A= Oy), b= O(V). | Soit v € Q(V,G), € = m(v) = r(x) avec x E V,,N ylisse, Puisque Gz et y\ lisse sont deux fermés G-stables disjoints il existe f € À valant 0

sur V\VISSe et 1 sur Gr. Donc V; = Spec By est un ouvert affine G-stable contenu dans Vi8S¢ et contenant x. Le comorphisme my : V; — V;//G de l'inclusion Ay = By + By est alors lisse ; on en déduit aisément la sur-

jectivité de l’application de restriction Der!(B;)® — Der'(Ay).

€ = (zx) est dans V;//G il en découle U; = eS

Comme

D'(Aë), prouvant

EENetvEe nO. Réciproquement soit v € m~'Q et x € Ves tel que m(v) = a(x) = €.

Par hypothèse £ € RegV//G il s’agit donc de prouver que da, :T;V —

460

THIERRY LEVASSEUR

TkV//G est surjective. Cette application s’identifie à l’application naturelle entre dérivations ponctuelles :

Der!(B,)

À

M:Der!(B;)

Der! (Ag)

MeDer! (A4)

Puisque € € 2 et A\Me C B\Maz, l'application Der’ (B)? — Der'(A¢)

est surjective et Der!(B)? C Der'(B;). Par conséquent dr, est surjective. Remarque. Par (3.4, remarque (a)), pour que (V,G) ait la propriété de relèvement il est nécessaire que Q(V//G) = Reg V//G. L’exemple de 2.4 montre que cela ne suffit pas. Signalons le corollaire qui suit, découlant facilement de la proposition

et de (3.3, proposition 2). Corollaire.

Il y a équivalence entre :

(i) (V,G) a la propriété de relèvement. (ii) py: D(V)% D(V//G) est surjective et gr J est un idéal premier de gr D(VYS. 3.6 Nous pouvons donner la condition suffisante pour le relèvement : Théorème.

Soient (V,G),U = ER,

C grU comme

en 2.1.

On

suppose : (i) gr U = 2

vérifie la condition (S2).

rU (ii) dim Awan

DEEE ye

< dimgrU — 2.

Alors (V,G) a la propriété de relévement. Preuve.

Compte tenu de (3.5, Proposition) et de la définition de

A(V//G) il suffit d’appliquer (3.4, Proposition). Remarque. Rappelons que si (i) et (ii) sont satisfaits alors grU est un anneau normal et Q(V//G) = Reg V//G, cf. (3.4, Remarques) et (3.5, corollaire). Problème.

L’inclusion gr JC K est évidente, a-t-on toujours gr J=

K ? De manière équivalente : si D € D"(V)S est tel que D restreint à O(VYS est d’ordre < m—1, a-t-on DE D™-1(V)F + J ?

RELEVEMENTS D’OPERATEURS DIFFERENTIELS

461

Le corollaire de 3.5 montre que la réponse est positive si (V,G) a la propriété de relévement.

4. Exemples : G fini et G = C*

4.1 Soit (V,G) comme en 2.1. Si (V,G) a la propriété de relévement alors A(V//G) = Z(Sing V//G) est de hauteur > 2, car V//G est normale, mais cela n’implique en général pas ht A(V,G) > 2, cf (2.4, exemple). Nous allons voir que si G est fini ou C* alors cette derniére condition assure la propriété de relévement. 4.2 On suppose ici G fini opérant sur B = O(V), régulière. On suppose, cf (2.2, remarque (b)), que Ae eet ni = V//G ; rappelons que 7 est L’ensemble V,, N lisse coincide avec V& Pouvert étale. On retrouve le résultat connu, cf. 2.4. : Théorème.

Sous les hypothèses précédentes,

C-algébre de type fini G C AutcB. Soient fini et que V,, = V. des points où 7 est

il y a équivalence en-

tre (i) (V,G) a la propriété de relèvement et (ii) codim(V\V®t) > 2. Si G — GL(V) est une représentation linéaire fidèle ces conditions sont équivalentes à (iii) G ne contient pas de pseudo réflexion (F 1).

Preuve. On remarque J = Ker = (0), cf ([Le], théorème 5, p. 171). Alors U = D(V)®, grU = gr D(V)® est normal.

Si

A= A(V//G) on a: A(V,G)= VAB, V(A(V,G)) = V\V*", htA =

htA(V,G) = htA gr D(V) = htAgrU.

Ceci montre que (i) implique (ii),

cf 4.1. Inversement (ii) signifie ht AgrU > 2 d’où (i) par (3.6, théorème). L’équivalence de (ii) et (iii) est classique. 4.3 On suppose ici que G = C* et que

G ~ GL(V) est une représenta-

tion linéaire fidèle. On choisit une base de V de sorte que O(V) = C[T] = C[T;,...,T;] et 9 -T; = g%T; avec a; € Z. Soit 0; = 0/07T;, 1; le symbole de 0; de sorte que D(V) = C[T,d] = C[T;,...,T;,01,..., Os], gr D(V) = CIT,r] = C[T1,...,T7:,71,-..-,7s]. L'image de l’algèbre de Lie de G dans

Ov est CO où 8 = 5), a;T;0;, donc @ est central dans DY)o et o = >; aj

757; est le symbole de 6.

Lemme. D)

Supposons dimV//G = dimV —1.

Alors

J = Kery =

Gale = her (ory) =grJ.= 6: gr D(V)S, et gr U = grU vérifie la

condition (S2).

462

THIERRY LEVASSEUR

Preuve. La première assertion résulte de ([Mu], Theorem 4.4). La seconde vient de la première et du fait que gr D(V) = C[T,r]® est de Cohen-M acaulay.

Théorème. Soit G — GL(V) une représentation linéaire fidèle de G=C*. Si htA(V,G) > 2 alors (V,G) a la propriété de relèvement. Preuve. On voit facilement que l’on peut supposer VS = (0), dimV > 2, et dimV//G = dimV — 1. Donc en particulier a; # 0 pour tout J, si

l’on adopte les notations précédentes. Le fermé F(V,G) = V — Q(V,G) est stable sous l’action du groupe des matrices diagonales de GL(s,C), ce qui implique que les idéaux premiers minimaux sur A(V,G) sont de la

forme Pe = (Ti;i € E)C[T] où E C {1,...,s} (l'inclusion est stricte car # htA(V,G) < dim V//G, cf. 2.3). Comme o ¢ Pg, pour E comme ci-dessus,

on en déduit que si A = A(V//G):

ht(A, o)CIT, r]% > ht(A, o)CIT, r] = ht(A(V, G), o)CIT, 7] = htA(V,G) +1 > 3. Le lemme précédent permet alors d’appliquer (3.6, Théorème) pour avoir le résultat.

Remarques.

(a) Dans les hypothèses du théorème on peut montrer que

la condition htA(V, G) > 2 est nécessaire. (b) Rappelons que [Mu] donne des conditions nécessaires et suffisantes pour que (V,G) ait la propriété de relèvement, ceci lorsque G est un tore quelconque.

5. Opérateurs différentiels sur les anneaux classiques d’invariants 5.1 On fixe une C-algèbre U munie d’une filtration {UT }mez croissante, exhaustive et discrète :

US = 0 sm

< 0; CCU

CUS CE

CUM

C=

UU

UE

On suppose également que U est graduée par {Uz}ecz : U = © Us, t Cc C Uo.

RELEVEMENTS D’OPERATEURS DIFFERENTIELS

463

On fait dans ce qui suit ’hypothése que la filtration et la graduation sont compatibles, c’est a dire :

Vm>0,U™=6

(U™NY).

LEZ

On pose alors : Uy” = U™ MU; et pour tout d € Z, U(d) = & Uf‘.

LEZ ‘3 Um. Il est facile de t+mGrA

mod

We

denoted by a += @, where @ = a

7, cata) A“) € Gra(a)A-

Given s € LE

denote by UW“) the linear span over C(q) of the

monomials M;,., such that d(Mi +4) < s. We define us?) C U, similarly.

PROPOSITION

1.7.

(a) TheU“), 5€ Zot, form a filtration of U (similarly for U, ). (b) [13] Elements E*, k € ZŸ, form a basis of Ut (resp. U+) over

C(q)(c) (resp. C). Elements

F*¥K1™...K™*E", where k,r € ZŸ, (m1,...,m) € Z”, form a basis of u over C(q) (resp. a basis of U. over C, provided

thate

tr Al for tn,

(d) The associated graded algebra GrU (resp. Grue: provided that e?di 41, i= 1,... ,n) is an associative algebra over C(q) (resp. €) on generators Ey, F, (a € Ry), K*!(i=1,... ,n) subject to the following relations:

Kp 1G, Ki; = 1, RES SUR ES, K; Eq = "Eg Ki, KiFa = CRUEK;, EqEg = QCM) EE, FaFs = QC) FgFa, ifa > B, (resp. same relations with q = €). PROOF:

It follows from (1.2.3) that

nation of the My,

Eg Fg = FgEq + (linear combi-

of degree less than

d(E Fg).

This together with

Lemma 1.7 imply (a) and the relations in (d). The fact that the E* span

480

DE CONCINI AND KAC

U+ (resp. Ut) follows also from Lemma 1.7. Their linear independence is proved as in [13]. This proves (b) and (c). (d) now follows from (c).

i) REMARK 1.7. (a) Let 8 = wa;. Applying T, to both sides of (1.3.1) and using (1.5.4), we see that (1.3.1) holds if we replace E;, F; and K; by Es,

Fe and

Kp.

(b) For different presentations tional. For example we have:

(1.7.2)

= wa; the Eg may be not propor-

By = T)E; = -E;E; +97 EE;

if aj; = —1.

1.8. An algebra P over € on generators z1,...,zx and defining relations 232; = A;;2;2; for 1 > j where À;; € C*, 7,7 = 1,... ,&, is called a quasipolynomial algebra. One may introduce a N*-gradation in P by letting degz; = (6;,... ,6;,). It is clear that P is spanned over C by monomials 27! ...2;*. Moreover, they form a basis of P. This follows by looking at the representation a of P in the polynomial algebra

C[t,,... ,tx] defined by —1

(C2) Re

SP ED) PATES Got HR ER ea

As usual, this implies that P has no zero divisors. Furthermore, let A be a commutative algebra on generators K1,...,K, with no zero divisors. Let P4 = A @c P with multiplication given by K;x; = pi;x; Ki, pi; an) C*. We may extend the N*-gradation from P to Py by letting deg K; = 0. This gives P, a structure of a free left A-module with basis x”? := tri, Mme hl It follows that P4 has no zero divisors as well. Hence GrU/ and Gr, have no zero divisors by Proposition 1.7d, and we deduce the following corollary.

COROLLARY

1.8. The algebras U and U. have no zero divisors.

O

Let A be an algebra with no zero divisors, let Z be the center of A

and Q(Z) the quotient field of Z, and let Q(A) = Q(Z) @z A. We shall call the algebra A integrally closed if for any subring B of Q(A) such that

AC BC

z7!A for some z € Z, z # 0, we have B = A. It is clear

that for a commutative algebra A this definition of an integrally closed algebra implies the usual one.

REPRESENTATIONS

OF QUANTUM

GROUPS

481

PROPOSITION 1.8. Let A be an integrally closed algebra with no zero divisors. Let C be an S-filtered algebra such that Co = A and Gr C = Pa. Assume

that each generator x; of P C PA has a preimage Z; in C

such that Z lies in the center Z of C. Then the algebra C is integrally closed.

PrRooF: Let z € Z, z # 0, and let B be a subalgebra of z~!C containing C. Let p € B, so that y := zy € C. We can write: y = az” + lower degree terms, where a € A,

hE Z*,

z = bz” + lower degree terms, where b € A, r€ Le

Pick N € Z such that N£ > h; for each 7 and set

t=(N—hy,...,N—hy)

€ 74.

Since x*z' = Ax"+*, À ECC*, we get:

Y := A~lyz' = az"** + lower degree terms

= A~!zpz'.

Letting v = \~!yz' € B, we get Y = zv. For each m € N we have:

(1.8.1)

ee

le nc,

Since, by the assumption, z*+* € Z, we get:

(1.8.2)

Y™ =a™z™("+4) 4 lower degree terms.

But 27 = z/-1(bz" + lower degree terms) = bz/—!Z" + lower degree terms. Hence we have by induction: (1.8.3) 21

— pm 1) L lower degree terms, where À,, € C*.

Since Gr,C is a free A-module of rank 1 generated by z°, s € Zi we deduce from (1.8.1), (1.8.2) and (1.8.3) that b”-1 divides a™ for all m € N. It follows that the subalgebra of Q(A) generated by A and a/b is contained in b-! A. Since A is integrally closed, we deduce that b divides a.

Note now that y" = z™—!(zp™), where zp™ € C. It follows that mh; > (m— 1)r; for all m € N and all i = 1,...,k. Hence h; > r; for all 2. Suppose now that B # C and choose y € B\C such that y= zp EC has minimal possible degree. Let c = a/b, s = h — r and consider the element 1

yi =p—p

CE,

where p € C* is taken from x” = yz'x°. But y’ € B\C and d(zy’) =

d(y — u7!cz&°) < degy, a contradiction. An immediate corollary of Propositions 1.8 and 1.7d is

CO

482

DE CONCINI AND KAC

THEOREM

1.8. The algebra U, is integrally closed.

O

1.9. Let Q3 be the group of all homomorphisms of Q to the group {+1}. Given À € P and 6 € Q3, define the Verma module of type

6, M*(A) over U (which is a vector space over C(q)) as the (unique) U-module having a vector v\ such that Utv

= 0, K;v1 =

5(a;)qO!*) vy,

and the vectors FF

(k (S 25)

form a basis of M*(A). Such a module exists due to Proposition 1.7c. Any quotient V of M®°(X), called a highest weight {-module, admits a weight space decomposition.

V = @ncq,Vn, where V, = {v

UNE

EV|Kjv = CAT

We denote the (unique) irreducible quotient of M$(XA) by L°(A) = M(à)/J(à). Given a highest weight module V over the algebra WU, let v\ be its highest weight vector and consider its {4-submodule (which is an Amodule) V4 =U,v,. This gives rise to a Uy—module V, = V4/(g—-1)V4. The K; act on Vi as 6(a;), hence by Proposition 1.5 the module Vi becomes a highest weight module with highest weight vector vy over U(g(ai;)), the enveloping algebra of the simple finite-dimensional Lie algebra g(a;;) over € associated to the Cartan matrix (a;;).

LEMMA 1.9. [12]. Let V be a Verma module of type 6 or an irreducible highest weight module with highest weight À over U and let n € Q:. Then dime) V, =

dime (V3) py.

O

Given a = a(q) € C(q), let à = a(q~'). A C-bilinear form H on a vector space V over C(q) with values in C(q) is called Hermitian if

(1.9.1)

H(au,v) = aH(u,v), H(u,av) = aH(u,v), and

GTS

A(u,v) = H(v,u), a E C(q), uve V.

The {-module M°(X) carries a unique Hermitian form A, called the contravariant Hermitian form, such that

(1.9.2)

;

H(vy,v,) = 1 and H(gu,v) = H(u,w(g}v) for g EU, u,v e M*(à).

REPRESENTATIONS OF QUANTUM GROUPS

We have: H(M°(\),,

483

M*%(A),) = 0 if p # v, and Ker H = J()).

Denote by H, the restriction of H to M*°(X),, n € Q4, and let det? (X) denote the determinant of the matrix of H, in the basis consisting of elements F*v,, k € Par (n). Here for 7 € Q we denote by Par (n) the set of all k € 7À) such that 5°; k;B; = n. Also, for given 6 € Q3, we

view Kg as a function on P defined by Kg(À) = 6(8)q!®); any y € U° thereby becomes a function on P with values in C(q), which we denote

by p°(à).

Given 6 € R* andr EN, let Tg = {A € P[2(A + |S) = r(B|B)}, and let TY, = {À € Trg |2(A + ply) # m(y1|7) for all y € Rt\{G} and MEN such that my < rB}. Note that if 6 € U° and y vanishes on iby, then y vanishes on 7,5.

PROPOSITION

1.9.

(a) dety = [pert Tmen((mla, (Ko; (018) — (818) Pertr-mP A,

(b) If A € TP, r EN,B € Rt, then M°(X) contains a submodule isomorphic to M*(A — r). PRoor:

Let

1€ T,g, r EN, BE Rt. By Shapovalov’s formula [22] for

the determinant of the contravariant form of the U(g(A))-module M (A) it follows that dime(L*(A)1)-3 < dime(M*(A):)-g. Hence by Lemma 1.9 we have: dimc(y) FO)

< dimeq) M*(A),g. It follows that det,g

is divisible by [Kg; (p|B)—5(818)] and that there exists a non-zero vector v € M*(A),g NJ*(A), such that U+v = 0, provided that A € Ser Since U- has no zero divisors it follows that v generates a submodule of M5()) isomorphic to M°(A — rf) which lies in J°(A), proving (b). Hence det, is divisible by [Kg;(p|2) — £(B|p))!Parn—r A Thus, det, is divisble by the right-hand side of the formula in question. To complete the proof of (a) we calculate the leading term of det,.

Using formula (1.3.1) it is easy to see (as in [22]) that it equals to

1)0) (m-4(a LL LI (role,LA Parcr-mat-tPan BER+mEN

which proves (a).

0

Let ¢ € C*. Given ahomomorphism À : U2 — C, we define the Verma module

M.(A) over U. as a vector space over C with an action of U,

having a vector vx such that U+v, = 0, uv, = A(u)v, for u € U? and the vectors F*v, (k € ZŸ) form a basis of M,(A). Such a module exists due to Proposition 1.7c. The module M,(A) admits a Q4-gradation M,(A) =

484

DE CONCINI AND KAC

PneQ+ Me(A)n, where M.(X), is the linear span over C of the FFvx with

k € Par(n). Note that M.(X), C {v € M-(A)|Kiv = A(Ki)e7 I) v}, and that this inclusion is an equality if € is not a root of 1. Note also

that M1(A)/(q—€)M1(A) = M.(X), where A,(Ki) = Ella), Let now defined as form). We on M,(A),

|e| = 1. The contravariant Hermitian form H on M,(A) is above (see (1.9.1) and (1.9.2)) (H is then a usual Hermitian define similarly det, -(A)(n € Q4) as the determinant of H in the basis F*v,, k € Par(n). Proposition 1.9a gives us

immediately the following formula (we assume that e?4i # 1 for all i):

(1.9.3)

det) = I] I (GS) emde

BERt mEN

ene e~mds

(E

.

|Par(n—m8)|

)

x (A(Kg)elO)- FEI) — N( Kg) te IEEE E18))Par(n=m8)le We define the U.—module L.(À) as the (unique) quotient of M,(A) by

the maximal Q4-graded submodule J,(A); this module is irreducible. Formula (1.9.3) implies the following

CoroLLARY 1.9. Let \(K;) = Ale),

i= 1,... ,n, where À € C @z P

and € is not a root of 1.Then M,(A) is an irreducible U.—module if and only if 2(A + p|B) # m(B18) for allmeEN, BERT. O

REMARK

1.9.

Using the usual argument (see [5] or [8]) one can

show that Proposition 1.9b holds for all À € Tg and derive the usual conditions for inclusions of Verma modules over U and occurence of

Lé(u) in M°(X) (cf. [1]). One can also prove similar results for the Ue-modules M.(À) provided that € is not a root of 1. 1.10. Let a;; = —a. Introduce the following notation: Er

= HE; Hoa=l

Se

E;; = (Tiny E;i; =

8; if,

1;7;E;

de

201), (a= 2:0r 3;

2, E;;; = T;1;7;E;

i a=

Ej = TiT;E; and Ey; = (TT)E; ifa= 3. Then one has the following important identities:

(1.10.1)

Ei

Ene DD k=0

eat ER CAES k times

3

REPRESENTATIONS

(1.10.2)

OF QUANTUM

GROUPS

485

a

Ee

ee ee Pe

a

k=0 k times

(1.10.3)

EYES = gf BO Ef + qf EF + [s]Ayj(q), if ays = —1,

where À;;(q) € U has no poles except for g = 0 and q = €, where ei = 1 forj EZ, |j|

F'orrE", where or, € U°.

nEQ+ k,rEPar(n)

As usual, the map z += 0,9 is a homomorphism h : Z — U°, called the Harish-Chandra homomorphism. Fix an element z € Z of the form (2.1.1). It is clear that z € Z if and only if z acts as a scalar equal to ¥0,9(A) on each Verma module M°*(X), AE P, 6 € Q5. Denote by ¢, the matrix (Pk,m)kmePar(n): We

will compute the matrix ¢, by induction on n € Q;. Denote by G$(,)

the matrix of the operator 3°},epar(y) Fy, ,E" on M*(A), in the basis F*v) (s € Par(n)). By the inductive assumption, we know G, for + < 7. We obviously have

(212)

Gy = $n Hn.

Hence the fact that z acts as a scalar y§,(A) on M°(X), can be written as follows:

(2.1.3)

bn Hy + >, Gy = pool. M*(A — rf) provided that AE BEN hence CT) — rf) = 95 o(A) ifA € la It follows that p$o(À — rB) = £$ o(À) for all À € T;g, hence (2.2.2) is necessary. Conversely, suppose that (2.2.2) holds. As has been mentioned above,

(2.2.4) holds for 8 € Rt and r € N. We shall prove by induction on n that the denominator of km with k,m € Par(n) does not contain factors [K'5;(p|B) — 5(818)]. In view of (2.1.4), this will imply the sufficiency of (2.2.2) for the ym to lie in 4°, and also will complete the proof of (b). Let Th, = {A € Trgl2(0 + ply) # m(yly) for all y € R+\{B} and m € N such that my < n. Let € theySe then, by Proposition 1.9, M*(X) > M*(A — rB) and, moreover,

(2.2.5)

M), N J°(à) = M90 —rB)n-re:

It is clear that z coincides on V := M°(A — rf),_-,g with the operator yen Lk mePary F*PkmE™ (and is defined on this subspace by the inductive assumption). On the other hand, z acts on V as the scalar (À — r@), which is CHE) by assumption (2.2.4). Hence the matrix B5()) := ph o(A)I — 1,0, GA) is zero on V. Thus, by (2.13) and Proposition 1.7a) we have an equality of matrix-valued functions on V:

$5(A)H3 (A) = B°(A), A EP, such that KerH?(A) C KerB*(X) (by (2.2.5)) for A € Ty,,, hence for all À € Trg, and dim KerH}(A) = multiplicity of zero of det H}(A) for AE Tega: Hence, using [7, Lemma 2], we see that the matrix ¢, has a removable singularity in the hyperplane T;g, as desired. Inequality (2.2.3) is clear by looking at degrees in (2.1.3), where we leideg i= 1nd 1, is 5: O

2.3. Let now € € C* be not a root of 1. Then for every y € ULW we denote by zy, the element zy in which q is replaced by €. This is an element

of the center

Z. of U,.

Defining the Harish-Chandra

homomorphism h, : Z, — U2 as in 2.1 and y: U? — U? by 7. Ki = ediK; we obtain from Proposition 2.2 that yr! oh. : Z.UIW is an isomorphism, the map y+—> Zy,- being the inverse homomorphism.

488

DE CONCINI AND KAC

§3. The center of U, and the group G, when € is a root of 1.

3.1. Let € be a primitive £-th root of 1. Let @ = £ if £ is odd and =

ze if 2is even.

We shall assume that

(3.321)

CoS d= max{d;}

Then U, is the algebra over C on generators E;, F;, K; and Kj 1 and

defining relations (1.2.1)-(1.2.5), where q = €. LEMMA

(SE)

3.1. The following relations hold in U,:

EY Eg =e CEE,

213

(3.1.4) PROOF:

FY Fy = (oll!

pe

Et Fs = Fekt, FLa Es“Bp = E3F*, arB Ba B

Kf Beale lOO ER

Ke Pye

ee BORG

It suffices to check the relations (3.1.2) and (3.1.3) for a = a;

and B = a; since these relations for arbitrary a, € Rt follow by using

automorphisms 73; (3.1.4) follows immediately from (1.2.2). If a = a; and 8 = a;, formula (3.1.3) follows from (1.3.2), formula (3.1.2) for the E’s follows from (1.10.1) and for the F’s it follows by applying the automorphism ¢. O Denote by Z, the center of U,.

COROLLARY 3.1. (a) All elements Ej, Fy (a € R*) and K£ (BE Q) lie in Z,. (b) If Lis even, then all elements Ee Be and K5 lie in Z, iffA is of

type By (ise Ue

el

3.2. Consider the Verma module M,(A) over the algebra U,. formula (3.1.3), we have:

By

Ut Fev, =0, ae Rt. It follows that Fv, € J.(À). Let M.(A) denote the quotient of M.(A) by the U,-submodule generators by all the vectors Fev), a € Rt. We call M.(A) a diagonal U.—module. Proposition 1:7b and formulas

(3.1.2)-(3.1.4) imply

REPRESENTATIONS PROPOSITION

3.2.

(3.2.1)

Fe Bion Fay en

OF QUANTUM

GROUPS

489

Vectors

where m; € Z4, m

form a basis over € of the space M,(X).

< t's

O

It is clear that M,(A) is irreducible if and only if M.(A),, n # 0, contains no singular vectors, i.e. no non-zero vectors v such that E;v =

0, i=1,...,n. Let À € C @z P be such that X(K$) = EC). formula (1.9.3) and Proposition 3.2 we obtain

From

THEOREM 3.2. The U,-module M,()) is irreducible if and only if

(3.2.2)

e2A+P18)-m(818) #1

for all BE Rt andmezZ, suchthatm m?, but the dimension of every irreducible representation of UX is less than m.

Proor: Let py: Z — C be a point of Spec Z. outside D. Then there exist elements u1,... ,Um2 in 4. such that p(det(u;,u;)) #0. We have: D := det((u;,u;)) € Z.; let d = y(D). Consider the algebra À :=U, @4 C. We claim that the elements %; := u; @ 1 form a basis of

A over C. Indeed, the u; being linearly independent over Q(Z,), form a

basis of Q(U.) over Q(Z.). Let {u'} be the dual basis with respect to the trace form. Then Du’ EU. for each i. For u € U we have:

Du= So (Du, u')ui = DC Du')uj. i i Passing to A we get:

du = 2s y((u, Du'))x. Thus, the @ span A over € and det((w;,u;))

# 0. Hence A is a m?-

dimensional semisimple algebra over C. On the other hand, each element of U, satisfies a polynomial of degree m with coefficients in Z,, hence each element of A satisfies a polynomial of degree m with complex ccefficients.

It follows that

A = Mat m(C) (since m? = n?+n3+...,

ny +n2+... 0 and

imply that m = n; for some i).

Let now p: U, — Matm(C) be an irreducible representation and let y = X(p): Ze — € be the corresponding element of Spec Ze. Let U1,... ,Um? be elements of A. such that p(u;) form a basis of Matm(C) over €, and let D = det((u;, u;)). Then (D) = det(tr p(ui)p(u;)) À 0, hence y ¢ D.

498

DE CONCINI AND KAC

In order to prove (b) notice that since each element of U, satisfies a polynomial of degree m with coefficients in Z,, every irreducible repre-

sentation of U. has dimension < m. This and (a) give the second part of (b), while the first part follows from the fact that U, is a torsion free rank m? module over Ze.

O

3.8. Consider a finite-dimensional irreducible representation 7 of U, in a complex vector space V. Since Zp acts by scalar operators on V, ma extends in an obvious way to U.. Given g € G, denote by 79 the “twisted” (irreducible) representation of U, in V defined by m(u) = r(gu),

vue.

Note that

Nas Xr 0979 CG: We call the representation m diagonalizable (resp. triangulizable) if there exists g € G such that 79 is a diagonal (resp. triangular) representation.

Denote by { the set of all À € Spec Z such that (r - X)~!A consists of €” irreducible diagonalizable representations of (maximal) dimension EN. Note that by Corollary 3.2a it follows Ho C Q. corollary of Lemma 3.6 and Corollary 3.2a is LEMMA

An immediate

3.8. { contains a non-empty open (in metric topology) subset

of Spec Zp.

O

Since, due to Corollary 3.3b:

(3.8.1)

dimgiz,) QU.) = N*",

Lemma 3.8 gives us immediately (3.8.2)

(3.8.3)

dimg(z,) Q(Z-)

= 1428

im = diniotz. QU) = to

The following theorem summarizes the obtained results.

THEOREM

3.8. Let { be an odd integer, { > max;{d;} and let € be a

primitive €-th root of 1. (a) Spec Ze is a normal affine algebraic variety and the map Tr : Spec Ze. — Spec Zp is finite (surjective) of degree ”.

REPRESENTATIONS

OF QUANTUM

GROUPS

499

(b) If x € Spec Z.\D (where the discriminant D is a closed proper

subvariety), then X—1(x) consists of a single irreducible representation ofU, of dimension € . If xED, then X~!(x) consists ofa finite number of irreducible representations of U, of dimension less than {N.

O

REMARK 3.8. Let the Cartan matrix A be of type B,, n > 1. Let £ be a positive even integer, and let, as before, ¢’ = £/2. Then elements

EC ,F£ (a € Rt) and ie (8 € Q) lie in the center Z, of the algebra U: (by Lemma 3.1). Denote by Zp the subalgebra generated by all these elements. Then all the results proved above still hold if £ is replaced by L'. In particular:

(3.8.4)

dimorz.) QUe) = 27%, dimgcz,) Q(Ze) = L”.

3.9. Given yg € UO”, let poo = y(e" Ki,... ,€2" Ky). Furthermore, for each pair, k,r € Par, construct elements ÿ4,. by formula (2.1.3), in which q is replaced by €. By formula (1.9.3) and the argument proving Proposition 2.2 (in which Verma modules are replaced by diagonal modules), we see that gx:

€ U2. We let

AD

AIR

k,r€Pare

LEMMA

3.9. Zye € Ze.

ProoF:

Note that the intersection of kernels of all triangular modules

ic

Pent Tale.

=

Since, by the construction,

z,,. acts on each

triangular module as a scalar, we obtain: [zy.-, Fi] € M. It follows, using Corollary 3.3b, and commutation relations (1.2.1)-(1.2.3), that [Zv.e, Fi] = 0. Since w(z,.) = 2, (see §2.1), we obtain that also (2

E;] 0:

O

Thus, we have an injective homomorphism U2” —

Pr

Z, defined by

ype.

§4. The example ofU and U, of type A. 4.1.

We consider here the simplest example,

that of the quantum

algebra U/, associated to the matrix (2), which first appeared in the work of Kulish-Reshetikhin and Sklyanin. This is a Q(q)-algebra on generators E, F, K and K~! and defining relations

(4.1.1)

i

aie hhh = qbeK eR

= 9° F,

500

DE CONCINI AND KAC

(4.1.2)

EF —-FE=(K—K7™!)/(q-q‘).

The center Z of U contains the well-known element Kq+

Cae

|eesket pee

TEFEUL

4.1.3

c= ————

+ FE.

Note that c = z,, where p = (K +K—')/(q—q7')? (see § 2.2). Since y generates U°™ |we It is well-known over Q(q) of U are sion n + 1, which Can 0 S044): (4.1.4)

deduce that c generates Z. that all finite-dimensional irreducible representations equivalent to representations a+, n€Z4, of dimenin some basis vp,... , Un are given as follows (we let

nt (Ko; = +q"~%0;, nE(E)v; = +[n—j4l]oj_1, t*(F)v; = [j+1]v;41. These facts still hold for the specialization of g to any complex number different from 0 and a root of 1.

4.2. Let now £ > 2 be an integer, and let, as before £’ = @ if £ is odd

and £’ = ¢/2 if £ is even. Let € be a primitive £’th root of 1, and let e’ =e if £is odd and = e? if £ is even. Denote by 4. the algebra over C on generators E, F, K and K~! and defining relations (4.1.4), (4.1.2) where q is replaced by €.

generated by x = Et, (3.8.3) and (3.8.4): (4.2.1)

Let Zp be the subalgebra of the center Z,

y= Ft, z = K® and z~!. We have by (3.8.2),

dimo(z.) Q(U.)

=

C2:

dimg(z,) Q(Z:)

=

L'.

Note that the norm of an element f(K) € U° over Q(Z.) can be calculated as follows:

ist

(4.2.3)

N(f(K)) = [J SEK). j=0

We also have:

(4.2.4)

N(E) = (-1)#1z, N(F) = (-1)¢ty.

Denote again by c the element of Z, given by formula (4.1.3) where q is replaced by €. Taking norm of both sides of the equalit Ke+K-te7}

AE Parsee

;

REPRESENTATIONS

OF QUANTUM GROUPS

501

we obtain:

ae

Kee’ mr eu nt = at) Say

(4.2.5)

The left-hand side of (4.2.5) is of the form: ct + act =! Te eat gia

+74,

where, clearly, a; € € fori=1,... ,@’—1, and

al)

(CA) (ere).

Hence we can rewrite (4.2.5) in either of the following two forms, where cr EC: £'-1

(4.2.64)

ee

Whe cj )=zy+(-1)

+2 -1 2

12

(e—e-1)2€"

In order to calculate the constants cF(e), consider the representations CE of U, over C defined as follows. If £ is odd (resp. even), ci (resp. my’) is defined for each n = 0,1,... ,€— 1 by formulas (4.1.4) for 7* (resp. +), where q is replaced by € and 0 Ly € Pica(Y(F)) est affine. Soit F une face ouverte de C. Nous montrons que tout point semistable pour F est stable ; donc si X est lisse, alors Y(F) n’a que des singularités quotients par des groupes abéliens finis. Les quotients associés aux faces ouvertes sont donc relativement simples. Lorsque F est une face quelconque, soit F’ une face ouverte dont l’adhérence contient F. Nous

définissons en 1.4 un morphisme mppr : Y(F’) > Y(F) , presque toujours birationnel. En 1.5, nous étudions le cas ot F est de codimension un, con-

tenue dans l’adhérence de deux faces ouvertes F_ et F,. Les morphismes TF,F_ et Tp,r, sont en général assez compliqués ; par contre, si Y désigne

le produit de Y(F_) et Y(F}) au-dessus de Y(F), les morphismes de Y vers Y(F_), Y(F) et Y(F,) sont des éclatements de sous-variétés, avec le méme diviseur exceptionnel (voir 2.3 pour un énoncé précis).

Prolongeons a C l’application affine

p —

Lp de Fy vers Picg(Y+).

Relevons chaque L, à Y, en une classe L*Pp ;d définissons de même L>. Nous montrons en 2.3 que la différence pe — L, est un multiple du diviseur

exceptionnel de suite, lorsqu'on faces, au-dessus recollent en une

l'éclatement Y > Y(F), et est nulle pour tout p € F. Par considère le produit Ÿ des quotients associés à toutes les de tous les morphismes Tr r', les applications p — L, se application affine par morceaux, et continue, de C vers

Pica(Y).

Dans la troisième partie de ce travail, nous appliquons ces résultats a l’étude asymptotique des modules multigradués ; nous considérons diverses extensions à ces modules des notions de multiplicité et de fonction d’Hilbert-Samuel pour un module gradué. Plus précisément, nous considérons une algèbre de polynômes A, graduée par N x Z! (la graduation par N étant définie par le degré des polynômes). Notons T le tore ayant Z! comme groupe des caractères. Tout A-module gradué par N x Z! définit

ACTION D’UN TORE

511

un faisceau T-linéarisé M sur l’espace projectif P(V) associé à A. Soit II l’ensemble des poids de T dans V, c’est-à-dire des opposés des Z!-degrés des générateurs de A. Pour toute face F associée à II, soit A4(F) le faisceau

formé des invariants de T dans (rr),(M), où mp : X°*(F) — Y(F) est le “quotient”.

Lorsque M est un A-module de type fini, nous montrons que

M) est un faisceau cohérent sur Y(F). Pour tout p € F, notons um (Pp) le degré de MF) relativement à la classe ample L, (voir [Kle; 1.3]). Nous montrons que la fonction y est polynomiale sur chaque face. En outre, elle intervient comme densité dans diverses expressions intégrales d’invariants asymptotiques de M : des “polynômes d’Hilbert-Samuel généralisés” (3.5),

les “polynomes de Joseph” ([Jos]; voir 3.6). Lorsque M est l’anneau des coordonnées homogènes d’une sous-variété projective lisse X de P(V), la fonction uw est continue, et sa transformée de Fourier ne dépend que des

points fixes de T dans X, et de leur fibré normal (3.4). Enfin, dans un appendice, nous exposons briévement les liens entre nos résultats et des théorèmes dus à Atiyah, Duistermaat-Heckman,

Guillemin-Sternberg en géométrie symplectique (voir [Ati], [DH1&2], [GS1 &2]). Il y a en effet d’étroites relations, étudiées dans [Kir], entre l’action algébrique d’un tore complexe dans une sous-variété X d’un espace projectif, et l’action hamiltonienne de son sous-tore compact maximal 7; dans X vue comme variété symplectique (la forme symplectique étant la partie imaginaire d’une forme kählerienne invariante par T, sur P(V). Les objets de notre étude s’interprètent en termes de l’application moment J : son image s’identifie au polyèdre convexe C; les faces ouvertes sont formées de valeurs régulières de J; le quotient Y(p) est “l’espace des phases réduit”

J71(p)/T.; la classe de Lp dans H?(Y(p), Q) provient de la classe de cohomologie de la forme symplectique.

De plus, si M est l’anneau des coor-

données homogènes de X, la fonction py est la densité de la mesure image de l’application moment. Après avoir obtenu les résultats de ce travail, nous avons pris connaissance (en mai 1989) de l’article [GS3] de V. Guillemin et S. Sternberg, qui démontrent des résultats voisins de ceux de notre deuxième partie, dans le cadre des actions hamiltoniennes de tores compacts dans les variétés symplectiques. 1. Quotients par un tore.

1.1. Notations. Soit T un tore opérant linéairement dans un espace vectoriel V (le

corps de base k étant algébriquement clos). Soit X une sous-variété fermée irréductible de l’espace projectif P(V), stable par T.

On suppose pour

512

BRION ET PROCESI

simplifier que T opère fidèlement dans V et ne contient pas les homothéties, et que de plus X n’est contenue dans aucun sous-espace linéaire de P(V). On note X(T) le groupe des caractères de T, et E (resp. Eq) l’espace

vectoriel X(T)@z R (resp. X(T) @z Q). Soit V = ®yexcr) Vy la décomposition de V en sous-espaces propres de T.

Notons

II l’ensemble des x

tels que V, # {0}, c’est-à-dire l’ensemble des poids de T dans V. Pour tout x € P(V), on note £ un représentant de x dans V, et 5 = ÿ_v, sa décomposition en vecteurs propres de T. Notons II(z) l’ensemble des x € II tels que v, # 0 ; on dit que c’est l’ensemble des poids de z. Pour tout p € E, soit F(p) l'intersection des simplexes dont tous les sommets sont dans II, et qui contiennent p. C’est un polyèdre convexe dans EF, à sommets dans Eg. Notons F(p) son intérieur relatif, c’est-àdire l’intérieur de F(p) dans l’espace affine qu’il engendre ; appelons F(p) la face de p. D’après le théorème de Carathéodory [Val; Theorem 1.21], tout point de l’enveloppe convexe C de II appartient à un simplexe ayant tous ses sommets dans II. Par suite, les F(p)pex forment une partition de C. De plus, toute face est contenue dans l’adhérence d’une face ouverte. On en déduit que toute face de codimension un est contenue dans l’adhérence d’au plus deux faces ouvertes. Tout hyperplan affine engendré par une face de codimension un, est aussi engendré par / points de II, où / est la dimension de E.

1.2. Faces et quotients.

Pour tout p € E, on note X**(p) (resp. X*(p)) l’ensemble des x € X tels que l’enveloppe convexe de II(z) contient p (resp. contient p dans son intérieur). Il est clair que ces deux ensembles ne dépendent que de la face de p ; pour toute face F, on peut donc définir X**(F) et X*(F). L’énoncé suivant est évident. Proposition. (i) X*(F), X**(F) sont des ouverts de X, stables par T.

(ii) X**(F) est non vide & (ili) X°(F) est non vide & (iv) X**(F) = X*(F) si F P(V) et X**(F) = X‘(F),

F est non vide. F est dans l’intérieur de C. est ouverte dans E. Réciproquement, si X = alors F est ouverte dans E.

Un point de X°(F) (resp. X**(F)) est dit stable (resp. semistable) pour F’. Le théorème ci-dessous, et sa démonstration; justifient cette terminologie. Théorème.

II existe une variété projective Y(F ), avec action triviale

ACTION D’UN TORE

513

de T, et un morphisme affine T-équivariant mp : X**(F) + Y(F), tels que Oy Fr) soit formé des invariants de T dans (™F)»(Oxee(7)). La restriction de mp à X*(F) a pour fibres les orbites de T. Démonstration. Remarquons d’abord que si 0 € C, alors X**(0) (resp. X*(0)) est l’ensemble des points semi-stables (resp. stables) de X C P(V) ; cela résulte en effet de [MF;Theorem 2.1]. Soit p un point de F'M Eq (un tel point existe car F est un polyédre convexe à sommets dans Eq). Ecrivons p = x/n où n est le plus petit entier positif tel que np € X(T). Définissons une action linéaire de T

dans la n-ième puissance symétrique SV par: t-m” = x(t)=!(t: m)". Plongeons X dans P(S”"V) par le plongement de Veronese. On vérifie immédiatement que X**(F) = X**(p) est l’ensemble des points semistables de X C P(S"V), et que X*(F) est l’ensemble des points stables. L’énoncé résulte alors de [MF; 1.4]. Remarque. Dans le langage de [MF; Chapitre 1], que nous adopterons, mp est un quotient catégorique universel, et sa restriction à X*(F) un quotient géométrique universel. 1.3. Une classe de faisceaux sur les quotients.

Le fibré en droites O(1) sur P(V) se restreint 4 X en un fibré Tlinéarisé, noté encore O(1). Pour tout caractère y de T, notons O(x) le fibré en droites T-linéarisé suivant : c’est le fibré en droites trivial sur X,

et T opère dans chaque est un homomorphisme classes d’isomorphisme cet homomorphisme de Reformulons

fibre via le caractère —x. L’application x — O(x) de groupes, de X(T’) vers PicT(X) (le groupe des des fibrés en droites T-linéarisés sur X). On étend Eq vers PicT(X) @z Q, en p > O(p).

la démonstration

du théoreme

1.2 dans le langage des

fibrés en droites. Soit A = 6&2 ,I'(X, O(n)); c’est une algèbre graduée dans laquelle T opère. D’après [MF; 1.11], on a Y(0) = Proj(AT) où AT est l'algèbre des invariants de T dans A. Donc Y (0) est muni canoniquement de faisceaux notés O(n) pour chaque entier n. D’après [EGA IT; 8.14.4], il existe un entier n > 0 tel que O(n) soit inversible et très ample. Soit

[O(n)] sa classe dans Pic(Y(0)). Posons [O(1)] := 1/n[O(n)] ; c’est un élément de Picg(Y (0)) :=Pic(Y(0)) ®z Q. D’après [EGA IT; 8.14.12], cette classe ne dépend pas du choix de n (voir [Dem] pour plus de détails sur cette construction). Par définition, [O(1)] est la classe de (ro)? O(1) dans Pica(Y(0)) (image directe invariante de O(1)). Plus généralement, soit p = x/n un point rationnel de C. Remarquons que X**(p) est l’ensemble des points semistables de X associés au fibré

514

BRION ET PROCESI

T-linéarisé O(n) @ O(x). Par suite, Y(p) est le Proj de l’algèbre graduée

@%_ oI(X, O(mn) ® O(mx))” = ORaol(X, O(mn))my (on rappelle que pour tout T-module rationnel W, et tout caractère # de T, on note Wy le sous-espace des vecteurs propres de T dans W, de poids ~). On a donc sur Y(p) un faisceau canonique O(1), qui dépend de x et de

n. Posons [O,(1)] = 1/n[O(1)] dans Pica(Y(F)); cette classe ne dépend que de p.

Proposition.

Soit p un point rationnel d’une face F. Alors [O,(1)]

est la classe de (mr)! (O(1) @ O(p)) dans Picg(Y (F)). Démonstration. Par définition, O, (1) est l’image directe invariante du faisceau canonique sur le Proj de ®%_4T(X,O(mn) © O(mx)), c’est-à-dire Oy n(1) = (t)2 (O(n) @ O(nx)). L’énoncé s’en déduit aussitôt.

Corollaire. Pour toute face F, l’application restriction d’une application affine. Démonstration.

p —

[O,(1)] est la

Soit y/n un point rationnel de l’espace affine en-

gendré par F. On vérifie immédiatement que la restriction de O(n) @ O(y) à toute orbite de T dans X°*(F) est le T-fibré en droites trivial. D’après [Kra2; Proposition 3], il existe un fibré en droites £, , sur Y(F) tel que O(n) ® O(x) = TF£nx. Par la formule de projection, on a Lay = (wr)? (O(n) ® O(x)), donc L,, est unique. Puisque l’application x/n > O(1) ® O(x/n) est affine, il en est de même de x/n — (1/n)Lh,. On conclut grâce au fait que (1/n)£h,, = O,7,(1) pour tout x/n € F. 1.4. Relations entre les quotients. Soient F, F’ deux faces distinctes telles que F C F’.

Théorème.

(DA) CXS (ED) Gl

CGR),

(ii) Il existe un morphisme tp,p: faisant commuter le diagramme

=|

XP

(EF)

YF)

|

X“(F)

EEE

(ili) mp,pr’ est birationnel si F n’est pas dans le bord de C.

ACTION D’UN TORE

515

Démonstration. (i) est une vérification directe. (ii) D’après [MF; 1.11]et la démonstration du théorème 1.2, mp : X**(F) 3 Y(F) est le quotient par la relation d’équivalence: x + y les adhérences des T-orbites de x et y se rencontrent. Par suite, l’inclusion de X**(F’)

dans X**(F) passe au quotient en tp p’ : Y(F’) + Y(F). (iii) Si F n’est pas contenue dans le bord de C, alors X*(F) n’est pas vide d’après 1.2 ; de plus les restrictions de mp et mp: à X*(F) coincident, car

X*(F) € X*(F"’). Donc mp Fr: induit un isomorphisme de mp/(X*(F))sur son image. Remarque. Supposons X lisse. Pour toute face ouverte F’, la variété Y (F’) n’a que des singularités quotients par des groupes finis cycliques. Par suite, si F est une face contenue dans F’ et non dans le bord de C,

le morphisme apr’ est une désingularisation partielle de Y(F). A l’aide du théorème du slice étale [Lun; III.1], on peut montrer que Y(F) n’a que des singularités quotients par des groupes diagonalisables, de dimension au plus la codimension de F dans V. Etudions maintenant le quotient associé à une face F incluse dans le bord de C. Soit (F) l’espace affine engendré par F. On peut choisir un sous-groupe à un paramètre À de C (c’est-à-dire un élément du réseau dual de X(T)) tel que À est constant sur F, égal à a € Z, et (À,p) > a pour

tout p € C\ (F). Soit XF l’ensemble des points fixes de À dans X**(F), c’est-à-dire l’ensemble des x € X tels que l’enveloppe convexe de II(z)

contienne F, et soit contenue dans (F). C’est un ouvert de l’ensemble X* des points fixes de À dans X.

Pour tout x € X, le morphisme t —>

X(t)x de k* vers la variété

complète X, se prolonge en un morphisme de k vers X. On note lim A(t)x

l’image de 0 par ce morphisme.

Proposition.

(i) X**(F) est l’ensemble des x € X tels que lim X(t}r € XF. (ii) La restriction r de mp à XF est un quotient géométrique universel, et Tr se factorise en q:

FORCES 3 LAs

CNT

lim X(t)z suivi de r.

(iii) Lorsque X est lisse, X F est aussi lisse, et q est une fibration localement triviale en espaces affines.

Démonstration.

(i) Soient x € X et

y= lim A(t)z. Par définition de à, on a:

516

BRION ET PROCESI

F C enveloppe convexe de I(x) + F C enveloppe convexe de II(y).

(ii) X? est une sous-variété fermée T-stable de X, et II(X*) C (F), donc F est ouvert dans l'enveloppe convexe de II(X*). De plus XF = X*(F).

La première assertion suit donc de 1.2. La deuxième est évidente.

(iii) résulte de [BB; Theorem 4.4]. Corollaire.

On suppose X lisse.

Soit F’ # F une face dont

l’adhérence contient F. Alors mp,r: n’est pas birationnel.

Démonstration. En considérant l’intersection de X**(F) et d’une fibre de mp, on se ramène au cas où F et XF sont des points, et où X**(F) est un T-module.

Alors Y(F') est un point, mais on vérifie sans peine que

Y(F’) a une dimension positive. 1.5. Cas des faces de codimension

un.

Soit F une face de codimension 1. Supposons d’abord F incluse dans le bord de C. Alors F est contenue dans l’adhérence d’une unique face ouverte F,. Choisissons comme précédemment un sous-groupe à un paramètre À

tel que l’équation de (F) soit (A,p)= a, et que (A,p) > a pour tout p € F} ; notons XF l’ensemble des points fixes de À dans X**(F). L’énoncé suivant est immédiat.

Proposition.

(i) X®(F4) = X®(F)\XF. (ii) tr,F, est une fibration localement triviale pour la topologie étale, avec

pour fibres les quotients par À des {x € X**(F)\XF

| lim A(t) = yt

(y € XF). Lorsque X est lisse, les fibres de ™F,F, sont des espaces projectifs avec poids. (Rappelons qu’un espace projectif avec poids est une variété de la

forme (V\{O})/k*, où k* opère linéairement dans l’espace vectoriel V, avec tous ses poids positifs). Considérons maintenant une face de codimension 1, rencontrant l’intérieur de C. Soient F_, Fy les faces ouvertes dont l’adhérence contient F. Soit À un sous-groupe à un paramètre de T tel que (À,p) = a sur F, et (A,p) > a sur Fy. Soit XF comme précédemment. Nous posons X_ =

{5 € X | Jim A(Dz € XP} ; X4 = {2 € X | limA(t)z € XF} ; Xy =

X4\XF ; X_ = X_\XF. Remarquons que X_ est inclus dans A (TES notons Y_ son image dans Y(F_). Notons enfin YF l’image de XF dans ¥CE):

ACTION D’UN TORE

517

Théoréme.

(HAE

CE)

XP) = Xe et REPAS

(FS) =X,

(ii) tr_,p : Y(F_) — Y(F) induit un isomorphisme de Y(F_)\Y_

sur

YUR\YS, (iii) La restriction de np_ pr à Y_ fait commuter le diagramme :

ZT

——

nO

Jim A(t)x

ET

rr.|

Y_

rr |

TFL,F

—

Nee

C’est une fibration localement triviale pour la topologie étale, avec pour

fibres les quotients par À de {rx € X_ | lim A(t)z = y} (YE XF). —+ OO

Démonstration. (i) se vérifie immédiatement. (it) D'après (ij, on a: X°(7-)\X- = X'*(F) C X4(F 2), donc rr_ et xp coincident sur X*(F_)\X_. (iii) Remarquons que X_ est un fermé T-stable de X**(F), et que 2,gh ib Gasca pa EE Dena

outre, la restriction de mp_ a X_ se factorise en

l’application z — Jim X(t)z, suivie du quotient de XF par T. L’assertion en résulte aussitôt.

On conclut que Y(F},) s’obtient à partir de Y(F_) en remplaçant Y_ par Y,. L'objectif de la seconde partie est de préciser ces résultats.

2. Structure des morphismes entre certains quotients. 2.1. Action linéaire d’un tore de dimension un.

Considérons une action linéaire de T = k* dans un k-espace vectoriel N de dimension finie. Décomposons N en Ny @No@®N_ où Ny (resp. No, N_) est la somme des sous-espaces propres associés aux caractères positifs

(resp.

nuls, négatifs) de T.

On suppose pour simplifier que No = {0},

c’est-à-dire que T opère dans V avec pour seul point fixe l’origine.

Soit

Z = N//T le quotient (au sens de Mumford) de N par T : si k[N] est l’algèbre des fonctions polynomiales sur N, et k[N]T la sous-algèbre formée des fonctions invariantes par T', alors Z =Spec k[N]T. C’est une variété affine, en général singulière en 0 (point associé à l’idéal maximal homogène de k[N]T). En-dehors de 0, elle n’a que des singularités quotients par des groupes finis cycliques.

518

BRION ET PROCESI

Notons Ny = N,\{0}, et P(N,) le quotient de Ny par T ; c’est un espace projectif avec poids. Soit 74 le quotient de Ny x N_ par l’action

de T. La première projection induit un morphisme p4 : 274 — P(N}) qui fait de Z4 le quotient d’un fibré vectoriel sur P(N},), par un groupe fini cyclique. ; L’inclusion de N, x N_ dans N induit un morphisme fy : Z} — Z.

On voit sans mal que f, contracte la section nulle (N4 x {0})/T sur 0, et se restreint en un isomorphisme sur le complémentaire de la section nulle. Définissons de même N_, P(N_), Z_, p- et f_. Soit Z le produit fibré Z4 Xz Z_. On a le diagramme

et

PLETE A MSEEO BTCE)

4 |

é

ONE

a

y au

L’algèbre k[N,] est graduée (par le poids par rapport à T'), et P(N,) est le Proj de k[N,]. Par suite, P(N}) est muni canoniquement de faisceaux qu’on notera O(n). Proposition.

Ona

Z = Spec a O(n) & O_(n)

n=0 (faisceau d’algébres graduées sur P(N,) x P(N_)), et Z+ = Spec Don) @ k[N_]n n=0 où k[N_], désigne les fonctions polynomiales sur N_, de poids —n par rapport aT. Le diagramme

ES #+ |

\ fers

de FA]

ACTION D’UN TORE

519

est dual de

Deo O+(n)@O-(n) ——

PEN: 8 0-(n)

Br=09+(2) SAN

pro EN

Démonstration.

——

@ k[N—]n

Il suffit de prouver que Z} = Spec

k[N_]n ; l’assertion sur Oz en résulte, car O; = Oz, oz

@ n 022

O4(n)®

Oz_.

Soit f

une fonction polynomiale de poids p sur Ny. Posons Z}} = Py (P(N+ }f); c’est un ouvert affine de Z,. On a

HZ+s] = RNa x NJ = OUIN AI /Sa © Nu D'autre part,

D(P(N+)5 D Os(n) AN) = DT(P(N4);: O4(n))@ AIN}, n=0

n=0

= PEN

/fIn ® AN}

Définition. Soit X une variété avec action de k*, telle que la graduation associée de Ox soit en degrés positifs ou nuls: Ox = OF Ox». Soit Y la sous-variété de X telle que Oy = Oxo. L’éclatement de Y dans X est le Proj de l’Algèbre graduée O72 pAn où An = Om>n Ox,m- Son

diviseur exceptionnel (considéré ici comme variété réduite) est le Proj du faisceau d’algèbres graduées Ox. Corollaire.

(i) f+ et f- sont propres. (ii) $4, ¢-, d sont des éclatements (resp. des sections nulles de py et p-,

et de 0), avec pour diviseur exceptionnel P(N) x P(N_). Démonstration.

Montrons que 7 est l’éclatement de 0 dans X, où la

graduation de k[X] est définie par k[X]n = k[Ni]n @k[N_]n (les assertions sur 4 et d_ se démontrent de façon analogue).

Cela résulte du lemme

suivant [Fle; 2.5]: Lemme. Soit B=@%,B, une algèbre graduée de type fini, avec Bo = k. L’éclatement de 0 dans B est le spectre du faisceau d’algébres E%00P(8)(n) où P(B) est le Proj de B.

520

BRION ET PROCESI 2.2. Comparaison de faisceaux sur les quotients.

On conserve les notations précédentes. Soit 14 : Nyx N2— Z, le quotient par T. Comme en 1.2, notons O(x) le fibré en droites T-linéarisé

sur N associé au caractère —y de T. Posons Of(1)= (14)x (O(x)); définissons de même O7(1). On a d’autre part un faisceau O(1) sur Z, associé au Oz—Module D. o O+(n) & O_(n). On peut encore voir O(1) comme le Since associé au diviseur (de Weil) #-1(0) sur Z, où #=1(0) est la fibre ensembliste de 7 en 0. Pour tout x € Z, on a donc la classe de

x.O(1) dans Picg(Z). Théorème.

Pour tout y € Z, ona

84.08(1)— 62 OZ(1) = x.O(1) = x#71(0) dans Pica(Z). Démonstration. Soit f une fonction polynomiale, vecteur propre de T de poids p, sur N_. Avec les notations de la preuve du théoreme 2.1, on a:

P(2Z45,0%(1)) = Nas x No = Autrement dit, Of (1) est le Module 4

RING IU/flntx © HINn O+(n + x) @k[N_]n sur Oz,.

De même, OY (1) est le Module BP, k[N4]n

@ O_(n — x) sur Oz_. Par

suite, #01) — $* OF (1) est le Module Oz, On + x) ® O-(n + x)

sur BP O4 (n) @ O_(n).

Remarque. Ces résultats sont encore valables dans un cadre un peu plus général. Soient en effet deux k-algèbres de type fini At et A7, graduées en degrés positifs, telles que At = A> = k. Le produit de

Segre [GW; Chapter 4] de At et A7 est la k-algèbre graduée A = @;,-0 At ® Az. Soient X4 (resp. X}) le spectre de At (resp. le spectre de At, privé de l’idéal maximal hompecne): le groupe multiplicatif T opère dans X4 et er Notons m4 : X, x X_ — Z, le quotient par T, et OŸ(1)= (x4)7(O(x) comme ci-dessus. On a un morphisme

f+ : Z4 — Z := Spec(A). Posons Z = Z+ XZ Z-. précèdent se penéraligent sans difficulté en la

Les énoncés qui

ACTION D’UN TORE

521

Proposition.

(i) Dans le diagramme

LAS

MENT

ez

#+|

N

fa

f+

f+ et f- sont birationnelles propres ; ~ (resp. ¢4, ¢- ) est 1’éclatement de

0 dans Z (resp. de Proj(A,)= X4/k* dans Z}; de Proj(A_) dans Z_). (ii) Pour tout x € Z, ona

= X_/k*

$4 OF (1) — PLOY (1) = x.-¥7*(0) dans Pica(Z). Dans le cas étudié précédemment, on prend Ay = k[Nx], avec la graduation induite par l’action de T dans N, et non la graduation comme algèbre de fonctions polynomiales. 2.3. Quotients associés à deux faces ouvertes contigües.

Revenons à une situation déjà abordée en 1.5. Soient Fy, F_ deux faces ouvertes contenant dans leur adhérence une même face F de codimension un. Soient À un sous-groupe à un paramètre de T’, et a € 2, tels que À > a sur F},et À = a sur F; on suppose de plus que À est indivisible dans le groupe dual de y(T). Soit Im(A) son image dans T. Notons pour

simplifier f4 : Y(F}) — Y(F) et f- : Y(F_) — Y(F) les morphismes définis en 1.4. Posons Y = Y(F,) xy(r) Y (F2). Soient X75 xe xs comme en 1.5, et YF, Y4, Y_ leurs quotients respectifs par T. Pour tout point rationnel p de Fy, on a la classe de O,(1) dans

Pica(Y(F+)), qui dépend de façon affine de p. On peut définir cette classe pour tout p € Eq par OX(1)= (rr,)7(O(1)@O(p)) ;si p € F on retrouve la définition nfescaentes d’après 1.3. Introduisons de même O; (1) dans Picg(Y(F_)), et O,(1) dans Pica(Y(F)). Le résultat qui suit Dértet de comparer ces classes, relevées dans Picq(Ÿ).

BRION ET PROCESI

522 Théoréme. (i) Dans le diagramme

o-

Ÿ

—

CHAT

Y(F_)

a

Ÿ

f+

Ÿ, 64, d- sont des éclatement (de YF, Y4, Y_ respectivement) avec pour

diviseur exceptionnel Yy Xyr Y_.

(ii) Le diagramme

VO)

#+ | Ve

=

i

hee

eens

provient de X4 XxrF ep

es

be

ra

Nee

—

|

z

—

lim A(t)z

par passage au quotient géométrique universel.

(iii) Dans Pica(Y), on a

$5, OF (1)— 6.05(1) = ((A,pp)— a)#7 (YF) pour tout p € Eq.

(iv) Les classes #} OF (1), 9° OF (1) et d*O, (1) coincident pour tout p € E tel que (À,p) = a. Démonstration.

Remarquons d’abord qu’il suffit de prouver le thé-

orème pour X = P(V). Décomposons V en No @ Ny ® N_ où No (resp. N+, N_) est la somme des espaces propres W avec (x, À) = a (resp. > a,

.

I] (x%i,2.9)7'(—n)P 2277 (-nY (ch(S7Nz), A) sl

7=0

Chaque intégrale sur Z est donc égale a exp(xz, À)

D-dz-1 II vez

Na (nye 4

s—1

2 |

|

De /É e"22 (ch(SNz),) ([] £i) TA(Tx)

j=0

La plus grande puissance de n qui apparaît dedans est donc nP-1, et son coefficient est D-dz-1

exp(xz,A)

[[ (xi,2,A)7” t=1

dz

DC-nP-é-14 =D

;

:

|(az — j)! lag" (ch(S!.Nz),À) Z

532

BRION ET PROCESI

On conclut grâce à la proposition précédente.

Lorsque T n’a qu’un nombre fini de points fixes dans Corollaire. X, la transformée de Fourier de y est D-1 d! Dis exp(xz, À) ARUP

zEXT

ee

i=1

où x, est le poids de z, et les 6;,, sont les poids de T dans l’espace tangent enzàX.

3.5. Une généralisation des polynômes d’Hilbert-Samuel Soit M un A-module multigradué de type fini, de dimension D. Soient y une fonction polynomiale sur FE, et s son degré. Posons Hu,e(n) :=

Se

dim(M, x) px)

XEX(T) En particulier, Hy,1 est la fonction de Hilbert-Samuel de M.

Théorème. La fonction n — Hyç(n) est polynomiale pour n assez grand ; le degré de ce polynôme est au plus D+s—1, et son terme de degré

D+s-—1

est d!-! Scat) um(p) ¥s(p) dp où y, est la composante homogène

de degré s de +. Démonstration. Lorsque M est égal à A, la dimension de My, est le nombre de solutions du système ro +---+2, =; toXo+---+2, x; = X

où les z; sont des entiers non négatifs. Par suite H 46(n) =

dE

p(xzoXo Ha

TrXr)

Zo+::-+rr=n

En prenant pour ¢ un monôme, on voit sans mal que la série génératrice Do 2" Ha,y(n) est une fonction rationnelle, ayant pour dénominateur (1 — z)"+#+1, On en déduit que Ha,y est un polynôme en n pour n assez grand.

Considérons maintenant le cas où M = A(n,6) pour un (n,6) € N x X(T). Alors Hu,y(m) =

Si dim(Am+n,x+5) P(X) = Hay.(m+ n) XEX(T)

ACTION D’UN TORE

533

où ps(x) := p(x — 4) . Par suite Hy,,(n) est encore polynomiale pour n assez grand. Tout A-module multigradué de type fini admet une résolution libre multigraduée par des sommes directes de modules de la forme A(n,6). La première assertion du théorème s’en déduit aussitôt. Pour montrer les ue autres, il suffit de prouver que n~?-* 5% _, Hyç(m) a pour limite

a" [em pu (Pes (p)dp quand n — oo. Or

Dem}

d dent

edim(Maiy).eGo

O0 1. Since Gv is closed, it follows that G, is

reductive, cf., e.g., [1]. Hence G, contains a one-dimensional torus T'. Let p : k* —

T be a fixed isomorphism.

Replacing v by gv for a suitable

g € G, we may assume that ¢(t) = diag(t?,t~*) xdiag(t™,...,t'r)

EG =

SL2xSL, for allt € k* and some a,b;,...,b, € Z. Clearly, b1+.. +b, = 0. We can find such a weight basis vo, v1,..., Vg, resp., U1,---, Up, relative to the diagonal torus in SL2-module R(q@1), resp., SL,-module R(æ:1), that

vi, Tesp., uj, is an eigenvector of diag(t*,¢~), resp., diag(t”:,...,t°?), with the eigenvalue 442), resp., ts. Then the vectors mou € V form a weight basis relative to T in V, and ¢(t)(vj @ uj) = 44-2)#b5u; @ u;. Therefore

544

V.L. POPOV

V7 is the linear span of all v;@u; for which a(q—2i)+b; = 0. We will prove that for some j the vector v; @u; does not lie in VT for any t. Assume the contrary; since q is odd, we then have b; = af; for any j =0,...,p, where f; is some odd number. Thus, 0 = 6, +...+6p = a(fi +...+ fp), and, therefore, since p is odd, a = 0. Consequently, VT is the linear span of those v; ® u; for which b; = 0. Clearly, we do not have 6; = 0 for all 7 (however there exists a j for which 6; = 0, since v € VT and therefore V7 # 0). Suppose, for definiteness, bo # 0. Consider in G the one-dimensional torus

{6(t) =diag(1,1)xdiag(t-?t!,t,...,t)

€ SL2 x SLp|t € k*}. If j> 0, then

6(t)(v; ® u;) = t(v; @ uj). Consequently, 6(t) acts on V7 as a homothety with coefficient t. Since v € V, this contradicts the fact that the orbit Gu is closed.

Thus, for some j the vector vj ®u; does not lie in VT for any i; suppose,

for definiteness, this is true for j = 0. Then, as above, 6(t) acts on V7 asa homothety with coefficient t, which contradicts the fact that Gv is closed.

Q.E.D. Corollary. The connected simply connected non locally transitive simple linear algebraic groups G € GL(V) which have the property that the stabilizer of each nonzero semisimple point of V is trivial are exactly the groups G = SLa,d >2,V = mR(œ1), where m > d. Proof of the Corollary. If G = SL2, V = R(pw1), p is odd, p > 3, then

the property does not hold (see Lemma 2.1 in [8]). Therefore it follows from Remark (a) above that one has only to check the property for the groups which are pointed out in the formulation of the Corollary. In this case the point v is the vector (v1,...,v»),v; € R(æ:1). It follows from here that G, is trivial if rk{v1,...,%m} = d, and that the condition rk{v1,...,um} #d implies for each t € C the existence of an element g € G which acts on the linear span of the set {v1,...,v,} as a homothety with the coefficient t. Hence 0 € Gv if rk{v1,...,um} # d. Therefore it is sufficient to check that rk{v,,...,Um} = d implies Gu = Gv. Assume that this is not the case and let Gu, u = (u1,...,um), be the (unique) closed orbit in Gv. It follows from dimGu < dimGv that dimG, > 0. Therefore rk{uj,...,um} < d, and hence u = 0. It follows from here that all the homogeneous nonconstant invariants of G-module V vanish at v. In particular, if one identifies V with

the coordinate space (of columns) C? then the determinant of any d vectors taken from v1,..., Um equals to zero. Therefore rk{v;,...,vm} < d which is a contradiction. Q.E.D.

The claim of the theorem follows immediately from the propositions 6, 7, 8, 9 and 10 which are proved in the next paragraphs. We consider

STABILIZERS OF SEMISIMPLE POINTS separately each of the types of simple groups; the case to be the most difficult one.

545

G = SLy appears

Notations

V# is the fixed point set of the group H which acts on V.

Za(H)) is the centralizer of a subgroup H of G. A(V) and © are resp. the weight system of V and the root system of G with respect to S.

X44 is the monoid of dominant weights of S. W is the Weyl group of G. The numbers aj,...,a; are called numerical labels of the element a,;m; +...+a. We denote by 1 the trivial one-dimensional module and by V* the dual of V. We abbreviate s.g.p. for the stabilizer of general position. 2. Some preliminary results. The cases B), Cj, Dg,, E7, Eg, F4 and Go. We shall say briefly that V is a G-module of type (F) if the G-stabilizer of each nonzero semisimple point of V is finite. Proposition

1. V is a G-module

of type (F) iff all of the fibers of the

canonical morphism ty,g, except the “zero-fiber” N = my G(tv,G(0)), are closed orbits with finite stabilizers, or, equivalently, iff each point of the set V\N is semisimple and its stabilizer ts finite.

Proof. The “if” part is evident, so assume that v € V \ N and Gu is the

(unique) closed orbit in X = y c(mv,c(v)) C V\N. Then G, is finite and therefore dim Gu = dimG. On the other hand, one has Gu C Gu, and hence dimG > dimGv > dimGu = dimG. Therefore dimGv = dim Gu, hence Gu = Gv = X. Q.E.D.

Proposition 2. If a direct sum summand is also of type (F). Proof. This is evident.

of G-modules

is of type (F) then each

Q.E.D.

Recall [13], that a module is called stable if its generic orbit is closed. If the group is semisimple then the stability of a module is equivalent to the reductivity of its s.g.p.,[13]; we shall use this systematically below.

V.L. POPOV

546

Proposition 3. Assume that there exists a reductive subgroup H of G such

that dimH > 0 and that the Zg(H)/H-module VX has a nonzero stable

submodule. Then V is not of type (F). Proof. It follows from the assumption that Zg(H)-orbit of some nonzero point of VA is closed. Therefore G-orbit of this point is also closed, [4], and Q.E.D. the assertion follows from the inequality dimH > 0. The group Zg(H)/H in the proposition 3 is always reductive. In what follows we shall systematically use this proposition together with the following assertion: Proposition 4. Let H be a reductive subgroup of G = G° and D = Ze(H)/H. Let R(A;), 1 < i < 8, be a set of simple G-modules and assume that D°-module R(A;)" contains a simple submodule M; with the multiplicity n; > 1. Then the Cartan product of D°-modules M1,...,M, is contained in R(A, +...+A,)" with a multiplicity > max{n,,..., ns}.

Proof. This is a special case of theorem 4.1 in [8].

Q.E.D.

Proposition 5. Assume that there exist such À1,...,)a € A(V) that

CL) eEkt Aga. A+) < dims: (2) zero is an inner point of the convex envelope of the set {A1,...,Aa};

(3) À — À; EX for eachiF j. Then V is not of type (F).

Proof.

Let v; is a nonzero weight vector of the weight A; and let v =

V1 +...+ va. Then it follows from [5] and (2) and (3) that Gu is closed. It follows from (1) that the intersection of all the subgroups Ker); of S, 1 1 and V be a module

of type (F) such that

kX € A(V) for some À € A(V) andk €EQ,k < 0. Then a/(1—k) € X44 for some a E UN X44. (Therefore if a/(1 — k) is not an element of X44 for each a €

UNX44 then each G-module is not of type (F)).

Proof. The set A(V) is W-invariant and each W-orbit in this set intersects the Weyl chamber. Therefore we can assume that À € X44. It follows from the inequalities rk{A, kA} < 1 < rkG and k < 0 that» the convex envelope of the set {A, kA} contains zero, therefore we derive from the proposition 5 that a= À — kA = (1 — k)A EE. Since 1 — k > 0, we have à € X44. Q.E.D.

STABILIZERS OF SEMISIMPLE POINTS

547

In the same way one obtains that if 0 € A(V) and rkG is arbitrary then V is not of type (F) (one has to take À = 0). Corollary 2. Let rkG > 1 and —1€ W. Then the highest weight of each

simple G-module V of type (F) is of the form a/2, where a E

UN X44.

Proof. Applying the transformation —1 € W to À € A(V) one obtains that kA € A(V) for k = —1 and then can argue as in the corollary 1.

Q.ED.

We consider now the connected simple groups which satisfy the assumptions of the corollary 2, i.e. the groups of the types Br, C;, Ds, Go, F4, Ez and Eg. In these cases one obtains the desired result just straightforwardly from this corollary:

Proposition 6. [f G is the group of type Bi(l > 3), Dos, Go, Fa, Er or Eg then each G-module is not of type (F). If G is the group of type Ci then the unique simple G-module of type (F) is R(w,), and this module is locally transitive. Proof. Let G be the group of type B (1 > 3). Then it is easy to see, [9], that 6M X44 consists only of €; and €; + €2 which are not divisible by 2 in X44. The claim now follows from corollary 2 of proposition 5 and from proposition 3.

Let G be the group of type C1 (1 > 2). Then UM X44 consists only of 2e1 and €; + e2.Therefore a/2 € X44 for a € UN X44 if and only if a = 2e,. The group G acts on R(æ1) \ 0 transitively because of Witt’s theorem. If G is the group of type Dos, Go, F4, E7, Es then UN X44 consists, resp., only of w2, &1 and w2, m and w4, m1, wg and all these elements

are not divisible by 2 in X+4.

Q.E.D.

3. The cases Eg and Dg,,4

In these cases the group W does not contain —1 and we have to use another arguments. These arguments are based on propositions 3 and 4. Proposition 7. If G is the group of type Eg then each G-module in not of type (F).

Proof. Let T be the one-dimensional torus in G defined by the highest root. Then Zg(T)/T is a group of type As and one has the following table

(see [8]):

548

V.L. POPOV

R(@2) As it follows from [13] and [10], R(w:) and R(ws) are the unique simple nonstable modules of the group of type As. Therefore it follows from proposition 4 and table 4 that for each simple G-module V there exists a nonzero stable submodule of Zg(T)°/T-module V7 .Now the claim follows from propositions 3 and 2. Q.E.D. Proposition

8. If G is the group of type Di,l is odd and > 5, then the

unique simple G-modules of type (F) are R(w4) and R(ws) for! = 5, and these modules are locally transitive. Proof. Let T be the one-dimensional torus in G defined by the highest root. Then Z¢(T) is a group of type Ai + Di-2 (where, by definition, D3 = A3)

and one has the following table (see [8]):

TOR) + (1@ R(&,-2))+

(1 @ R(wp_4)) + (R(2m1) © R(wp_2)),

where, by definition,

R@;) = haf ¢=0) R(e@;) = 01f

R( mp)

R(wi-3 + wi-2)

Ra) © Rw»)

R(2m_3)

+ R(2a_2)

2. If ag = 1 then VT contains y° @1 and therefore again V is not of type (F). Finally, | being even, V is locally transitive if

a2 = 0 and therefore V is of type (F). (le) Let V = R(ws),! > 5. If 1= 5 or 6 then, as it follows from [11] and [12], V is stable and s.g.p. of V has positive dimension; therefore V is not of type (F) in this case and one can assume that | > 7. Take in

G = SLi41 two-dimensional torus Ti = {diag(t;,t2,t3,1,...,1)|titet3 = 1} . Then Za(Ti) = Ti x Zi x Z6(T1), where ZG(T1) = diag(1, ie 1,SLi-2) and

Z, = {diag(t~'*?, 1,1,t,...,t)} . Let p be the character of Z, defined by the formula 7(diag(t~'*+?, 1, 1,t,...,t)) = t. Now the same arguments as in

[8] show that if one identifies ZG(T1)/T1 with Z1 x SLj_2 then Zg(T;)/T}module V7" is of the form (p~'t? @ 1) + (3 @ R(æ3)). It follows from the ere F').

1 > 7 that this module is stable and therefore V is not of type

(1f)

Let V = R(wæ4),l be even and | > 8. Take in G = SLiy:

STABILIZERS OF SEMISIMPLE POINTS

951

a three-dimensional torus T} = {diag(t1,t2,t3,ta,1,...,1)ltitotsta = 1}. Then

Za(T2)

= To x 22 X ZG(T2), where ZG(T2)

=

{diag(1, 1: Ls ds SLi-3)}

M Zoteadiae({sl 4151) 1,t,...,t)} . Let 4 be the character of Z2, defined by the formula y(diag(t3—',1,1,1,t,...,t)) = t. Now, as in [8], one can show that if one identifies ZG(T2)/T2 with Z2 x SLi_3 then Zg(T>)/To-

module V7? is of the form (y?—! @ 1) + (4 @ R(wa)). It follows from the restrictions on | that SL;_3-module

R(w4) is locally transitive only for

! = 8. Therefore, if / is even and | > 8 then V is not of type (F). Consider now the case | = 8 separately. We return back to the torus T.

Then Zg(T)/T can be identified with Z x SL7. It follows from table 3 that Ze(T)/T-module V7 is of the form (y~> ® R(w2)) + (x4 @ R(w4)). We obtain from [11] and [13] that SL7-module R(w2)+ R(w4) is stable. Use now that the algebra of invariants of SL7-module R(w4) is not trivial (because this module follows from weight x°,a invariant of

is not locally transitive and the group SL7 is semisimple). It this fact that there exist in this algebra a Z-semi-invariant of < 0. Let us show that one can find in this algebra also a Z-semiweight x?,b > 0. Indeed, it is clear that the algebra of poly-

nomial functions on the space of SL7-module R(w2) + R(w4), considered as SL7-module, contains a submodule of the form S?(R(w2)") ® R(wa4)* an which Z acts scalarly with the weight x!!. But one has the following equality of SL7-modules, [6]: S?R(w2) = R(3m2) + R(œ1 + w2) + R(w3) and hence S?(R(w2)") = R(3m5)+ R(ws + we) + R( wa). Therefore S3(R(w2)") © R(w4)* D R(w4) ® R(w4)* D 1. This means that the algebra of invariants of SL7-module R(w2) + R(w4) contains a homogeneous invariant of degree (3,1), which is a Z-semi-invariant of the weight y!?. Therefore, Zg(T)/T-module V7 is stable and hence V is not of type (F). So we proved in this section that if ay = a; = 0,1 > 3 then only R(w2) and R(m_1) for even | are unique simple G-modules of type (F).

(2) The case 1 = 2.

Let V = R(pæi). It follows from table 3 that if we identify 7¢(T)/T with a one-dimensional torus Z then Zg(T)/T-module V7 is of the form xP + P34 xP 8 +... + yP-ÙP/28, and hence contains a submodule x? + x? b/28, We have p — [p/2]3 < 0 for p > 2 (and the equality takes place only for p = 3). Therefore V is not of type (F) if p > 2. If p = 1 then V is locally transitive and hence of type (F). It follows from the equality

R(pw1)* = R(pw2) that Z¢(T)/T-module V7 for V = R(qw2) is of the form x77 + x7 1t3 + y~ It 84. 7949/18, We can conclude from here and from proposition 4 that Zg¢(T)/T-module V7 for V = R(pwi+qm2),pq # 0, contains a submodule of type x* + x?, ab < 0, if either both of p and q p= 3,q > 2, and it contains a p= 1,q > 2, or are not equal to 1 and 3, or

552

V.L. POPOV

submodule of the type x° if p = q = 1. This submodule is stable in each of the cases and therefore V is not of type (F).

So we conclude that if 1 = 2 then R(mw1) and R(w2) are unique simple G-modules of type (F). . (3) The case ay #0, a =0,123.. (3a) Let at first be a, = p > 2. It follows from table 3 that ZG(T')/Tmodule V7 contains a submodule of type (x?-(P/21-1)1+1)+a @ R((p —

2([p/2] — 1))æ1 + a)) + (x?t? © R(pmi + B)) + (xP—P/I Dt @ R((p —

2{p/2]æ1 + y)) for some a, B,y € X44 and a,b,c € Za < 0,b > 0,c < 0. If we consider now the dependence on p of the signs of the exponents of x and of the coefficients of w1 in this sum, then we obtain, after some calculations, that this submodule contains a stable submodule when either 1 = 3,a; >6 orl > 4,a1 > 4 and a; # 5 for |!= 4. Therefore V is not of type (F). Let 1 = 3 and a, = p = 5. If a2 # 0, then it follows from table 3 that

Za(T)/T-module VT contains a stable submodule of type (v7 @ R(5m1))+ (x? @ R(3m1)). Therefore V is not of type (F) in this case. If ag = 0, then V7 contains a submodule L of type (x! @ R(3m1)) + (x7? @ R(@1)). In

this case one can identify (Zg(T)/T)’ with SL». It follows from [13] and [11] that SL2-module R(3m,) + R(w,) is stable. It is known, [14], that (1,3), (2,2), (3,3), (4,0) are the degrees of elements of a minimal system of homogeneous generators of algebra of invariants of this SL2-module. Therefore these elements, as Z-semi-invariants, have, resp., the weights x78,

x74, 7°, x4. This shows that L is stable and hence V is not of type

Let

|= 4,a;

=

p=5. It follows from table 3 that if ag + a3 # 0 then

Za(T)/T-module V7 contains a submodule of type (x ® R(5w1 + a)) + (x? ® R(3m1 + B)), ab < 0, a,8 € X44. This submodule is stable and therefore V is not of type (F). Otherwise, if ag = a3 = 0, then VT contains

a stable submodule of type x° @ R(3@), and hence V is again is not of type (F). Let | = 3,a; = 4. Then Zg(T)/T-module VT contains a ubmosule of

type (x* © R(4m1)) + (x? @ 1), ab < 0, which is stable. Therefore V is not of type (F). Let | > 3,a, = p = 3. It follows from table 3 that ZG(T)/T-module

VT contains a submodule of type (x° @ R(3m1 + a)) + (x? © R(w + B)),

where ab < 0,a,8 € X44, and B £0 if a3+...+ 1

# 0. In the latter

case this submodule is stable and hence V is not of type (F). Assume

now that a3 = ... = aj_1 = 0. Consider at first the case ag = d > O0. It

follows from table 3 that ZG(T)/T-module

VT contains a submodule of

STABILIZERS OF SEMISIMPLE POINTS

593

type (x +24 @ R(3m1 + 2m2)) + (x3+40-) @ R(w, + w2)), which is stable

when (/,d) # (3,1) or (4,1). If (1, d) = (4,1) then the second summand of this submodule is stable, and if (1, d) = (3,1) then it follows from table 3 R(w1)). The that VT contains a submodule of type (x°@ R(3m1))+(x73@ p=5 same arguments which we use when considered the case | = 3,a1 = show that this latter submodule is stable. Finally, assume that a) = 0 and

consider the same torus T; as in (le). Then it is not difficult to see that

Za(Ti)/T1-module V7" is of the type (y~'+?@1)+(x3@R(3m@)) and hence is stable. Therefore we see that V is not of type (F) if 1 > 3, a,

=p=3.

Let now az = 2. It follows from table 3 that Zg(T)/T-module V7 contains a submodule of type (x? ® R(2m: + a)) + (x? ® R(B)); a > 0,

b < 0; a,8 € X44, where R(B) is locally transitive only when either (Gas arr) = (1,0, :.:5,0) or lis even and (a3, 9.5.42 4) = (0p 105... ,0), (0,...,0,1). In all of the other cases this submodule is stable and therefore V is not of type (F). We shall consider these special cases separately. As-

sume that (a3,...,a;-1) = (1,0,...,0) (and hence | > 4). Denote az = d. If d > 0 then it follows from table 3 that Z¢(T)/T-module VT contains a submodule of type (x? ® R(2m, + a)) +(x? @ R(w; + w2)),@ € X44,a> 0,b < 0, which is stable. If d = 0 then V7 contains a submodule of type (x4-'! @ R(3m1)) + (x° @ R(2m1 + w3)) which is stable when! > 4. If 1= 4 then the first summand of this submodule is stable. Therefore V is not of type (F) in the case under consideration.

Assume that | is even an (az,...,a;-1) = (0,1,0,...,0) (and hence 1 > 6). It follows from table 3 that if d > 0 and, resp., d = 0 then Zg(T)/T-module V7 contains a submodule of type (x?+24 @ R(2m, +

daw)) + (x°~'+40-) @ R(2w2)) and, resp., of type (x° @ R(2m1 + wa)) + (x°-' @ R(2m1 + w2)). Both of these submodules are stable and hence V is not of type (F).

Finally, assume that | is even and (a3,...,ai-1) = (0,...,0,1), 1 > 6. We obtain from table 3 that if d > 0 and, resp., d = 0 then ZG(T)/T-

module V7 contains a submodule of the type (x'*+?4t' @ R(2m1 + dw2)) + (x)

@ R(w2 + w-3)) and, resp., of the type x° @ 1. Both of them are

stable and hence V is not of type (F). So we conclude that V is not of type (F) if a; > 2, a; = 0,1>3.

(3b) Let consider the case aj = 1, | > 3. Assume at first that | > 5 and a3+...+a-2 # 0. Then it follows from table 3 that Z¢(T)/T-module VT contains a submodule of type (x7 @ R(m1 + a)) + (x? © R(œ1 + B)), a

0; a,8 # 0. This submodule is stable. Hence V is not of type

(F). Now assume that | > 4, ag = 5, &-1

= t and a; = 0 if 1 #1,2,/-1.

554

V.L. POPOV

It follows from table 3 that Zg(T)/T-module V7 contains a submodule of

! @R(m1+502))+ type (x2°+-Dt+

(x 0 2441 @ R(w +t _3)) if st > 0,

DIS R(w + a1-3)) +(x *@R(wi+taw1-3)) ifs = 0,t >

of type (xD

2, and oftype (xO-2G6-)+3g R(w14+m2))+ (xt @R(w1+sw2)) ifs> 2,

t = 0. These submodules are stable if ({,s) # (4,2). On the other hand, the first summand of the first of these submodules is stable if (I,s) = (4,2). Therefore V is not of type (F) in the cases under consideration. Assume

at last that

| = 3, ag

= s > 0. If s = 1 then, as it follows

from [15], V is not of type (F). Let s > 2. Then, using the equalities of SL4-modules R(w2) = A?(m1) and S?R(w2) = R(2m1) + 1, [16], one can easily show that Zg(T)/T-module R(2@2)" is of the form (x°@1)+(x°® 1) + (x74 @1)+(x° ® R(2m)). It follows from here and from table 3 that VT contains a submodule of type (y!~7* @ R(æ1)) + (x77? ® R(3m1)). Now the same arguments as in the case (3a), / = 3,a; = 5, show that this submodule is stable. Hence V is not of type (F). So we conclude that if a, # 0, a; = 0,1 >3,(a1,...,æ) # (1,0,...,0), (1,1,0,...,0) and (1,0,...,0,1,0) then V is not of type (F). Notice also that G-module R(@) is locally transitive and hence of type (F).

(4) The case aya; #0 ,12>3. (4a) Let ay = a; = 1. If all the other numerical labels are equal to zero then V is adjoint G-module and therefore, as it is well known, is not of type (F). If ag +...+ai_1 # 0 then it follows from table 3 that

Za(T)/T-module V7 contains a submodule of type (y* © R(w; + wi-2 + @)) + (x? © R(w1 + mi-2 + B)), a> 0;b 2,a; > 2. It follows from table 3 that Z¢(T)/Tmodule V7 contains a submodule of the type (x? © R(pæ1 + a)) + (x? @ R(qwi-2 + B)), a > 0; b < 0; p> 2; q > 2; a,8 € X44, which is stable. Therefore V is not of type (F).

(4c) Let ai = p > 3, a; = 1. It follows from table 3 that ZG(T)/Tmodule V7 contains a submodule of type (x7 @ R(pwi + wi-2 +a))+(x°®@

R((p—2[p/2])m1 + m1-2+P))+(x° @R(p—2([p/2]—1)@1+ w-2+7)), a > 0; b 2, and of type (x° @ R(2m1 + wo + wi-2)) + (x 742 © R(w + @i-2)) when d = 1. The exponents of y in the first summands of these

submodules are positive and in the second summands are negative, and these summands, as SL;_,-modules, are not locally transitive. Therefore V is not of type (F) in this case also. If 1= 3 and aj = d > 1 then it

follows from table 3 that ZG(T)/T-module VT contains a submodule of type (x74t+! @ R(3m1)) + (x7 24+! @ 2R(w,)). This submodule is stable because SL2-modules R(3@;),2R(@1) are not locally transitive, and 2d + 1>0,—-2d+1

< 0. Hence V is not of type (F) again.

So we conclude that if aya;

0,1 > 3 and (a1,...,a) # (2,0,...,0,1)

then V is not of type (F). (5) The case (ar,

a)

— (2,0,..-,0,1)

1 > 3.

One has the following equalities of G-modules (e.g., [16]): R(2~,) ® Ru) = R(œ1) + R(2m1 + mi), R(m1)* = R(w) and R(2m) = S?R(m). It is well known, [10], that the weights of G-module R(w:) are €1,...,€141 and all of them have multiplicity 1. It easily follows weights of G-module R(2@; + a) are €; +€; —€x, and the multiplicity of the weight equals to 1 if otherwise. Now consider the following | weights

from these facts that the where 1 < i,j,k 3. Using the equalities

of G-modules R(w,) ® R(w2) = R(w1+ w2) + R(w3), R(w2) = A? R(m1), R(w3) = A? R(mw,), see [16], one can show that the weigts of G-module R(œ1 + w2) are ei + €; — ex, where at least two of the indices 7,j,k are

different, and also that the multiplicity of the weight is equal to 2 if all the indices are different and is equal to 1 otherwise. Let [+1

we

=

3k, k € Z. Then

jAk = €3k-2 + E3k-1

À1 =

€1

+ €2 + €3, An =

+ €3x are the weights of G-module

€4 +E5 + Ee,

V which have

the properties A; — A; ¢ U for each i # j, and A; +...À; = 0. As in n.(5), it follows from here that V is not of type (F). Let 1+1=3k+1, G-module

V: À

=

k € Z. Then we consider the following weights of

Gy or ED ap Sy

A2 =

ap GR AP EB

coos

Àk-1

NÉS

a

556

V.L. POPOV

Esk4 + €3k—3) Ak = 2€3e—-2 + Esk-1, Av41 = 2Esk-1 + Esk, Ak+2 = 2E3k +

€3k41, Ak43 = 2ZE3e4+1 + Eak—-2. We have A; — À; € Z for each 7 # j, and 3A, + 3rot+... + 3Ag—1 HAR + ARG +AR42 +Ak+3 = 0. Therefore, as above,

we conclude that V is not of type (F). Let 1+1 = 3k+2, k € Z. Then we consider the following weights of Gmodule V: Ay = €1 +€2+€3, Ao = Ca FES HEC, ..., Ak = E3k-2+E3k-1 +E3k,

+E3K-1Agti = Esk +Esk+1 + €3k4+2) Ak42 = 2ZE3k41 + E3k-2, Ak43 = 2€3k42 We have À; — A; ¢ Z for each i # j, and 3A; + 3A2 +... +BAR-1 dee +An+2 + An+3 = 0. Hence V is not of type (F).

(7) The case (a,...,a]) = (1,0,...,0,1,0)

+ 244+

,1> 4.

Here we use the following equalities of G-modules: R(w; ® R(wi-1) =

R(w,+m1-1)+R(a1), R(w1) = R(m1)*, R(wi-1) = (A? R(œ1))* (see [16]). Using these equalities, one can easily see that the weights of G-module R(@1 + mi-1) are ex —€; —€;, 1 < i,j,k < 141,17 F J, and also that the multiplicity of the weight is equal to 1 if 7,7 # k and equal to /— 1 otherwise.

Let |+1= 2k, k € Z. We consider the following weights of G-module V: Ar = Ex — €1 — €2, A2 = Ene — €3 — €4, AZ = Ext — 5 — &, ---, Ap—1 = ok — E2k=3 — C2b=25 Ak = £1 — €2:— Enk—15 Ar C2 — £1 = EE), Àg+2 = —€2k. Then we have À; — A; ¢ E for each i # j, and 2A; + 212 + weet 22861 + À + Angi + (2k — 1)An42 = 0. Hence V is not of type (F).

Let 1+1= G-module

V: À

2k +1, k € Z. Now we consider the following weights of =

€ok41

— €1 — €2, À9 =

€2k41

— €3 —

€4,A3 =

€2k41

— €5 —

£6,---) Ak = E2k41 — E2k-1 — E2k) Ak41 = —E2k41- Then À; —à; ¢ Dd ifi Fj,

and Ay +Ag+...+ An + (k + 1)AxR41 = 0. Hence V is not of type (F).

(8) The case 1 = 1. If a; is even then zero is the weight of V = R(aiw1) and therefore

V is not of type (F) (see section 2 above). If a; is odd then zero is not the weight of V. It is well known that the stabilizer of any semisimple

point is reductive (e.g., see [1]). But it is also well known that the identity component of any reductive subgroup of SL: is either identity or a maximal torus of SL. This shows that V is G-module of type (F) if a; is odd. The results of this section give, all together, the proof of the following assertion which is the main result of the section: Proposition 9. If G = SLiy1 then, to within of taking the duals, the unique simple G-modules of type (F) are R(m1);R(w2) if l is even; R(pw;) ifl=1 andp is odd.

STABILIZERS OF SEMISIMPLE POINTS

557

5. Nonsimple G-modules of type (F) Our starting point here will be proposition 2 and the description of simple

G-modules of type (F) given in propositions 6, 7, 8, 9. (1) Let G be the group of type Ds. It follows from [11] and [13] that G-modules 2R(w4), 2R(æ5) and R(w4) + R(ws) are stable and that their s.g.p. have positive dimension. Therefore these G-modules are not of type (F). This shows that there are no nonsimple G-modules of type (F).

(2) module V either m = (and H is

Let G be the group of type C1. Denote by H the s.g.p. of G= mR(@1), m > 2. Then it follows from [11] that dimH > 0 iff 2k,k 2. It follows from [11], [13] that G-modules R(œ1) + R(@) and, if l'is even, 1 > 4, R(w1) + R(w2), R(w2) + R(w-1) are stable and have infinite s.g.p. Hence these G-modules are not of type

(F). It follows from [11] that 2R(@2) for | even, | > 4, is locally transitive, and hence of type (F). If 1is even, 1 > 4, and V = 3V(mz2), then it follows from table 3 that

VT is Zg(T)/T-module of type (x!~' @ (3 - 1)) + (x? @ (3R(w2)). Using that SLi_1-module 3R(w2) is not locally transitive, [11], we obtain from

here that V7 is stable, and hence V is not of type (F). It follows from [11] that mR(@1) is locally transitive (and hence of type (F)) ifm < 1+1. Otherwise it is not locally transitive but still of type

558

V.L. POPOV

(F) (see the proof of corollary to theorem in section 1). Finally, let lis even, | > 4, and V = R(w2) + mR(a). It follows from the tables in [11] that V is locally transitive (and hence of type (F’)) if m = 1, and that s.g.p is infinite and dimV/G > 0 (and hence V is not of type (F), see n.(2) of this section) if 2 < m < I. If m >I then s.g.p. is finite and we need another arguments. We consider the two-dimensional torus T1 of G which we intriduced in n.(1e) of section 4. Then it is not difficult to check that VT is Zg(T;)/Ti-module of type (9? @ R(w2)) + (#7! @ MR(w-3). Using that SLi-2-modules R(w2) and mR(w-3) for m > 1 > 1-3 are not locally transitive, we obtain that V7 is stable and hence V is not of type

Gis). Therefore we proved the following Proposition 10. Nonsimple G-modules V of type (F) of connected simple groups G are exhausted, to within of taking the duals, by the following list:

G=Slo,V = R(piwm1)+...+ R(p.ws), s > 2, allp; are odd, G=

SLi4i,;

l = 2 we

mR(æ1),

m

> 2

G = SLi4i, | is even > 4, V = 2R(we) or R(w2) + R(w). Acknowledgement: I wish to thank the Mathematical Institute of Basel University, where this paper was written up, for its hospitality. Especially I am grateful to H. Kraft and also to D. Wehlau for the help on preparing the paper.

REFERENCES

[1] Luna, D.: Slices étales. Bull. Soc. Math. France, Mem. 33 (1973), 81105 [2] Kac, V.G.; Popov, V.L. Vinberg, E.B.: Sur les groupes linéaires algébriques dont l’algébre des invariants est libre. C. R. Acad. Sci. Paris 283 A (1976), 865-878 [3]

Nakajima, H.: Representations of a reductive algebraic group whose algebras of invariants are complete intersections. J. reine angew. Math. 367 (1986), 115-138

[4] Luna, D.: Adherences

d’orbite et invariants. Inv. math. 29 (1975),

231-238

[5] Dadok, J.; Kac, V.G.: Polar representations. J. Algebra 92

504-524

[6]

|

(1985),

Schwarz, G.W.: Representations of simple Lie groups with regular rings

of invariants. Inv. math. 49 (1978), 167-191

STABILIZERS OF SEMISIMPLE POINTS [7] [8]

959

Gordeev, N.L.: Finite linear groups whose algebra of invariants are complete intersections. Math. USSR-Izv. 28 (1987), 335- 379 Popov, V.L.: Syzygies in the theory of invariants. Math. USSR-Izv. 22 (1984), 507-585

[9] Bourbaki, N.: Groupes et algèbres de Lie. Ch. 4-6. Hermann, Paris (1968) [10] Bourbaki,

N.: Groupes et algébres de Lie. Ch. 7-8. Hermann,

Paris

(1975) [11] Elashvili, A.G.: A canonical form and stationary subalgebras of points in general position for simple linear groups.

Funct.

Anal. Appl. 6

(1972), 44-53 [12] Elashvili, A.G.: Stationary subalgebras of points in general position for irreducible linear Lie groups.. Funct. Anal. Appl. 6 (1972), 139-148 [13] Popov, V.L.: Stability criteria for the action of a semisimple group on a factorial manifold. Math. USSR-Izv. 4 (1970), 527-535

[14] Springer, T.A.: Invariant Theory. Lect. Notes Math., Springer Verlag 585 (1977) [15] Panyushev, D.I.: Classification of four-dimensional anti-commutative algebras with a nontrivial group of automorphisms. In: Questions of theory of groups and of homological algebra. Yaroslavl. (In Russian) (1978), 98-106 [16] Onishchik, A.L.; Vinberg, E.B.: Seminar on algebraic groups and Lie

groups (1967/68). Izdat. Moskov. Gos. Univ. (In Russian)

(1969)

[17] Gordeev, N.L.: Coranks of elements of linear groups and complexity of algebras of invariuants. To appear (In Russian) (1989)

[18] Wolf, J.A.: Spaces of Constant (1967)

Curvature. McGraw-Hill,

New York

[19] Popov, V.L.: Modern developments in Invariant theory. Proceedings of the International Congress of Mathematicians. Berkeley,U.S.A.

(1986), 394-405 Received February 9, 1990

Moscow Institute of Electronical Constructions (MIEM)

per B. Vuzovskii 3/12 Moscow

109028, USSR

17.

4 eg

sd \

je (etiweell al}. via cu)

Ve eteavetngs Laos serré wel (BAO) tv

es

+

wet

rÆhet(À

in tdewtad

MT

L

ao aieores, LUN

‘A

a

Son

Arr re

lgMR

ednm)

(anwesyft nF) dréquui d'Air de. fq"

"be nn

orion

=m

“Ve Latest

à

PAPER

bith vbafl ne WOT QIN Lharts 14 it

@

Méeisiw

fs th

CIRE

UNE

es

| Gee

ter oat + eine wei

C*-Actions On Affine Space HANSPETER KRAFT* Dedicated to Jacques Dixmier on his 65th birthday

Introduction

Over the last few years algebraic group actions on affine space have been intensively studied. One of the central! questions in this area is the following.

(We refer to [K2] for a survey and related problems). Linearization Problem. Is every action of a reductive algebraic group G on affine space C” linearizable, 1.e., G-equivariantly isomorphic to a linear action? For example, every reductive group action on C? is linearizable. This is a consequence of the structure of the group of algebraic automorphisms of C? as an amalgamated product, a result which can be traced back to VAN DER KULK. Such a structure theorem does not hold in higher dimension, due to an example of BAss. Nevertheless, the linearization problem has got a positive answer for semisimple groups in dimension n < 4, by the work

of KRAFT, Popov and PANYUSHEV ([KP], [Pa]). All these results depend on some “smallness” assumptions. E.g., linearization always holds whenever the only G-invariant functions are con-

stant. The more general situation where the ring of invariants has (Krull-)

dimension 1 has been studied by LUNA, SCHWARZ and KRAFT (see [KS]1], [KS2]). It turns out that in many cases one can still prove linearization, but in general there exist infinite families of nonequivalent actions on affine space. The first examples of such non-linearizable actions were found by ScHwARZ [Sch], e.g., actions of O2 on C* and of SL: on LE

Using these examples and techniques KNOP showed that for every semisimple group there exist non-linearizable actions on some C”, and *

Partially supported by Schweizerischer Nationalfonds

562

HANSPETER

KRAFT

MASUDA and PETRIE constructed non-linearizable actions of certain finite

groups on C* ([MP)). For tori the first results about the linearization problem go back to BIALYNICKI-BIRULA [BB1], [BB2]. He showed the existence of fixed points and proved linearization in case there is an orbit of codimension < 1.

In this paper we consider the smallest open case, namly C*-actions on

C3. The fixed point set (C%)©" is (non-empty and) acyclic (cf. [K2, 3.4]). If dim(C?)©” = 2 then the action is “fix-pointed” and therefore linearizable (see 6.4). In case of a one-dimensional fixed point set we have (CHA

C

since C is the only acyclic smooth curve, and linearization can also be proved (see 6.7). It remains the case of an isolated fixed point xo € Cc

which is the main object of this paper.

One of our results is the following (see Theorem 5.3; a more general

version was obtained by KORAS and RUSSELL, see [KR1], [KR2], [KR3]). Theorem. Assume that the quotient CYC* linearizable.

is smooth. Then the action is

It is still an open question whether every C*-action on C* is linearizable. In fact, we don’t know of any example of a non-linearizable action of a

torus on C”. Outline of the paper. In §1 we show that a semifree action on a smooth affine variety X with a unique fixed point zo € X and with quotient

X//C* = C? is linearizable (Theorem 1.3). In §2 we give an equivariant version of this where the extra group is finite cyclic (Theorem 2.4). A crucial step in the proof of these two results is a decomposition property for the equivariant automorphisms of the tangent space T;,X (Proposition 1.7

and 2.3). This is discussed in §3. Now we drop the assumption that the action is semifree and assume instead that X is smooth and factorial of dimension 3 with an isolated fixed point x9 € X. Then the “zero fiber” 1~!2(x9) of the quotient map x: X — X/C* contains a plane Fo ~ C?; it is given by an irreducible semi-invariant function f. Define F := f-1(1). We show in §4 that the action on X is linearizable if (and only if) F ~ C? (Theorem 4.4).

It turns out that this is always the case if X is in addition acyclic and the quotient isomorphic to C? (Corollary 5.2). From this the Theorem above follows easily by using s result of FUJITA, MIYANISHI and SUGIE which gives an algebraic characterization of C? (see 5.3). The lacking of a similar characterization of C”, n > 3, is one of the main obstructions for extending our results to higher dimension.

The last paragraph handles some cases where dimX©

> 1. More

C*-ACTIONS ON AFFINE SPACE

563

The last paragraph handles some cases where dimX©° > 1. More generally we show that a torus action on C” with one-dimensional fixed point set and two-dimensional quotient is linearizable (Corollary 6.7). Conventions.

Throughout the paper the base field is the field C of com-

plex numbers. If X is a variety we denote by O(X)

the algebra of (global)

regular functions on X. Every action of an algebraic group G on X is assumed to be algebraic, i.e., the corresponding map G x X — X is a mor-

phism. It then follows that G acts on the regular functions O(X) by means of algebra automorphisms and that this representation is locally finite and

algebraic. (For this and the following see [K1]). We denote by O(X)® the subalgebra of invariant functions.

By HILBERT’s famous theorem the invariant algebra O(X)° is finitely generated in case X is affine and G reductive. We write X//G for the corresponding affine variety, and mx : X — X//G for the morphism associated to the inclusion O(X)° + O(X). The variety X//G is called the quotient of X by G, and rx is the quotient map. The quotient map has a number of remarkable properties, e.g., it is surjective and parametrizes the closed

orbits in X (cf. [K2, §4]). An important tool for algebraic transformation

groups is the “slice

theorem” of Luna ([Lul]; cf. [K2, §7]). We assume that the reader is familiar with this result and refer to [SI] for a detailed discussion and some applications.

Acknowledgments: I wish to thank Gerald Schwarz and Friedrich Knop for helpful discussions about the matter of this paper.

§1.

Semifree actions

Let X be an affine variety with an action of the multiplicative group C*. 1.1. Definition. The action of C* on X is called semifree if the isotropy group of a point of any closed orbit is either trivial or the whole group C*.

1.2. Example. Consider the linear action of C* on C* with weights (—a, 8,7) where a, 3,7 > 0. This means that TAZ)

= (t~% x,t? y, t7z),

forte

C*, (x,y,z) € C7

We assume that the representation is faithful, i.e., that gcd(a, 8,7) = 1. This action is semifree if and only if ged(a,B) = gced(a,y) = 1. If, in

564

HANSPETER

KRAFT

addition, the quotient C3/C* is smooth then a = 1, and vice versa. In this case the quotient map 7 : C? — C? is given by (z, y,z) + (x? y, 27z). 1.3. Theorem. Let X be a smooth affine variety of dimension 3. Assume that C* acts semifreely on X with a unique fired point ro € X and that XJ/C* = C?. Then the action is linearizable, t.e., X is C*-equivariantly isomorphic to the tangent representation T;,(X). For the proof we need some preparation. First, we remark that the represen-

tation on V := T;,(X) is semifree and has a smooth quotient: V/C* + C?; this follows from the assumptions and the slice theorem. As a consequence, the weights on V are (—1,8,7) where 8,7 > 0 and the quotient map my : V = C? — C? is given by my(z, y, z) = (28y,27z) with respect to suit-

able coordinates (see Example 1.2). We fix an identification X//C* = C? such that rx(xo) = 0. In order to obtain a C*-equivariant

isomorphism

y : X = V we proceed in three steps:

(I) We construct an ep aaey g:X > V of the corp loments of the zero fibers X := X \ rx1(0) and V := V \rÿ!(0) such that Tyop

=

TX.

(II) From the slice theorem we obtain an isomorphism ¢ : XIV

of

étale neighborhoods X 3 7x1(0) and V D 21(0) of the zero fibers. (III) Finally, we modify the isomorphisms ¢ and ¢ in such a way that they coincide on the intersection XX. This will define, by gluing, an isomorphism y: X => V. The same method applies to higher dimensional varieties X and yields the following result. Only step III needs some modification; we leave the details to the reader. 1.4. Theorem. Consider a semifree action of C* on a smooth affine variety X with a unique fized point ro € X. Assume that XfC* = C* for some k. Then X is C*-equivariantly isomorphic to T;,X.

1.5. We now give the details of the construction outlined above.

Step I: Define C? := C? \ {0}. Since the actions on X and V are both semifree (with a unique fixed point) it follows from the slice theorem that the induced maps

my : V — C2? and

rx : X — C2

are principal C*-fibrations. They are trivial, because C? is factorial and so every line bundle on C? or C? is trivial. Thus we get a C*-equivariant

C*-ACTIONS ON AFFINE SPACE

565

isomorphism ¢ : X © V with a commutative diagram

FAR

rx | C2

oS

lv

(+)

Cc?

This finishes the first step. Step II:

We

put C? =

Spec C[u, v]. There is a canonical morphism

C? — C? from which we obtain the following fiber products (which can be seen as limits over all “saturated” étale neighborhoods! of the zero fiber):

Vee

VV

Nixes ©

CAG?

OY Ces

ee

C7

The coordinate ring of V is given by

O(V) = O(V) ®ctu,v] C[u,v] where

u:=2%y, v:= 272,

1.e., it is the C*-finite part of the completion of O(V) in the ideal generated by u and v (cf. [Ma]), and similarly for X, where we use the identification O(X)S == C[u, v]. It is again a consequence of the slice theorem that there is a C*-equivariant isomorphism ¢ : X © V. This finishes the second step. Step III:

The two isomorphisms ¢ and ¢ both induce C*-equivariant

isomorphisms X © V of the “intersections” Ù

X

V

XX =

= X\7z!(0) = X xer C?

and

VOnV:=V\r; (0) =V xe C2.

This is clear for @ since @ maps rx (0) onto Ty (0). For ¢ it follows from the diagram (*) by base change. The next lemma is easy. 1.6. Lemma. Assume that ¢ and $ induce the same tsomorphism LGA Then there is a C*-equivariant isomorphism y : X = V such that ply = 9

and ply = 9 Proof. Clearly, we have O(V),O(V) € OV) in a canonical way, and O(V) = O(V)NO(V) since every function f € O(V)\O(V) has poles along 1

saturated means inverse image of a subset of the quotient

566

HANSPETER

KRAFT

the plane {x = 0} C my (0) and does therefore not belong to O(V). Similarly, O(X), O(X) € O(X), and OX) = O(X) N O(X).. By assumption,

the two isomorphisms ¢* : O(V) © O(X) and ¢* :O(V) TZ= O(X) are both = O(X ). Hence, Ÿ deterinduced from the same isomorphism Ÿ : O(V) =>

mines an isomorphism y* : O(V) = O(V)NO(V) = O(X)NO(X)= O(X). Q.E.D. We cannot assume a priori that y and ¢ induce the same peut Sn x = V. Therefore, we consider the automorphism p := go! of V. By the following proposition there is a C*-equivariant automorphism Ÿ of V such that the composition op extends to:an automorphism Ÿ of V. Putting Pi pow : X SV and ÿ: := pop :À > V we obtain

Pilz = dopop = Wiyopy = Pilz. By Lemma 1.6 this finishes the last step of the proof of Theorem 1.3. Q.E.D.

1.7. Proposition. Let p be a C*-equivariant automorphism of V. Then there is a C*-equivariant automorphism Ÿ of V such that the composition pop extends to an automorphism Ÿ of V. This is a special case of Proposition 2.3 of the next paragraph whose proof will be given in 3.3.

§2.

A generalization

For our main results we need a more general version of Theorem 1.3.

2.1. Consider the linear action of C* on W := C® with weights (—1,8,“7) where 8,7 > 0 (see Example 1.2). Let a be a positive integer. Assume that the cyclic group fa := {s € C* | s* = 1} acts on W by C*-equivariant automorphisms. The zero fiber W° := 14/(0) is the union of the (y, z)plane and the z-axis, and both components are stable under the action of Ha. In particular, there is a À € Z such that s -(z,0,0) = (s*z,0,0) for all SE flag, TEC. 2.2. Theorem.

The C* x u,-action on W 1s linearizable.

Proof. We follow the same steps as in the proof of Theorem 1.3. Let V := ToW be the tangent representation of C* x yz in the origin. By assumption,

V/C* and W/C* are both isomorphic to C?. Using the slice theorem we see that the .-actions on the quotients are locally isomorphic (i.e., induce

C*-ACTIONS ON AFFINE SPACE

567

equivalent tangent representation at my (0) and my(0)). Since every action of a reductive group on C? is linearizable (see [K2, 1.3 Corollary 2]) we can identify W/C* and V/C* with C? where pa acts linearly on C?. There is a C*-equivariant isomorphism C* x W/C* = W \ {x = 0} given by (t, (u, v)) + (t,t?u,t7v), and similarly for V. It is easy to see that every lift of the p,-action on W//C* to an action on C* x W//C* commuting with the C*-action is of the form s - (t, w) = (s*t,s - w) with some À € Z. But this À is the same for W and V: It is given by the action of y, on the z-axis (see 2.1). Hence there is a C* x u,-equivariant isomorphism

gp: W\{z =0}

> V \ {x = 0} with a commutative diagram

W\{z=0}

—

V\{z=0}

This finishes the first step. The second step is again a consequence of the slice theorem: There is a C* x pa-equivariant isomorphism W — V. For the last step we want to apply again Lemma 1.6. It is clear that we can follow the same lines as in the proof of Theorem 1.3 once we have established the following generalization of Proposition 1.7. This will finish the proof of Theorem 2.2. Q.E.D.

2.3. Proposition. Let p be a C* x pa-equivariant automorphism of V. Then there is a C* x p,-automorphism Ÿ of V such that the composition pop extends to an automorphism ÿ of V. (The proof will be given in the next paragraph.)

Combining 1.3 and 2.2 we obtain the following result. 2.4. Theorem. Let X be a smooth affine variety of dimension 3. Assume that C* x fla acts on X satisfying the following conditions:

(a) The C*-action is semifree with a unique fired point ro € X. (b) X/C* = C?. Then X is C* x pa-equivariantly isomorphic to T,,X.

568

HANSPETER §3.

KRAFT

Automorphisms

of V

Let Auto C[u, v] denote the group of those automorphisms of the C-algebra C[u,v]which stabilize the homogeneous maximal ideal. We have a canonical inclusion Auto C[u, v] — Aut Cu, v].

3.1. Lemma.

Given an automorphism o € AutCl[u,v]

number k.> 0, there is an n € AutgC[u,v] & := noo satisfies the following condition:

and a natural

such that the composition

(Ce) &(v) = vf + ukh for suitable h € Cfu], f € C[u,v], f(0,0) = 1. (Condition C; essentially means that the power series G(v)|,-0 has a zero of order > k in u = 0.)

Proof. First we find an m € GL2(C) C Auto Clu,v] such that o1 := moo has the form oi(u)=u+a, œi(v) = v +b where a,b € (u,v)? C Cfu,v]. In particular, o1 satisfies condition Co. Next, by induction, we may assume that 0, satisfies condition C;. Define m € Auto Clu,v] by no(u) = u, n2(v) = v — h(0)u'. An easy calculation then shows that & := 7200 satisfies condition C;,1. Q.E.D. 3.2. Remark. In the notations of the lemma above assume in addition that the cyclic group fg acts linearly on Clu,v] with weights 8,7, 1.e., s-u=s?uand s-v = s7v. Then an equivariant version of the lemma holds as well: Given a pA-equivariant automorphism o of Clu,v] and a natural number

k > 0 there is a ptg-equivariant automorphism n € Auto C[u, v] such that the composition noo satisfies condition Cy of Lemma 3.1. (In fact, in the proof above the linear automorphism

1

is clearly pg-

equivariant. Hence, we can write o1(v) in the form o1(v) = vf + u'h where h € C[u], h(0) # 0 and f(0,0) = 1. Since œi(v) is a semi-invariant of weight y (with respect to fq), u’ is also a semi-invariant of weight y, ie., fi = ymoda. Hence, the automorphism m2 defined by mo(u) = u, m2(v) = v — h(0)u’ is ua-equivariant.) 3.3. We come back to the situation of § 1: V = C* is the C*-module with weights (—1,8,7) and quotient map ty : V > C?, (z,y,z) + (2%y, 212).

The zero fiber (0)

has two components, the (y, z)-plane {x = 0} and

the z-axis {y = z = 0}. We have a C*-equivariant isomorphism

C*x C?SV\{x=0},

(t,(u,v)) 6 (t,t? u,t7v)

C*-ACTIONS ON AFFINE SPACE

569

In particular, O(V \ {x = 0}) = C[u,v, 2,271] where u = xy, v = 272, and O(V \ {x = 0})© = C[u, v]. Define V := V \ {x = 0} ~ C* x C2.

Then V = V \ {z-axis}, hence O(V) = O(Ÿ) = O(V), = Cu, v][z, 271] C Cr, y, z][z7"]. Clearly, O(V)S" = OV) == C[u,v]. On the other hand, O(V) is the completion of O(V) in the ideal generated by u and v (see 1.5, Step II), and so O(V) C Cfr, y, z] in a canonical way. It follows that

O(Ÿ) = O(V)n Cr, y,z]. Proof of Proposition 2.3: The C* x s,-automorphism p of V induces an

automorphism p* of O(V) which stabilizes the C*-invariants C[u,v]. We have to find a C* x u,-automorphism w of V such that the composition yop extends to an automorphism of V. The automorphism p defines a u,4-

automorphism o of Clu,v] and satisfies p*(x) = xe, where e € Cu, v]* is a unit which is fixed under y. (This follows from the fact that the units of O(V) are all of the form z‘e where i € Z and e € C[u, v]*.) Clearly, there is a C* x ty-automorphism of V which fixes the invariants and sends z to ex: It is given by z + ex,y + e~fy,z + e~7z. Hence, we may assume that p* fixes z,

p'(z)=z,

p(ul=p,

P(r) =4,

where p,q € C[u, v]. Such an automorphism

p extends to a morphism

p:V — V if (and only if) the two functions p*(y) and p*(z) belong to

O(V), i.e.,

2

(2% y,2%z) € Cfz,y,z],

2 q(xy,z2) € Cr,y,z].

(++)

(We use the fact mentioned above that O(V) = O(V)n Cfr, y, z]). We claim that this condition can always be satisfied by a suitable change of coordinates. In addition, we will see that this extension p is automatically an isomorphism.

Case 1: B = y. Here condition (++) is automatically satisfied because p and q belong to the maximal ideal (u,v) C C[u,v]. In addition, the Jacobian of f in the origin (with respect to the variables x, y, z) is given by

ye at 0 Jacp(0) = det | x galo gelo | — Ja p(0) 4.0.

* fl

lo

570

HANSPETER

KRAFT

Hence, p extends to an automorphism p of V. Case 2: 3 < y. Here the first condition in (**) is clearly satisfied, and the second is equivalent to the fact that the power series q(u,v) has the form

q(u,v) = vf +ufh where f € C[u,v], h € Clu] and kB > 7. It follows from 3.1 and 3.2 that there is a p,-equivariant automorphism 7 of C[u,v], stabilizing the homogeneous maximal ideal (u,v), such that the second component q := p(v) of the composition ÿ = nop has the required form. This automorphism

n extends to a C* x u,-automorphism % of C[u, v,z,2~'] by putting 7(z) = z. It is clear now that the corresponding automorphism of V\{x = 0} leaves

the z-axis invariant (since 7 stabilizes the ideal (y, z) C O(V)) and defines an C* x pA-equivariant automorphisin Ÿ of V which has the property that the composition Wop satisfies the condition (++), 1.e., extends to a morphism : V — V. It remains to see that f is an isomorphism. Again, one easily calculates the Jacobian and finds

1 0 Jacp(0) = det | x lo *

0

0 x

| =Jacp(0) #0

Q

oe

(We use the fact that q(u,v) = vf + u*h where k > 2). Hence, f is an isomorphism. This finishes the proof of Proposition 2.3. Q.E.D.

§4.

The hyperplane F

We begin with an example which will serve as a guideline for the further considerations in this paragraph.

4.1. Example. Consider the linear action of C* on V = C® with weights

(—a, 8,7), a, 8,7 > 0, gcd(a, 8,7) = 1. The plane

E := {(1,y,z)|y,z€ C} ~ CO? is stable under the subgroup pa of C*, and the quotient map my : V —

V/C* induces an isomorphism 7 : E/u, — V//C*. Consider the (reduced)

fiber product TE

V Wit E

—

V

E

=

V/C*

C*-ACTIONS ON AFFINE SPACE

571

Claim. Then the normalization of V Xvyc: E is C*-equivariantly isomor. phic to the representation V with weights (—1, 3,7). Proof. Consider the finite C*-equivariant morphism € : V — V given by (æ; y,2) b> (x 1Y,2), and the quotient morphism 7 : VE; (x,y,z) bh (1, 2% y, z7z). It is easy to see that the compositions Ty 0G and TEOTÿ are equal. Hence, we obtain a finite C*-morphism n : V V Xvyc* E which induces the tie on the quotient:

iB

V

i

ets

—

neni E

E

—

SED

ise

V

VIC*

Since ¢ and 7g have both the same degree a the morphism 7 is birational, and the claim follows. Q.E.D. We remark that C* x pg acts on V, where the action of Ha is given by s-(z,y,z) = (s*%z,y,z), and that ¢ = Ton is the quotient under pa. 4.2. For the rest of the paragraph we consider an affine variety faithful C*-action which satisfies the following assumptions:

X with a

(i) X is irreducible of dimension 3; (ii) X is smooth and factorial; (iii) zo € X is an isolated fixed point. The tangent representation V := T,,X has weights (—a, 8,7), a, 8,y > 0, gcd(a,,7) = 1. As before, we denote by mx : X — X//C* and zy : V — V/C* the quotient maps. As a consequence of the slice theorem

we know that the zero fibers X° := 2y'(4x(zo)) and V° := my'(v(0)) are C*-isomorphic.

Clearly, V° is the union of the (y,z)-plane and the

z-axis. Since X is factorial the two-dimensional component Fo(~ C?) of X° is defined by an irreducible function f € O(X), and this function is a semi-invariant of weight —a, i.e., f(t: w) = t~*f(w) for w € X. Define Bees aix.

4.3. Lemma. F is stable under fig and (C* x F)/p1q = X \ f~'(0) where Ba acts by s-(t,w) := (ts~',s-w) fors € pa, t € C*, we F. In particular, F is smooth and mx induces an isomorphism 7 : F/u, = XfC*. Proof. Since f : X — C is a semi-invariant of weight —a, we see that X = C*FU f-{(0). Hence, we have a surjection C* x F — Vie 1(0) sien

by (t, w) + t-w, which induces an isomorphism (C* x F)/a = X\f-1(0). Now the ni

follows easily.

Q.E.D.

572

HANSPETER

4.4. Theorem.

Assume

KRAFT

that F = C?. Then the C*-action

on X

is lin-

earizable, i.e., X is C*-equivariantly isomorphic to T,X.

Proof. (a) Since every reductive group action on C? is linearizable (see [K2, 1.3 Corollary 2]) we can assume that F = C? with a linear action of pa with weights (8,7). Consider the diagram

Peis

es

ee

[rs

erat

Wx

fa

F

os [ex

——

KXfC*

where tp := 7x |p is the quotient by ta, X Xxyc- F is the (reduced) fiber product, and 7: X > X X xfC* F the normalization (cf. Example 4.1). Clearly, C* x fq acts on X such that 7 is equivariant. The composition ¢ := pon is the quotient by wa, and mz is the quotient by C*. Since

C* x F is smooth and X \ f-1(0) = (C* x F)/pa we obtain

tp (X\FT (0) & Ct x F > GTX\ f-1(0)). Hence, X is smooth outside the zero fiber X° := wy (0).

(b) We now claim that X is everywhere smooth. If the C*-action on X is linearizable this follows from Example 4.1. In general, X is smooth in a neighborhood of the zero fiber X° by the slice theorem. In fact, setting

Q := Spec O(X//C* )x(p) and F := Spec O(F)o where the hat À denotes the completion of the local ring in the maximal ideal, we obtain by base change the following commutative diagram: n ——

D) Ay4——

Again, # is the normalization of the (reduced) fiber product X x, F’. This diagram is isomorphic to the corresponding diagram in the linear situation, hence X is isomorphic to V with a C* x pa-equivariant isomorphism. On

the other hand, we have seen in (a) that X \ X° is smooth, hence the claim. (c) The considerations above also show that the C*-action on X is semifree with a unique fixed point ro € 1 (0). Thus we can apply Theorem 2.4 to obtain a C* x pg-equivariant isomorphism ¢ : X3V:= TAX By construction, the composition X — X Xxyc+ F — X is the quotient by

C*-ACTIONS ON AFFINE SPACE

ka and therefore

573

X = X/u, = V/ua = V, where the isomorphisms are all

C*-equivariant.

Q.E.D.

4.5. Remark. Assume that X is isomorphic to C?. Then the semi-invariant f € Cfr,y, z] defining the two-dimensional component Fy of the zero fiber

in C® (see 4.2) has the following properties:

(i) Fo = f-1(0) = C’; (ii) f : C°\ Fo — C is an (étale) fibration with fiber F := TRE) Question. Do these conditions imply that F is isomorphic to C?2? As we have seen above, a positive answer would imply that any C*-action on C? is linearizable.

§5.

Reduction to the semifree case

Let X, V, a, B, y, etc. be as in 4.2. In addition to the conditions (i), (ii)

and (iii) of 4.2 we assume that (iv) X is acyclic. Here acyclic means

that X has the Z-homology of a point. Similarly we

define Z/p-acyclic. Let d be one of the numbers gcd(a, B), gcd(a,y), gcd(8, 7), and consider the subgroup pg C C* acting on X. 5.1. Proposition. If the C*-action on X/ig is linearizable, then the C*action on X 1s linearizable as well. This result allows to reduce the linearization problem to the case where gcd(a,3) = gcd(a,y) = gcd(B,7) = 1. This means that we can assume that the action on X is semifree. In fact, if X is acyclic and the action on the tangent space T,,X semifree, then the action on X is semifree, too. (The fixed point set in X of every cyclic p-group is connected ([Bo, III.4.6]); thus, it is contained in the zero fiber). A first consequence is the following result: 5.2. Corollary. If X//C* is isomorphic to C? then the C*-action on X is linearizable.

Proof. By 5.1 we can assume that the action is semifree. Since the quotient

is smooth we have a = 1 (see Example 1.2), and so F > F/pa — C?. Now the claim follows from Theorem 4.4.

Q.E.D.

574

HANSPETER

KRAFT

From this we obtain the theorem mentioned in the introduction.

5.3. Theorem. Consider a C*-action on C® and assume that the quotient C°/C* is smooth. Then the action is linearizable.

Proof. If the quotient is zero-dimensional this is an immediate consequence of the slice theorem [Lul]. For a one-dimensional quotient we can apply

the methods developed by KRAFT and SCHWARZ ([KS1, Theorem 0.2(3)], cf. [KS2]). Finally, the characterization

of C? due to FUJITA,

MIYANISHI

SUGIE shows that a smooth two-dimensional quotient of the form is isomorphic to C? (see [K2, Theorem 2.2]). If the fixed point set dimensional then the claim follows from Proposition 5.1. Otherwise apply Corollary 6.7.

and

C°/C* is zerowe can Q.E.D.

5.4. Remark. In [KR2] one can find more general conditions which imply linearization. They are formulated in terms of the type of the singularity

of mx (x9) € (C*)/C*.

For the proof of Proposition 5.1 we need the following two results. 5.5. Proposition. Let Z be an affine smooth factorial C*-variety, and let pe € C* be the cyclic subgroup of order e. Assume that there is a C*-equivariant isomorphism ç@ : Z/ue = W where W is a C*-module, such that the image of the fixed point set Z4* in W is a linear subspace of codimension 1. Then the C*-action on Z 1s linearizable.

Proof. By assumption, we have O(Z)#« = C[z,,22,...,2%,] where the 2; are algebraically independent semi-invariants. Let f = 0 be the equation defining the hypersurface Z4 C. Choose a C*-equivariant linear projection V = ToV — ToS. This projection induces an isomorphism y : S > TS. In particular, there is a C*-equivariant co; ofV which maps S onto a coordinate plane.

C*-ACTIONS ON AFFINE SPACE

575

Proof. It follows from the assumptions that W := TyS has a one-dimension-

al quotient W//C* = C, and we get the following diagram:

SF

“|

Peso,

SIGE

AoE

ie

Since S//C* = C we see that ¢ is a finite (surjective) morphism. On the other hand, ¢ is étale at 0, hence ¢ is étale at m5(0) ([Lu2, 1.3 Lemme fondamental]; cf. [S1, § 4]). Furthermore, y~!(0) = {0} because y is C*equivariant, and so ¢ is an isomorphism. But this implies that ¢ is bijective, hence an isomorphism, too.

The second claim is clear: S is the graph of the function f := prog”! : W = C.

Q.E.D.

Proof of Proposition 5.1: If d divides gcd(G,7) then X“¢ is equal to Fo, the two-dimensional component of the zero fiber of X (4.2). It follows that the image of X#¢ in the C*-module X/y4 is the coordinate plane spanned by the positive weight vectors. Hence, the claim follows from Proposition 5.5.

Now consider the case where d divides gcd(a, B), the remaining one is similar. Here, the image of X#4 in the C*-module X/pq is a smooth surface S containing the origin. Let p be a prime divisor of d. Then X#¢ =

Xe, hence X44 is Z/p-acyclic ([Bo, III.4.6]). This implies that SYÿC* = X#a/C* = C ([KPR, Theorem A]). Hence, we can apply Lemma 5.6, and the claim follows again from Proposition 5.5.

§6.

Q.E.D.

Actions with many fixed points

6.1. Let X be a smooth affine variety with an action of a torus 7’. Assume that the fixed point set X7 is connected and non-empty. The aim of this

paragraph is to compare X with the normal bundle N := N(X7) of the fixed point set X7 and to find conditions under which they are isomorphic. Recall that the normal bundle is a T-vector bundle over X7 and that N is trivial, i.e., N © XT x W where W is a T-module, in case every vector

bundle on XT is trivial ([K2, Corollary 2.1]). We denote by p: N — XT the canonical projection. The zero section of N induces an isomorphism oo : XT — NT and the quotient map 7x : X — X/T an isomorphism XT > rx (XT). We use these maps to identify

XT with NT and with 7x(X7).

576

HANSPETER KRAFT

6.2. Proposition. Assume that there is a retraction p: X — XT which is T-equivariant.

Then there exists a T-equivariant morphism p : X — N

such that plxr = oo, and an open neighborhood U of ax(X7) in XT such that the diagram

Xe

nel (U)

FX

À TR SAV

ae

Tx

hn

OW AN

SN PT)

is cartesian with étale morphisms yp and G.

Outline of Proof: Let a C O(X) be the ideal of XT and put À := O(XT) = O(X)/a. Then a/a? is a projective A-module and O(N) ~ Sx(a/a?), the symmetric A-algebra of a/a?. The retraction p defines an inclusion

p* : A

O(X) such that O(X) = AG

a. It is easy to see that there exists

a T-stable A-submodule P C asuch that P@a? = a. Hence, P is projective

and S4(P) = O(N). It follows that the canonical homomorphism S4(P) — O(X) gives rise to a T-equivariant morphism y : X — N extending oo. By construction, ¢ is étale in a neighborhood of X7. Now the claim is a consequence of the slice theorem (see [S], § 4]). Q.E.D. An action on X is called fir-pointed if every closed orbit is a fixed point, or equivalently, if the quotient map mx induces an isomorphism X7 = X/T. The next two corollaries follow immediately from the proposition above (cf. [BH, 10.3, 10.5] or [K2, 5.5]). 6.3. Corollary. Assume that the action on X is fiz-pointed. T-equivariantly isomorphic to the normal bundle N.

6.4. Corollary.

Consider a fiz-pointed action of T on C”.

Then X is

Then C” is

T-equivariantly isomorphic to (C")T x W where W is a T-module. In particular, if dim(C”)? < 2 then the action is linearizable. Proof. Since every vector bundle on C” is trivial by QUILLEN and SUSLIN,

the same holds for (C”)”, because there is a retraction p : C” — (C”)?. It follows that the normal bundle N is a trivial T-vector bundle (6.1), hence the first claim. The second part is a consequence of the characterization of C? due to FUJITA, MIYANISHI and SUGIE which implies that CT = C?

(cf. 5.3).

Q.E.D.

6.5. Remark. It is clear that Proposition 6.2 and its corollaries hold for any reductive group G.

Now assume that the image 1(X7) of the fixed point set is of codimension

C*-ACTIONS ON AFFINE SPACE

977

lin X/T. Then N/T is a line bundle over 1(X7). Hence, if XT is factorial then N/T= XT x C. This suggests to make the following assumptions: 6.6. Theorem.

Assume that the following conditions hold:

(i) The action on X is semifree; (ii) There is an isomorphism yp: XT x C © X//T which sends (2,0) to

n(x); (iii) XT (or X) is factorial and O(XT)* = C*. Then X is T-equivariantly isomorphic to the normal bundle N.

Proof. It follows from (iii) that the line bundle N/T — x(X7) is trivial. Hence, the quotient map can be written as ty : N — X7™xC, 2+

(p(z), z)

where ty (Az)= (p(x), Az) for a suitable d and all À € C. Similarly, we can assume that 7x :X — XT x C satisfies x(x)= (x,0) for x € XT. Let X and N denote the completions with respect to the ideal defining the

Teron

A

t= ay (A) and N° t= ey CX); respectively. Then, by

Proposition 6.2, there is a T-equivariant isomorphism ¢ : X = N which induces an italics phish OX x C = XT x C of the quotient.

We claim that we can arrange that ¢ = id. In fact, the automorphism

~ is given by a morphism r : XT — C[{]*. Since O(XT)* = C* there exists a 7: XT — AGE such that F(z)° = r(x). Now define ¢; : À > N by $1(z):= F(p(z))* -@(x). Then ¢ induces the identity on the quotient. Define X :=X \ X° and N := N \ N°. By assumption (i), X and N are principal T-fiber bundles overr XT x |C, hence trivial by assumption (iii). Thus, we get an isomorphism Ÿ : ies N which induces the identity on the quotient X7 x C.

Consider the automorphism p := goy™! of N:= N\N°. It remains to see that every such p can be written as a product p = pop where p extends

to an automorphism of N and p to an automorphism of N (cf. 1. 6). But this is clear: p is given by a morphism p : XT x CEA

(where C :

CNC = Spec C((t)) which does not depend of X7, and since (y = C[t] : Cft,t-!] we obtain the required decomposition. Q.E.D. 6.7. Corollary.

Let T act on C”. Assume

that there exists an orbit of

codimension < 2 and that dim(C”)? > 1. Then the action is linearizable. Proof. By assumption, we have dimC"/T < 2. Since the fixed point set has dimension > 1 if follows from the slice theorem that the quotient is smooth. Hence, either the action is fix-pointed and therefore linearizable

by 6.4, or else C"//T = C? and (C”)? = C (see 5.3). But this implies that

578

HANSPETER

KRAFT

C'ÎT = 1(C”) x C by the theorem of ABHYANKAR-MOH [AM]. It is clear now that all conditions of Theorem 6.6 are satisfied, and the claim follows

Q.E.D.

since the normal bundle N is trivial (see 6.1).

REFERENCES

[AM] [BH] [BB1]

Abhyankar, S.S; Moh, T.-T.: Embeddings of the line in the plane.

J. Reine Angew. Math. 276 (1975), 149-166 Bass, H.; Haboush, W.: Linearizing certain reductive group actions.

Trans. Amer. Math. Soc. 292 (1985), 463-482 Bialynicki-Birula, A.: Remarks on the action of an algebraic torus

on k", I. Bull. Acad. Polon. Sci. Sér. Sci. Math. 14 (1966), 177-181

[BBQ]

Bialynicki-Birula, A.: Remarks on the action of an algebraic torus

on k”, IT. Bull. Acad. Polon. Sci. Sér. Sci. Math. 15 (1967), 123125

[Bo] [KR1]

Borel, A.: Seminar on transformation groups. Ann. of Mathematical Studies 46, Princeton University Press, 1960 Koras, M.; Russell, P.: G-actions on A. Canad. Math. Soc. Con-

fer. Proc. 6 (1986), 269-276

[KR2]

Koras, M.; Russell, P.: On linearizing “good” C*-actions on C®. Proceedings of the Conference on “Group Actions and Invariant Theory”, Montreal 1988. Canad. Math. Soc. Confer. Proc. 10 (1989), 93-102

[KR3]

Koras, M.; Russell, P.: Codimension

2 torus actions on affine n-

space. Proceedings of the Conference on “Group Actions and Invariant Theory”, Montreal 1988. Canad. Math. Soc. Confer. Proc. 10 (1989), 103-110

[K1] [K2]

Kraft, H.: Geometrische Methoden in der Invariantentheorie. Aspekte der Mathematik D1, Vieweg-Verlag, Braunschweig 1984 Kraft, H.: Algebraic automorphisms of affine space. In: “Topological methods in algebraic transformation groups,” edited by H.

Kraft et al. Progress in Math. 80 (1989), 81-105, Birkhauser Verlag

[K3]

Kraft, H.: G-vector bundles and the linearization problem. Proceedings of the Conference on “Group Actions and Invariant Theory”,

Montreal 1988. Canad. Math. Soc. Confer. Proc. 10 (1989), 111123

[KPR]

Kraft, H.; Petrie, T.; Randall, J.: Quotient varieties. Adv. in Math. 74 (1989), 145-162

C*-ACTIONS ON AFFINE SPACE

[KP]

[KS1]

579

Kraft, H.; Popov, V. L.: Semisimple group actions on the three dimensional affine space are linear. Comment. Math. Helv. 60 (1985), 466-479 Kraft, H.; Schwarz, G.: Linearizing reductive group actions on affine space with one-dimensional quotient. Proceedings of the Conference on “Group Actions and Invariant Theory”, Montreal 1988.

Canad. Math. Soc. Confer. Proc. 10 (1989), 125-132

[KS2]

Kraft, H.; Schwarz, G.: Reductive group actions on affine space with one-dimensional quotient. Preprint 1990 , to appear

[Lu1]

Luna, D.: Slices (1973), 81-105

[Lu2]

Luna, D.: Adérences (1975), 231-238

[Ma]

Magid, A. R.: Equivariant completions and tensor products. Proceedings of the Conference on “Group Actions and Invariant Theory”, Montreal 1988. Canad. Math. Soc. Confer. Proc. 10 (1989), 133-136 Masuda, M; Petrie, T.: Equivariant Serre conjecture and algebraic

[MP]

étales.

Bull.

d’orbite

actions on C”. Preprint

Soc. Math.

France,

et invariants.

Mémoire

Invent.

Math.

33 29

(1990)

[Pa]

Panyushev, D.I.: Semisimple automorphism groups of four-dimensional affine space. Math. USSR-Izv. 23 (1984), 171-183

[Sch]

Schwarz, G. W.: Exotic algebraic group actions. C. R. Acad. Sci. Paris 309 (1989), 89-94 Slodowy, P.: Der Scheibensatz fur algebraische Transformationsgruppen. In: Algebraic Tranformation Groups and Invariant The-

[Sl]

ory, DMV Seminar 13 (1989), 89-113, Birkhauser Verlag Received April 2, 1990

Hanspeter Kraft Mathematisches Institut Universitat Basel Rheinsprung 21 CH-4051 Basel Switzerland

igs SALe Set Ai: à (te Me

a:

ph

oe eli

i

‘

an

nes

‘

ae

Debit. 7 FA Med MA dry et RAIN PRPs D TUE LES FANPRES ALAM Lege we nase id sg Ho

‘ur nn On , alter qe ÉD

ee ral

DELL cuis gris = Ko vifighs aa PART TEL im ae =< a 2 LE Mmpind bte Ari ne dem cn wr meaty A at px is sf

HAUT

©

re, ete b

NX Ben

;

CHIEN

dl