Unitary Representations of Reductive Lie Groups. (AM-118), Volume 118 9781400882380

Citation preview

Annals o f Mathematics Studies Number 118

U N IT A R Y REPRESENTATIONS OF REDUCTIVE LIE GROUPS

BY

D A V I D A. V O G A N , JR.

PRINCETON UNIVERSITY PRESS

PRINCETON, NEW JERSEY

1987

Copyright © 1987 by Princeton University Press ALL RIGHTS RESERVED

The Annals of Mathematics Studies are edited by W illiam Browder, Robert P. Langlands, John Milnor, and Elias M. Stein Corresponding editors: Stefan Hildebrandt, H. Blaine Lawson, Louis Nirenberg, and David Vogan, Jr.

Clothbound editions of Princeton University Press books are printed on acid-free paper, and binding m aterials are chosen for strength and durability. Pa­ perbacks, while satisfactory for personal collections, are not usually suitable for library rebinding

ISBN 0-691-08481-5 (cloth) ISBN 0-691-08482-3 (paper)

Printed in the United States of America by Princeton University Press, 41 W illiam Street Princeton, New Jersey

☆

Library of Congress Cataloging in Publication data will be found on the last printed page of this book

To my parents, for keeping the first draft

CONTENTS

ACKNOWLEDGEMENTS

ix

INTRODUCTION

CHAPTER 1

3

COMPACT GROUPS AND THE BOREL-WEIL THEOREM

19

CHAPTER 2

HARISH-CHANDRA MODULES

50

CHAPTER 3

PARABOLIC INDUCTION

62

CHAPTER 4

STEIN COMPLEMENTARY SERIES AND THE UNITARY DUAL OF GL(n,(C)

CHAPTER 5

COHOMOLOGICAL PARABOLIC INDUCTION: ANALYTIC THEORY

CHAPTER 6

105

COHOMOLOGICAL PARABOLIC INDUCTION: ALGEBRAIC THEORY

INTERLUDE:

S2

THE IDEA OF UNIPOTENT REPRESENTATIONS

vii

123 159

viii

CHAPTER 7

CONTENTS

FINITE GROUPS AND UNIPOTENT REPRE­ SENTATIONS

CHAPTER 8

LANGLANDS’ PRINCIPLE OF FUNCTORIALITY AND UNIPOTENT REPRESENTATIONS

CHAPTER 9

CHAPTER 13

REFERENCES

235

K-MULTIPLICITIES AND UNIPOTENT REPRESENTATIONS

CHAPTER 12

211

THE ORBIT METHOD AND UNIPOTENT REPRESENTATIONS

CHAPTER 11

185

PRIMITIVE IDEALS AND UNIPOTENT REPRESENTATIONS

CHAPTER 10

164

258

ON THE DEFINITION OF UNIPOTENT REPRESENTATIONS

284

EXHAUSTION

290

302

ACKN0WLEDG3EMENTS

T h is book is b ased v e ry L e c tu re s g iv e n a t P rin c e to n ,

th e I n s t i t u t e

in January,

1986.

L ang lan d s and th e I n s t i t u t e th o se

le c tu re s ,

l o o s e l y o n t h e H erm a n n Weyl

and fo r

f o r Advanced S tu d y in

I am v e r y g r a t e f u l for

th e i n v i t a t i o n

to R o b ert

to d e l iv e r

t h e i r h o s p i t a l i t y d u r i n g my v i s i t .

D u rin g 1985 an d 1986,

t h e r e w as a s e m i n a r a t MIT o n u n i p o -

te n t re p re se n ta tio n s;

I am g r a t e f u l

for

to a l l

th e p a r t i c i p a n t s

t h e i r u n f la g g in g d e d i c a t i o n to m a th em atical

f o r s t a y i n g awake m ost o f th e tim e .

tru th ,

S u san a S ala m an c a-R ib a

a n d J e f f Adams h a v e e x a m i n e d p a r t s o f t h e m a n u s c r i p t , p ro v id ed h e lp fu l a d v ic e

and

th ro u g h o u t i t s w r itin g .

m ade d e t a i l e d c o m m e n ts o n t h e e n t i r e m a n u s c r i p t ; few p l a c e s w h e re I d i d n o t f o l l o w h i s a d v i c e ,

and

T o n y K n app in th o se

th e re a d e r

w i l l p r o b a b ly w ish I h ad . M o st o f a l l ,

I m u s t t h a n k D an B a r b a s c h .

Many o f t h e

id e a s on u n i p o t e n t r e p r e s e n t a t i o n s d i s c u s s e d h e r e a r e

ix

jo in t

X

ACKNOWLEDGEMENTS

work, and all of them are influenced by the deep results he has obtained on unitary representations of classical groups Our collaboration is a continuing mathematical and personal pleasure.

This work was supported in part by grants from the National Science Foundation and the Sloan Foundation.

Unitary Representations of Reductive Lie Groups

INTRODUCTION

Perhaps the most fundamental goal of abstract harmonic analysis is to understand the actions of groups on spaces of functions.

Sometimes this goal appears in a slightly dis­

guised form, as when one studies systems of differential equations invariant under a group; or it may be made quite explicit, as in the representation-theoretic theory of automorphic forms.

Interesting particular examples of problems

of this kind abound.

Generously interpreted, they may in

fact be made to include a significant fraction of all of mathematics.

A rather smaller number are related to the

subject matter of this book. Let

X

be a pseudo-Riemannian manifold, and

group of isometries of sure, and

G

Here are some of them.

acts on

X. Then

2

L (X)

X

G

a

carries a natural mea-

by unitary operators.

Often

(for example, if the metric is positive definite and com­ plete) the Laplace-Beltrami operator adjoint.

In that case,

G

A

on

X

is self-

will preserve its spectral

3

INTRODUCTION

4

decomposition.

Conversely, if the action of

tive, then any

G-invariant subspace of

preserved by

A.

The problem of finding

G

2

L (X)

is transiwill be

G-invariant sub­

spaces therefore refines the spectral problem for

A.

The prototypical example of this nature is the sphere Sn

with

G

the orthogonal group

0(n).

least 2, the minimal invariant subspaces for on

2 n ^ L (S )

Laplacian.

If

n

0(n)

is at acting

are precisely the eigenspaces of the spherical (This is the abstract part of the theory of

spherical harmonics.) Fourier series.

If

n

is 2, we are talking about

The fundamental importance of these is

clear; but they may of course be analyzed without explicit discussion of groups.

For

n = 3, the theory of spherical

harmonics leads to the solution of the Schrodinger equation for the hydrogen atom.

Here the clarifying role of the

group is less easy to overlook, and it was in this connec­ tion that the "Gruppenpest" entered quantum physics in an explicit way. A second example, still in the framework of pseudoRiemannian manifolds, is the wave operator.

Viewed on a

four-dimensional space-time manifold, this is just the Laplace-Beltrami operator for a metric of signature (3,1). If the manifold has a large isometry group (for instance, if

5

INTRODUCTION

it is Minkowski space), then the space of solutions can often be described terms of this group action. An example with a rather different flavor is the space X

of lattices (that is, discrete subgroups isomorphic to

Zn ) in

IRn .

An automorphic form for

smooth function on

G = GL(n,IR)

X, subject to some technical growth and

finiteness conditions.

(Actually it is convenient to consi­

der at the same time various covering spaces of (for fixed basis of

p) the space of lattices

L/pL.)

is a

L

X, such as

endowed with a

It is easy to imagine that functions on

X

have something to do with number theory, and this is the case.

One goal of the representation-theoretic theory of

automorphic forms is to understand the action of space of automorphic forms. sure on

X

Because the

G

on the

G-invariant mea­

has finite total mass (although

X

is not com­

pact), this problem is closely connected to the corresponding

L

2

problem.

An introduction to this problem may be

found in [Arthur, 1979], Finally, suppose space.

X

is a compact locally symmetric

(Local symmetry means that

-Id

on each tangent

space exponentiates to a local isometry of is a compact Riemann surface.) deRham cohomology groups of

X.

X.

An example

We seek to understand the Here there is no group

6

INTRODUCTION

action in evidence, and no space of functions.

However,

Hodge theory relates the cohomology to harmonic forms on so the latter defect is not serious. over

X

For the former, we

consider the bundle

Y

whose fiber at

(compact) group

of local isometries of

Harmonic forms on

X

valued functions.

On the other hand,

transitive group

G

pull back to

acting on it.

Y

(G

cohomology of of

G

G X

X

p

is the

fixing

p.

as certain vectorY

has a large

may be taken to be

the isometry group of the universal cover of the quotient of

X,

X;

by the fundamental group of

Y X.)

is then The

can now be studied in terms of the action

on functions on

Y.

Perhaps surprisingly, this has

turned out to be a useful approach (see [Borel-Wallach, 1980]). With these examples in mind, we recall very briefly the program for studying such problems which had emerged by 1950 or so.

The first idea was to formalize the notion of group

actions on function spaces.

In accordance with the general

philosophy of functional analysis, the point is to forget where the function space came from.

Definition 0.1.

Suppose

sentation of

is a pair

G

topological vector space

G

is a topological group. A repre­ (tt,V)

consisting of a complex

V, and a homomorphism

ir from

G

7

INTRODUCTION

to the group of automorphisms of from

GxV

to

V.

We assume that the map

V, given by (g,v) -» ir(g)v

is continuous. is a subspace ators

ir(g)

An invariant subspace of the representation W

of

(for

g

V

which is preserved by all the oper­

in

G).

The representation is called

reducible if there is a closed invariant subspace than V

V

itself and {0}.

is not zero, and

ir

We say that

tt

W

other

is irreducible if

is not reducible.

The problem of understanding group actions on spaces of functions can now be formalized in two parts: we want first to understand how general representations are built from irreducible representations, and then to understand irredu­ cible representations.

This book is concerned almost exclu­

sively with the second part.

Nevertheless, we may hope to

gain a little insight into the first part along the way, much as one may study architecture by studying bricks. If we take

G

to be

Z, then a representation is deter­

mined by a single bounded invertible operator, tt(1). only interesting irreducible representations of

G

The are the

one-dimensional ones (sending 1 to a non-zero complex num­ ber).

The decomposition problem in this case amounts to

trying to diagonalize the operator

tt(1).

There are some

8

INTRODUCTION

things to say about a single operator; but on infinite­ dimensional spaces, one needs more hypotheses to begin to develop a reasonable theory.

The easiest assumption to use

is that the space is a Hilbert space and that the operator commutes with its adjoint. mal.)

(Such operators are called nor­

In that case, the space can be decomposed in some

sense into an "integral" of eigenspaces of the operator. Once we decide to focus on group actions on Hilbert spaces, it is easy to see how these could arise naturally in our original problem.

If G acts in a measure-preserving

way on a measure space

X, then it acts by unitary operators

on

2

L (X).

The continuity condition in the definition of

representation comes down to this: if of finite measure, and g.S

differs from

S

g

is a

only in a

S

is a subset of

small element of

X

G, then

set of small measure.

This

is clear for smooth Lie group actions on manifolds with smooth densities.

The following definition therefore admits

many examples.

Definition 0.2. tary if

#

The representation

(tt,Sff)

is called uni­

is a Hilbert space and the operators

7r(g)

unitary (that is, they preserve the inner product in #).

are

INTRODUCTION

All of the

L

2

9

spaces mentioned in the examples provide

examples of unitary representations. If

^

is a closed invariant subspace of the unitary

representation of

(tr,#), then the orthogonal complement

is also invariant, and

3f =

y

e sr1 .

We would like to iterate this process, and finally write as a direct sum of irreducible unitary representations. see why this is not possible, take

2

translation on

L (IR).

G

to be

S

of

To

IR, acting by

Any invariant subspace

corresponds to a measurable subset

#

^

of

H

IR, by

2 i ^ ^ = (f € L (IR) I f vanishes almost everywhere outside S}.

A

(Here

f

denotes the Fourier transform of

measure space

f.)

Since the

IR has no atoms, it follows that any non-zero

invariant subspace of

2

L (IR)

has a proper invariant sub­

space. On the other hand, the Fourier transform in this exampie does exhibit

2

L (IR)

as a sort of continuous (or measur­

able) direct sum of translation invariant "subspaces," con­ sisting of functions with Fourier transform supported at a single point

f.

These spaces are one-dimensional (consist­

10

INTRODUCTION

ing of multiples of the function

exp(ixf)) and therefore

irreducible. A fundamental theorem, going back to [Mautner, 1951], guarantees the existence of such a decomposition in great generality.

The proof is based on von Neumann’s theory of

rings of operators.

THEOREM 0.3 (cf. [Dixmier, 1981]).

Let

separable locally compact group, and let unitary representation of

G.

Then

G

be a type I (ir,3if)

be a

ir may be written

uniquely as a direct integral of irreducible unitary repre­ sentations of

G.

For the definitions of direct integral and type I, we refer to [Dixmier, 1981].

All Lie groups (even over local fields)

with countably many connected components are separable and locally compact.

Type I Lie groups include nilpotent, reduc

tive, and algebraic ones.

Examples of groups not of type I

are free groups on more than one generator, and certain solv able Lie groups. There is no analogous general theorem for decomposing non-unitary representations; yet these may be of the most direct interest in applications.

(We are rarely content to

know only that a solution of a differential equation exists

INTRODUCTION

in

2

L ).

Fortunately,

L

2

11

harmonic analysis often provides

a guide and a tool for studying other function spaces, like L

D

00

or

C .

We will not pursue this topic further; one

place to begin to look is in [Helgason, 1984]. Because of the theorem of Mautner, it is of particular interest to understand the irreducible unitary representa­ tions of a group

Definition O.k.

G.

Suppose

G

is a topological group.

The

set of equivalence classes of irreducible unitary represenA

tations of

G

is written

G . u

A

We will later use the notation

G

for a certain larger

class of irreducible representations of a reductive Lie group

G.

The irreducible unitary representations of characters of the circle.

IR, the continuous homomorphsms of

y

IR

into

These are just the functions of the form x

with

IR are the

real.

e

ixy J

With proper hindsight, the characters of fi­

nite abelian groups can be found in classical number theory. By the late nineteenth century, Frobenius and Schur were beginning to study the irreducible representations of nonabelian finite groups.

In the 1920's, Weyl extended their

12

INTRODUCTION

ideas to compact Lie groups ([Weyl, 1925], [Peter-Weyl, 1927]).

(In fact, because of the simple structure of com­

pact groups, Weyl’s results in that case were substantially more complete than those available for finite groups.

Even

today, the irreducible representations of compact connected Lie groups are far better understood than those of finite simple groups.)

Weyl’s work is in many respects the begin­

ning of the representation theory of reductive groups; parts of it are summarized in Chapter 1. In the 1930*s, the representation theory of noncompact groups began to be studied seriously. The books [Pontriagin, 1939] and [Weil, 1940] each contain (among other things) a rather complete treatment of the unitary representations of locally compact abelian groups.

At the same time, quantum

mechanics suggested the problem of studying the Heisenberg group by

Hn *

n+1

This is the

2n+l-dimensional Lie group of

matrices having l ’s on the diagonal, and

n+1

0 ’s every­

where else except in the first row and the last column. is nearly abelian: its commutator subgroup coincides with its one-dimensional center.

Its unitary representations

were determined completely by the Stone-von Neumann theorem (cf. [von Neumann, 1931]). In [Wigner, 1939], the physicist Eugene Wigner (still motivated by quantum mechanics) made a study of the irredu-

13

INTRODUCTION

cible representations of the Poincare group.

This is the

group of orientation-preserving isometries of (the pseudo4

Riemannian manifold)

IR

with the Lorentz metric.

He found

all of those which were of interest to him, but did not suc­ ceed in obtaining a complete explicit description of

G^.

The missing representations he constructed in terms of the irreducible unitary representations of two subgroups of the Poincare group: the Lorentz group

S0(3,l)

(the isotropy

4

group of the origin in

IR ), and its subgroup

S0(2,l).

Wigner*s analysis was based on studying the restriction of a unitary representation to normal subgroups. that it could not succeed for

S0(2,l)

and

The reason

S0(3,l)

is

that these groups (or rather their identity components) are simple.

Within a few years, their irreducible unitary repre­

sentations had been determined, in [Bargmann, 1947] and [Gelfand-Naimark, 1947].

We will return to this branch of

the development in a moment. Around 1950, Mackey extended Wigner*s methods enormous­ ly, into a powerful tool ("the Mackey machine") for studying unitary representations of subgroup

N

of

G

in terms of those of a normal

G, and of the quotient

work may be found in [Mackey, 1976].

G/N.

Some of his

His results were ap­

plied to successively larger classes of groups over the next thirty years; two high points are [Kirillov, 1962] and

INTRODUCTION

14

[Auslander-Kostant, 1971].

For algebraic Lie groups, this

study was brought to a fairly satisfactory culmination by the following theorem of Duflo.

(Notice that it is a direct

generalization of the result of Wigner for the Poincare group.)

THEOREM 0.5 ([Duflo, 1982]).

Let

G

be an algebraic group

over a local field of characteristic 0. ble unitary representations of

G

Then the irreduci­

may be explicitly para­

metrized in terms of those of certain reductive subgroups of G.

This is a slight oversimplification, but the precise state­ ment is as useful as (and more explicit than) this one. There is a similar statement for all type I Lie groups, involving semisimple Lie groups instead of reductive ones. Some problems remain - for example, the explicit construc­ tion of the representations is not as satisfactory as their parametrization - but to some extent the study of

G^

is

reduced to the case of reductive groups.

Here the subject

matter of the book itself really begins.

We will conclude

the introduction with a quick outline of the material to be discussed. We begin with a definition.

INTRODUCTION

Definition 0.6.

A Lie group

Gis called simple if

a)

dim G

b)

G

c)

any proper normal subgroup of the identity

is greater than one;

has only finitely many connected components;

component of We say that

G G

is finite. is reductive if it has finitely many con­

nected components, component groups.

15

Gq

and some finite cover of theidentity

is a product of simple and abelian Lie

It is semisimple if there are no abelian factors in

this decomposition.

Finally,

G

is said to be of Harish-

Chandra’s class if the automorphisms of the complexified Lie algebra defined coming from

Ad(G)

are all inner.

This definition of reductive is chosen to please no one.

It

does not include all the semisimple groups needed in Duflo*s theorem for non-algebraic groups; but it is significantly weaker (in terms of the kind of disconnectedness allowed) than most of the definitions generally used. The lowest dimensional noncompact simple group is SL(2,IR), the group of two by two real matrices of determi­ nant one. of

S0(2,l)

It is a two-fold cover of the identity component and its irreducible unitary representations

were determined in the paper [Bargmann, 1947] mentioned

16

INTRODUCTION

earlier.

We will not give a detailed account of his work;

this may be found in many places, including [Knapp, 1986] and [Taylor, 1986].

The answer, however, contains hints of

much of what is now known in general.

We will therefore

give a qualitative outline of it, as a framework for describ­ ing the contents of the book. First, there are two series of representations with an unbounded continuous parameter (the principal series).

The

representations are constructed by real analysis methods. An appropriate generalization, due to Gelfand and Naimark, is in Chapter 3.

These representations tend to contribute

to both the discrete and the continuous parts of direct integral decompositions of function space representations. Second, there is a series of representations with a discrete parameter (the discrete series). nally constructed by complex analysis.

These were origi­

They can contribute

only as direct summands in direct integral decompositions. Harish-Chandra found an analogue of this series for any reductive group in [Harish-Chandra, 1966]; this paper is one of the great achievements of mathematics in this century. brief discussion of its results appears in Chapter 5.

The

most powerful and general construction of such representa­ tions now available is an algebraic analogue of complex

A

17

INTRODUCTION

analysis on certain homogeneous spaces, due to Zuckerman. It is presented in Chapter 6. Third, there is a series with a bounded continuous para­ meter (the complementary series).

They are constructed from

the principal series by an analytic continuation argument. One hopes not to need them for most harmonic analysis prob­ lems; for automorphic forms on

SL(2,R), this hope is called

the Ramanujan-Petersson conjecture.

The construction of com­

plementary series for general groups will be discussed in Chapter 4; but we will make no attempt to be as general as possible.

Complementary series constructions have been

intensively investigated for many years (see for example [Baldoni-Silva-Knapp, 1984] and the references therein), but the precise limits of their applicability are still not at all clear. Finally, there is the trivial representation.

For a

general reductive group, the trivial representation belongs to a finite family of "unipotent representations.”

In addi­

tion to a name, we have given to these objects a local habi­ tation in Chapters 7 through 12.

All that they lack is a

complete definition, a reasonable construction, a nice gen­ eral proof of unitarity, and a good character theory. information can be found in [S, 1594].)

(More

INTRODUCTION

18

The latter part of the book is largely devoted to a search for a definition of "unipotent."

A vague discussion

of what is wanted appears in the Interlude preceding Chapter 7.

Chapters 7 through 11 discuss various subjects closely

related to the representation theory of reductive groups, to see what they suggest about the definition. Chapter 12 is a short summary of some of the main results of the search; it contains a partial definition of unipotent. Implicit in this discussion is the hope that the ideas described here suffice to produce all the irreducible uni­ tary representations of any reductive group G.

Because the

constructions of complementary series and unipotent represen­ tations are still undergoing improvement, this hope is as yet not precisely defined, much less realized.

Neverthe­

less, a wide variety of partial results (for special groups or representations) is available. cussed in Chapter 13.

Some of these are dis­

I hope that the reader will be not

disappointed by this incompleteness, but enticed by the work still to be done.

Chapter 1 COMPACT GROUPS AND THE BOREL-WEIL THEOREM

Our goal in this chapter is to recall the Cartan-Weyl description of the irreducible unitary representations of compact Lie groups, together with the Borel-Weil realization of these representations.

A good general reference for the

material is [Wallach, 1973].

The absence of more specific

references for omitted proofs in this chapter is not in­ tended to indicate that the results are obvious. We begin with a little general notation.

If

G

is a

Lie group, we will write G0 = identity component of go = Lie(G) (1.1)

3 = go

G

(the Lie algebra) ^

(the comp 1exi f icat ion of

U(g) = universal enveloping algebra of (go) g

gQ) g

= real-valued linear functionals on

gQ

= complex-valued linear functionals on

19

gQ .

20

CHAPTER 1

The circle group

¥

of complex numbers of absolute

value 1 has Lie algebra exponential map for tion.

ilR.

¥

With this identification, the

is just the usual exponential func­

We often identify

¥

with the unitary operators on a

one-dimensional Hilbert space. tive group

X (D

Similarly, the multiplica-

of non-zero complex numbers has Lie algebra

_a

a€ A+ Since

t

is the comp1exification of a real subalgebra, it

is equal to its complex conjugate. fc

The complex conjugate of

is therefore

(c)

fc” = t © n~

By (1.12), (d)

ifl

b ’ = t.

We define the Carton subgroup to be (e)

the normalizer in

K

T

associated to

T0

and

A+

of fc:

T = {t € K| Ad(t)fc C *}.

Later (Definition 1.28) we will generalize this definition, allowing certain subgroups between

T

and

T0 .

The special

case defined by (e) will then be called a large Carton sub­ group .

27

COMPACT GROUPS AND BOREL-WEIL THEOREM

It is easy to show that the Lie algebra normalizer in

t.

fc

is its own

Conseqently the Lie algebra of

fl ?0 , which equals

fQ

identity component of

T

T is

by (d) in Definition 1.14. is therefore

The

T0 . A more subtle

point is that (1.15)

T PI K0 = T0 .

For connected compact Lie groups, a Cartan subgroup is therefore just a maximal torus. Fix now a negative definite inner product *o, invariant under compact.

Ad(K); this is possible since

Its restriction to

duality

i(*o)acquires a

(which we also

is

A

in

itQ

?, still denoted

is positive definite, so by

positive definite inner product

call ).

Definition 1.16. weight

K

on

The comp1exification of this inner product is a

non-degenerate symmetric bilinear form on < , >.

< , >

In the setting of Definition 1.14, a

T0

is called dominant if > 0,

for every root

a

in

A+ .A representation

(tt,V) of

T

is called dominant if every weight occurring in its restric­ tion to

T0

is dominant.

28

CHAPTER 1

THEOREM 1.17 (Cartan-Weyl). Suppose group, and

T

K

is a compact Lie

is a Carton subgroup (Definition 1.1*t). A

There is a bijection between the set unitary representations of

K

and the set of irreducible

dominant unitary representations of defined as follows. representation of hilated by under

n

If

(tt.V)

K, define

of irreducible

T

(Definition 1.16),

is an irreducible unitary

V+

to be the subspace anni­

(Definition 1.1k).

Then

V+

is invariant

T, and the corresponding representation

ir+

on

V+

is irreducible and dominant.

A

Let us consider the extent to which this "computes" K

K^.

If

is connected, then Theorem 1.17 says that the irreducible

representations of weights.

K

are parametrized by the dominant

This is a completely computable and satisfactory

parametrization (cf. Example 1.18 below), although of course one can ask much more about how the representation is re­ lated to the weight.

If

K

is finite, then

T

is equal to

K, and the theorem is a tautology; it provides no informaA

tion about

K^.

In general, if

K

is not connected then

T

is not connected either, and its representation theory can be difficult to describe explicitly.

Essentially the prob­

lem is one about finite groups, and we should not be unhappy to treat it separately.

29

COMPACT GROUPS AND BOREL-WEIL THEOREM

Example 1.18.

Suppose

K

is

U(n), T = T0

is the group

of diagonal matrices, and A+ = { e j - e j l

(cf. Example 1.8).

i

< j}

If we Identify

f*

with

C11, we may

take < , > to be the standard quadratic form.

The set of

dominant weights is then {X = (A, 1 v 1

X ) I X. € Z, and X. > X. if i < j}, ny1 l i " J

the set of non-increasing sequences of n integers.

We turn now to the problem of realizing the representa­ tions described in Theorem 1.17.

To do so requires con­

structing a certain complex manifold on which transitively.

closed subgroup.

h

G/H of

acts

Here is a general recipe for doing that.

PROPOSITION 1.19.

on

K

Suppose

G

The set of

Is a Lie group and

H

is a

G-invariant complex structures

is in natural bijection with the set of subalgebras (the complexified Lie algebra of

q

g), having the

following properties:

b>~

a)

b

contains

b)

the intersection of

is precisely

bf, and

bf; and

Ad(H)

preserves

b>;

b> with its complex conjugate

CHAPTER 1

30

c) g/fy.

the dimension of

is half the dimension of

In the usual identification of

plexified tangent space of and

G/H

at

with the com­ eH, the subspaces

h/fy

correspond to the holomorphic and antiholomorphic

tangent spaces, respectively.

Proposition 1.21 will explain how to describe the holomor­ phic functions on G/H in terms

PROPOSITION 1.20.

Suppose

of fc.

G

is a Lie group and

H

Is a

00

closed subgroup. on

G/H

The set of

C

is in natural bijection with the set of finite­

dimensional representations of bundle

W

to its fiber

we may identify the space W

homogeneous vector bundles

with the space of

W

H, by sending a vector

over

eH.

oo C (G/H,W)

Using this bijection, of smooth sections of

W-ualued smooth functions

f

on

G,

satisfying a) Here of course

f(gh) = T(h_1)f(g).

denotes the

isotropy

action ofH onW.

A proof of this result will be

sketched

in Chapter3(Propo­

t

sition 3.2 and Corollary 3.4).

COMPACT GROUPS AND BOREL-WEIL THEOREM

PROPOSITION 1.21.

Suppose

G/H

complex structure given by homogeneous vector bundle 1.20).

Then to rnahe

W

carries an invariant (Proposition 1.19) and a

given by

W

(Proposition

W a holomorphic vector bundle amounts representation (also called t )

to giving a Lie algebra on

fc

31

of

W, satisfying

a)

the differential of the group representation

agrees with the Lie algebra representation restricted

of H to 1/;

and b)

for

h

in

H, X

in

h ~ , and

w

in

r(h)[r(X)w] = T(Ad(h)X)[r(h)v] The space

on

.

T(G/H,Jf)of holomorphic sections of

identified with the space of

W,

W

may

W-valued smooth functions

be £

G, satisfying Proposition 1.20(a) and the following con­

dition.

For every

X

in

we require that

(Xf)(g) = -T(X)(f(g)). Here the action on the left comes from regarding the Lie algebra as left-invariant vector fields on

G.

To set up the Borel-Weil theorem, we need just one more observation.

LEMMA 1.22. complement of

In the setting of Theorem 1.17, the orthogonal V+

In

V

Is

32

CHAPTER 1

V° = 7r(n")V. Consequently,

V+

may be identified (as a representation of

T) with the quotient

Proof. of

Write

a

V/V°.

for the complex conjugation automorphism

t:

(1.23)

cr(X + iY) = X - iY

The inner product

< , >

on

V

(X,Y

in

*)•

satisfies

= - (Here as usual we write it for the differentiated represen­ tation of the Lie algebra.)

Since

cr(n’)

is

n, an easy

formal argument now shows that the orthogonal complement of 7r(n~)V

is

V +. The assertion of the lemma follows.

THEOREM 1.24 (Borel-Weil). Suppose group,

T

K

Is a Carton subgroup, and

algebra normalized by

T

irreducible representation

differentiated representation of act trivially. Write vector bundle on

K/T

is a compact Lie b

is a Bor el sub­

(Definition 1.1k). (t ,W)

of t

□

Fix an

T, and extend the to

b~

by making

n~

Hf for the resulting holomorphic (Propositions 1.19, 1.20, and 1.21).

Let V = r( K/T,fT),

33

COMPACT GROUPS AND BOREL-WEIL THEOREM

K

act on

V

by left translation of sections; the resulting

representation is denoted and only if that case,

(t,W) (tt.V)

attached to

tt.

Then

(tt.V)

is non-zero if

is a dominant representation of

is the irreducible representation of

(t,W)

by Theorem 1.17.

That is,

naturally isomorphic to the representation on the subspace of

Example 1.25.

OP1

V

annihilated by

Suppose

diagonal matrices. space

K

K

is

(t,W)

(ir+ fV +)

In K is

of

T

n.

U(2), and

T

is the group of

acts transitively on the projective

of complex lines in

(C2 . The stabilizer of the

line through the second coordinate is identified with

T.

(DP1.

T; so

K/T

may be

This defines the complex structure.

A typical homogeneous line bundle is the tautological bundle, which puts over every point of it "is."

(DP1

the line which

In the identification of homogeneous bundles with

characters of

T, which are in turn identified with

tautological bundle corresponds to

(0,1).

Z2 , the

That weight is

not dominant (cf. 1.18); so the Borel-Weil theorem says that the bundle should have no sections.

This well-known fact

simply means that the only holomorphic way to pick a point in each line in

C2

is to pick zero everywhere.

CHAPTER 1

34

The dual line bundle to the tautological bundle asso­ ciates to each line the space of linear functionals on it; this corresponds to the weight

(0,-1), which is dominant.

We can find global sections of the bundle by fixing a linear functional on

(C2

and restricting it to each of the lines.

All the holomorphic sections arise in this way; so the space of sections forms a two-dimensional representation of

K.

We are going to prove the Borel-Weil theorem; more pre­ cisely, we will show how to deduce it from Theorem 1.17.

It

is helpful to introduce a definition.

Definition 1.26.

Suppose

complex Lie algebra.

H

is a Lie group, and

h

is a

Assume that we are given

i) an inclusion of the complexified Lie algebra H

into

of

h ; and

ii) an action, denoted

Ad, of

H

on

fc by automor­

phisms, extending the adjoint action on A

(fc,H)-module is a complex vector space

V

(possibly

infinite-dimensional), carrying a group representation of and a Lie algebra representation of lowing three conditions.

H

fc, subject to the fol­

(For the moment we will write

for these representations, but later it will usually be

ir

35

COMPACT GROUPS AND BOREL-WEIL THEOREM

conenient to drop this in favor of module notation: instead of a) smooth.

h*v

7r(h)v.)

The group representation is locally finite and That is, if

v € V, the vector space span of

is finite-dimensional, and

H

7r(H)v

acts smoothly (or, equivalent­

ly, continuously) on this space. b)

The differential of the group representation (which

exists by (a)) is the restriction to

bf of the Lie algebra

representation. c)

The group representation and Lie algebra represen­

tation are compatible in the sense that 7r(h)7r(X) = 7r(Ad(h)X)7r(h) For elements h in If

V

and

H0 , condition (c) follows from (b).

W

are

(16 ,H)-modules, we can form

Horn^ jj(V,W). This is the space of linear transformations from

V

to

W

that are compatible with both representa­

tions .

The next result is a version of Frobenius reciprocity appropriate for the Borel-Weil theorem.

PROPOSITION 1.27. closed subgroup,

Suppose 6

G

is a Lie group, H

is a

defines a holomorphic structure on

(Proposition 1.19), and

W

G/H

is a (b~,H)-module corresponding

36

CHAPTER 1

to a holomorphic vector bundle 1.21).

Let

V

W

on

G/H

(Proposition

be a finite-dimensional representation of

G; by differentiation and restriction, we may regard a

V

as

(fc~ ,H)-module. Then there is a natural isomorphism HomG (V,r(G/H),f) = Hom^- H (V,W) .

Proof.

Write

4> for a typical element on the left, and

for a typical element on the right.

If these correspond

under the isomorphism, then *(v) = [*(v)](e)

[ * ( v ) ]( g ) = ♦ ( f ( g ' 1W . (We are using the description of Proposition 1.21.)

T(G/H,ff)

contained in

The verification that these formulas

define the isomorphism we want is left to the reader.

Proof of Theorem 1.2k.

Let

dimensional representation of

(f,X) K.

be any finite­ By Proposition 1.27,

HomG (X,V) = Hom^- T (X,W). Now

n“

acts trivially on W; so the right side is HoiOj^X/f (n~)X],W)

By Lemma 1.22, we therefore have HomG (X,V) = Hoiar(X+,W).

□

37

COMPACT GROUPS AND BOREL-WEIL THEOREM

By Theorem 1.17, the right side is always zero unless

W

is

dominant; and in that case it is one-dimensional for a unique irreducible representation

X.

□.

We will conclude this chapter with two reformulations of Theorem 1.17, each of which will be convenient or instruc­ tive later on.

Definition 1.28.

Suppose

K

is a compact Lie group.

the notation of Definition 1.14 (a)-(d).

Write

T+

Use for the

large Cartan subgroup of Definition 1.14 (e). The small Carton subgroup associated to (a)

T0

is

T “ = {t € K| Ad(t) is trivial on t}.

A general Cartan subgroup associated to subgroup

T

between

Suppose

T

T“

and

for the normal izer of

T

K

in

(b)

If

Ad(K)

and

A+

is a

T+.

is a small Cartan

N^(T)

T0

T

in

subgroup of K.

K.

Write

The Weyl group of

is the quotient W(K,T) = Nk (T)/T.

consists of inner automorphisms (that is, if

is of Harish-Chandra*s class in the sense of Definition

K

38

CHAPTER 1

0.6), then the notions of small and large Cartan subgroup coincide. For the next definition, we need to make sense of the length of a weight. < , >

on

i(*Q)*

Definition 1.29. and

(tt,V)

This is defined using the inner product (defined before (1.16).

Suppose

T

is a Cartan subgroup of

K,

is a finite-dimensional representation of

K.

An irreducible representation in

7r

r

of

T

is called extremal

if

a)

t

occurs in the restriction of

b)

the length of any weight

A

of

tt

to

T; and

r

(cf. (1.4) is

greater than or equal to the length of any weight of

tt.

Here is the first reformulation of Theorem 1.17.

THEOREM 1.30 (Cartan-Weyl). Suppose group. pairs

Write

$(K)

(T,t), with

for the set of T

K

is a compact Lie

K-conjugacy classes of

a small Cartan subgroup of

K

and

t

A

in

T. a)

Fix a particular small Cartan subgroup

T, with A

Weyt group set of

W

W.

Then

orbits on

$(K) /V T.

may be identified with

T/W, the

COMPACT GROUPS AND BOREL-WEIL THEOREM

39 A

b) onto

There is a finite-to-one correspondence from

K

$(K). defined by associating to w the set of extremal

representations of small Cartan subgroups occurring in c)

If

K

tt.

Is of Harish~Chandra's class (Definition

0.6), then the correspondence in (b) Is a bijection.

We will not prove this result in detail.

Note, however,

that (a) is a formal consequence of the conjugacy of all maximal tori in

K.

The other fact needed in the proof (or

rather the reduction to Theorem 1.17) is that every represen­ tation of

T

is conjugate under

W

to a dominant one.

This in turn follows from Proposition 1.13. The second reformulation of Theorem 1.17 is motivated by the theory of characters (Definition 1.38 below).

It

turns out that characters of compact groups are most natu­ rally expressed as quotients of two multi-valued functions (Theorem 1.40).

These functions will be single-valued on

certain coverings of Cartan subgroups.

In order to define

these coverings, we recall a standard general construction.

Definition 1.31.

Suppose

H~

is a topological group,

is a closed normal subgroup, and H~/F.

Let

G

H

F

is the quotient group

be another topological group, and

r

a

40

CHAPTER 1

homomorphism from pullback from

G

H

to

to

G

H.

Define a new group

of

G~, the

H~, by

G~ = {(x,g) € (H~xG)| tt(x ) = T(g)}. Here

denotes the quotient map from

tt

contains a copy of

F

(as

Fx{e}).

F.

map

from

t~

to

H. Then

G~

Projection on the

second factor defines a surjection from kernel

H~

G~

to

G, with

Finally, projection on thefirst factor gives a G~

to

H~.

We therefore have a commutative

diagram

An important case is the extraction of acters. group

In that case, we take (IR)

Suppose

G

is a projective variety acts transitively on S^(IR),

S^(IR). Then the stabilizer in

is a parabolic subgroup of

is

GL(n), and

n-dimensional representation of

the Grassman variety of

Let

is the stan­ &

denote

k-dimensional subspaces of

This satisfies the hypotheses above. of the standard

G.

V

G(IR).

V.

The stabilizer in

k-dimensional subspace is the group

G

PARABOLIC INDUCTION

with

A

matrix.

in

GL(k), D

in

GL(n-k), and

The natural Levi factor

consisting of matrices with phic to

73

B

L

B

of

P

any

k

by

n-k

is the subgroup

equal to zero; it is isomor­

GL(k) x GL(n-k).

To get all parabolic subgroups of

GL(n), it suffices

to replace the Grassmanian by any partial flag variety (con­ sisting of increasing sequences of subspaces of

V

of speci­

fied dimensions). For a slightly different example, let 0

(n,n)

of linear transformations of

IR^n

G

be the group

preserving the

quadratic form

X2n> = (*1>2 + -

Let

&

+ 2 - (Xn+ 1>2 -

be the variety of totally isotropic

subspaces; that is, subspaces on which the hypotheses are satisfied.

Q

2 -

n-dimensional vanishes.

Again

We take as our base point the

subspace w = {(v,v) I v € IRn }. Restriction of linear transformations to isomorphism of the Levi subgroup

L

of

W P

provides an with

GL(n).

We

CHAPTER 3

74

leave to the reader the task of describing

P

and

L

more

explicitly.

Notice that in both of these examples, the standard maximal compact subgroup of on

#(IR).

case.)

G(IR)

still acts transitively

(This requires a small argument in the second

This is a general phenomenon, described in the fol­

lowing lemma.

LEMMA 3.15.

Suppose

G

is a reductive Liegroup, K

maximal compact subgroup, and

P

is a

is a parabolic subgroup of

G. a)

G = KP.

That is, every element of G

ten (not uniquely) in in

the form

kp, with

may be writ­

k in

K and

p

P. b)

The homogeneous space

G/P

may be identified with

K/(K n P). c)

The modular function

K fl P; so there is a is a function on

G

6

of

G/P

is trivial on

K-invariant measure on transforming according to

G/P. If 6

Definition 3.8), then L p u(x)dx = f w(x)dx = f w(K)dK. J^/r JK/(KTIP) JK

Here is a little more useful structure.

(cf.

a)

75

PARABOLIC INDUCTION

Definition 3.16.

Suppose

reductive Lie group as in (3.13).

G.

P

is a parabolic subgroup of the

Choose a Levi subgroup

of

Recall from (2.1) the Cartan subspace

the -1 eigenspace of

0.

A

G sQ .

Define

aQ = (center of

(3.17)(a)

L

tG) H sQ

= exp(tt0)

®(m )0 = orthogonal complement of aQ in mo = (*o H tQ) © s(m )0 M = (L fl K)exp(s(m)0)

The abelian group

A

is isomorphic by the exponential map

to its Lie algebra; we call N

A

a vector group.

for the unipotent radical of

If we write

P, then we have a Langlands

decomposition (3.17)(b)

P = MAN,

a semidirect product with each factor normalizing the suc­ ceeding ones.

In particular,

L = MA.

If

G

is connected,

or is the group of real points of a connected algebraic group, then M commutes with

A.

In any case,

M

is a reduc­

is the semidirect product of

M

and

tive group with compact center.

Because

L

and the identity component of

M

acts trivially on

A,

A, it

76

CHAPTER 3

is very easy to describe the representations of ly in terms of those of

M

and

A.

L

explicit­

We leave this task to

the reader, however.

Definition 3.18.

Suppose

G

is a reductive Lie group, and

P = LN = MAN is a Langlands decomposition of a parabolic subgroup of Let

be a unitary representation of

as a representation of

L.

We can regard

P, by making N act trivially; we may

occasionally denote this extended representation by The representation (parabolically) induced from by

G.

L

$ ® to

1

.

G

is = Indp(* ® 1).

The space of tions

f

4> will be written

from

G

to the space

It consists of func­ Ik. of . as follows. from

G

to

a) in

The space 9

of

(g)f](x) = ) = Indp(if 0 u 0 1),

a series of representations parabolically induced from to

G.

MA

In accordance with Definition 4.1, we can regard all

of these representations as realized on the same Hilbert space, with the same restriction to

(4.10)(b)

K.

By Proposition 4.9,

Ip (f ® u)h = Ip(f 0 (uh )).

By Lemma 1.2, we may identify

A A

with

X a .

In this identi­

fication, the Hermitian dual of a representation corresponds the to negative complex conjugate of a linear functional; so

(4.10)(b)’

Ip (f ® v)h = Ip(f 0 (-v)).

At least in the case that Ip(f ® v) is irreducible,Proposi­ tion 4.8 therefore says that we are seeking conditions under which

Ip(f ® i>)

isequivalent to

Fix an element w for the image of

w

in

Ip(? ® (~v)) •

K, normalizing MA.

Write

in

W(G,MA) = Nk (MA)/(M n K). Then w acts on (4.10) (c)

A M

and (linearly) on wfSf.

x a .

Assume that

w

90

CHAPTER 4

For unitary characters

v

of

A, Theorem 3.19(a) now guar­

antees that (4.10)(d)

Ip (f * ») S Ip (f « t o ).

The next theorem meromorphically continues this equivalence to non-uni tary

v.

THEOREM 4.11 ([Knapp-Stein, 1980]).

In the setting just

described, there is a rational family of intertwining oper­ ators {A(w-v)| v € a*}, with the following properties. Write Hilbert space of all the a)

For v

H

for the common

Ip(f ® v).

outside a countable

locally finite union

Z

of proper algebraic subvarieties of a , A(w*d) is a linear isomorphism from b)

Forv

equivalence of c)

Any

to itself.

not in

Ip(f ® v)

If w2*v = v, and

In particular, if If

with

is an infinitesimal

Ip(f ® wu).

v

is not in

Is a

Z, then

= A(w'--wv).

wu = - d , then

Ip(f®u)

A(w-u)

v.

A(w'-v)

e)

A(w'-v)

K-finite matrix entry of

rational function of d)

Z,

A(w’v)

is irreducible, then

is Hermitian. v

is not in

Z.

STEIN COMPLEMENTARY SERIES

91

Knapp and Stein prove this result by constructing the inter­ twining operators as explicit integral operators for certain u, then proving that they can be continued meromorphically. This approach leads to a wealth of detailed information about the operators and their connections with harmonic anal­ ysis.

We will be content with the properties listed here,

however.

For that, a fairly easy non-construetive proof can

be given.

Here is an outline of it.

Sketch of proof.

We begin by writing explicitly the inter­

twining conditions being imposed.

They are

(4.12)(a)

A(w:u)Ip (f®u)(k) = Ip(f®wu)(k)A(w:u)

(k € K)

(4.12) (b)

A(w:u)Ip (f®i>}(Z) = Ip (jf®wi>) (Z)A(w: u)

(Z € 9 ) .

Of course (4.12)(b) may be replaced by

( 4 . 1 2 ) ( b ) ' A(w:iOIp (f*i>)(u) = I p (f*wiO(u)A(w:i>)

(u € U ( g ) ) .

For simplicity, assume that there is an irreducible re­ presentation

p

1i.

for the set of

Write

&

ducible, and

3>Q

of

(e),

R R

that occurs with multiplicity one in v

such that Ip(f 0 v) is irre-

for the intersection of

It is known that union

K

is the complement of a locally finite

of algebraic subvarieties in will turn out to contain

be strictly larger.)

with (ia0) •

a .

(Because of

Z; it will almost always

In particular,

is Zariski dense

92

CHAPTER 4

in

a .

For

v

in

$Qt define

infinitesimal equivalence from

A(w:d) to be the unique Ip(f ® v)

that restricts to the identity on prove (c) for

v

Zariski dense.

LEMMA 4.13. tion of way.

Ht . The main point is to

$0 \ this makes sense since

in

%

Z

in

g.

is

In the setting (k.10), the ac­

depends on the parameter

More precisely, choose a basis

Then there are operators linearly on

$0

This fact in turn depends on

Fix

Z

in

to Ip(f ® wi>)

v

X^,...,X

Tq.T^,...,^

on

in an affine of

a.

(depending

Z ) , such that r Ip(f ® »)(Z) = T 0 +

J

«(X.)T.

i=l

The lemma may be proved in a straightforward way by differen­ tiating (4.5); we omit the details.

COROLLARY 4.14. {v.} l

of

tors in

In the setting of Theorem k.ll, fix a basis

H . Suppose

(w.)

p

H^.

Is any other finite set of vec-

J

Then we can find elements

depending rationally on

{tjLj }

of

U(g),

v, so that

w . = >u ..v. J

L

ji

l

i for all

v

for which the rational functions are defined.

93

STEIN COMPLEMENTARY SERIES

Sketch of proof.

Possibly after expanding the set of

somewhat, one can find finitely many elements

of

w.

J

U(g),

such that

(4.15)

qkv. = I fikjfuKj.

with the coefficients f polynomial in irreducibility of inal v).

v.

Because of the

Ip(f ® v), we can arrange for the orig­

w ’s to be in the span of the various

Qj^i

(for most

Solving (4.15) by row reduction gives the corollary. □

The fact that A(w-u) depends rationally on

v

is now a

formal consequence of the intertwining condition (4.12)(b)’, Corollary 4.14, and Lemma 4.13. Because

A(w-v)

decomposition of

^

This is Theorem 4.11(c).

commutes with into

K, it preserves the

K-primary subspaces (cf. (2.8));

we write

(4.16)

A(w'v) = ^ A(w-v)^

accordingly. tion from 6

a

Each of the summands is now a rational functo

of the poles of

Obviously

Z

End(^).

Define

Z

to be the union over

A(w*d)^, and the zeros of det(A(w^u)^).

satisfies the last assertion of Theorem

4.11(a); that the union is locally finite will follow from (e) and the remarks after (4.12).

94

CHAPTER 4

By Theorem 4.11(c) and Lemma 4.13, both sides of the equations (4.12) depend rationally on

v.

Since they are

true by definition on the Zariski dense set true everywhere.

they are

This is Theorem 4.11(b).

A formal argument shows that the adjoint intertwines

Ip(f ® (”wu)}

and

Ip(f ® (-u)).

the adjoint is still the identity on therefore agrees with w

(-w d )

is equal to

lent to

A(w:(-wu))

A(w^u) Furthermore,

X . By definition, it

for

v

in

such that

-v . This latter condition is equiva­

w2*v = v, proving Theorem 4.11(d) on

5>°.

The gen­

eral case follows by a density argument. The proof of Theorem 4.11(e) requires a little care. Suppose

vQ

is not in

Z; we want to prove that

is reducible.

If

A(w-*u0)

definition of

Z), it must not be invertible.

is well defined, then (by the

therefore a proper invariant subspace of kernel cannot be all of

K.

We may therefore assume that

Its kernel is

Ip(f ® u0)*

(The

because it does not meet

# .) JJ,

A(w*u)

has a pole at

is natural to consider the subspace on which finite at

Ip(f ® vQ)

A(w'-v)

vQ .

It

is

vQ , and to think that it should provide a proper

invariant subspace of 3!f^.

It is not invariant, however, as

a careful attempt to write the obvious argument reveals. The correct approach is to define

STEIN COMPLEMENTARY SERIES (4.17)

(# K ) 0

= {v €

95

there is a polynomial function f

of v such that f(u0) = v, and A(w-u)f(u) is finite at uQ} This is a proper (e).

Ip(? ® uQ)“invariant subspace, proving

□

The definition in (4.17) can be generalized, giving a family of invariant subspaces parametrized by the submodules of the local ring at

in its quotient field.

ly is called the Jantzen filtration.

This fami­

It plays an important

part in representation theory (cf. [Jantzen, 1979] and [Vogan, 1984], for example) but is very far from being well understood. Our efforts in the proof of Theorem 4.11 are now reward­ ed by unitary representations growing on trees (or at any rate on crosses).

THEOREM 4.18.

In the setting (k.10), assume that

Is irreducible.

Define a real subspace

S w

of

Ip(fOO) a

by

Sw = {u| wv = -v). Write ble.

R

for the set of

Finally, let

the origin in

S

W

C

w

- R.

v

for which Ip(? 0 v) Is reduci­

denote the connected component of Then

Ip(f 0 v) is infinitesimally r

96

CHAPTER 4

equivalent to an irreducible unitary representation of for every

Proof.

v

Fix

in

v

C . w

in

C . By Theorem 4.11, w

tian intertwining operator from it depends rationally on sit ion 4.8, A(w:u)

G

A(w-*0)

v.

A(w-*u) is a Hermi

Ip(f ® u)

to

Ip(f ® wd);

By the "only if” part of Propo

must be definite.

must be definite as well.

By continuity,

The theorem now follows

from the ”if" part of Proposition 4.8.

□

The representations constructed in Theorem 4.18 are called complementary series.

The term is also applied in a

variety of generalizations; for this style of argument can be pushed much further.

Unitarity occurs either everywhere

or nowhere on a connected component of

- R.

It often

happens that such a component will contain a few points where unitarity is easier to prove or disprove.

One can get

an immediate improvement by replacing R with the set Theorem 4.11.

I have used

easier to compute.

R

Z

of

only because it is sometimes

Another possibility is to investigate

the nature of the poles and zeros of

A(w:d)

along

to use this information to study the positivity of

R, and A.

(Zeros of even order do not affect positivity, for example.)

97

STEIN COMPLEMENTARY SERIES

For some hints about how complicated this study can become, the reader may consult [Knapp-Speh, 1983] or [Duflo, 1979]. Here

isa basicexample.

linear groupGL(2m,(C)

Suppose

G

of invertible

is the general

2mx2m matrices.

We

can take as a Car tan involution the map

(4.19) (a)

0g = (g-1)*.

the inverse conjugate transpose. (4.19)(b)

K

Then

= U(2m),

the group of unitary operators on

c2"1. Let

bolic subgroup of Example 3.14, with there, the Levi factor

MA

is

P

be the para­

k = m; as discussed

GL(m,C)xGL(m,C). To discuss

such groups, it is convenient to use the following notation: if

n = Pj + ...+Pr# and

IL

is a

d(Bj

P^P^

square matrix, then

Br)

denotes the block-diagonal matrix with the indicated diag­ onal blocks, and zeros elsewhere. for the

pxp

identity matrix.

product of two copies of

(4.19)(c)

We will also write

Then the group

A

I P

is a

DR, by

A = d(etlIm ,et2 Im )

(t € R).

Similarly, M is a product of two copies of what might be (but never is) called

UL(m,(C):

98

CHAPTER 4

(4.19)(d)

M = d(m 1 >m2)

(m^ an mxm matrix,

The group

M

characters

has unitary

|det(nu)| = 1).

f(klfk2) (k. in Z ) ,

def ined by

(4.20) (a)

f(k 1 ,k2 )(d(m1 ,m2) = [detOniJ^fdetOna)]1*2 .

Similarly, the characters of (4.20)(b)

A

are parametrized by (C2 , by

(ui,u2 )(d(x1 ,x2)) = [det(x1 )]Ul[det(x2)]|U2.

In these coordinates, the

modular function

trivial on

is given by

M, and on

A

(4.20)(c)

5

for G/P

is

6 = (2m,-2m).

Consider now the element 0

I m

w = -I

of

GL(2m,(C).

SL(2m,(C).

0

m

(We use the minus sign only to put

w

in

For our purposes that is unnecessary, but the

possibility of doing so shows that the construction works for

SL

as well as for

and normalizes

(4.21)(a)

This element belongs to

K

MA; it acts there by permuting the two

GL(m,(D) factors. (cf.(4.10)).

GL.)

Write

w

for the image of

w

in

W(G,MA)

Then w(f(kj,k2)) = ffka.kj)-

To be in the setting of (4.10), we should therefore restrict attention to the case the set

S w

k A = k2 .

of Theorem 4.18 is

Similarly, one sees that

STEIN COMPLEMENTARY SERIES (4.21)(b)

99

Sw = {(a+it.-a+it)}.

Finally, we need to know the set

R

of reductibility

points for the non-unitarily induced representations.

We

will take the result as a gift from non-unitary represen­ tation theory, without discussing a proof.

LEMMA 4.22.

In the setting (k.19)-(k.21), the induced repre­

sentation Ip(f(k,k)®u) Is reducible if and only if

v±-v2

is a non-xero even inte­

ger.

Now we can apply Theorem 4.18, to get

THEOREM 4.23 ([Stein, 1967]).

Suppose

notation as in (k.19)-(k .22).

For

o

strictly between -1 and

1

k

G = GL(2m,(D). in

Z,

t

in

Fix IR, and

, the representation

C 2 m (k» t :cr) = Indp(f(k,k)

8

(cr+it ,--

We return now to our discussion of the proof of Theorem 6.8.

Part (b) follows from Theorem 6.34(b).

For (d)

through (g), we may as well assume (perhaps after replacing Z

by

Z © Z*1) that

Hermitian form. well.

Z

carries a non-degenerate invariant

By Proposition 6.29,

Notice that

S

X = pro(W)

is half the dimension of

does as K/LT1K.

The

decomposition

t/x n * = (u n t) © (tT n *), and the invariant pairing between the two summands induced by our fixed bilinear form on

g, show that

L H K

acts

COHOMOLOGICAL PARABOLIC INDUCTION

153

trivially on the top exterior power of i/l fl t.

By Theorem

g

6.28,

# (Z) carries a non-degenerate form, as required by

Theorem 6.8(e).

In addition,

3iJ (Z )h = If

j

is greater than

6.34(a).

If

j

the same reason.

S, the left side is zero by Theorem

is less than

S, the right side is zero for

Theorem 6.8(d) follows.

The statement

about definiteness in Theorem 6.8(e) is rather hard.

Proofs

may be found in [Vogan, 1984] or [Wallach, 1984]. For the remaining statements, notice that Theorem g

6.8(d) and Theorem 6.34(c) compute tion of

K

# (Z) as a representa­

(under the hypothesis of Theorem 6.8(d)).

The

non-vanishing statement in Theorem 6.8(f) is proved using Corollary 6.35, by exhibiting a representation of

L fl K

g

that is not killed by

LTK^ ’

^°r

* [^c^un^^»

1975] contains a characterization of any discrete series representation in terms of its restriction to

K. Theorem

g

6.34 allows one to check that

# (t ) satisfies the condi­

tions of that characterization. To conclude this chapter, we discuss the unitary repre­ sentation theory of [Vogan, 1986b]. of

GL(n,IR).

Details may be found in

To see how the theory differs from the case

GL(n,C), we begin with a series of representations

154

CHAPTER 6

studied by B. Speh in [Speh, 1981].

Suppose

n = 2m, and

consider the Levi subgroup (6.36)(a)

L = GL(m,(C)

of

G

(cf. (5.3)).

gQ

corresponding to multiplication by

identified with to be 0 and

Recall that

X

denotes the element of i

(when

2m IR

(D™). The eigenvalues of

ad(X)

turn out

+2i.

Put

(6.36)(b)

u = +2i eigenspaces of ad(X)

(6.36)(c)

q = I + u.

Then

q

is a parabolic subalgebra of

complex conjugate. G

is

g, opposite to its

If we take the Car tan involution

to be inverse transpose, then

0X is

X; so

q

is

0

on

0-

stable. For each integer L

k, define a unitary character

of

by

(6.37)

xk (£)

= det(£)/|det(£)|.

(Here we refer to the determinant function on

GL(m,C).)

In

the notation of Definition 5.7, one can calculate that (6.38) (a)

2p(u) = x ^

This has a square root (namely metaplectic cover of phic to L

L

of

L

x^ ) .

It follows that the

(Definition 5.7) is isomor­

L x Z/2Z, and that metaplectic representations of

may be identified with representations of

L.

(To use

155

COHOMOLOGICAL PARABOLIC INDUCTION

this identification in Definition 6.20, for example, one sim­ ply replaces tensoring with

^y tensoring with

\m -)

We are going to consider the representations (6.39) of

^

GL(2m,IR).

= i*S (xk )

Theorem 6.34 suggests looking first at

which in this case is the orthogonal group

K,

0(2m). Because

of Proposition 6.31, the following result is a special case of the Bott-Borel-Weil theorem (cf. [Bott, 1957]).

LEMMA 6.40.

In the setting above, put

a)

If k < 0, then

Is zero.

b)

If k = 0, then

Is the sum of the trivial

the determinant characters of c) 0(2m)

If k = 1, then on

d) tion of

and

0(2m). is the representation of

Am (C2m). If k > 1, then

0(2m).

It is the

is an irreducible representa­ k ^ 1 Cartan power of

V±.

In

appropriate standard coordinates, it has highest weight (k,...,k).

The shift by -1 in the definition of fy the remaining statements.

is made to simpli­

CHAPTER 6

156

Corollary 6.35 now guarantees that long as

k

is at least -1.

X^

is non-zero as

On the other hand, the condi­

tion in Theorem 6.8(d) amounts to k > m - 1 in the present case. of

k

There is therefore a range of values

for which the general theory does not guarantee nice

behavior of

X^, but for which the restriction to

K

looks

reasonable.

It is possible to improve substantially on the

general theory, to make it cover most of this range. We will return to this point in Chapter 13 (Theorem 13.6).

For now,

we simply state what happens here.

PROPOSITION 6.41.

In the setting of (6.36) and (6.37), the

Harish-Chandra module

*k - VL for

GL(2m,C)

(notation (6.9)) is irreducible and infinites

imally unitary for of

0(2m)

k > 0.

It contains the representation

(cf. Lemma 6 .k0) as its lowest

K-type in

the sense of [Vogan, 1981]. When

k

is -1,

esimally unitary.

X^

is neither irreducible nor inf init

OOHOMOLOGICAL PARABOLIC INDUCTION

The main point here is that negative

k.

This is due to Speh.

157

is unitary for non­ (One must bear in mind

that this result was proved before Theorem 6.8(e) was avail­ able.)

What Speh did was show that

actually appears as

a constituent of the (unitary) representation of

GL(2m,IR)

on a certain space of square-integrable automorphic forms. When

k

is zero,

X^

is equivalent to a representa­

tion induced from the real parabolic subgroup of with Levi factor

GL(2m,IR)

GL(m,IR) x GL(m.IR), by a certain one­

dimensional unitary character.

For

k

positive,

X^

is

not equivalent to any representation induced from a proper real parabolic subgroup of

G.

This behavior shows the

differences in the unitary representation theory of and

GL(n,(C).

GL(n,IR)

It is necessary to introduce cohomological

induction to get all the representations; and when that is done, the problem of finding all equivalences among the var­ ious constructions becomes rather delicate.

We will not try

to formulate the answer precisely here, but the following theorem contains the main point.

THEOREM 6.42 ([Vogan, 1986b]). representation of

GL(n,IR)

Any irreducible unitary

may be obtained by iterating the

processes of parabolic induction (Definition k.l) and coho-

CHAPTER 6

158

mological parabolic induction (Definition 6.20), starting from two kinds of representations: one “dimensional unitary characters, and Stein complementary series (cf. Theorem k .23).

We actually need the Stein complementary series over both and

(C.

As was pointed out after Theorem 4.23, the choice

of ground field hardly affects their construction.

IR

Interlude THE IDEA OF UNIPOTENT REPRESENTATIONS

Theorem 6.42 is a model of what one would like to know for any reductive group:

that any unitary representation is

obtained by systematic processes from a small number of building blocks.

The systematic processes should certainly

include unitary induction from real parabolic subgroups (Definitions 3.8 and 4.1), and cohomological parabolic induc­ tion (Definition 6.20).

For our purposes, we will regard

the formation of complementary series (as in Theorem 4.23) as another "systematic process,” even though the limits of its applicability are not nearly so well understood as in the first two cases.

Problem 1.1. class

We are therefore led to

For each reductive group G, describe a nice

36(G) of unitary representations of G, with the fol­

lowing property.

Let

ir be any irreducible unitary repre-

159

INTERLUDE

160

sentation of G.

Then there is a Levi subgroup

(Definition 5.1), and a unitary representation $(L), such that

7r

is obtained from

L of G 7r^

in

by a complementary

series construction, followed by real parabolic induction, followed by cohomological parabolic induction.

(The letter

$

may be taken to stand for building block.)

In the sense intended here, the Stein complementary series for

GL(2m,C)

character

are obtained from the one-dimensional unitary

f(k,k)®(it,it) of GL(m)xGL(m) (cf. Theorem 4.23).

(Replacing this by the non-unitary character (a+it,-cr+it)

f(k,k) 8

(from which one actually induces in the end)

is regarded as part of the complementary series construc­ tion. ) Dan Barbasch has pointed out that this problem is almost certainly phrased too optimistically in at least one important respect.

Under special conditions, a non-unitary

Hermitian representation can give rise to unitary representa­ tions by a complementary series kind of procedure: one deforms an indefinite induced inner product through some poles until it becomes definite.

If this is really the most

natural construction of these representations, then they will not fit into the scheme proposed in Problem 1.1.

It is

INTERLUDE

161

not yet clear how to deal with this difficulty.

It should

not interfere with the less ambitious and more precise conjectures stated later, however. For

G

a product of copies of

GL(n,IR)

or

GL(n,C),

Theorems 4.28 and 6.42 suggest that one can take

$(G)

to

consist of the set of one-dimensional unitary characters of G.

At any rate we must include these char-acters in

$(G);

for they cannot be obtained from any smaller group in the manner suggested. This immediately suggests the hope that one might take $(G) al.

to consist of the unitary characters of This hope fails for the first time when

group

Sp(2n,(C), of linear transformations of

ing a non-degenerate symplectic form. unitary representation tic representation.

w

on

G

G G

in gener­ is the

(C?n

preserv­

has a beautiful

2 n L ()( trivial) (with the transfer tr defined by Corollary 8.20).

Now

Theorem 8.24 becomes evidence for Conjecture 9.15 as well; for most of the ideals involved here are not induced. I do not know any counterexamples to Conjecture 9.15 for the classical groups. problem.

In type

G2 , however, there is a

Joseph has shown that there are exactly two com­

pletely prime primitive ideals

I±

and

ing algebra, such that each quotient

I2

in the envelop­

UfgJ/I^

has Gelfand-

Kirillov dimension 8 (see [Joseph, 1981] and [Vogan, 1986a], section 5).

These ideals are not induced.

irreducible Harish-Chandra modules

X*

and

There are unique X2

such that

CHAPTER 9

226

LAnn(X.) = RAnn(X.) = I.. However, Duflo*s results in [Duflo, 1979] show that only one of these two representations (say

X 4) is unitary.

Accord­

ing to the Dixmier conjecture as formulated in [Vogan, 1986a],

I2

variety, and

corresponds to a certain non-normal algebraic It

to its normalization.

This is a small and

subtle distinction, and it is hard to see how it can direct­ ly affect representation theory. ring

R2 = U(g)/I2

(One might hope that the

is itself "non-normal" in some sense,

and that this causes the problem.

Joseph’s work suggests

that a reasonable definition of normal for a primitive quotient

R

of

U(g)

is that

Harish-Chandra module in satisfied for

R

Fract(R).

should be the largest This condition is

R2 , however.)

Without understanding the

G2

example, we can still

use it to guide a reformulation of Conjecture 9.15.

We

begin by recalling from [Vogan, 1986a] a part Qf the Dixmier conjecture alluded to above.

CONJECTURE 9.17.

Suppose

G

is a connected reductive alge­

braic group, with Lie algebra on

g0 » aad write

Y

Fix an orbit

for its closure.

affine algebraic variety 1)

gQ -

G, and

of

Fix an irreducible

V, endowed with

an algebraic action of

Y°

G

227

PRIMITIVE IDEALS

2)

a finite,

G-equivariant morphism

ir from

V

onto

Y. Then there is canonically associated to prime primitive algebra i)

V

a completely

A = A(V), endowed with

an algebraic action (called

Ad) of

G

by automor-

phi sms, and ii)

a

G-equivariant (for the adjoint action on

algebra homomorphism finitely generated

and

Y

is the closure of

Y°, and put

Hy = stabilizer of y H0 = identity component of Hy ir^G.Y) = Hy/H0 . Write

II(Y)

for the set of equivalence classes of irreduci­

ble, normal, affine algebraic varieties

V

structures (1) and (2) of Conjecture 9.17.

endowed with the Then

IT(Y)

is

in bijection with the set of conjugacy classes of subgroups of

tt1(G,Y), as follows.

Fix such a covering

V, and a pre­

231

PRIMITIVE IDEALS

image In

v

G.

of

y

in

V.

Write

Hv

for the stabilizer of

v

Then H0 C H

v

C H ; y

so M G . V ) = h v /H o may be regarded as a subgroup of sends the equivalence class of

Y) . The bijection V

to the conjugacy class of

TTi (G, V) .

Example 9.22. of

Suppose

G' = S0(9,(C)).

G

is

Spin(9,C)

We may regard

(go)*

9x9 complex skew-symmetric matrices.

(the double cover as the space of

Let

Y°

be the

coadjoint orbit consisting of nilpotent matrices with Jordan blocks of size 5, 3, and 1. M G ’.Y) = TTi(G.Y) = Write

D

the ideals

(Z/2Z) x (Z/2Z)

(Z/4Z) x (Z/2Z) (the dihedral group).

for the dihedral group. I(V)

Here is a description of

corresponding to various coverings of

(and henceto subgroups mal ideal,

Then one can check that

S

of

D).

Each

Y

I(V) is a maxi­

and therefore (by an observation of Dixmier)

determined by its intersection with the center of

is

U(gQ).

By Harish-Chandra’s theorem (Theorem 6.4), this intersection is determined by an element (fyo)

A

of a Cartan subalgebra.

(or

A(S)) in the dual

There is a standard way to

232

CHAPTER 9

identify

(*)o)*

finally

I(V)

written

A.

To

Y

with

C4

(see [Humphreys, 1972]); so

is determined by an element of

on

gQ

This induces an isomorphism

k

: go ■* So.

defined by the property that (10.18) (b) for all

Y

= A(Y) in

gQ .

We may write Xx = x(A).

The map

is a linear

k

isomorphism, intertwining the coad­

joint and adjoint actions: (10.18) (c)

(c(Ad*(g)(A)) = Ad(g)(»c(A)).

Recall (from [Humphreys, 1972], for example) that every element

X

of

gQ

has a Jordan decomposition

(10.19) '

X = X

s

+ X , n

characterized by the properties that

ad(Xg)

(as an automorphism of

gG);

is nilpotent;

belongs to

[^s »^n ] = 0*

[g,g]; and

ad(X^)

is semisimple X

follows that

X^

acts nilpotently in every finite-dimensional representation and that

Xg

acts semisimply in every completely reducible

finite-dimensional representation.

We say that

X

is semi-

CHAPTER 10

252

simple (respectively nilpotent) if (respectively

(respectively nilpotent) if gQ

is equal to

Xg

X^).

We say that an element of

element of

X

gQ

*c(X)

is semisimple is.

By (10.18), every

^SiS a Jordan decomposition

(10.20)

X = Xg + X .

Here are some of its properties.

PROPOSITION 10.21. belongs to

Suppose

G

s

for the Jordan decomposition of The Isotropy group

is a Leui subgroup of

G

+ X n X.

G(Xg)

is reductive; in fact it

(Definition 5.1).

b) The restriction of

X^

to

g(Xg)0

is nilpotent.

Its isotropy group for the coadjoint action is G(xs)n(V c) The restriction of d) The

X

gQ . Write X = X

a)

is a reductive group, and

G

orbit of

= G(x)-

X^ X

to

g(X)Q

is algebraically polarizable

(Definition 10.12) if and only if the is algebraically polarizable.

is zero.

G(Xg)

orbit of

X^

THE ORBIT METHOD

253

The first three parts of this proposition are routine, and the last is a fairly easy consequence of them.

Perhaps the

most serious fact one needs to know about the Jordan decom­ position in

go

commutes with

is that if

Y commutes with

X, then

X . s

COROLLARY 10.22.

Suppose

G

is reductive and

X

potent element (cf. (10.20) in gQ . Then the orbit A

Y

is integral (Definition 10.5).

Is a nllA

of

It is admissible

(Definition 10.16) if and only if the metaplectic double cover of

Proof.

G(Xg)0

is disconnected.

By Proposition 10.21(c),

The trivial character of

G(X)0

irof

G(X)

is equivalent to find one perties.

(If we have

For admissibility, we need with certain properties. on

ttq

tt0 ,

is trivial on g(X)Q .

therefore satisfies the

requirement in Definition 10.5. a representation

X

G(X)0 with

It

thesepro­

then

IndG(X) Jir0) G (X)o works for

tt; and if we have

the right properties for Because

X

ir, then its restriction has

irQ .)

is trivial on g(X)Q , tt0

on the identity component of

must be trivial

G(X)0 - This is compatible

CHAPTER 10

254

with the requirement that group is disconnected.

7r0( 0

be -1 exactly when the

□.

This corollary provides additional evidence that admis­ sibility is a more appropriate condition than integrality. It is fairly well known that certain nilpotent coadjoint orbits ought not to be associated to any unitary represen­ tation. group

An example is the minimal orbit for the symplectic Sp(2n,IR)

(with

n

at least 2).

It is associated to

the metaplectic representation of the metaplectic double cover

Mp(2nfIR)

but not to any representation of the sym­

plectic group itself.

The reason offered by the orbit

method is that the orbit is not admissible (except for the covering group). On the other hand, admissibility alone is still not a sufficient condition to guarantee the existence of a repre­ sentation attached to the orbit. PSL(2,IR), and

A

to be a non-zero nilpotent orbit (cf.

Example 11.3 below). and even polarizable.

One can check that

PSL(2,IR)

A

is admissible,

(The polarization fails to satisfy

the Pukanszky condition, however). tion of

To see this, take G to be

There is no representa­

attached to A - the best candidate is a

limit of discrete series representation for this fails to pass to the quotient

PSL(2,IR).

SL(2,IR), and

255

THE ORBIT METHOD

The nature of the relationship we want between the orbit method and unipotent representations is this.

"Definition” 10.23.

Suppose

G

An irreducible representation of

is a reductive Lie group. G

is called unipotent if

it is a constituent of a representation attached to an admis­ sible nilpotent coadjoint orbit.

This should be compared to Definition 9.19.

The quotation

marks are needed because we do not know how to define the representation attached to an orbit. A natural guess is that the set of representations attached to the orbit of

A

should be parametrized by the

irreducible admissible representations of tion 10.16).

G(A)

(Defini­

Theorem 1.37 can be interpreted as evidence of

this, along with its generalization Theorem 5.12.

There are

two kinds of problems with the existence of such a parametrization.

First, there are the considerations of Chapter 9

for complex groups; these should certainly be compatible with the orbit method.

Example 9.22 seeks to attach at

least ten different representations to a single orbit; there are only five irreducible admissible representations of G(A) .

CHAPTER 10

256

There is a much more serious problem, however. M = G/P

is a homogenous space and

bundle on P.

M

^; and c)

ir is infinitesimally equivalent (Definition

2.1k) to the induced representation Indp(Tr ® C ) , with the induced Hermitian form (defined in analogy with (3.9)). In particular, w

is unitary if and only if

nr is.

This result allows us to restrict attention to repre­ sentations having real infinitesimal character.

Roughly

speaking, such representations ought to come from the

L,

CHAPTER 13

296

derived functor construction of Theorem 6.8.

One of the

problems with that construction was that it did not always take unitary representations to unitary representations.

We

can now repair that problem to some extent.

THEOREM 13.6 (Theorem 7.1 and Proposition 8.17 in [Vogan, 1984]).

In the setting of Theorem 6.8, assume that

in Har ish-Chandra's class.

Write

I

G

is

as the direct sum of

its center and its commutator subalgebra: t = c + I'. Assume that i)

the restriction of Z

(Definitions 12.1 ii)

to

X'

Is

weakly unipotent

and 12.10); and

the

weight in c

by which

c

acts on

Z

satisfies Re > 0 c for any

root a

of

a) &J(Z)

c

in

a.

Then

is zero for not equal to

S;

b) any non-degenerate Hermitian form


^

The mostpleasant feature Lare fixed,L

$ (Z); and

is positive, then so Is

< , >^.

of this result is that (if

is a Levi factor for some

Z

and

0-stable para-

297

EXHAUSTION

bolic

g ’, and

Z

is unitary

sis (ii) is always satisfied 0-stable parabolic

q

on the

center of

L)hypothe­

for at least onechoice of

with Levi factor

X.

We therefore

have a rather complete way of passing from weakly unipotent unitary representations of Levi factors, to unitary repre­ sentations of

G.

The picture would be satisfactory indeed

if the following result were true.

FALSE THEOREM 13.7.

Suppose

Harish-Chandra1s class and

G X

is a reductive Lie group in is an irreducible

(g,K)-

module having real infinitesimal character (Definition 13. k).

Assume that

Hermitian form i)

X

admits a positive definite invariant

< , >^.

Then

a Levi subgroup

parabolic subalgebra ii)

we can

L

find

of G, attached to a

0-stable

q = X + u; and

an irreducible

(X,(L fl K) )~module

positive definite invariant Hermitian form following properties.

Z, with a

< , >^, with the

Use the notation of Theorem 10.6.

Then a) the restriction of

Z

to

I'

Is weakly uni­

potent ; b)

Z

Hermitian form

admits a positive definite invariant < , >^;

298

CHAPTER 13

c) Z

the weight

in

c

by which

c

acts on

satisfies > 0

for any

root

a

of

c

in

u; and g

d)

X

ts isomorphic

to # (Z), with the induced

Hermitian form.

The main reason this is false is that mentary

not all comple­

series representations areweaklyunipotent.

example, if

G

is

SL(2,IR)f the complementary series

(Theorem 4.23) is weakly unipotent only for equal to

A.)

o

(For C(cr)

less than or

Here is a weaker result, which follows from

[Vogan, 1984].

THEOREM 13.8

Suppose

Chandra*s class.

G

is a semisimple group in Harish-

Then there is a computable constant

with the following property. Suppose unitary

X

c^,,

is an irreducible

(g,K)-module, with real infinitesimal character

(Definition 13.k). Assume that > cQ . Then we can find

q,

L, and

Z

ing a)

q

is not equal to

q;

as in Theorem 6.8, satisfy­

299

EXHAUSTION

b)

Z

ts unitary and irreducible;

c) the positivity hypothesis of Theorem 6.8(d) holds; and g

d)

X

Is isomorphic to

Sketch of proof.

$ (Z).

Using Theorem 13.1, write

X

presentation of some induced representation.

Use the nota­

tion established for Theorem 4.11; in our case, discrete series representation of (13.9)(a) Since

M.

as a subre­

f

is a

Then

X C Ip(f ® v).

X

has real infinitesimal character,

is real­

d

valued. The analysis described before Proposition 13.3 shows that the Hermitian an element

w

form on X

arises as follows: there is

of W(G,MA),

(13.9) (b)

w f = f,

w u = -d .

The Hermitian

form on

(13.9)(c)

^ = h .

Here

v^

and

v^

X

of order 2, suchthat

is given by

are elements of the space

1k of the in­

duced representation that belong to the subspace < , > over

is the inner product on

1k given by integration

K/(KDP). The proof proceeds by analyzing the form

(13.10)

X, and

t = ^

300

on

CHAPTER 13

#, as a function of the real variable

guarantees that it is meromorphic in zeros only when

Ip(f ® tv)

t.

Theorem 4.11

t, with poles and

is reducible.

When t is zero,

the induced representation is unitary, and its reducibility is precisely known; the signature of lated exactly.

< , >^

can be calcu­

The way that the signature varies with

t

is controlled by the reducibility of the induced represen­ tation. On the other hand, [Speh-Vogan, 1980] allows one to describe the reducibility of

Ip(f ® tv)

analogous problem on some Levi factor parabolic, as long as

tu

infinitesimal character

A^

L

in terms of an of a

0-stable

is not too large compared to the of

f.

(This is the most diffi­

cult step in the argument; I will not discuss the ideas in­ volved.

They are one of the main topics of [Vogan, 1981].)

The description is implemented by the cohomological inducg

tion functor

# . The conclusion of the theorem now drops

out, as long as

v

is not too large compared to

A^.

On the other other hand, the proof of Theorem 13.1 produces

v

from

coefficients of

X

X

in terms of the behavior of matrix

at infinity.

If

X

is unitary, its

matrix coefficients must be bounded; so the corresponding cannot be too big.

The infinitesimal character of

that of the induced representation, which is (A^.u).

X

is Since

v

EXHAUSTION

X

is assumed to have large infinitesimal character, this

forces to

301

A^

to be large.

A ^ f as desired.

Now

v

is not too large compared

□

This argument is replete with information even when X has small infinitesimal character; many of the recent re­ sults on classifying unitary representations are based in part on it.

REFERENCES

J. Arthur, ’’Eisenstein series and the trace formula,” in Automorphic Forms, Representations, and L~ functions, Proceedings of Symposia in Pure Mathematics 33, part 2, 27-61. American Mathematical Society, Providence, Rhode Island, 1979. J. Arthur, ”0n some problems suggested by the trace formu­ la,” in Proceedings of the Special Year in Harmonic Analysis, University of Maryland, R. Herb, R. Lipsman, and J. Rosenberg, eds. Lecture Notes in Mathematics 1024, Springer-Verlag, Ber1in-Heidelberg-New York, 1983. M. Atiyah and W. Schmid, ”A geometric construction of the discrete series for semisimple Lie groups,” Invent. Math. 42 (1977), 1-62. L. Auslander and B. Kostant, ”Polarization and unitary representations of solvable Lie groups,” Invent. Math. 14 (1971), 255-354. M. W. Baldoni-Silva and A. Knapp, ’’Indefinite intertwining operators,” Proc. Nat. Acad. Sci. U.S.A. 81 (1984), 1272-1275. D. Barbasch and D. Vogan, ’’The local structure of charac­ ters,” J. Funct. Anal. 37 (1980), 27-55. D. Barbasch and D. Vogan, ’’Unipotent representations of complex semisimple Lie groups,” Ann. of Math. 121 (1985), 41-110. V. Bargmann, ’’Irreducible unitary representations of the Lorentz group,” Ann. of Math. 48 (1947), 568-640. A. Borel and N. Wallach, Continuous Cohomology, Discrete Subgroups, and Representations of Reductive Groups. Princeton University Press, Princeton, New Jersey, 1980. R. Bott, ’’Homogeneous vector bundles,” Ann. of Math. 66 (1957), 203-248. F. Bruhat, ”Sur les representations induites des groupes de Lie,” Bull. Soc. Math. France 84 (1956), 97-205. 302

REFERENCES

303

N. Conze, ’’Quotients primitifs des algebres enveloppantes et algebres d ’operateurs differentiels,” C. R. Acad. Sci. Paris Ser A-B 277 (1973), A1033-A1036. P. Deligne and G. Lusztig, ’’Representations of reductive groups over finite fields,” Ann. of Math. 103 (1976), 103-161. J. Dixmier, Algebres Enveloppantes. Gauthier-Viliars, Paris-Brussels-Montreal 1974. J. Dixmier, Von Neumann Algebras, translated by F. Jellett. North-Holland Publishing Co., New York, 1981. M. Duflo, ’’Representations irreductibles des groupes semi­ simples complexes,” in Analyse Harmonique sur les Groupes de Lie (Sem. Nancy-Strasbourg 1973-75). Lec­ ture Notes in Mathematics 497, Springer-Verlag, BerlinHeidelberg-New York, 1975. M. Duflo, ’’Representations unitaires irreductibles des groupes semi-simples complexes de rang deux,” Bull. Soc. Math. France 107 (1979), 55-96. M. Duflo, ’’Construction de representations d'un groupe de Lie,” Cours d ’ete du C.I.M.E., Cortona 1980. M. Duflo, ’’Theorie de Mackey pour les groupes de Lie algebriques,” Acta Math. 149 (1982), 153-213. M. Duflo, ’’Construction de gros ensembles de representa­ tions unitaires irreductible d ’un groupe de Lie quelconque,” in Operator Algebras and Group Repre­ sentations , volume I. Monographs Stud. Math. 17. Pitman, Boston, Massachusetts-London, 1984. T. J. Enright and N. R. Wallach, ’’Notes on homological algebra and representations of Lie algebras,” Duke Math. J. 47 (1980), 1-15. I. M. Gelfand and M. A. Naimark, ’’Unitary representations of the Lorentz group,” Izv. Akad. Nauk S.S.S.R. 11 (1947), 411-504. V. Guillemin and S. Sternberg, Geometric Asymptotics, Math Surveys 14. American Mathematical Society, Providence, Rhode Island (1978).

304

REFERENCES

Harish-Chandra, ’’Representations of semi-simple Lie groups I,” Trans. Amer. Math. Soc. 75 (1953), 185-243. Harish-Chandra, ’’Spherical functions on a semisimple Lie group II,” Amer. J. Math. 80 (1958), 553-613. Harish-Chandra, ’’Invariant eigendistribtions on a semi­ simple Lie group,” Trans. Amer. Math. Soc. 119 (1965), 457-508. Harish-Chandra, ’’Discrete series for semi-simple Lie groups II,” Acta Math. 116 (1966), 1-111. S. Helgason, Differential Geometry, Lie Groups, and Symme­ tric Spaces. Academic Press, New York, San Francisco, London, 1978. S. Helgason, Groups and Geometric Analysis. Press, Orlando, Florida, 1984.

Academic

R. Howe, ”0n the role of the Heisenberg group in harmonic analysis,” Bull. Amer. Math. Soc. (N.S.) 3 (1980), 821-843. J. E. Humphreys, Introduction to Lie Algebras and Represen­ tation theory. Springer-Verlag, Berlin-Heidelberg-New York, 1972. J. C. Jantzen, Moduln mit einem Hochsten Gewicht, Lecture Notes in Mathematics 750. Springer-Verlag, BerlinHeidelberg-New York, 1979. A. Joseph, ’’The minimal orbit in a simple Lie algebra and its associated maximal ideal,” Ann. Scient. Ec. Norm. Sup. (4) 9 (1976), 1-30. A. Joseph, ’’Goldie rank in the enveloping algebra of a semi­ simple Lie algebra I, II,” J. Alg. 65 (1980), 269-316. A. Joseph, ’’Kostant’s problem and Goldie rank,” 249-266, in Non-commutative Harmonic Analysis and Lie Groups, J. Carmona and M. Vergne, eds., Lecture Notes in Mathema­ tics 880. Springer-Verlag, Berlin-Heidelberg-New York, 1981. A. Joseph, ”0n the classification of primitive ideals in the enveloping algebra of a semisimple Lie algebra,” in

REFERENCES

305

Lie group representations I, R. Herb, R. Lipsman and J. Rosenberg, eds. Lecture Notes in Mathematics 1024. Springer-Verlag, Berlin-Heidelberg-New York, 1983. D. Kazhdan and G. Lusztig, "Representations of Coxeter groups and Hecke algebras," Inventiones Math. 53 (1979), 165-184. A. Kirillov, "Unitary representations of nilpotent Lie groups," Uspehi Mat. Nauk. 17 (1962), 57-110. A. Knapp, Representation Theory of Real Semisimple Groups: An Overview Based on Examples. Princeton University Press, Princeton, New Jersey, 1986. A. Knapp and B. Speh, "The role of basic cases in classifi­ cation: theorems about unitay representations applic­ able to SU(N,2)," 119-160 in Non-commutative Harmonic Analysis and Lie Groups, J. Carmona and M. Vergne, eds., Lecture Notes in Mathematics 1020. SpringerVerlag, Berlin-Heidelberg-New York-Tokyo, 1983. A. Knapp and E. Stein, "Singular integrals and the princi­ pal series. IV," Proc. Nat. Acad. Sci. U.S.A. 72 (1975), 2459-2461 A. Knapp and E. Stein, "Intertwining operators for semisim­ ple groups II," Inventiones Math. 60 (1980), 9-84. A. Knapp and D. Vogan, "Duality theorems in relative Lie algebra cohomology," preprint, 1986. A. Knapp and G. Zuckerman, "Classification theorems for representations of semisimple Lie groups," 138-159 in Non-commutative Harmonic Analysis, J. Carmona and M. Vergne, eds., Lecture Notes in Mathematics 587. Springer-Verlag, Berlin-Heidelberg-New York, 1977. B. Kostant, "The principal three-dimensional subgroup and the Betti numbers of a complex simple Lie group," Amer. J. Math. 81 (1959), 973-1032. B. Kostant, "On the existence and irreducibility of certain series of representations," Bull. Amer. Math. Soc. 75 (1969), 627-642. B. Kostant, "Coadjoint orbits and a new symbol calculus for line bundles/ in Conference on Differential Geometric

306

REFERENCES

Methods in Theoretical Physics (G . Denardo and H.D. Doebner, eds.). World Scientific, Singapore, 1983. B. Kostant and S. Rail is, ’’Orbits and representations associated with symmetric spaces,” Amer. J. Math. 93 (1971), 753-809. S. Lang, SL(2,IR). 1975.

Addison-Wesley, Reading, Massachusetts,

R. P. Langlands, ”0n the classification of representations of real algebraic groups,” preprint, Institute for Advanced Study, 1973. J. Lepowsky, ’’Algebraic results on representations of semi simple Lie groups,” Trans. Amer. Math. Soc. 176 (1973) 1-44. G. Lusztig, Characters of Reductive Groups over a Finite Field. Annals of Mathematics Studies 107. Princeton University Press, Princeton, New Jersey, 1984. G. Mackey, Theory of Unitary Group Representations. Univer sity of Chicago Press, Chicago, 1976. F.I. Mautner, ’’Unitary representations of locally compact groups II,” Ann. of Math 52 (1951), 528-556. E. Nelson, ’’Analytic vectors,” Ann. of Math. 70 (1959), 572-615. H. Ozeki and M. Wakimoto, ”0n polarizations of certain homogeneous spaces,” Hiroshima Math. J. 2 (1972), 445-482. F. Peter and H. Weyl, ’’Die Vollstandigkeit der primitiven Darstellungen einer geschlossenen kontinuierlichen Gruppe,” Math. Ann. 97 (1927), 737-755. Pontriagin, Topological Groups, translated by E. Lehmer. Princeton University Press, Princeton, New Jersey, 1939. J. Rawnsley, W. Schmid, and J. Wolf, ’’Singular unitary representations and indefinite harmonic theory,” J. Funct. Anal. 51 (1983), 1-114.

REFERENCES

307

W. Shakespeare, A Midsummer -Ni ght ’s Dream, act V, scene i, lines 12-17. W. Schmid, "Homogeneous complex manifolds and representa­ tions of semisimple Lie groups," Ph.D. dissertation, University of California at Berkeley, Berkeley, 1967. W. Schmid, "Some properties of square-integrable representa­ tions of semisimple Lie groups," Ann. of Math. 102 (1975), 535-564. W. Schmid, "L2 cohomology and the discrete series," Ann. of Math. 103 (1976), 375-394. D. Shale, "Linear symmetries of free boson fields," Trans. Amer. Math. Soc. 103 (1962), 149-167. B. Speh, "Unitary representations of SL(n,IR) and the coho­ mology of congruence subgroups," 483-505 in Non-commutative Harmonic Analysis and Lie groups, J. Carmona and M. Vergne, eds., Lecture Notes in Mathematics 880. Springer-Verlag, Berlin-Heidelberg-New York, 1981. B. Speh and D. Vogan, "Reducibility of generalized princi­ pal series representatons," Acta Math. 145 (1980), 227-299. T. A. Springer, "Reductive groups," in Automorphic Forms, Representations, and L~functions, Proceedings of Sym­ posia in Pure Mathematics 33, part 1, 3-28. American Mathematical Society, Providence, Rhode Island, 1979. E. M. Stein, "Analysis in matrix spaces and some new representations of SL(n,IR)," Ann. of Math. 86 (1967), 461-490. M. J. Taylor, Noncommutative Harmonic Analysis. American Mathematical Society, Providence, Rhode Island, 1986. P. Torasso, "Quantification geometrique, operateurs d ’entrelacement et representations unitaires de SL3(IR)~," Act Math. 150 (1983), 153-242. D. Vogan, "Gelfand-Kiri1lov dimension for Harish-Chandra modules," Inventiones math. 48 (1978), 75-98.

308

REFERENCES

D. Vogan, Representations of Real Reductive Lie Groups. Birkhauser, Boston-Basel-Stuttgart, 1981. D. Vogan, "Unitarizability of certain series of represen­ tations," Ann. of Math. 120 (1984), 141-187. D. Vogan, "The orbit method and primitive ideals for semi­ simple Lie algebras/* in Lie Algebras and Related Topics, CMS Conference Proceedings, volume 5, D. Britten, F. Lemire, and R. Moody, eds. American Mathe­ matical Society for CMS, Providence, Rhode Island, 1986. D. Vogan, **The unitary dual of GL(n) over an archimedean field,** Invent. Math. 83 (1986), 449-505. J. von Neumann, **Die eindeutigkeit der Schroderschen Operatoren,** Math. Ann. 104 (1931), 570-578. N. Wallach, Harmonic Analysis on Homogeneous Spaces. Marcel Dekker, New York, 1973. N. Wallach, **0n the unitarizability of derived functor modules,** Invent. Math. 78 (1984), 131-141. G. Warner, Harmonic Analysis on Semisimple Lie Groups. Springer-Verlag, Berlin, Heidelberg, New York, 1972. A. Weil, L*integration dans les Groupes Topologiques et ses Applications. Actual. Sci. Ind. 869. Hermann et Cie., Paris, 1940. H. Weyl, *’Theorie der Darstellung kontinuierlicher Gruppen. I, II, III,** Math. Z. 23 (1925), 271-309; 24 (1925), 328-376, 377-395. E. Wigner, *’0n unitary representations of the inhomogeneous Lorentz group,*’ Ann. of Math. 40 (1939), 149-204.

Library of Congress Cataloging-in-Publication Data Vogan, David A ., 1954Unitary representations of reductive Lie groups. (Annals of mathematics studies ; no. 118) Bibliography: p. 1. Lie groups. 2. Representations of groups. I. Title. II. Series. QA387.V64 1987 512'.55 87-3102 ISBN 0-691-08481-5 ISBN 0-691-08482-3 (pbk.)