Hall polynomials for classical groups

Citation preview

INFORMATION TO USERS

This manuscript has been reproduced from the microfilm master. UMI films the text directly from the original or copy submitted. Thus, some thesis and dissertation copies are in typewriter face, while others may be from any type o f computer printer. The quality o f this reproduction is dependent upon the quality o f the copy subm itted. Broken or indistinct print, colored or poor quality illustrations and photographs, print bleedthrough, substandard margins, and improper alignment can adversely afreet reproduction. In the unlikely event that the author did not send UMI a complete manuscript and there are missing pages, these will be noted.

Also, if

unauthorized copyright material had to be removed, a note will indicate the deletion. Oversize materials (e.g., maps, drawings, charts) are reproduced by sectioning the original, beginning at the upper left-hand comer and continuing from left to right in equal sections with small overlaps. Each original is also photographed in one exposure and is included in reduced form at the back o f the book. Photographs included in the original manuscript have been reproduced xerographically in this copy. Higher quality 6” x 9” black and white photographic prints are available for any photographs or illustrations appearing in this copy for an additional charge. Contact UMI directly to order.

UMI A Bell & Howell Information Company 300 North Zeeb Road, Ann Arbor MI 48106-1346 USA 313/761-4700 800/521-0600

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

R e p ro d u c e d with perm ission of the copyright owner. Further reproduction prohibited without permission.

HALL POLYNOMIALS FOR CLASSICAL GROUPS

Franklin Miller Maley

A DISSERTATION PRESENTED TO THE FACULTY OF PRINCETON UNIVERSITY IN CANDIDACY FOR THE DEGREE OF DOCTOR OF PHILOSOPHY

RECOMMENDED FOR ACCEPTANCE BY THE DEPARTMENT OF MATHEMATICS

November 1996

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

UMI Number: 9701217

UMI Microform 9701217 Copyright 1996, by UMI Company. All rights reserved. This microform edition is protected against unauthorized copying under Title 17, United States Code.

UMI

300 North Zeeb Road Ann Arbor, MI 48103

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

© Copyright by Franklin Miller Maley, 1996. All rights reserved.

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

ABSTR A CT The Hall polynomial g^u{q) counts submodules of type // and cotype v in a finite nilpotent Fg[t]-module of type A. We derive a formula for g^u(q), simpler than previous ones, which shows that its expansion in powers of q —1 has nonnegative coefficients. We then define Hall functions h^u(q) that are related to the Green functions for Spn (Fq), O j(F (?), and Un (F^2) just as ordinary Hall polynomials are related to the Green polynomials for GLn (F9). The functions h^u{q) count totally isotropic submodules of a finite nilpotent module over Fq[t] (or Fq 2 [£]) equipped with a nondegenerate bilinear (or sesquilinear) form compatible with the action of t. We show that Hall functions are polynomials in q and provide an effective algorithm for computing them. In the unitary case, h^u has integer coefficients; in the symplectic and orthogonal cases, we prove that certain signed sums of the polynomials h ^u have integer coefficients.

in Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

ACKNOW LEDGEM ENTS I received substantial encouragement and advice, and important references, from Bhama Srinivasan, A1 Hales, Hale Trotter, and especially Lynne Butler. I am very grateful to all of them.

iv Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

CONTENTS A b stra ct....................................................................................................iii 1: Ordinary Hall Polynomials .............................................................1 1.1: B ack g ro u n d ................................................................................. 1 1.2: The category of p a i r s .................................................................4 1.3: Reduction to F -p o ly n o m ials..................................................... 8 1.4: A new formula for F-polynom ials...........................................10 2: Symplectic and Orthogonal G rou p s...............................................16 2.1: Definition of Hall f u n c t i o n s ................................................... 16 2.2: Unipotents versus n ilp o te n ts ...................................................21 2.3: Nilpotent conjugacy c l a s s e s ...................................................23 2.4: The Springer-Steinberg b a s is ...................................................26 3: Computing General Hall P olynom ials...........................................30 3.1: Overview of the c a lc u la tio n ...................................................30 3.2: Action of the centralizer...........................................................32 3.3: Details of the c a lc u la tio n ...................................................... 39 3.4: Details of the r e c u r s io n .......................................................... 43 4: Main Results ...................................................................................46 4.1: Dependence on q mod 4 .......................................................... 46 4.2: Integrality and Fourier transforms .......................................52 4.3: The one-block c a s e ...................................................................59 4.4: Directions for further s t u d y ...................................................63 5: Unitary Hall P o ly n o m ia ls...............................................................65 5.1: Classes and o r b i t s ...................................................................66 5.2: Computations involving unitary g r o u p s .............................. 68 5.3: The one-block c a s e .................................................................. 69 R eferen ces...............................................................................................71 A p p e n d ix ............................................................................................... 73

V

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

1. O rdinary H all P o ly n o m ia ls In this section, we study the Hall polynomials g^v associated with the general linear group GLn . Sometimes we call them ordinary Hall polynomials, to distinguish them from the generalized Hall polynomials considered in later sections.

1.1. B ackground Let R be a discrete valuation ring with maximal ideal P and finite residue field R /P . Every finite module L over R is a direct sum of cyclic modules: L w R / P Xl © • • • ® R / P XK If we arrange the lengths

of the summands in descending order,

then the partition A = (Ai, . . . , An ) is determined by L, because the conjugate partition A' is given by A' = dirafl/P (P i- 1L )/(P i L).

(1- 1)

Thus A, the type of L, is a complete isomorphism invariant for L. Each submodule M C L has a type p and a cotype u, which is the type of L / M , and it follows from (1-1) that /i, v C A as partitions. (We often equate a partition A with its shape { { i , j ) : l < j < \ i } . )

By

convention Az- = 0 when i > I, and |A| denotes Yli ^ i • Given three partitions A, p, and v, we may ask how many submod­ ules of type p and cotype v appear in a finite i?-module of type A. The answer depends only on the cardinality q of the field R / P ; it is given by the Hall polynomial gfh,{q), which is a polynomial in q with integer 1 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

coefficients [5, 9]. Taking R to be the ring Zp of p-adic integers, we obtain Philip Hall’s original definition: g^u(p) is the number of sub­ groups of type p and cotype v in a finite abelian p-group of type A. For another example, let V be a vector space over the field Fq, and let T be an endomorphism of V whose minimal polynomial is a power of an irreducible polynomial / G Fq[t\ of degree d. Then the action of T makes V into a module over R = Fq[t] a n d the T-invariant subspaces of V are counted by the polynomials

where A is the

type of V. In this guise, the Hall polynomials play an important role [4, 9] in the construction of the irreducible characters of GLn (F9). Hall polynomials are also closely related to the Green polynomials Qp for GLn (F«j) [4]. Let a U r denote the partition formed by taking the multiset union of the parts of cr and r, and write p h k to indicate that p is a partition of k. Then

QaUr =

;})

(1-3)

5 u r= {l,...,n } #S = m , # T = n —m

P

and hence is a polynomial in p with nonnegative coefficients. Formula (1-3) comes from counting subspaces by their standard matrices, which are m x n matrices over Fp. We define the standard matrix of a subspace to be the unique matrix whose rows form a basis for the subspace and which is in row-reduced reverse echelon form: the rightmost nonzero entry in each row is 1, each trailing 1 lies to the right of the trailing 1 in the preceding row; and all entries below a trailing 1 are zero. Standard matrices look like "1 0

00 0 *1 0

0 0 O' 0 0 0

0

* 0

*

*

1

0

0

*0 *

*

0

1

5

so the number of standard matrices whose trailing l ’s fall in certain columns S is a power of p. We will use a similar idea to calculate Hall polynomials.

1.2. T he category o f pairs Before embarking on the calculation, we provide an example to show 4 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

that finite abelian p-groups (or finite i?-modules) are more complicated than they might seem. One way to compute g^u{p) would be to choose a finite abelian p-group G of type A and classify the subgroups H C G up to automor­ phisms of G. In some cases, all subgroups H of type p and cotype u are equivalent under Aut(G), and the result is a simple formula for g^v . We will see examples later. In general, the subgroups of type p and cotype v comprise a union of Aut(G)-orbits. If the number of these orbits, call it n j^ p ) , were always independent of p, we could obtain a formula for g^u{p) by computing the order of the stabilizer of a sub­ group H in each orbit. (In fact, to prove that the expansion of g^u(p) in powers of p —1 had nonnegative coefficients, it would suffice to show that the order of each stabilizer was a polynomial in p. Each such polynomial would have to divide # Aut(G), which is a polynomial in p whose complex roots lie in the half-plane Re z
p-

2.

A h24,i/H 6

It follows th at not all the functions n^v can be bounded. We can be more precise. The group G is generated by the elements xi, x2, x3, £ 4 , and u. Applying Lemma

1.2

to these elements in succes­

sion, starting from the trivial group, we find that G has type A = 87531. Similarly, the type of G /H is v = 7531. Thus

>

P

- 2.

6

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

By elaborating this example, we can take any invariant for arrange­ ments of subspaces in a vector space over Fp, and construct a corre­ sponding invariant for pairs of groups G 3 H of certain types. Thus there is no hope of finding a (useful) complete set of invariants for general pairs G D H of finite abelian p-groups. On the other hand, pairs G 2 H can sometimes be classified when the type of G or H is restricted. Many such results flow from the following criterion, which is part of the proof of Ulm’s Theorem; see Exercise 38 of [7]. (We state only the finite case.) T h eorem 1.2. Let H and H' be subgroups of isomorphic finite abelian p-groups G and G '. An isomorphism cj>: H —■> H r extends to an isomorphism every n >

: G —►Gf if and only if 4>(H C\pnG) = H' f\p nG' for

0.

In other words, any height-preserving isomorphism H —>H r extends, if G « Gf, to an isomorphism G —>Gf. We mention a few applications. • If A = kl, then H fl pnG = ker(pk~n : H —> i7), so all subgroups H of type p are equivalent by automorphisms of G. Their cotype v is given by

= k—

and g^u is given by the well-known

formula (see [1]) for the number of subgroups of type p in a group of type A. • If /if is cyclic, then H is characterized up to G-automorphisms by the orders of the groups H fl pnG. (It follows that the orbits of Aut(G) on G correspond to Ulm sequences. The Ulm sequence of x G G specifies the heights of pix for i > 0.) 7 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

• If H is elementary (pH = 0), then the filtration H C\pnG is a flag of Fp-subspaces of H. Knowing the dimensions of these subspaces is equivalent to knowing the cotype of H. Consequently, all elemen­ tary subspaces of a given cotype are equivalent by automorphisms of Aut(G).

1.3. R ed u ctio n to F-polynom ials Both Klein [8 ] and Macdonald [9] express g^u(p) as a sum of products of simpler polynomials Fa^ (p), and we will follow the same strategy. The first step is to classify subgroups H according to the cotypes of plH for i >

Let G be a finite abelian p-group of type A, and

0.

let H be a subgroup of type p and cotype v. We associate with H a semistandard tableau T of shape Xf —v' and weight p! (a filling of — v' with pi ones, p£> twos, and so on, the numbers

the skew shape

increasing weakly along rows and strictly along columns) as follows. The subgroups H

D

pH

D

p2H D • • • D pfllH = 0

of G have cotypes v = v° C v 1 C v2 C - - - C and for

1

= \,

(1-4)

< z < p i we place the integer i into the squares (viY —(z/- 1 )'

of T. As shown in [9], the tableaux T th at arise are precisely those such that the word of T, obtained by reading successive rows of T from right to left, is a ballot sequence, meaning that in every prefix, each number i >

0

occurs at least as often as i + 1 . Such tableaux are counted by 8

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

the Littlewood-Richardson coefficient c ^ , = c^u. Let gx(p) count the number of subgroups H of G with associated tableau T. The quantities 9t (p ) t 11111 out to be monic polynomials in p of the same degree, which explains the appearance of c^u as the leading coefficient of g^u. The second step is to factor the polynomials gp, as explained in Macdonald [9]. If the tableau T comes from the partitions (1-4), then we have

Pi 9t (p )

Cp)> i=

1

where i/ ^ 1+1 = A, and the F-polynomials are defined as follows. L em m a 1.3. [8,9] Let C be a finite abelian p-group o f type j', and let B be an elementary subgroup of C of cotype (31. The number of subgroups A C C of cotype a f such that pA = B is given by a polynomial Fap^(p) with integer coefficients. (The type of B is

The factorization of gp arises because one

may choose the subgroup K\ = plH C G of cotype vl inductively for i = pi — 1, . . . , 1, 0 ; and in choosing

one may take everything

modulo K i+i. At this point the analyses of Klein and Macdonald diverge, but nei­ ther applies directly to the p —1 question. Klein obtains a formula for Fa/3'y(p) in terms of rational functions of p that are not obviously poly­ nomials, though she proves they are. Macdonald observes instead that in the situation of Lemma 1.3, counting subgroups A

D

B with pA C B

is easy—it amounts to counting elementary subgroups A / B of cotype a 1 in C / B —and he then uses Mobius inversion on the lattice of sub­ spaces of B to complete the calculation. The Mobius inversion, which 9 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

is essentially an inclusion-exclusion argument, introduces inconvenient signs into his final formula.

1.4. A new form ula for F -polynom ials We now present a new calculation of the polynomial Fa^ f based on counting standard matrices for the subgroups in question. Let us adopt the notation of Lemma 1.3. As observed in §1.2, the cotype of the elementary subgroup B C C determines B up to an automorphism of C. We may therefore choose an isomorphism C « Z/p7iZ © • • • © Z/p7cZ such that, if e2- represents the generator of the zth summand of C , then B has as Fp-basis the elements { p l~ l ej : ( i j ) For a picture, think of

35

6 7

—/?}.

the square in row i and column j of

the shape 7 . 7'

B

0 = 3333221

B B The squares in

7

= 4433321

— (3 give generators for B, and the squares in (3 form

a picture of C /B . Multiplication by p moves each square to the square below. Multiplying a lowermost square (an element

by p sends

it to zero. Our task is to count subgroups A of C of cotype a! such that p A = B. Let S ( C /B ) denote the socle, or maximal elementary subgroup, 10 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

of C /B . It is equivalent to count elementary subgroups A / B of C /B of cotype a ' such that 4>(A/B) = B , where . In addition, for A / B to be elementary, (3 —a must be, like

7

—/?, a horizontal strip: so we assume

*>1. These assumptions hold automatically if a, /3, and

7

(1-5) arise from a semi­

standard tableau as described in §1.3. We sort subspaces A / B of S (C /B ) according to the dimensions of their intersections with certain subspaces Vi C S (C /B ). S ( C/ B) comes with a basis

The socle

... ,pP'n~ l en }, where n = (3\,

corresponding to the squares on the lower edge of /?, and hence it has a standard flag of subspaces V{ =

(i = 0 , 1, . . . ,n)

whose i-dimensional member is spanned by the lowermost squares in the first 2 columns of 0. The dimensions of the intersections Vi fl ( A/ B) determine the cotype of A in C. L e m m a 1.4. Let G be a finite abelian p-group o f type X, and H a subgroup o f G. The cotype o f H in G is v if and only if pi _ l G n H w , dimz/?z W n T = X i ~ Ui'

. 2~

,

L

Proof. A simple calculation using the isomorphism theorems. 11 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

□

C orollary 1.4. Let A / B C S (C /B ). The cotype of A in C is

ol

if and only if VQ. D (A/ B) dimz/pZ Vf ch n ( A/ B)

1- L

Proof. Apply the Lemma to G = C /B and H = A /B , observing that Pl- l G fl S( C/ B) = Vp. and hence picture of

fl H = Vp. n (A/ B). (The

is the portion of (5 lying in rows > i.)

□

To count subspaces of S ( C / B ), we represent each one by its stan­ dard matrix. The standard matrix records a basis for the subspace in row-reduced reverse echelon form, as in §1.1. Let A / B be an elementary subgroup of C /B of cotype ot. The dimension of A / B is necessarily m = \(31 — |a|, and hence there are exactly m indices i € { 1 , . . . ,n} such that dimZ/Pz ( ^ H A/ B) = 1 + dimz /pZ(Li_i fl A/ B) . Call these indices i\ < • • • < im . Then A / B has a basis {up . . . , um } in which Uj e Vi-, and if we require that the ijth coordinate of be

1

if j = k and 0 otherwise, then this basis is unique. The vectors

u \ , . . . , Um form the rows of the standard matrix M for A /B . The columns i \ , . . . , im of M are called pivotal. The cotype of A can be read off from the pivotal columns of the standard matrix for A /B . By the Corollary, the condition cotype(A) = of means that (3\ — cq of the last (3i —/% columns of M are pivotal, /?2

—ol2 of the preceding fa ~ @3 columns are pivotal, and so on; finally

fa — cty, of the first fa columns of M are pivotal, where b = (3[. 12 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

It remains to impose the condition that pA = B, or equivalently 0. If degenerate, the form has a nonzero | radical, |

rad X = { y € X : [x, y] = 0 for all x € X },

\

and [•, •] descends to a nondegenerate form on the quotient X /(rad X). • An alternating form on X is characterized by its rank, which is an even integer 2m. If the form is nondegenerate, its isometry group is called the symplectic group Sp2m(F9). • A symmetric form on X is characterized by its rank and the dis­ criminant of the induced form on X / (rad X ). The discriminant of I

a nondegenerate form is an element of the group F?x/(F gx )2, repre­ sented by the determinant of a matrix for the form. Since q is odd, F* is cyclic of even order, and Fgx/(F gx ) 2 has order 2. Hence there are two kinds of nondegenerate symmetric forms on AT. If n is odd, then the two forms are proportional and hence give rise to the same isometry group On(Fq). If n is even, then X admits split forms and nonsplit forms, which are quite different. A split form has Witt index n/2; its isometry group is denoted O j(F 9). A nonsplit form has W itt index (n / 2 ) — 1 ; its isometry group is denoted 0 ^ (F g). All these groups are called orthogonal groups. Therefore, to specify a nilpotent class in g, we need a partition A of n, together with an element of Fq / (F?x ) 2 for each even integer (if G is sym­ plectic) or odd integer (if G is orthogonal) that appears with nonzero 24

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

multiplicity in A. The other parts of A (the odd parts if G is symplectic, the even parts if G is orthogonal) must appear with even multiplicity. Which discriminants can appear in combination with a partition A? The construction in §2.4 will show that anything is possible, except that the form (•, •) on V can be reconstructed from the forms [•, -]z. If G is symplectic, then there are no restrictions. If G is orthogonal, it turns out that the discriminant of (•, •) is the product of the discriminants of the forms [•, -]z for odd i. This condition rules out half the possible combinations of discriminants when A has an odd part. When com­ puting Hall polynomials, however, our focus is the module V rather than the group G, so we need not specify the discriminant of (•, •) in advance. It remains to index the nilpotent classes by objects that do not involve q. Naturally, other authors have confronted this problem; the scheme that works for Hall polynomials appears in a paper of Shoji [10]. Let mi denote the multiplicity of the part i in A. We attach a or ‘o’) to each nonempty block imi of equal parts of A,

sign

and call the result a signed partition. (Typical signed partitions for G = Spg(F9) are 4+ 22~ and 33°11°.) If [•, •],• is symplectic, there is no

discriminant, so the sign is ‘o’. Now suppose that [-,•]* is orthogonal. If mz is even, the sign is ‘+ ’ if [-,-]z is split and

’ otherwise. The

complications arise when mz is odd. Let dz be the discriminant of the form [•,

and define ez = [_z/2 j + |_m z/2j + # { j > i ■m j is odd}.

The sign is ‘+ ’ if (—1)&idi is a square, and If q =

1

(mod 4), so that

—1

(2-4)

if ( —1 )eidi is a nonsquare.

is a square mod q, then the factor (—l) e* 25

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

is trivial, but if q = 3 (mod 4), then it alters the parameterization.

2.4. T he Springer-Steinberg basis A good way to understand the module V is to choose an Fg-basis for V in which both the map T and the form (•, •) are easy to describe, i Springer and Steinberg [12 ] show that V decomposes as an orthogonal : direct sum of Fg[[£]]-modules with very simple descriptions. Each piece is either • a cyclic module with Fg-basis {e, Te, . . . , T z - 1e}, where (—l ) 2 = —e and (T ae, T be) ^ 0 if and only if a +

6

= i — l;o r

• a bicyclic module with Fg-basis {e, Te, . . . , T*- 1e, e', T e . . . . where (—1)* = e and (Tae, T be') = (—l)b if a + b = i — 1 , all other pairings of basis elements being zero. The form [•, •],• is orthogonal in the first case, symplectic in the second case. The union of the bases for the pieces of V is called a diagram basis (or Springer-Steinberg basis) of V.

The pictures above illustrate diagram bases for a symplectic module of type 4332211 and an orthogonal module of type 322221. Each dot represents a basis element, and T sends each dot to the one below it (or to zero if there is no dot below it). Two dots x, y are connected by 26

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

an arc if (x, y) ^ 0. In each picture, the dots labelled i yield a basis for the space By taking a direct sum of appropriate cyclic and bicyclic modules, we can arrange that when ( - l ) z = e, the symplectic form [•, -]t- has any ! desired even dimension, and when (—l)z 7^ e, the orthogonal form [•, •],• i

! is any desired diagonal form. Since every symmetric form over a field can be diagonalized, this construction is general. Thus every signed partition that satisfies the parity conditions set down in §2.3 occurs as the class of some nilpotent endomorphism T. E x a m p le . We can use a diagram basis to compute an interesting Hall polynomial for G = Spg(F?), namely ^ 2 2 +*" ^ ^ ^as c^ass 4+ 2+ , then V has a diagram basis {re, T x , T 2 x, T 3 a;, y, T y } with the following properties. (Using equation (2-4), we find that —[T3 x , T 3 x]4 and —[Ty,Ty] 2 are squares, so we can choose x and y to make them equal to 1 .) f

X

[C

t

(x , T 3 x ) = 1

I

„

\ t 3x \ T y

(Vi'Ty)

-

1

We wish to count 2-dimensional, cyclic, totally isotropic subspaces W C V according to the class of W ^ / W . Such a subspace has a basis {u;, Tw} with w = aT^x 4 - by 4- cT3x 4 - d T y , where a, b,c,d £ either a or

6

is nonzero. We have (w ,T w ) = (by^bTy) =

terms vanish), so 1 . See §4.3 for further explicit calculations of Hall polynomials.

ii

29

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

3. C om p u tin g G en eral H all P o ly n o m ia ls We now present a systematic method for computing the symplectic and orthogonal Hall polynomials h^u. We describe the algorithm mainly in English, avoiding complicated formulas and routine calculations where possible. The algorithm has been implemented as a fully documented Mathematica package (see the Appendix) and used to compute all the Hall polynomials for groups Spn and On with n < 12. The data help us understand, and the algorithm lets us establish, how the polynomial depends on the signs that appear in the class A and the coclass v.

3.1. O verview o f th e general calculation Experience with ordinary Hall polynomials suggests th a t we should consider, for each totally isotropic submodule W C V of type ji, the nested sequence of modules W D T ( W ) D • • O T m~ l (W) 3 {0}, where m = fi\. If m =

1,

(3-1)

then our problem is to count |/x|-dimensional,

totally isotropic subspaces W of ker(T) according to their coclass. The problem that arises for m >

1

is more general, so let us assume m > 1 .

We will count the submodules W recursively, by summing over the possibilities for the elementary module A =

Each totally

isotropic subspace A of ker(T) gives rise to a quotient module V' = A 1 -/A, which inherits a form

and a nilpotent endomorphism T'.

The map T also induces a surjection S' : ker(T') —> A,

v 4 - A ^ T{v). 30

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

If m > 1, then the map W

W /A is a bijection between totally

(•,•)-isotropic, T-stable subspaces W C V such th at T m - 1 (W) = A, and totally (•, -/-isotropic, Testable subspaces W f C V ' such that T m ~ 2 {Wf) C ker(T) and S ( T m~ 2 (W f)) = A.

Furthermore, if W

has type p and coclass v, and if dim A = p!m, then the type of W ’ is fJL with its last column removed, and the coclass of W ' is again v because W ^ / W « (W/A)-L/(W /A ) . To count such submodules W ' of V \ we sum over elementary submodules A' =

C

ker(T/), now with the added restriction that S f(Af) = A, and (if m > 2) we recursively count the appropriate submodules W " = W /A 1 of V " = ( A ^ ^ / A '. No further comphcations appear. (Each subspace A' gives rise to an endomorphism T " of [A') 1 -[A 1 and a surjection S" : ker(T") -> A', and we require that

= A', but

there are no extra conditions on W n.) Removing some primes from the notation, we reach the following problem. P ro b le m 3.1. Given a module V , a surjection S : ker(T) —►B, and k > 0, count k-dimensional subspaces A C ker(T) with 5(A) = B according to the class o f V f = A^~/A and a complete invariant for the induced map S f : ker(T/) —* A. In other words, we consider two subspaces A\, A2 Q ker(T) to be equiv­ alent if there is an isometry / : V/ —> V!^ of the resulting Fg[[t]]-modules and an isomorphism i : A\ —» A2 such that i o S[ = S '2 o f . Our prob­ lem is to compute the number of subspaces A in each equivalence class such that 5(A) = B. (If we put G = GLn (Fg) and let the form (•, •) be zero, this problem amounts to computing the polynomials Fa^ of §1. 31

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

Here a and (3 correspond to the types of V 1 and V, respectively; they determine the map S f: while (3 and

7

together determine the map 5.)

In §3.2, we divide the subspaces A C ker(T) into orbits consisting of equivalent subspaces. In §3.3, we show how to count the number of spaces A in each orbit such that 5(A) = B. The answer turns out to be a polynomial in q. Finally, in §3.4, we compute the class of A 1-/A and characterize the map S' in terms of the orbit of A. Thus we obtain an algorithm for computing general Hall polynomials. (To begin the recursion, let 5 be the zero map and let k = fs!m = dim T m~^{W).) We could convert the recursive calculation of Hall polynomials into something more conventional by enumerating all possible sequences of coclasses (z/, i/1, . . . ,

A) of the modules (3-1), together with

the possible orbits for the spaces T i ( W ) / T i+1 (W), and writing each possible combination as a “signed tableau”. The result would be a “formula” , akin to that in §1.3, expressing the polynomial h^u as a sum over signed tableaux. This point of view is helpful in §4.2, but we do not pursue it very far. The conditions defining signed tableaux are very complicated, and moreover, the polynomials corresponding to different signed tableaux can have different degrees. Consequently, our techniques do not immediately yield formulas for the degree and leading coefficient of the polynomial h^u.

3.2. A ctio n o f th e centralizer The most natural way to show that two subspaces A \ , A 2 C ker(T) are equivalent is to find an isometry g : V —►V commuting with T such that g(A\) — A 2 . In this case, any invariants derived from A\ and A 2 will be the same. The set of isometric automorphisms of V 32

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

th at commute with T is the centralizer of T in G , denoted C q (T). In this section, we show that the action of Cq (T) on ker(T) partitions the subspaces of ker(T) into orbits indexed by a set th a t depends only on the class of V, not on q. First we need a more abstract view of the Springer-Steinberg de­ composition, as found in [12]. We can regard V as a module over the ring R = F9[£]/(^), where I = A*. Let a : R —> R be the in­ volution that is trivial on F9 and sends t to —£, and let I : R —►F9 be the map that picks off the coefficient of tl~ l . Then there exists a unique cr-sesquilinear form [*, •] on V. taking values in R, such that (u, v) = ^([tJ,v]) for all vectors u,v

6

V. The term “cr-sesquilinear”

means that [•, •] is F9-bilinear and [au, v] = a[u: v],

[u, av] = a(a)[u, u]

(3-2)

for all a G R and all u, v € V. If Rq, « R / (£*) is a cyclic piece of the Springer-Steinberg decomposition of V, then [q,q] = t l~l (Ti~ l ci ,ci). If Rbi © Rb^ & R / ( t *) © R/{fi) is a bicyclic piece, then

All other pairings are determined by (3-2), and the relation (u,v) = i([u,v\) is easily checked. A short calculation shows that [u,v] = e(—1)l~l a[v,u].

(3-3)

(Check it on the generators, and extend to all vectors u, v using (3 - 2 ).) 33

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

To exhibit an element g £ C q (T), it suffices to define g on a set of .R-generators of V and check that [g(x),g(y)\ = [x,y] for each pair {.x , y} of these generators.

The next lemma and its corollary are crucial for everything that follows. L e m m a 3.2.

Let K = kerT, and let Mi = K fl im T \

The

set { g\% : g £ C q (T) } consists of all maps f £ GL(i^) such that f{M{) C Mi and the induced map fo :

—►Mi—i/M i is an

isometry of [•,•]* for every i > 0 . The lemma says that, in terms of a basis for K obtained from a diagram basis of V, the image of the restriction map C q (T) —►GL(K) is block upper triangular: it looks like

*

(

0 SPm3

*

*

*

*

\ 0

*

*

*

0 m3

*

*

\

or

0

0

Om2

*

0

SP7712

*

0

0

0

SPmi

0

0

Omi

V where

/

\

/

is the multiplicity of the part i in the type of T.

Proof If g commutes with T, then clearly f = g \ x preserves each space M{. If g also preserves the form (•,•), then / preserves the derived forms [•,

Now let / £ GL(K) be any invertible map satisfying these

conditions. Our task is to show that / = g\% for some g £ Cq (T). The Springer-Steinberg decomposition of V shows that V = ® i V i , where Vi is a direct sum of cyclic R- modules having F^-dimension i. Let 34

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

Ki = T l

1 (V5).

Then K = © z- K{ and Mi = © j >z Ay, so the natural

| map Ki —►M i - i / M i is an isomorphism. We can therefore identify i ■ M i_ i/M i with K i . As the picture indicates, / is the product of an automorphism h : K —►K such that h(K{) = Ki and [h(u),h(v)]i = ; [it, v\i for each i > 0, and a unipotent automorphism u : K —* K such that (it — l)(M j_ i) C Mi for each i > 0. It suffices to express both h and u in the form g\j^. The first part is easy. Let hi be the restriction of h to Ki. Choose a diagram basis of V£, and let T : V{ —►Vi be the F^-linear map that sends each basis element to the one above it (if any). Call the uppermost basis elements the generators of Vi. Let gi(e) = (T

o hi o T l

)(e) for

each generator e, and extend /^-linearly to obtain a map 9 i : Vi

Vi.

Then gi\K{ — hi- For any two generators e and er of Vi, we have [ffi(e),0 i(e')] = t?~t (hi(T,~ 1e),gi (e1)) = tl- i [hi (Ti~ 1e)1hi (T i- 1e % = tl~i [Ti~ 1e , T i~ 1e>]i = t1- i (Ti~ 1e,e') = [e,e']. Thus gi preserves the form on VJ. The direct sum of the maps gi is an element g G C q {T) such that g\% = h. The group of unipotent maps u : K —>K such that (it—l)(A ^_i) C Mi for each i is generated by maps 1 4- fij such that faj sends a single basis element of Ki to a basis element of Ay, where j > i, and sends all other basis elements of K to zero. Let X C Vi and Y C Vj be pieces of a Springer-Steinberg decomposition of V, with i < j, and let T2-1# € Ki and T J~ l y G K j be elements of the standard bases for X and Y, respectively. It is enough to find an R-module isometry g \X® Y

X ( & Y such that g(rK ~^x) = T^~^x + T^~^y and g is the 35

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

identity on all other basis elements of X

fl

K and Y

fl

K . (Extending

g to be the identity on all other pieces of V , we find th at g\% is the desired map

1

+ / y .)

There are four cases, depending on whether X is cyclic or bicyclic and whether Y is cyclic or bicyclic. For the purpose of calculating with the form [•,•], we may assume I = j. Then, by equation (3-3), we have [u,v]

=

a[v,u]

if Y is cyclic (e(—1)^_1

=

1)

and

[u,v] =

—

2) |_raz/ 2 J , if raz is even, else \ i / 2 \ + \m i/ 2 \ + # { j > i : m j is odd}.

(4-1)

{

+ m,i+i 4-

Here m >z is an abbreviation for p=

2^1

2

+ • • •. If we let

£ F2 , then we can write 6Z = d{ + pq.

The main idea of the proof is to find similar corrections that allow us to index the C q(T )-orbits in a consistent way. By Corollary 3.2, the orbit of a subspace A is determined by the dimensions xz, the ranks y{, and the discriminants t z

E

F2 of the bilinear spaces Az =

( A n i m T i~ ^ ) / ( A n i m T i ). We represent the orbit of A by the sequence £ of triples (xz, yz, q ) where q = t{ + psi and sz is a correction defined by Si = Ly*/2J + y{ • (|_z/2 j + m>i). We must prove two things: 47

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

(4-2)

• The number of subspaces A in a given orbit such that A + Z = ker T depends only on the orbit of A, the orbit of Z , and the class of V, not on q mod 4. • The class of V' = A ^/A and the orbit of Z 1 = ker S' are determined by the orbit of A and the class of V, and do not depend on q mod 4. We tackle the second problem first. Suppose (—l)z = —e, so that [•, -]i is orthogonal. Let m ', &'•, cj-, and

denote the multiplicity, sign,

correction, and discriminant associated with the i 1**1 block of the class of V ' . We have a Springer-Steinberg decomposition V' = 0 7- V( in J J which V- is the direct sum of contributions from V^, Vj+i, and Vi+2 By the results of §3.4, the dimension of V[ is

m i = im i ~ 2xi + Vi) + 2 ixi+l ~ Vi+l) + Vi+ 2 and its discriminant, in additive notation, is = (di +

4 - p(xi — Vi)) + p{%i+1 —2/i-+-l) + (ti + 2 + PVi+2 ) •

This equation gives rise to a formula for the sign 6' in terms of the sign b{ and the parameters r t- and r ^+ 2 °f the orbit of A, namely —

^

+

^

+

^

+

2

+

p

[

c i

+

q

+

s

z- +

( a ; i —

Vi) + (xi+ i

—

2 / i + i ) + s ; +

2

+

2/ 2+ 2 ]

•

For the class of V' to be independent of q mod 4, the formula in brackets must vanish. Expand it using equations (4-1) and (4-2), observing that

and yi+\ are even because [•, -]^+ i is symplectic. 48

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

Therefore

= m >, + y, (mod 2) and m >,-+ 2 =

+ m z (mod 2),

and the result is [ ( m + Vi + Vi+2 )/2 j -

+

5+1

+ {rrii + y, + yz+2) ( L V 2 J + m >i + Vi)

+ \m i/ 2 \ + m i{ \i/2 \ +m>,-) + [y,-/2J + 2f i ( |i / 2 j + m>z) + :ez- y z + z z+1 + h / i + 2 / 2J H- 2/ih-2 ( L^/2J -F 1 -f m > , + m z) + 3/i+2-

The underlined terms cancel, as do all terms involving [_z/2J and m >z. Rearranging terms and changing signs, we get rrii

+ Vi + Vi+ 2

.

2

Vi .

.2.

Vi+ 2 I

2

J

+ (m z + yz + Vi+2)yi + 2/i + yi+2 m i-

Now observe that

CL-\-b-\rC “~ T ~

maj{a, b, c} =

= [f J + 0 1

5 -F [§J + m a j { a , 6, c}, where

if at most one of a, 6 , c is odd, if two or three of a, 6 , c are odd.

Working in F2 , we have maj{a, 6 , c} = ab+ac+bc. Thus our expression becomes (771*2/,- 4-mzyz+2 + 2/i2/i+2) + (mz2/z + Vi + Vi+2 Vi) + Vi + Vi+ 2 Everything cancels! The proof that the orbit of Z' is independent of q mod 4 is similar but simpler. By §3.4 again, the component Z & '■— Z ' n V jC comes from Vi and Vi+i . It has rank ra, — -F y, and discriminant d, + 1, + p[x{ —y,-). Equation (4-2) implies that the correction when q = 3 (mod 4) is l ( m i - 2 xi + y i)/ 2 \ + (m ,+ yz) ( [ i/ 2 j 4 - m ^ ) . In terms of the signs bz and rz, therefore, the sign associated with Z 1- is bi+n+p Ci+Si+(xi-yi) + l ( m i - 2 x i+ y i) / 2 \ +(ra*+y,-)( [i/ 2 J + m > z) 49

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

Expand the term in brackets, using (4-1) and (4-2), to get |m z-/2j + m i([ i/2 \ + m > 2-) + |j/z-/2J + V i(li/2\ + ra>*) + & + L(m i + Vi) / 2 J + (m i + Vi)( L V 2 J + m >i +

2/z)*

All terms involving \i/2J and m>z- cancel, along with a pair of yi s, leaving mi + yi

2

L

J

rrii

Vi

. 2 .

.2 .

which is zero. Thus the orbit is independent of p. It remains to show that the enumeration algorithm of §3.3 gives the same answers regardless of p. Discriminants came into play only in the solution to the main subproblem, when [•, -]j is orthogonal. That subproblem was to count subspaces

C K{ having a given isometry

type and having /c-dimensional intersection with a fixed subspace Z\ C K{. By equations (4-1) and (4-2), the discriminant of each space U{, A i, and

is corrected by a term that depends only on its rank r,

namely p(|_r/ 2 J +rj. We show th at corrections

e F2 is constant, guarantee invariance.

The proof recapitulates the solution of Subproblem 3.3. (It may also help to refer to the Mathematica code in the Appendix.) 8

Let

E F2 be fixed, and let U be an orthogonal space of dimension u and

discriminant s + p([u/ 2 J -f-u