Combinatorial properties of quasisymmetric Schur functions and generalized Demazure atoms

Citation preview

UNIVERSITY OF CALIFORNIA, SAN DIEGO Combinatorial Properties of Quasisymmetric Schur Functions and Generalized Demazure Atoms A dissertation submitted in partial satisfaction of the requirements for the degree Doctor of Philosophy in Mathematics by Janine LoBue Tiefenbruck

Committee in charge: Professor Professor Professor Professor Professor

Jeffrey Remmel, Chair Adriano Garsia Ronald Graham Ramamohan Paturi Jacques Verstraete

2015

UMI Number: 3714267

All rights reserved INFORMATION TO ALL USERS The quality of this reproduction is dependent upon the quality of the copy submitted. In the unlikely event that the author did not send a complete manuscript and there are missing pages, these will be noted. Also, if material had to be removed, a note will indicate the deletion.

UMI 3714267 Published by ProQuest LLC (2015). Copyright in the Dissertation held by the Author. Microform Edition © ProQuest LLC. All rights reserved. This work is protected against unauthorized copying under Title 17, United States Code

ProQuest LLC. 789 East Eisenhower Parkway P.O. Box 1346 Ann Arbor, MI 48106 - 1346

Copyright Janine LoBue Tiefenbruck, 2015 All rights reserved.

The dissertation of Janine LoBue Tiefenbruck is approved, and it is acceptable in quality and form for publication on microfilm and electronically:

Chair

University of California, San Diego 2015

iii

DEDICATION

To my smart, strong, and inspiring aunt, Paula Van Riper.

iv

TABLE OF CONTENTS Signature Page . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

iii

Dedication . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

iv

Table of Contents . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

v

List of Figures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . vii Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

ix

Vita . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

xi

Abstract of the Dissertation . . . . . . . . . . . . . . . . . . . . . . . . . . . xii Chapter 1

Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 1.1 Background . . . . . . . . . . . . . . . . . . . . . . 1.1.1 Symmetric Functions . . . . . . . . . . . . . 1.1.2 Generalized Demazure Atoms . . . . . . . . 1.1.3 Quasisymmetric Functions . . . . . . . . . . 1.2 Overview . . . . . . . . . . . . . . . . . . . . . . . . 1.2.1 Murnaghan-Nakayama Rules . . . . . . . . . 1.2.2 Enumeration of Permuted Basement Fillings 1.2.3 Bruhat Order . . . . . . . . . . . . . . . . .

. . . . . . . . .

1 2 2 9 20 28 28 34 38

Chapter 2

A New Proof of the Classical Murnaghan-Nakayama Rule . . .

42

Chapter 3

A Murnaghan-Nakayama Rule for Generalized Demazure Atoms 50

Chapter 4

Murnaghan-Nakayama Rules for Quasisymmetric Schur Functions and Row-Strict Quasisymmetric Schur Functions . . . . 4.1 Murnaghan-Nakayama Rule for Quasisymmetric Schur Functions . . . . . . . . . . . . . . . . . . . . . . . . . . 4.2 Murnaghan-Nakayama Rule for Row-Strict Quasisymmetric Schur Functions . . . . . . . . . . . . . . . . . . . . .

Chapter 5

Enumeration of Permuted Basement Fillings . 5.1 Lattice Paths . . . . . . . . . . . . . . . 5.2 Enumeration of Generalized Rectangular Shapes . . . . . . . . . . . . . . . . . . . 5.3 Enumeration of Other Shapes . . . . . .

v

. . . . . . . . .

. . . . . . . . .

. . . . . . . . . . . . . . . . . .

63 64 84 91 92

. . . . . . . . . 102 . . . . . . . . . 109

Chapter 6

Bruhat Order . . . . . . . . . . . . . . . . . . . . . . . . . . . 116

Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 128

vi

LIST OF FIGURES Figure 1.1: Ferrers diagrams for λ = (5, 3, 2, 1) and µ = (4, 2), and the skew-diagram for λ/µ = (5, 3, 2, 1)/(4, 2). . . . . . . . . . . . . . Figure 1.2: The column-strict tableaux that generate s(2,1) (x1 , x2 , x3 ). . . . Figure 1.3: The row insertion procedure for a column-strict tableau. . . . . Figure 1.4: Skew columns of size 2 around the diagram of (4, 2). . . . . . . Figure 1.5: (5, 5, 4, 1)/(5, 3, 2, 1) is a rim hook of (5, 5, 4, 1) with sign −1. . . Figure 1.6: The diagram and augmented diagram of (2, 3, 0, 2, 0, 1). . . . . . Figure 1.7: The reading order of the cells in an augmented diagram. . . . . Figure 1.8: Type A and B triples. . . . . . . . . . . . . . . . . . . . . . . . Figure 1.9: The B-increasing condition. . . . . . . . . . . . . . . . . . . . . Figure 1.10: An example of a permuted basement filling of shape (2, 3, 0, 2, 0, 1) and basement 261345. . . . . . . . . . . . . . . . . . . . . . . . Figure 1.11: The transformation from column-strict tableau to PBF with basement n . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Figure 1.12: The insertion algorithm for permuted basement fillings. . . . . . Figure 1.13: The shift map and inverse shift map for PBFs. . . . . . . . . . Figure 1.14: Transposed skew rows and transposed skew columns for γ = (2, 0, 3, 1, 1, 3, 1, 0, 0) relative to basement 127346589. . . . . . . Figure 1.15: The collection CSCT ((2, 1), 3) of all CSCTs of shape (2, 1) with entries in {1, 2, 3}. . . . . . . . . . . . . . . . . . . . . . . . . . Figure 1.16: The collection RCST ((2, 1), 3) of all RSCTs of shape (2, 1) with entries in {1, 2, 3}. . . . . . . . . . . . . . . . . . . . . . . . . . Figure 1.17: dg(4, 3, 2, 4) v dg(4, 3, 1, 3, 6) but dg(4, 3, 2, 4) 6⊆ dg(4, 3, 1, 3, 6). Figure 1.18: Examples of a (2, 2, 1, 4)-column and a (2, 2, 1, 4)-row. . . . . . Figure 1.19: A satisfactory labeling of dg((3, 2, 0, 2, 2, 3)/(2, 1, 0, 1, 0, 2)) . . . Figure 1.20: Unique satisfactory labelings of dg(δ/γ) for δ = (5, 0, 1), (3, 0, 3) and γ = (2, 0, 1) . . . . . . . . . . . . . . . . . . . . . . . . . . Figure 1.21: Two satisfactory labelings of dgC (β/α) for α = (2, 2, 1, 4), β = (3, 1, 4, 2, 5), and coloring C as shown. . . . . . . . . . . . . . . Figure 1.22: Diagrams (a) and (b) are not uniquely colored. Diagram (c) is uniquely colored. . . . . . . . . . . . . . . . . . . . . . . . . . . Figure 1.23: The contents and hook lengths for λ = (5, 3, 2, 1). . . . . . . . . Figure 1.24: The three 2-paths of length 6. . . . . . . . . . . . . . . . . . . . Figure 1.25: An example of the map 1-Θ3 . . . . . . . . . . . . . . . . . . . . Figure 1.26: The weak Bruhat order on S3 . . . . . . . . . . . . . . . . . . . . Figure 1.27: The grouping of Demazure atoms by shape when shifting to basement 213 is a refinement of the grouping when shifting to basement 231. . . . . . . . . . . . . . . . . . . . . . . . . . . . . Figure 2.1: Examples of rim hooks and broken rim hooks. . . . . . . . . . . Figure 2.2: The involution I. . . . . . . . . . . . . . . . . . . . . . . . . . .

vii

4 4 6 6 7 10 11 12 12 13 14 15 17 19 22 24 25 27 29 30 32 33 35 36 37 39

41 44 45

Figure 2.3: The involution J. . . . . . . . . . . . . . . . . . . . . . . . . . . Figure 2.4: The involution K. . . . . . . . . . . . . . . . . . . . . . . . . .

47 49

Figure Figure Figure Figure Figure

3.1: 3.2: 3.3: 3.4: 3.5:

Proof that δ is an acceptable shape. . . . . Proof that L satisfies rules (a) through (c). Proof of σ-compatability. . . . . . . . . . . The involution J, when r = 6, k = 5. . . . . A rim hook and its satisfactory labeling . .

55 55 57 59 62

Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure

4.1: 4.2: 4.3: 4.4: 4.5: 4.6: 4.7: 4.8: 4.9: 4.10:

The diagrams that result from the multiplication e1 (X)S(2,1) (X). The cell labeled h is not removable. . . . . . . . . . . . . . . . . The colored labeled diagrams used to compute p2 (X)S(2,1) (X). . Proof that C is an acceptable coloring. . . . . . . . . . . . . . . Proof that L is a Pieri labeling. . . . . . . . . . . . . . . . . . . Proof that cj is a removable cell. . . . . . . . . . . . . . . . . . Any recoloring must move gray cells in one of these two ways. . Any recoloring creates a relabeling, in one of these two ways. . The colored labeled diagrams used to compute p2 (X)RS (2,1) (X). The Murnaghan-Nakayama diagrams or the row-strict MurnaghanNakayama diagrams can be used to compute p2 (X)RS (2,1) (X).

Figure Figure Figure Figure Figure

5.1: 5.2: 5.3: 5.4: 5.5:

. . . . .

. . . . .

. . . . .

. . . . .

. . . . .

. . . . .

. . . . .

. . . . .

. . . . .

. . . . .

. . . . .

The three 2-paths of length 6. . . . . . . . . . . . . . . . . . . . Q as in the proof of Theorem 7. . . . . . . . . . . . . . . . . . . 2-path3 (P ) if row 3 of P contains 1, 2, 3, and 5 . . . . . . . . . An example of the map 1-Θ3 . . . . . . . . . . . . . . . . . . . . An example of the map R : P BF (((s + u)t , u)k , (t+1)k ) → P BF (((s)t , 0)k , (t+1)k ). . . . . . . . . . . . . . . . . . . . . . . Figure 5.6: The unique permuted basement filling of shape (stair4 )3 . . . . . Figure 5.7: If t = 4 and e < 5m + 4, then d > e . . . . . . . . . . . . . . . . Figure 5.8: An example of the Γ−1 3,3 map. . . . . . . . . . . . . . . . . . . . .

65 65 66 69 70 72 77 79 87 89

94 103 103 105 108 109 111 113

Figure 6.1: The shape γ(τ, σ) for τ = 23765481 and σ = 12763485. . . . . . 121 Figure 6.2: The PBFs A and B in basement σ = 12763485, and when shifted to basement τ = 23765481. . . . . . . . . . . . . . . . . 122

viii

ACKNOWLEDGEMENTS There are many people to whom I am indebted for their support and encouragement throughout my studies. My advisor, Jeff Remmel, has been incredibly helpful, encouraging, and patient. I am always impressed by his mathematical wisdom, his meticulously organized file cabinets, and his ability to make plenty of time for his students while being committed to so many other causes. Being his student has been a privilege for which I am very grateful. I have been fortunate to find a great network of friends in the math department at UCSD, and it has been fun to share this graduate school journey with all of them. In particular, Hooman and Michael have been wonderful friends from day one. My family members, especially my parents Cindy and Tony and my husband Mark, have showered me with encouragement and love. My mother has constantly cheered me on, and my father has shown me firsthand what hard work and dedication can accomplish. Mark’s confidence in me has never wavered, and he has been a source of strength when I needed it most. I am so thankful for his constant support throughout the entire process of earning this degree. A portion of Chapter 1 has been published in DMTCS Proceedings. LoBue, Janine; Remmel, Jeffrey. “A Murnaghan-Nakayama Rule for Generalized Demazure Atoms”, DMTCS Proceedings (Online), 2013. In addition, a portion of Chapter 1 is currently being prepared for submission for publication. Remmel, Jeffrey; Tiefenbruck, Janine LoBue. The dissertation author is an author of all of this material. The contents of Chapters 2, 4, 5, and 6 are currently being prepared for submission for publication. Remmel, Jeffrey; Tiefenbruck, Janine LoBue. The dissertation author is an author of this material. An abbreviated version of Chapter 3 also appeared in DMTCS Proceedings. LoBue, Janine; Remmel, Jeffrey. “A Murnaghan-Nakayama Rule for Generalized Demazure Atoms”, DMTCS Proceedings (Online), 2013. The dissertation author is an author of this paper. The complete version of Chapter 3 is currently being prepared for submission for publication. Remmel, Jeffrey; Tiefenbruck, Janine

ix

LoBue. The dissertation author is an author of this material.

x

VITA 2009

Bachelor of Science in Mathematical Sciences, summa cum laude, Loyola College, Baltimore

2009

Bachelor of Arts in Computer Science, summa cum laude, Loyola College, Baltimore

2009-2014

Graduate Teaching Assistant, University of California, San Diego

2012-2013

Associate Instructor, Department of Mathematics, University of California, San Diego

2013-2014

Senior Teaching Assistant, Department of Mathematics, University of California, San Diego

2015

Associate Instructor, Department of Computer Science and Engineering, University of California, San Diego

2015

Doctor of Philosophy in Mathematics, University of California, San Diego

PUBLICATIONS J. LoBue and J. Remmel, A Murnaghan-Nakayama Rule for Generalized Demazure Atoms, DMTCS Proceedings (Online) (2013). J. LoBue Tiefenbruck and J. Remmel, The mu pattern in words, Journal of Combinatorics 5.3 (2014), 379–417. J. LoBue Tiefenbruck and J. Remmel, q-analogues of convolutions of Fibonacci numbers, Australasian Journal of Combinatorics, to appear.

xi

ABSTRACT OF THE DISSERTATION

Combinatorial Properties of Quasisymmetric Schur Functions and Generalized Demazure Atoms

by

Janine LoBue Tiefenbruck Doctor of Philosophy in Mathematics University of California, San Diego, 2015 Professor Jeffrey Remmel, Chair

The Schur functions, which form an important basis for the ring of symmetric functions, have been shown in recent years to have various quasisymmetric and nonsymmetric refinements. The quasisymmetric Schur functions and row-strict quasisymmetric Schur functions each refine the Schur functions in separate ways, and the generalized Demazure atoms further decompose the Schur functions into nonsymmetric pieces. The broad goal of this dissertation is to determine which algebraic and enumerative properties of the Schur functions are retained by these refinements. A main result of this work is the development of new Murnaghan-Nakayama rules for the these refinements. The classical Murnaghan-Nakayama rule gives a

xii

combinatorial interpretation to the coefficients in the product of a Schur function and a power symmetric function when expanded as a sum of Schur functions. We prove new Murnaghan-Nakayama rules for the quasisymmetric and nonsymmetric setting. That is, we give combinatorial interpretations to the coefficients that arise in expanding a power symmetric function times a quasisymmetric Schur function as a sum of quasisymmetric Schur functions and in expanding a power symmetric function times a generalized Demazure atom as a sum of generalized Demazure atoms. We also explore the enumeration of the combinatorial objects called permuted basement fillings that generate the generalized Demazure atoms. We show that certain permuted basement fillings can be counted by nice determinantal formulas, and we also uncover connections to lattice paths including Dyck paths and their generalizations. Finally, we show that the nonsymmetric decomposition of the Schur functions into generalized Demazure atoms corresponds in a certain way to the weak Bruhat partial order on the symmetric group.

xiii

Chapter 1 Introduction

1

2

1.1

Background Symmetric functions are an important area of study in algebraic combina-

torics because they are closely linked with other key combinatorial objects, such as partitions and permutations. Further, symmetric functions are a bridge between combinatorics and other areas of mathematics, including linear algebra, group theory, and representation theory. Symmetric functions are so named because they have the characteristic property that rearranging the order of the input variables produces the same function. While the study of symmetric functions has been said to date back to as early as the start of the 16th century [Fun30], there have been many more recent developments in the area that are the subject of much active research. In fact, in the past ten years, a new field of study has emerged with the discovery of various refinements of arguably the most important type of symmetric functions, the Schur functions. These refinements include the quasisymmetric Schur functions, the row-strict quasisymmetric Schur functions, and the nonsymmetric generalized Demazure atoms. Our broad question of interest is determining which properties of the symmetric Schur functions are retained by these quasisymmetric and nonsymmetric refinements.

1.1.1

Symmetric Functions Let N denote the set of natural numbers and let P denote the set of positive

integers. We say that λ = (λ1 , λ2 , . . . , λ` ) is a partition of m into ` parts if each ` X λi ∈ P, λ1 ≥ λ2 ≥ · · · ≥ λ` , and λi = m. We say that the size of λ is i=1

m, which we write as |λ| = m. Similarly, we say that α = (α1 , α2 , . . . , α` ) is a ` X composition of m into ` parts if each αi ∈ P and αi = m. Finally, we say that i=1

γ = (γ1 , γ2 , . . . , γ` ) is a weak composition of m into ` parts if each γi ∈ N and ` X γi = m. Thus in a weak composition, parts of size zero are allowed. We let i=1

λ ` m denote that λ is a partition of m, α  m denote that α is a composition of m, and γ w m denote that γ is a weak composition of m. If γ is a weak composition

3 of m, then we let α(γ) be the composition of m that arises by removing the zeros from γ, which we call collapsing γ. For example, if γ = (2, 0, 3, 0, 2, 0, 0, 4), then α(γ) = (2, 3, 2, 4). Similarly, if α is a composition of m, then we let λ(α) be the partition of m induced by reordering the parts of α in weakly decreasing order, which we call rearranging α. For example, if α = (2, 3, 2, 4), then λ(α) = (4, 3, 2, 2). Given a weak composition of m into ` parts, we can define γi = 0 for all i > `. Then a bounded-degree formal power series in indeterminates x1 , x2 , . . . is a for∞ X Y mal power series f (x1 , x2 , . . . ) = cγ xγi i with rational coefficients cγ , where γw m

i=1

the sum is over all weak compositions of m. Call this set of bounded-degree formal M power series BDm [[x1 , x2 , . . . ]]. Then let BD[[x1 , x2 , . . . ]] = BDm [[x1 , x2 , . . . ]]. m≥0

An element f of BD[[x1 , x2 , . . . ]] is said to be a symmetric function if and only if for all N ≥ 1 and for all permutations σ in the symmetric group SN , f (x1 , x2 , . . . ) = f (xσ1 , xσ2 , . . . , xσN , xN +1 , xN +2 , . . . ). To each f ∈ BD[[x1 , x2 , . . . ]], we can define a corresponding polynomial f (x1 , x2 , . . . , xn ) in finitely many variables by setting xi = 0 for all i > n. Thus f (x1 , x2 , . . . , xn ) is symmetric if and only if f (x1 , x2 , . . . , xn ) = f (xσ1 , xσ2 , . . . , xσn ) for all σ ∈ Sn . A symmetric function in finitely many variables x1 , . . . , xn is called a symmetric polynomial. We will use X to denote the infinite set of variables x1 , x2 , . . . . We will fix a positive integer n and use Xn to denote x1 , . . . , xn . A symmetric function f is said to be of homogeneous degree m if every term of f has degree m. The set of symmetric functions of homogeneous degree m forms M a vector space Λm , and the direct sum Λ = Λm forms a graded ring called the m≥0

ring of symmetric functions. The vector space Λm has many known bases, all of which are indexed by partitions of m. A partition λ = (λ1 ≥ λ2 ≥ · · · ≥ λ` ) of m into ` parts can be pictured visually as a diagram of cells or boxes, called the Ferrers diagram of λ, or dg(λ), where the ith row from the top contains λi left-justified cells, for i = 1, . . . , `. Given two partitions λ = (λ1 ≥ λ2 ≥ · · · ≥ λ` ) and µ = (µ1 ≥ µ2 ≥ · · · ≥ µk ), we say that µ is contained in λ and write µ ⊆ λ if k ≤ ` and µi ≤ λi for i = 1, . . . , k. The skew-diagram dg(λ/µ) is the diagram of λ with the cells of dg(µ) removed.

4 The shape λ/µ is called a skew shape. Trivially, any partition λ is a skew shape, if we take µ to be the empty partition. Figure 1.1 depicts two Ferrers diagrams and a skew-diagram.

λ

μ

λ/μ

Figure 1.1: Ferrers diagrams for λ = (5, 3, 2, 1) and µ = (4, 2), and the skewdiagram for λ/µ = (5, 3, 2, 1)/(4, 2). We can define the Schur function basis of Λm in terms of certain fillings of the cells in the Ferrers diagram of λ. Fill the cells of dg(λ) with positive integer entries in such a way that the rows weakly increase from left to right and the columns strictly increase from top to bottom. Such fillings F are called columnstrict tableaux. We let F (i, j) denote the entry in row i and column j, and to each Y such filling F of dg(λ), we associate the weight wt(T ) = xF (i,j) . If CST (λ) (i,j)∈dg(λ)

is the set of all column-strict tableaux of shape λ, we can then define the Schur function sλ combinatorially as X

sλ (X) =

wt(T ).

(1.1)

T ∈CST (λ)

In the case of finitely many variables, sλ (Xn ) has an equivalent definition as a quotient of determinants. Figure 1.2 shows all eight column-strict tableaux of shape (2, 1) with entries in {1, 2, 3}. Summing their weights gives s(2,1) (x1 , x2 , x3 ) = x22 x3 + x2 x23 + x21 x2 + x21 x3 + x1 x22 + x1 x2 x3 + x1 x2 x3 + x1 x23 . 2 2 3

2 3 3

1 1 2

1 1 3

1 2 2

1 2 3

1 3 2

1 3 3

Figure 1.2: The column-strict tableaux that generate s(2,1) (x1 , x2 , x3 ).

5 The Schur functions and the tableaux that generate them have been wellstudied, and many properties about them are known. For one, as λ ranges over all partitions of m, the set of Schur functions sλ forms a basis for Λm . The coefficients in the transition matrices between the Schur function basis and many other known bases of Λm are well-understood. Schensted defined an algorithm known as row insertion, which takes a column-strict tableau T and a positive integer k and inserts k into T to produce a new tableau with one additional cell [Sch61]. This insertion procedure was later extented to become the Robinson-Schensted-Knuth algorithm (RSK), which gives a bijection between N-valued matrices with a finite number of nonzero entries and ordered pairs of column-strict tableaux of the same shape [Knu70]. The RSK algorithm has a wide variety of applications in combinatorics, for example, in the study of generating functions for statistics on permutations. Row insertion itself has been instrumental in the proofs of various combinatorial rules for multiplying two symmetric functions, which provide information about the representation theory of the symmetric group Sm . For example, the Murnaghan-Nakayama rule can be used to give a combinatorial interpretation for the characters of Sm , and the Littlewood-Richardson rule gives a way of computing the coefficients that appear when an induced representation is decomposed into its irreducible components. Next, we present the row insertion procedure and several of these well-known multiplication rules for the symmetric functions because we wish to develop analogues for them in the quasisymmetric and nonsymmetric settings. The row insertion procedure takes as input a column-strict tableau T and a positive integer k. The operation T ← k is defined recursively as follows. To insert k into row 1, let j be minimal such that k < T (1, j). If no such j exists, then place k at the end of the first row. Otherwise, place k in cell (1, j) and bump T (1, j) into the second row. That is, perform the procedure to insert T (1, j) into row 2, and continue in this fashion one row at a time until some element is inserted at the end of a row, possibly as the first cell in a new row. Figure 1.3 shows an example of this procedure, where the circled elements are the ones that are bumped during the process. This row insertion procedure always produces a column-strict tableau

6 and is instrumental in the proofs of the following multiplication rules. 2

1 1 2 3 5 2 3 4 4 5 5

1 2 4 5

=

1 2 2 5 3 3 4 5

Figure 1.3: The row insertion procedure for a column-strict tableau. The classical Pieri rules give combinatorial definitions of the coefficients that arise when we multiply sµ by certain other important symmetric functions and expand the result in terms of Schur functions sλ . Let hr be the rth homogeneous symmetric function, defined as hr (X) :=

X

xi1 . . . xir

1≤i1 ≤···≤ir

and let er be the rth elementary symmetric function, defined as er (X) :=

X

x i 1 . . . xi r .

1≤i1 0, where γi ≥ γj . A type B triple is a set of three cells a, b, c of the form (j, k + 1), (i, k), (j, k) for some pair of columns i < j of the diagram and some row k ≥ 0, where γi < γj . Note that basement cells can be elements of triples. As noted above, our fillings F have weakly increasing column entries, reading from top to bottom, so that F (a) ≤ F (c) for all such triples. We say that a triple

12 Type A

a c

Type B

b b

γi ≥ γj

a c

→

F (a) ≤ F (c) < F (b) or F (b) < F (a) ≤ F (c)

γi < γj

Figure 1.8: Type A and B triples. b a

→ F (a) < F (b)

γi < γj

Figure 1.9: The B-increasing condition. of either type is an inversion triple if the relative order of the entries is either F (b) < F (a) ≤ F (c) or F (a) ≤ F (c) < F (b). See Figure 1.8. A filling F is said to satisfy the B-increasing condition if, whenever i < j and γi < γj , then F (i, k − 1) < F (j, k) for all 1 ≤ k ≤ γi + 1, as in Figure 1.9. b If γ = (γ1 , . . . , γn ) is a weak composition, we say that a filling F σ of dg(γ) with positive integers is a permuted basement filling (PBF) of shape γ and basement σ if 1. the basement is filled with σ1 , σ2 , ..., σn from left to right, 2. column entries weakly increase reading from top to bottom, 3. every triple of type A or B is an inversion triple in F σ , and 4. F σ satisfies the B-increasing condition. It was observed in [HMR12] that the B-increasing condition plus the fact that column entries are weakly decreasing automatically implies that all type B triples are inversion triples. An example of a PBF of shape (2, 3, 0, 2, 0, 1) and basement 261345 is given in Figure 1.10, though it takes some effort to check that all four conditions are satisfied.

13

2 1 4 2 1 6 3 5 2 6 1 3 4 5

Figure 1.10: An example of a permuted basement filling of shape (2, 3, 0, 2, 0, 1) and basement 261345. Let P BF (γ, σ) be the set of all PBFs with shape γ and basement σ ∈ Sn . Since the largest basement entry is n and entries increase down columns, every PBF with basement σ ∈ Sn necessarily has entries in {1, . . . , n}. The weight of Y a PBF F σ ∈ P BF (γ, σ) is defined to be wt(F σ ) = xF σ (i,j) . Note that (i,j)∈dg(γ)

the basement cells do not contribute to the weight of the PBF. The generalized b σ (Xn ) is then defined by Demazure atom E γ b σ (Xn ) = E γ

X

wt(F σ ).

F σ ∈P BF (γ,σ)

We say that a shape γ is σ-compatible if γi ≥ γj whenever i < j and σi > σj . In b σ (Xn ) = 0 unless γ is σ-compatible. Thus, there are [HMR12], it is shown that E γ

no PBFs of shape γ and basement σ if γ is not σ-compatible. When σ = n , a PBF is exactly a semi-standard augmented filling, as defined previously by Mason [Mas08, Mas09]. When σ = n , a PBF is strictly decreasing from left to right and weakly decreasing from bottom to top. The only n -compatible shapes are partition shapes λ, so that PBFs become “reverse rowstrict tableaux” with a basement attached. In fact, column-strict tableaux are in bijection with PBFs with basement n , or these reverse row-strict tableaux. To see this, first we can take some T ∈ CST (λ, n), where CST (λ, n) is the set of all column-strict tableaux with entries in {1, . . . , n}, and replace each element i in T by n + 1 − i. This will result in a reverse column-strict tableau T 0 . That is, the entries in the rows of T 0 will weakly decrease as we read from left to right and the entries in each column will strictly decrease as we read from top to bottom. If we let RCST (λ, n) denote the set of all reverse column-strict tableaux of shape λ

14 with entries from {1, . . . , n}, then we can express the Schur function sλ as X X sλ (Xn ) = wt(T ) = wt(T 0 ). T ∈CST (λ,n)

T 0 ∈RCST (λ,n)

If one takes a reverse column-strict tableau T 0 , rotates by 90 degrees, and adds a basement filled with the permutation n (n − 1) . . . 2 1, then one gets the PBF T 00 corresponding to T 0 . This process is pictured in Figure 1.11. Note that since this process involves rotation of the usual Ferrers diagram, in which there are λi cells in the ith row, we read the shapes of PBFs as the heights of the columns from left to right. Thus, the augmented diagram of λ contains λi cells in the ith column. This b n (Xn ) = sλ (Xn ). Thus the generalized Demazure atoms bijection shows that E λ interpolate between Mason’s Demazure atoms and the Schur functions. T 1 2

1 2

T’ 3

3 3 1 2 2

T’’ 1 3 3

2

1 3

2

2

3

2

3

2

1

Figure 1.11: The transformation from column-strict tableau to PBF with basement n . Mason [Mas08] defined an insertion procedure k → F analogous to the Schensted row insertion procedure that inserts a positive integer k into a semistandard augmented filling to produce another semi-standard augmented filling. Haglund, Mason, and Remmel [HMR12] generalized this procedure to PBFs F σ with arbitrary basements. To define this insertion k → F σ , let F σ be the filling that extends the basement permutation by first adding a j in each cell (j, 0) with n < j ≤ k and then adding an extra cell filled with a 0 on top of each column. Let (x1 , y1 ), (x2 , y2 ), . . . be the cells of F σ listed in reading order. To insert k into F σ , go through the cells of F σ in reading order looking for the first (xi , yi ) such that F σ (xi , yi ) < k ≤ F σ (xi , yi − 1). Replace F σ (xi , yi ) with k and insert the cell’s previous value into the remaining cells in reading order, beginning with (xi+1 , yi+1 ). Continue in this way until some 0 is replaced by a positive integer. Finally, remove any zeros from the tops of the columns. Notice that k → F σ

15 3 0 1 1 2

0 2 0 4 2 0 6 0 3 0 5 6 1 3 4 5

=

3 1 4 2 2 6 1 3 5 2 6 1 3 4 5

Figure 1.12: The insertion algorithm for permuted basement fillings. creates a new cell at the top of some column in F σ . Figure 1.12 gives an example of this insertion procedure, where the circled elements are the ones that are bumped during the process. A fundamental result of [HMR12], which was used to prove many properties about generalized Demazure atoms and PBFs, is the fact that this insertion procedure k → F σ is well-defined and always produces a PBF. An important question addressed in [HMR12] is whether this insertion procedure can be reversed. To answer this question, the authors defined what they called removable cells. Let γ and δ be weak compositions with n parts such that dg(γ) ⊆ dg(δ), that is γi ≤ δi for all 1 ≤ i ≤ n. Suppose dg(δ/γ) consists of a single cell c = (x, y) which is in dg(δ) but not dg(γ). Then c is a removable cell from δ if it is at the top of column x and there is no cell to the right of c that is at the top of a column in dg(δ). That is, there is no j with x < j ≤ n and δj = y. It was shown in [HMR12] that if F σ ∈ P BF (γ, σ), and Gσ = k → F σ ∈ P BF (δ, σ), then the cell c in dg(δ/γ) is a removable cell. This terminology is used because it means that the insertion procedure can be reversed starting with c. That is, begin with the entry, say a, in cell c and read through the cells of Gσ in reverse reading order starting with c until an entry, say b, is found which is greater than a and positioned below a number less than or equal to a. Now replace b with a and continue reversing the insertion procedure with b. The entry that emerges from the first cell in reading order is the positive integer k that was originally inserted into F σ to produce Gσ . So long as cell c is removable, the insertion can be reversed in this way. Also, in the case that σ = n , the authors of [HMR12] show that this insertion algorithm reduces to a reverse row-strict version of the usual Schensted row insertion algorithm. Further, the insertion of words into PBFs with arbitrary

16 basements can be factored through this reverse row-strict version of Schensted row insertion. To state this result more precisely, we must first define an important function called the shift map. The shift map ρ was originally introduced by Mason for semi-standard augmented fillings, or permuted basement fillings with basement n [Mas08]. The shift map ρ sends any permuted basement filling F with basement n to a reverse row-strict tableau ρ(F ) by simply rearranging the elements of F in each row j ≥ 1. Row j of the tableau ρ(F ) has the elements appearing in row j of F arranged in decreasing order, reading from left to right. We can then add a basement below ρ(F ) containing the permutation ¯n to produce a permuted basement filling with basement equal to ¯n . Haglund, Mason, and Remmel extended this map for semistandard augmented fillings in a very natural way to apply to permuted basement filling F with any basement σ ∈ Sn . They called this map ρσ . Given any permuted basement filling F with basement σ, create a reverse row-strict tableau ρσ (F ) by reordering the elements of F in each row j ≥ 1 into decreasing order from left to right. Then append a basement filled with ¯n to get a permuted basement filling. An example of this map appears in Figure 1.13. Furthermore, it was shown in [HMR12] that ρσ is invertible, so that ρσ is a bijection. This extends the analogous result of Mason for semi-standard augmented fillings [Mas08]. In fact, the inverse of the shift map, ρ−1 σ can be easily described. Given a reverse row-strict tableau T , first, construct an empty PBF P with basement σ. Beginning with row 1 and moving up, process the elements of row j of T one at a time from left to right. To process an element, place it in the same row j of P but at the leftmost unoccupied column i so that column i remains weakly increasing from top to bottom. In this way, rows are reordered one at a time, from the bottom up, and within each row, larger numbers are placed first. Since the row sets have stayed the same, it should be clear that the resulting shape of P is some rearrangement of λ and also that the weight of T and P are the same. We call the act of applying the map ρ−1 σ shifting to basement σ. An example of shifting a reverse row-strict tableau to basement 261345 is given in Figure 1.13. A main result of [HMR12] is that the inverse shift map commutes with the

17 ρ−1 261345

← →

2 1 4 2 1 6 3 5 ρ261345 2 6 1 3 4 5

2 4 2 1 6 5 3 1 6 5 4 3 2 1

Figure 1.13: The shift map and inverse shift map for PBFs. insertion procedure for reverse row-strict tableaux. That is, suppose that we are given a word w = w1 . . . wk and σ ∈ Sn . Let E σ be the empty permuted basement filling with basement σ and let w → E σ = wk → (· · · (w2 → (w1 → E σ )) · · · ). Then w → E σ can be computed by finding w → E ¯n via the Schensted row insertion algorithm for reverse row-strict tableaux and then shifting to basement σ by applying the map ρ−1 σ . Since every PBF with basement σ comes from inserting some word into E σ , this gives an alternate characterization of PBFs. That is, P is a PBF with basement σ if and only if P = ρ−1 σ (T ) for some reverse row-strict tableau T . This characterization is very useful because it sidesteps the complicated triple rules and B-increasing condition, while still capturing all of the information those conditions encode. There is an important algebraic consequence of fact that ρ−1 σ commutes with row insertion. We have seen that complementing the entries of a column-strict tableau, rotating the diagram by 90 degrees, and attaching a basement filled with n gives a bijection between CST (λ, n) and P BF (λ, n ). For every permutation ]  ) and P BF (γ, σ). σ ∈ Sn , the map ρ−1 is a bijection between P BF (λ, n σ λ(α(γ))=λ

Since we could express the Schur function sλ (Xn ) as a sum of the weights of column-strict tableaux, we can instead express sλ (Xn ) as a sum of the weights of PBFs whose shape is some σ-compatible rearrangement of λ. Grouping by shape b σ (Xn ). Thus, a gives a way of decomposing sλ (Xn ) into nonsymmetric pieces E γ

main result of [HMR12] is that if λ = (λ1 , . . . , λ` ) is a partition of m with ` ≤ n parts and σ is any fixed permutation in Sn , then X bγσ (Xn ) sλ (Xn ) = E γw m λ(α(γ))=λ

(1.5)

18 This decomposition extends equation (1.4) to the permuted basement setting. Haglund, Mason, and Remmel also used the insertion algorithm for PBFs to prove extensions of the Pieri rules for generalized Demazure atoms. To state these extensions of the Pieri rules, we must define γ-transposed skew rows and γ-transposed skew columns relative to a basement permutation σ. To define a γ-transposed skew row, suppose that γ and δ are weak compositions of length n such that dg(γ) is contained in dg(δ) and σ is a permutation in Sn . Let c1 = (x1 , y1 ), . . . , ck = (xk , yk ) be the cells of dg(δ/γ) listed in reverse reading order. Let dg(δ (i) ) consist of the diagram of γ plus the cells c1 , . . . , ci . Then we say that δ/γ is a γ-transposed skew row relative to basement σ if 1. y1 < y2 < · · · < yk (no two cells are in the same row), 2. for i = 1, . . . , k, δ (i) is a σ-compatible weak composition, 3. dg(γ) ⊂ dg(δ (1) ) ⊂ dg(δ (2) ) ⊂ · · · ⊂ dg(δ (k) ), and 4. ci is a removable cell from δ (i) for i = 1, . . . , k. To define a γ-transposed skew column, suppose that γ and δ are weak compositions of length n such that dg(γ) is contained in dg(δ) and σ is a permutation in Sn . Let d1 = (x1 , y1 ), . . . , dk = (xk , yk ) be the cells of dg(δ/γ) listed in reading order. Let dg(δ (i) ) consist of the diagram of γ plus the cells d1 , . . . , di . We say that say that δ/γ is a γ-transposed skew column relative to basement σ if 1. for i = 1, . . . , k, δ (i) is a σ-compatible weak composition, 2. dg(γ) ⊂ dg(δ (1) ) ⊂ dg(δ (2) ) ⊂ · · · ⊂ dg(δ (k) ), and 3. di is a removable cell from δ (i) for i = 1, . . . , k. Note that in any γ-transposed skew column, it will be the case that no two cells of dg(δ/γ) are in the same column. If there were two cells di , dj in the same column with di above dj , then we would have i < j since the cells were labeled in reading order. Then δ (i) is not a weak composition shape so δ/γ is not a δ-transposed skew column.

19 For example, if n = 9, σ = 127346589, and γ = (2, 0, 3, 1, 1, 3, 1, 0, 0), then, in Figure 1.14, we have pictured a γ-transposed skew row relative to basement σ in the top left and a γ-transposed skew column relative to basement σ in the bottom left. The diagram on the top right is not a γ-transposed skew row relative to basement σ since c3 is not a removable cell from δ (3) = (2, 0, 3, 3, 1, 3, 1, 1, 0) and the diagram on the bottom right is not a γ-transposed skew column since the diagram consisting of γ plus cells d1 and d2 does not correspond to the diagram of a weak composition. When γ is a partition shape, it is easy to check that a γ-transposed skew row relative to basement ¯n is just a transposed skew row and a γ-transposed skew column relative to basement ¯n is just a transposed skew column. Thus the Pieri rules given below reduce to the classical Pieri rules when the shapes are partitions and the basement is the reverse of the identity. c4

c5

c3

c4 c3 c2

c2 c1

1

2

7

3

4

6

5

8

c1 9

A transposed skew row

d1

1

2

7

3

4

6

5

8

9

Not a transposed skew row

d2

d1 d2

d3

d3 d4 d5

1

2

7

3

4

6

5

A tranposed skew column

8

9

d4 d5 1

2

7

3

4

6

5

8

9

Not a transposed skew column

Figure 1.14: Transposed skew rows and transposed skew columns for γ = (2, 0, 3, 1, 1, 3, 1, 0, 0) relative to basement 127346589. The Pieri rules for the generalized Demazure atoms are as follows [HMR12].

20 Theorem 1. Let γ = (γ1 , . . . , γn ) be a weak composition of m and σ ∈ Sn . Then b σ (Xn ) = hr (Xn )E γ

X

b σ (Xn ), E δ

(1.6)

δ

where the sum runs over all weak compositions δ = (δ1 , . . . , δn ) of size m + r such that dg(γ) ⊆ dg(δ) and δ/γ is a γ-transposed skew row relative to basement σ and bγσ (Xn ) = er (Xn )E

X

bδσ (Xn ), E

(1.7)

δ

where the sum runs over all weak compositions δ = (δ1 , . . . , δn ) of size m + r such that dg(γ) ⊆ dg(δ) and δ/γ is a γ-transposed skew column relative to basement σ.

1.1.3

Quasisymmetric Functions A quasisymmetric function f (x1 , x2 , . . . ) is a bounded-degree formal power

series with rational coefficients, that is, an element of BD[[x1 , x2 , . . . ]] such that for all k and 1 ≤ i1 < i2 < · · · < ik , the coefficient of xα = xα1 1 xα2 2 . . . xαk k equals the coefficient of xαi11 xαi22 . . . xαikk for all compositions α with k parts. Similarly to symmetric functions, the set of quasisisymmetric functions of homogeneous degree M m forms a vector space QSymm ⊇ Λm , and the direct sum QSym = QSymm m≥0

forms a graded ring called the ring of quasisymmetric functions. There are two natural bases for QSymm , both introduced by Gessel [Ges84]. Bases for QSymm are indexed by compositions α of m. Since the definition of a quasisymmetric function requires each monomial of the form xαi11 xαi22 . . . xαikk to have the same coefficient whenever 1 ≤ i1 < i2 < · · · < ik , a natural way to express a quasisymmetric function is in terms of functions of the form Mα (X) =

X

xαi11 xαi22 . . . xαikk .

1≤i1 0. In other words, dg(δ/γ) can be labeled in accordance with rules (a) through (d) if and only if it avoids the configuration

.

Now suppose σ ∈ Sn , γ is a σ-compatible weak composition of m with n parts, and δ is an acceptable shape such that |δ/γ| = r. Then we say that δ

30

v v v

h

3 1 2

3 1 2

v v

Figure 1.20: Unique satisfactory labelings of dg(δ/γ) for δ = (5, 0, 1), (3, 0, 3) and γ = (2, 0, 1) has a unique satisfactory labeling if dg(δ/γ) has exactly one satisfactory labeling. For example, the shape shown in Figure 1.19 has a unique satisfactory labeling as (r)

pictured. We let Uγ,σ be the set of δs which are acceptable shapes for σ and γ such that |δ/γ| = r and dg(δ/γ) has a unique satisfactory labeling. For such a δ, we let sgn(δ/γ) equal (−1)h(v) where h(v) is the number of cells labeled h in the unique satisfactory labeling of dg(δ/γ). Then the Murnaghan-Nakayama rule for generalized Demazure atoms is as follows. Theorem 4. If σ ∈ Sn and γ = (γ1 , . . . , γn ) is a σ-compatible weak composition of m, then b σ (Xn ) = pr (Xn )E γ

X

b σ (Xn ). sgn(δ/γ)E δ

(1.24)

(r) δ∈Uγ,σ

b 312 (x1 , x2 , x3 ). The shapes For example, consider the product p3 (x1 , x2 , x3 )E (2,0,1) δ that appear on the right hand side of (1.24) must be 312-compatible weak compositions of 6 into 3 parts containing (2, 0, 1). Also, for δ to be an acceptable shape, we cannot have δ = (4, 1, 1) or δ = (3, 2, 1) otherwise γ2 < γ3 = δ3 ≤ δ2 . Since the first and second basement entries are out of order, we require δ1 ≥ δ2 . Similarly, we require δ1 ≥ δ3 . Take dg(2, 0, 1) and consider all the ways to add three cells around the outside of the diagram. Adding three cells to the first column creates the 312compatible shape (5, 0, 1). Moreover, this shape has a unique satisfactory labeling b 312 (x1 , x2 , x3 ) appears with no cells labeled h, as shown in Figure 1.20. Thus, E (5,0,1) on the right hand side of (1.24) with coefficient (−1)0 = 1. If instead we add two cells to the first column and one cell to another column, the lower of the two cells in the first column will not be assigned a v or h label by rules (a)-(d). Thus, there will be more than one satisfactory labeling of the diagram of such a shape, and it

31 will not contribute to the sum. If one cell is added to the first column and two cells are added elsewhere, note that rules (b) through (d) will not apply to the cell in the first column. Therefore, in order to get a unique satisfactory labeling, one where each cell’s label is forced by rules (a) through (d), we must make sure that the cell in the first column is not the rightmost cell in its row. The only shape that achieves this requirement is (3, 0, 3). This shape has a unique satisfactory b 312 (x1 , x2 , x3 ) labeling as shown in Figure 1.20, and this labeling has one h. So E (3,0,3) appears on the right hand side of (1.24) with coefficient (−1)1 = −1. Finally, if we add no cells to the first column, the only way to keep the height of the first column greater than or equal to the heights of the other columns, as required by the basement permutation 312, is to create the shape (2, 2, 2). In this case, however, dg((2, 2, 2)/(2, 0, 1)) contains the configuration

. Lemma 1 says there is no sat-

isfactory labeling, so this shape will not contribute to the sum. Putting this all tob 312 (x1 , x2 , x3 ) = E b 312 (x1 , x2 , x3 )− E b 312 (x1 , x2 , x3 ), gether gives p3 (x1 , x2 , x3 )E (2,0,1)

(5,0,1)

(3,0,3)

which can be verified by direct computation. Next, we define the necessary background to state the Murnaghan-Nakayama rules for the quasi-Schurs and row-strict quasi-Schurs. Let α be a composition of m. Suppose β is a composition of m + r. We say dg(β) has an acceptable coloring if there is a way of coloring the cells of dg(β) so that 1. m cells are gray, 2. r cells are white, 3. the gray cells form the diagram of a weak composition shape γ such that α(γ) = α, and 4. there is no j < k such that column j has a white cell in row ` and column k has a gray cell in row ` with no cell above it. The first three conditions say that dg(α) v dg(β), and the last condition forces that all the white cells are removable. Any such shape β with an acceptable coloring will be called an acceptable shape. Let β be an acceptable shape. For each acceptable coloring C, we let dgC (β/α) be the skew diagram consisting of the white cells of C. We will define a

32 Pieri labeling of dgC (β/α) to be a labeling of the white cells only with v’s and h’s that is consistent with the following three rules: (a) Assign an h to all but the rightmost white cell in each row. (b) Assign a v to any white cell above another white cell. (c) Assign an h to any white cell above a gray cell and having a white cell one row below and anywhere to the left. If a given shape β has more than one acceptable coloring, then β may have Pieri labelings for each of those colorings. Next, we define a satisfactory labeling of dgC (β/α) as a Pieri labeling that satisfies one additional condition. (d) Assign a v to the last white cell in reading order. For some examples of satisfactory labelings, see Figure 1.21. Note that in this example, cell (3, 3) is not determined by any of rules (a) through (d), so that this cell can be labeled with either a v or an h.

v v

h v

h h

v h

h v

h h

Figure 1.21: Two satisfactory labelings of dgC (β/α) for α = (2, 2, 1, 4), β = (3, 1, 4, 2, 5), and coloring C as shown. The following lemma gives a characterization for when dgC (β/α) has a satisfactory labeling. Lemma 2. Let α be a composition of m, and let β be an acceptable shape with acceptable coloring C. Then dgC (β/α) has a satisfactory labeling if and only if there is no cell (x, y) in dgC (β/α) such that (x + j, y) and (x, y − 1) are both cells in dgC (β/α) for some j > 0. In other words, dgC (β/α) can be labeled in accordance with rules (a) through (d) if and only if it avoids the configuration

.

33 Now suppose dgC (β/α) has a satisfactory labeling. We say that dgC (β/α) b 6= C of is a uniquely colored diagram if there is no other acceptable coloring C dg(β) such that dgCb (β/α) has a satisfactory labeling. For example, diagram (a) in Figure 1.22 is not a uniquely colored diagram, as diagram (b) has a different acceptable coloring, in which the white cells have a satisfactory labeling. Diagram (c) is a uniquely colored diagram, as there is no other acceptable coloring that permits a satisfactory labeling. The only other possible colorings are depicted in diagrams (d) and (e). However, diagram (d)’s coloring permits no satisfactory labeling because the white cells contain the configuration

, and diagram (e)’s

coloring is not acceptable because the white cell (2, 1) is not removable, which violates rule 4 of an acceptable coloring. Thus, diagram (c) is a uniquely colored diagram.

v (a)

v h

v v v (b)

hh

v

(c)

(d)

(e)

Figure 1.22: Diagrams (a) and (b) are not uniquely colored. Diagram (c) is uniquely colored. We say that dgC (β/α) is a Murnaghan-Nakayama diagram of r and α if it is a uniquely colored diagram and it has a unique satisfactory labeling. We will associate each Murnaghan-Nakayama diagram, D, with the sign (−1)h(D) , where h(D) is the number of cells labeled h in the unique satisfactory labeling. With these definitions, we can state the main theorems. Theorem 5. If α is a composition of m, then X pr (X)Sα (X) = (−1)h(D) Ssh(D) (X)

(1.25)

D∈MN(r,α)

where MN(r, α) is the set of all Murnaghan-Nakayama diagrams of r and α. (r)

Theorem 5 says that the nonzero coefficients uα,β in (1.22) are ±1, depending on the number of h’s in the satisfactory labeling of the Murnaghan-Nakayama diagram.

34 Theorem 6. If α is a composition of m, then pr (X)RS α (X) =

X

(−1)v(D)+1 RS sh(D) (X)

D∈MN(r,α)

where MN(r, α) is the set of all Murnaghan-Nakayama diagrams of r and α. (r)

Note that Theorem 6 says how the coefficients uα,β in the Murnaghan(r)

Nakayama rule for quasi-Schurs (1.22) are related to the coefficients vα,β in the (r)

Murnaghan-Nakayama rule for row-strict quasi-Schurs (1.23). First, uα,β = 0 (r)

if and only if vα,β = 0, which happens when there is no Murnaghan-Nakayama (r)

diagram of r and α that is of shape β. Otherwise, Theorem 5 says that uα,β = (−1)h(D) where h(D) is the number of cells labeled h in the Murnaghan-Nakayama (r)

diagram D of shape β. Likewise, Theorem 6 says that vα,β = (−1)v(D)+1 where v(D) is the number of cells labeled v in the Murnaghan-Nakayama diagram D (r)

of shape β. However, since D has r labeled cells total, we can write vα,β = (r)

(−1)v(D)+1 = (−1)r−h(D)+1 = (−1)h(D) (−1)r−1 = uα,β (−1)r−1 . This says that the coefficient of RS β in the Murnaghan-Nakayama rule for row-strict quasi-Schurs is (−1)r−1 times the coefficient of Sβ in the Murnaghan-Nakayama rule for quasiSchurs. Thus, when r is odd, these coefficients are the same, and when r is even, these coefficients have opposite sign. While the relationship between the coefficients in the Murnaghan-Nakayama rule for quasi-Schurs and the coefficients in the Murnaghan-Nakayama rule for row-strict quasi-Schurs turns out to be a straightforward sign change, this simple relationship is actually a nontrivial fact.

1.2.2

Enumeration of Permuted Basement Fillings Another somewhat different area in which we have made progress concerns

the enumeration of PBFs. Most of the prior research in this area of study has focused on the algebraic properties of Demazure atoms and generalized Demazure atoms, such as the insertion procedure, transitions to other bases, and various multiplicative rules. However, the enumerative properties of semi-standard augmented fillings and permuted basement fillings have not previously been explored. Our motivation for this study comes from the fact that each Schur function breaks up as

35 a sum of generalized Demazure atoms (1.12). The column-strict tableaux that generate the Schur functions can be counted by a well-known formula called the hook-content formula, so it is a natural question to ask whether there is a similar formula for semi-standard augmented fillings or permuted basement fillings. The hook-content formula, due to Stanley, gives the number of columnstrict tableaux of partition shape λ in terms of two statistics [Sta71]. For each cell u = (i, j) in the Ferrers diagram of λ, let the content of u be c(u) = j − i. Let the hook length of cell u, h(u), be given by one plus the number of cells directly to the right of u plus the number of cells directly below u. For example, Figure 1.23 shows the contents and hook lengths for the cells in the Ferrers diagram F(5,3,2,1) . The hook-content formula says that the number of column-strict tableaux of shape Y n + c(u) . λ with entries in {1, 2, . . . , n} is given by h(u) u∈F λ

0 1 2 3 4 -1 0 1 -2 -1 -3

content, c(u)

8 6 4 2 1 5 3 1 3 1 1

hook length, h(u)

Figure 1.23: The contents and hook lengths for λ = (5, 3, 2, 1). Thus, our goal is to determine whether there is some similar formula, perhaps involving statistics related to the cells of weak composition diagrams, that holds for permuted basement fillings. It turns out that we can show that two very different weak composition shapes can have the same number of permuted basement fillings, indicating that there is no simple analogue of the hook-content formula. For example, we have the following theorem, which says that two different shapes with a different number of cells can have the same number of permuted basement fillings when the basement is the identity permutation. Theorem 7. Let γ be any shape such that γi ≤ 1 for all i. Then |P BF (γ, n )| = 1. Similar phenomena with multiple-row shapes indicate again that there is no analogue of the hook-content formula for PFSs. Despite this, we can show that

36 for certain weak compositions γ, there are some nice determinantal formulas for the number of PBFs of shape γ in the case where the basement permutation is the identity n . In particular, we are able to count what we will call generalized rectangular shapes, which are weak compositions of the form ((s)t , 0)k , as well as some slight variations of generalized rectangular shapes. Here (s)t is the weak composition shape that arises by concatenating s with itself t times, and ((s)t , 0)k concatenates ((s)t , 0) with itself k times. We also consider staircase shapes of the form (0, 1, 2, . . . , s − 1)k . The resulting enumerative formulas have many close connections to lattice paths, such as Dyck paths and their generalizations, and we give bijective proofs between some of these combinatorial objects. A lattice path is a directed path on the integer lattice Z × Z. Define a t-path of length (t + 1)k to be a lattice path starting at (0, 0) consisting of tk up-steps in the (1, 1) direction and k down-steps in the (1, −t) direction. Let a t-Dyck path be a t-path that does not fall below the x-axis. In general, the number of t-Dyck paths of length (t + 1)k is given by
(t+1)k 1 . When t = 1, t-Dyck paths become ordinary the Fuss-Catalan number tk+1 k Dyck paths counted by the Catalan numbers. Given two t-paths A and B of the same length, we say that A is below B if there is no point x on the path for which the y coordinate associated with path A exceeds that associated with path B. For example, in Figure 1.24, the three 2-paths are each below one another from left to right. An r-tuple of noncrossing paths is a tuple (P1 , P2 , . . . , Pr ) of t-paths such that Pi is below Pi+1 for all 1 ≤ i < r. These tuples are very closely related to other well-studied combinatorial objects called watermelons [Fis84, Fei12].

Figure 1.24: The three 2-paths of length 6. We show that the number of PBFs of shape ((s)t , 0)k and basement (t+1)k is given by the number of (s − 1)-tuples of non-crossing t-Dyck paths of length

3 3 3 3 4

2 4 5 5 6

6 7 7 7

(

8

=

,

(

1 1 1 1 2

37

Figure 1.25: An example of the map 1-Θ3 . (s−1)

(t + 1)k. In fact, there is a nice bijective proof of this fact. Let t-N CDP (t+1)k denote the set of all (s − 1)-tuples of non-crossing t-Dyck paths of length (t + 1)k. If Q ∈ P BF (((s)t , 0)k , (t+1)k ), let r be some row in Q and define t-pathr (Q) to be the t-path of length (t + 1)k such that the ith step is an up-step if and only if the number i appears in row r of Q. Define a map t-Θs (Q) that sends Q ∈ P BF (((s)t , 0)k , (t+1)k ) to the tuple (t-path2 (Q), . . . , t-paths (Q)). An example of the map 1-Θ3 is given in Figure 1.25. Then we can prove the following theorem. Theorem 8. For all s ≥ 2 and t, k ≥ 1, t-Θs is a bijection from P BF (((s)t , 0)k , (t+1)k ) onto t-N CDP s−1 (t+1)k . In addition, we show that there is a nice determinantal formula to count the number of non-crossing r-tuples of t-Dyck paths of length (t + 1)k. To this end, let Ai = (0, (t + 1)(i − 1)) and Bi = ((t + 1)k, (t + 1)(i − 1)) for i = 1, . . . , r. For any 1 ≤ i, j ≤ r, let Pt (Ai , Bj ) denote the number of t-paths that start at Ai , end at Bj , and stay weakly above the x-axis. Using the standard lattice path involution arguments of Lindstrom [Lin73] and Gessel and Viennot[GV85], we can prove the following theorem. Theorem 9. For any k, r, t ≥ 1, the number of r-tuples of non-crossing t-Dyck paths of length (t + 1)k is equal to det(M (k,r,t) ) where M (r,t,k) = (Pt (Ai , Bj ))i,j=1,...,r . We also show how to obtain the numbers Pt (Ai , Bj ) using recursions and generating functions, so that this theorem may be useful for computation. We

38 note that in the case t = 1, when we are counting r-tuples of non-crossing Dyck paths, our matrix M (k,r,t) is not the same as the matrix in a previously discovered determinantal formula due to Desainte-Catherine and Viennot [DCV86]. We can also enumerate some other PBF shapes. For example, we can prove the following theorem with an easy bijection. Theorem 10. For all k, s, t, u ≥ 1, |P BF (((s + u)t , u)k , (t+1)k )| = |P BF (((s)t , 0)k , (t+1)k )|. Let stairs = (0, 1, 2, . . . , s − 1) for s ≥ 2 and consider shapes of the form (stairs )k . Then we have the following theorem. Theorem 11. For any s ≥ 2 and k ≥ 1, there is a unique permuted basement filling of shape (stairs )k with basement ks . Finally, this next theorem also has a bijective proof, though the bijection is not so straightforward. Theorem 12. For all t ≥ 2 and k ≥ 1, |P BF (((2)t−1 , 3, 0)k , (t+1)k )| = |P BF (((2)t , 0)k , (t+1)k )|.

1.2.3

Bruhat Order We also address a question of how the Schur functions decompose into

nonsymmetric pieces for two different basement permutations σ and τ . We know that for any fixed basement σ ∈ Sn , the Schur function sλ (Xn ) breaks up as a sum of generalized Demazure atoms according to equation (1.5), sλ (Xn ) =

X

bγσ (Xn ). E

γw m λ(α(γ))=λ

b σ (Xn ) appearing in this decomposition correspond to σThe nonzero terms E γ compatible shapes γ whose nonzero parts rearrange to λ. In particular, when the basement is the identity n , every weak composition shape γ is n -compatible, so the Schur function sλ (Xn ) breaks up into its most granular pieces, the Demazure

39 atoms. Since equation (1.5) holds for any basement, it must be the case that for b σ (Xn ) as a sum of terms of the form E b n (Xn ). Since a all γ and σ, we can write E γ

δ

generalized Demazure atom for any basement can be written as a sum of Demazure atoms, we want to know when this sum for one basement σ is actually a refinement of this sum for another basement τ . We show that one sum is a refinement of the other if and only if the basement permutations are comparable in a well-known partial order called the weak Bruhat order. For any permutation σ ∈ Sn , define the inversion set of σ as Inv(σ) = {(σi , σj ) : i < j, σi > σj } and its cardinality as inv(σ) = |Inv(σ)|. The weak Bruhat order is a partial order on the symmetric group, or more generally, on any Coxeter group. For Sn , we say τ covers σ in the weak Bruhat order, written written σ ≺ τ , if for some 1 ≤ i < n, τ = σ(i, i + 1) and inv(τ ) = inv(σ) + 1. For example, 14235 ≺ 14253 in S5 because 14253 = 14235(4, 5) and inv(14235) = 2 while inv(14253) = 3. Figure 1.26 shows the Hasse diagram for the weak Bruhat order on S3 . We say σ < τ if there is a sequence of permutations {σ (i) }ki=1 such that σ = σ (1) ≺ σ (2) ≺ · · · ≺ σ (k) = τ . That is, we can get to τ from σ by a sequence of adjacent transpositions that increase the number of inversions at each step. Note that when σ < τ in the weak Bruhat order, it means Inv(σ) ⊆ Inv(τ ). When σ and τ are incomparable in the partial order, that is σ 6< τ and τ 6< σ, it means that there exists some inversion in σ that is not an inversion in τ and vice versa.

321 312

231

132

213 123

Figure 1.26: The weak Bruhat order on S3 . The main result of Chapter 6 is the following theorem, which says that generalized Demazure atoms with different basements refine one another if and

40 only if the basement permutations are comparable in the weak Bruhat order. Theorem 13. Given two distinct permutations σ and τ in Sn , σ < τ in weak Bruhat order if and only if for all weak composition shapes γ = (γ1 , γ2 , ..., γn ), X bτ = bσ . there exists a set of shapes Sγ such that E E γ

δ

δ∈Sγ

For example, we can write the Schur function s(2,1) (X3 ) as a sum of Demazure atoms, corresponding to permuted basement fillings with basement 123, or semi-standard augmented fillings. The eight semi-standard augmented fillings shown in Figure 1.27 come from shifting the eight reverse row-strict tableaux of shape (2, 1) to the identity basement. They are organized according to their shape, where the solid rounded boxes group fillings of the same shape with basement b 123 (X3 ) + E b 123 (X3 ) + E b 123 (X3 ) + 123. That is, we can write s(2,1) (X3 ) = E (1,0,2) (0,1,2) (0,2,1) 123 123 123 b b b (X3 ). If σ = 213 and τ = 231, then since all per(X3 ) + E (X3 ) + E E (2,0,1)

(1,2,0)

(2,1,0)

mutations are greater than the identity in weak Bruhat order, Theorem 13 says that there is a way to group the Demazure atoms to form generalized Demazure atoms with basements σ and τ . The dashed lines in Figure 1.27 show which semi-standard augmented fillings shift to PBFs of the same shape with basement σ = 213. That is, b 123 (X3 ) + b 213 (X3 ) = E b 123 (X3 ), E b 213 (X3 ) = E b 123 (X3 ) + E we can write E (2,0,1) (2,0,1) (0,1,2) (1,0,2) (1,0,2) 123 213 123 123 b b b b E (X3 ), and E (X3 ) = E (X3 ) + E (X3 ). When we shift these (2,1,0)

(2,1,0)

(2,1,0)

(1,2,0)

eight semi-standard augmented fillings to basement τ = 231, only two shapes arise. The dark solid lines in Figure 1.27 show which of these diagrams have the b 231 (X3 ) = same shape when shifted to basement 231. This means we can write E

(1,2,0) 123 123 231 123 123 b b b b b b 123 E (1,0,2) (X3 )+E(0,1,2) (X3 ) and E(2,1,0) (X3 ) = E(0,2,1) (X3 )+E(2,0,1) (X3 )+E(1,2,0) (X3 )+ b 123 (X3 ). However, we can also express both E b 231 (X3 ) and E b 231 (X3 ) in E (2,1,0) (1,2,0) (2,1,0)

terms of generalized Demazure atoms with basement σ = 213 because this grouping of semi-standard augmented fillings according to their shape with basement τ = 231 did not separate any of the groups that were made by shifting to basement σ = 213. In other words, the grouping of Demazure atoms into generalized Demazure atoms with basement σ is a refinement of the grouping of these same Demazure atoms into generalized Demazure atoms with basement τ . Theorem 13 says that this happens because σ < τ in the weak Bruhat order. This is visible in

41 the diagram because the dark solid lines collect whole groups of Demazure atoms with basement σ. If σ were not comparable to τ in the weak Bruhat order, the solid dark lines corresponding to basement τ would separate diagrams that had been grouped together by σ.

3 3 1 1 2 3

1 3 1 1 2 3

2 2 3 1 2 3

1 2 3 1 2 3

2 1 3 1 2 3

3

2 1 2 1 2 3

1 1 2 1 2 3

2 3 1 2 3

Figure 1.27: The grouping of Demazure atoms by shape when shifting to basement 213 is a refinement of the grouping when shifting to basement 231. A portion of Chapter 1 has been published in DMTCS Proceedings. LoBue, Janine; Remmel, Jeffrey. “A Murnaghan-Nakayama Rule for Generalized Demazure Atoms”, DMTCS Proceedings (Online), 2013. In addition, a portion of Chapter 1 is currently being prepared for submission for publication. Remmel, Jeffrey; Tiefenbruck, Janine LoBue. The dissertation author is an author of all of this material.

Chapter 2 A New Proof of the Classical Murnaghan-Nakayama Rule

42

43 In this chapter, we shall give a new combinatorial proof of the classical Murnaghan-Nakayama rule from the classical Pieri rules and set up the framework for the coming chapters. This approach to proving the Murnaghan-Nakayama rule is very different from the standard proof of the Murnaghan-Nakayama rule in, say [Sta99] or [Mac95]. The standard proof does not use the combinatorial interpretation of the Schur function sλ as the generating function for columnstrict tableaux of shape λ, but rather an alternative definition as a quotient of determinants. Let ||ai,j ||n denote the n by n matrix whose entry in row i and column j is given by ai,j . Then the Schur function sλ (Xn ) can be defined by λ +n−j

sλ (Xn ) =

det||xi j ||n . n−j det||xi ||n

(2.1)

It is not obvious that this quotient is symmetric, or even a polynomial, but it has been established in several ways that this quotient does indeed equal the Schur function sλ as defined combinatorially through column-strict tableaux [Mac79, Pro89, CRK95]. Note that this definition of sλ only makes sense for finitely many variables, so to use it to define sλ (X), one must take the limit as n approaches infinity. Thus, the standard Murnaghan-Nakayama rule proof actually proves the Murnaghan-Nakayama rule for finitely many variables, then extends the result to infinitely many variables by letting n approach infinity. One benefit to the new proof presented here is that it can directly prove the Murnaghan-Nakayama rule for infinitely many variables, since the Pieri rules hold for infinitely many variables. Additionally, this new proof allows the Murnaghan-Nakayama rule to be extended to various refinements of the Schur functions. In subsequent chapters, we shall see that the same method can be used to derive the Murnaghan-Nakayama rules for generalized Demazure atoms, quasi-Schurs, and row-strict quasi-Schurs from their respective Pieri rules. This was not possible before, as these functions have only combinatorial definitions in terms of fillings of diagrams. Without an analogue of the quotient of determinants, it was not clear how to develop a Murnaghan-Nakayama rule for these refinements. The new proof provides a prototype for developing Murnaghan-Nakayama rules from Pieri rules in a variety of different settings.

44 Let λ be a partition of m and recall that a skew shape λ/µ is a rim hook of λ if dg(λ/µ) contains no 2 × 2 subdiagram and the cells can be ordered so that any two consecutive cells share a common edge. A skew shape λ/µ is a broken rim hook of λ if it is a union of rim hooks. For example, in Figure 2.1, the shaded squares on the left correspond to the rim hook (6, 5, 4, 2, 1, 1, 1)/(4, 3, 3, 2, 1, 1, 1) and the shaded squares on the right come from a union of three rim hooks and so correspond to the broken rim hook which is (6, 5, 3, 3, 3, 2, 2, 1, 1)/(4, 3, 3, 2, 1, 1, 1). We let RH(λ, r) (BRH(λ, r)) denote the set of rim hooks (broken rim hooks) of λ with r cells. Recall that the sign sgn(λ/µ) of a rim hook λ/µ is (−1)row(λ/µ)−1 where row(λ/µ) is the number of rows spanned by dg(λ/µ).

A rim hook

A broken rim hook

Figure 2.1: Examples of rim hooks and broken rim hooks. Our proof of the Murnaghan-Nakayama rule from the Pieri rules proceeds in two steps. First, it is well-known that pr (X) =

r−1 X

(−1)k s(r−k,1k ) (X),

(2.2)

k=0

where (r − k, 1k ) is a partition shape with a single part of size r − k and k parts of size 1, called a hook shape because of the shape of its Ferrers diagram. Indeed there is a simple combinatorial proof of this fact. That is, the right-hand side of (2.2) can be interpreted as the sum of sgn(T )wt(T ) over all column-strict tableaux T of hook shape where if T is a column-strict tableau of shape (r − k, 1k ), then sgn(T ) = (−1)k . For such a T , we let r(T ) be the rightmost element in the first

45 row and c(T ) be the lowest element in the first column. We can define an involution I on this collection of column-strict tableaux by letting I(T ) = T 0 where (i) T 0 is the result of moving r(T ) to the bottom of the first column of T if r(T ) > c(T ) and (ii) T 0 is the result of moving c(T ) to the end of the first row of T if r(T ) ≤ c(T ) and T has more than one row. For example, Figure 2.2 pictures two examples of this involution. If neither (i) or (ii) applies, then we let I(T ) = T . 1

1

2

4

1 I

2

1

2

4

5

1

1

1

2

3

3

5

I 1

1

1

2

2

3

3

2

2

3

3

Figure 2.2: The involution I. It is easy to see that T is a fixed point if and only if T is of shape (r) and all of its entries are equal to some i ∈ P, in which case sgn(T ) = 1 and wt(T ) = xri . It is also easy to see that if I(T ) 6= T , then I(I(T )) = T and sgn(T )wt(T ) = −sgn(I(T ))w(I(T )). Thus, I shows that r−1 X

(−1)k s(r−k,1k ) (X) =

k=0

=

X

sgn(T )wt(T )

I(T )=T ∞ X xri i=1

= pr (X). The first step of our proof is to give a combinatorial description of the coefficients that appear in the Schur function expansion of the product s(r−k,1k ) (X)sµ (X).

46 Since hr−k = s(r−k) , applying the Pieri rule (1.3) for ek gives that ek (X)hr−k (X) = s(r−k,1k ) (X) + s(r−k+1,1k−1 ) (X) so that s(r−k,1k ) (X) = ek (X)hr−k (X) − s(r−k+1,1k−1 ) (X). Iterating this recursion, we see that k X s(r−k,1k ) (X) = (−1)k−i ei (X)hr−i (X).

(2.3)

i=0

Thus, putting together (2.2) and (2.3) means that we want to compute pr (X)sµ (X) = =

r−1 X

k

(−1)

k=0 k r−1 X X

k X

(−1)k−i ei (X)hr−i (X)sµ (X)

i=0

(−1)i ei (X)hr−i (X)sµ (X).

(2.4)

k=0 i=0

By the Pieri rules, ei (X)hr−i (X)sµ (X) =

X

sλ (X)

λ

where the sum is over all partitions λ such that λ comes from µ by first adding a skew row of size r − i on the outside of µ to get some partition shape  and then adding a skew column of size i on the outside of  to get λ. If we put h’s in the cells of /µ to record that they come from a horizontal skew-row and v’s in the cells of λ/ to record that they come from a vertical skew-column, then we would get a diagram D like the one pictured on the left in Figure 2.3. In general, it is easy to see that λ/µ must be a broken rim hook. Moreover, it is easy to see that any cell of λ/µ with a cell of λ/µ directly to its right must be filled with an h and any cell of λ/µ with a cell of λ/µ directly above it must be filled with a v. However, the highest and then rightmost cell of any rim hook comprising the broken rim hook λ/µ, can be filled with either an h or a v. Thus k X (−1)i ei (X)hr−i (X)sµ (X) i=0

47 can be interpreted as the sum of sgn(D)ssh(D) over all diagrams D obtained in this way such that the number of v’s in D is less than or equal to k. Here sh(D) is the shape of D and the sign of D, sgn(D), is equal to (−1)v(D) where v(D) is the number of v’s in D. There is a simple involution J that we can apply to the set of such diagrams. Given any diagram D, let top(D) be the highest and then rightmost cell of D that can be filled with either an h or a v. Then J(D) is defined as follows. (i) If the entry in top(D) is v, then J(D) comes from D by replacing that v by h, and (ii) if the entry in top(D) is h and D has less than k v’s, then J(D) comes from D by replacing that h by v. If neither (i) or (ii) applies, then J(D) = D. For example, if k = 6 and D is the diagram pictured on the left in Figure 2.3, then J(D) is pictured on the right in Figure 2.3. top(D)

h h

h

v

h

v

h

h

h

v

J h

h

h

v h

h

v

v

h

v

Figure 2.3: The involution J. It is easy to see that J is an involution, i.e, J 2 is the identity map. Further, if J(D) 6= D, then sgn(D)ssh(D) = −sgn(J(D))ssh(J(D)) since we change the number of v’s by one in going from D to J(D), but the shape does not change. Finally, note that if D is a fixed point of J, then the element in top(D) must be an h and v(D) = k so that sgn(D) = (−1)k . Thus, we can write k X i=0

(−1)i ei (X)hr−i (X)sµ (X) =

X D

ssh(D) (X)

(2.5)

48 where the sum is over all diagrams D such that (a) the cells filled with h’s form a skew row of shape γ on the outside of µ, where |γ/µ| = r − k, (b) the cells filled with v’s form a skew column of size k on the outside of γ, and (c) the cell top(D) is filled with an h. It follows from (2.4) and (2.5) that we can write pr (X)sµ (X) =

X

sgn(D)ssh(D) (X)

(2.6)

D

where the sum is over all diagrams D such that there is a k with 0 ≤ k < r such that (a) the cells filled with h’s form a skew row of shape γ and size r − k on the outside of µ, (b) the cells filled with v’s form a skew column of size k on the outside of γ, and (c) the cell top(D) is filled with an h. In this situation, the cells containing the v’s and h’s in D will form a broken rim hook. Once again, it will be the case that if a cell of sh(D)/µ has a cell of sh(D)/µ directly to its right, then it must be filled with an h, and if a cell of sh(D)/µ has a cell of sh(D)/µ directly above it, then it must be filled with a v. By rule (c), the cell top(D) must be filled with an h. This implies that if sh(D)/µ is a single rim hook, then the filling of sh(D)/µ is completely forced. Such a filling is pictured in the second row of Figure 2.4. However, if sh(D)/µ is a union of more than one rim hook, then we are free to change the filling of the highest and then rightmost cell of any rim hook except for the rim hook that contains top(D). This leads to our second involution K. That is, if D is a diagram such that (a), (b), and (c) hold and sh(D)/µ is a union of more than one rim hook, we let 2-top(D) be the highest and then rightmost cell that is not contained in the rim hook containing top(D). That is, 2-top(D) is the highest and then rightmost cell whose label is

49 not determined by rules (a) through (c). Let K(D) be the diagram that results by changing the label of 2-top(D) from an h to a v or vice versa. If sh(D)/µ consists of single rim hook, then we let K(D) = D. Figure 2.4 pictures an example of this involution K and a fixed point of K. top(D)

h h

h

h

h

v

h

h

h

v

K h

h

h

v h

v

v 2−top(D)

v

h h

h

v

h

v

h A fixed point of K

v h

h

v

v

Figure 2.4: The involution K. It is easy to see that K is an involution and, if K(D) 6= D, then sgn(D)ssh(D) (X) = −sgn(K(D))ssh(K(D)) (X) since we change the number of v’s by one in going from D to K(D) and leave the shape unchanged. Finally, note that if D is a fixed point of I, then sh(D)/µ is a rim hook and v(D) = row(sh(D)/µ) − 1 so that sgn(D) is the same as the sign of the rim hook sh(D)/µ. Thus K shows that the Murnaghan-Nakayama rule (1.1.1) holds. The content of Chapter 2 is currently being prepared for submission for publication. Remmel, Jeffrey; Tiefenbruck, Janine LoBue. The dissertation author is an author of this material.

Chapter 3 A Murnaghan-Nakayama Rule for Generalized Demazure Atoms

50

51 In this chapter, we shall prove our Murnaghan-Nakayama rule for generalized Demazure atoms, which was given in Theorem 4. This rule gives a combina(r,σ)

torial description of the coefficients dγ,δ in the expansion X (r,σ) bγσ (Xn ) = bδσ (Xn ). pr (Xn )E dγ,δ E

(3.1)

δ

This proof proceeds from the Pieri rules, following the general method of Chapter 2. First we need to define the analogues of the broken rim hook diagrams filled with h’s and v’s that were used in the proof of the classical MurnaghanNakayama rule. Before stating our definitions, we note that the roles of h’s and v’s will be reversed in this setting as compared with the classical setting. The reason for this change comes from how Schur functions are represented as generalized Demazure atoms with basement n . To convert a column-strict tableau to a PBF with basement n , one must complement the entries, rotate the diagram by 90 degrees, and attach a basement, as in Figure 1.11. Since this process involves rotation of the usual Ferrers diagram, in which there are λi cells in the ith row, we read the shapes of PBFs as the heights of the columns from left to right. Thus, the augmented diagram of λ contains λi cells in the ith column. In such a process, skew rows become λ-transposed skew rows and skew columns become λ-transposed skew columns of the Pieri rules for generalized Demazure atoms. However, it is more natural to use h’s to fill shapes which have no two elements in the same column (thus forming a horizontal strip) and v’s to fill shapes which have no two elements in the same row (thus forming a vertical strip). For this reason, we will reverse the roles of h’s and v’s in this chapter as compared with Chapter 2. Let σ ∈ Sn and let γ be a σ-compatible weak composition of m with n parts. In the introduction, we defined Pieri and satisfactory labelings of dg(δ/γ). Recall that a satisfactory labeling must follow these four rules, whereas a Pieri labeling need only follow the first three: (a) an h is assigned to all but the rightmost cell in each row of dg(δ/γ), (b) a v is assigned to any cell above another cell of dg(δ/γ), (c) an h is assigned to any cell that is above a cell of dg(γ) and has a cell of dg(δ/γ) one row below and anywhere to the left, and

52 (d) a v is assigned to the last cell in reading order of dg(δ/γ). We also gave a characterization for when dg(δ/γ) has a satisfactory labeling, which we now prove. Lemma 1. Suppose σ ∈ Sn , γ is a σ-compatible weak composition, and δ is an acceptable shape. Then dg(δ/γ) has a satisfactory labeling if and only if there is no cell (x, y) in dg(δ/γ) such that (x + j, y) and (x, y − 1) are both cells in dg(δ/γ) for some j > 0. In other words, dg(δ/γ) can be labeled in accordance with rules (a) through (d) if and only if it avoids the configuration

.

Proof. Suppose dg(δ/γ) has a cell (x, y) in dg(δ/γ) such that (x+j, y) and (x, y−1) are both cells in dg(δ/γ) for some j > 0. Then rule (a) says that an h must be assigned to cell (x, y) because it is not the rightmost cell in its row. Also, rule (b) says a v must be assigned to cell (x, y) because it is above another cell of the diagram. Since both of these rules cannot be satisfied simultaneously, it is not possible to label dg(δ/γ) in a way that follows rules (a) through (d). Now suppose dg(δ/γ) has no satisfactory labeling. This means that there is some cell (x, y) for which rules (a)-(d) disagree on how to label (x, y). Rules (a) and (c) cannot disagree on how to label a cell, since they both assign an h label. Similarly, rules (b) and (d) must always agree. Rule (c) can never conflict with rule (d) because rule (c) deals with cells above the lowest row of dg(δ/γ) and rule (d) handles assigning labels to cells in the lowest row only. Rules (b) and (c) can also never apply to the same cell, because rule (b) involves cells that are positioned above another cell of dg(δ/γ), whereas rule (c) involves only cells positioned above cells of dg(γ). Rules (a) and (d) can also never conflict, because rule (d) only applies to the last cell in reading order, and rule (a) applies to cells that have another cell to their right, or cells that are not the last cell in reading order. Rules (a) and (b), however, conflict exactly when there is a cell of dg(δ/γ) with another cell immediately below it and another cell somewhere to its right. Therefore, if rules (a)-(d) disagree on how to label cell (x, y), it means that there is some j > 0 such that (x + j, y) and (x, y − 1) are both cells in dg(δ/γ). To prove Theorem 4, we note that by equations (2.2) and (2.3) from Chapter

53 2, bγσ (Xn ) = pr (Xn )E =

=

r−1 X k=0 r−1 X

bγσ (Xn ) (−1)k s(r−k,1k ) (Xn )E k

(−1)

k=0 r−1 X k X

k X

b σ (Xn ) (−1)k−i hr−i (Xn )ei (Xn )E γ

i=0

b σ (Xn ). (−1)i hr−i (Xn )ei (Xn )E γ

(3.2)

k=0 i=0

By the Pieri rule for ei , bγσ (Xn ) = ei (Xn )E

X

bνσ (Xn ), E

ν/γ∈T SC(i)

where the sum is over all shapes ν such that ν/γ is a γ-transposed skew column of size i. Label each cell in dg(ν/γ) with an h to record that it is part of a transposed skew column, or horizontal strip. Now, by the Pieri rule for hr−i , X b σ (Xn ) = hr−i (Xn ) b σ (Xn ) hr−i (Xn )ei (Xn )E E γ ν ν/γ∈T SC(i)

=

X

X

b σ (Xn ), E δ

ν/γ∈T SC(i) δ/ν∈T SR(r−i)

where the inner sum is over all shapes δ such that δ/ν is a ν-transposed skew row of size r − i. Label each cell in dg(δ/ν) with a v to record that it is part of a transposed skew row, or vertical strip. Thus bγσ (Xn ) hr−i (Xn )ei (Xn )E b σ (Xn ) over all labeled diagrams D of shape δ can be interpreted as the sum of E δ that arise from dg(γ) by placing a transposed skew column of size i marked with h’s, followed by a transposed skew row of size r − i marked with v’s. Our next result says that such labeled diagrams are equivalently character(i,r−i)

ized by our rules for Pieri labelings. Let Pδ/γ

be the set of diagrams of shape δ

with a Pieri labeling of dg(δ/γ) with i cells labeled h and r − i cells labeled v. Lemma 3. For any fixed r, if 0 ≤ k ≤ r − 1 and 0 ≤ i ≤ k, then X X b σ (Xn ) = b σ (Xn ). hr−i (Xn )ei (Xn )E E γ δ δ:|δ/γ|=r

(i,r−i) D∈Pδ/γ

(3.3)

54 Proof. We have seen that b σ (Xn ) hr−i (Xn )ei (Xn )E γ b σ (Xn ) over all labeled diagrams D of shape δ can be interpreted as the sum of E δ that arise from dg(γ) by placing a transposed skew column of size i marked with h’s, followed by a transposed skew row of size r − i marked with v’s. Thus, if we (i,r−i)

let Dδ/γ

be the set of diagrams that arise in this way, we have bγσ (Xn ) = hr−i (Xn )ei (Xn )E

X

X

bδσ (Xn ). E

δ:|δ/γ|=r D∈D(i,r−i) δ/γ

(i,r−i)

We claim that Dδ/γ

(i,r−i)

= Pδ/γ (i,r−i)

Suppose D ∈ Dδ/γ

.

. We must show that D’s labeling, say L, is a Pieri (i,r−i)

labeling of dg(δ/γ) with i cells labeled h, and thus D ∈ Pδ/γ

. To do this, we

must show that δ is an acceptable shape and L satisfies rules (a), (b), and (c). By the definition of γ-transposed skew column and γ-transposed skew row, if δ comes from adding a transposed skew column of size i followed by a transposed skew row of size r − i to γ a σ-compatible weak composition of m, then δ is a σ-compatible weak composition of m + r with dg(γ) ⊆ dg(δ). Further, let c1 = (x1 , y1 ), . . . , ci = (xi , yi ) be the cells labeled h listed in reading order. Let ci+1 = (xi+1 , yi+1 ), . . . , cr = (xr , yr ) be the cells labeled v listed in reverse reading order. Let dg(δ (i) ) be the diagram of γ plus the cells c1 , c2 , . . . , ci . If it were the case that we could find 1 ≤ j < k ≤ n with γj < γk = δk ≤ δj , as in Figure 3.1, then the cell at position (j, γk ) would be cell c` for some 1 ≤ ` ≤ r. Since c` is not removable from δ (`) , this contradicts the definition of γ-transposed skew column if ` ≤ i, or γ-transposed skew row if ` > i. So no such j, k exists, and δ is an acceptable shape. Now we show L satisfies rules (a) through (c). It may be helpful to refer to Figure 3.2. Suppose that (x, y) ∈ dg(δ/γ) and there is a cell to its right, (z, y) with z > x, which is also a cell in dg(δ/γ). If L labels (x, y) with a v and (z, y) with a v, this contradicts the fact that the cells labeled v form a transposed skew row, with no two cells in the same row. If L labels (x, y) with a v and (z, y) with an h, then the cell (x, y) = cj for some i + 1 ≤ j ≤ r. In the diagram of shape δ (j) , there

55

δj γk = δ k cl γj j

k

Figure 3.1: Proof that δ is an acceptable shape. must be no cell in position (z, y + 1). If there were, it could not be labeled with an h, since the h’s form a transposed skew column with no two cells in the same column, and it could not be labeled with a v since the cells labeled v are assigned in reverse reading order, so cj is the highest cell labeled v in δ (j) . Thus, the height of column z in δ (j) equals the height of column x. This means cell cj = (x, y) is not removable, which is a contradiction. Thus, L does not label (x, y) with a v and (z, y) with an h. Thus, it must be the case that any cell (x, y) to the left of another cell of dg(δ/γ) is assigned an h by the labeling L, so L satisfies rule (a).

y+1 y y-1

cj cl

u

x

z

Figure 3.2: Proof that L satisfies rules (a) through (c). Now suppose that (x, y) ∈ dg(δ/γ) and the cell below it, (x, y − 1), is also a cell in dg(δ/γ). If L labels (x, y) with an h and (x, y −1) with an h, this contradicts the fact that the cells labeled h form a transposed skew column with no two cells in the same column. If L labels (x, y) with an h and (x, y − 1) with an v, then the cell (x, y) = cj for some i + 1 ≤ j ≤ r. Then δ (j) includes (x, y) but not the cell below it, so δ (j) is not a weak composition shape. Thus, it must be the case that any cell (x, y) above another cell of dg(δ/γ) is assigned a v by the labeling L, so L satisfies rules (b).

56 Now suppose that (x, y) ∈ dg(δ/γ), the cell below it, (x, y − 1) is in dg(γ), and there exists some u < x for which (u, y − 1) ∈ dg(δ/γ). That is, suppose (x, y) is a cell to which rule (c) would apply. In the labeling L, (u, y − 1) = c` for some 1 ≤ ` ≤ r and (x, y) = cj for some 1 ≤ j ≤ r. If L assigns (x, y) a v, then since the h’s are labeled first in reading order, then the v’s in reverse order, it must be the case that ` < j. Then, c` is not a removable cell in δ (`) since column x is to the right of column u and both columns are of height y − 1 in δ (`) . Thus, (x, y) must be labeled h, so L satisfies rule (c). We have shown that L is a Pieri labeling. (i,r−i)

Therefore, Dδ/γ

(i,r−i)

⊆ Pδ/γ

.

Now, suppose P is a Pieri-labeled diagram of shape δ with i cells labeled h and r − i cells labeled v. Then δ is an acceptable shape and the labeling of P satisfies (a) through (c). We must show that the cells labeled h in P form a γ-transposed skew column and that the cells labeled v in P form a ν-transposed skew row, where ν is the shape formed by the cells of γ plus the cells labeled h. Consider the shape ν/γ formed by the cells labeled h in P . Call these cells c1 = (x1 , y1 ), . . . , ci = (xi , yi ), listed in reading order. Next consider the shape δ/ν formed by the cells labeled v in P . Call these cells ci+1 = (xi+1 , yi+1 ), . . . , cr = (xr , yr ), listed in reverse reading order. Let dg(δ (j) ) be the diagram of γ plus the cells c1 , c2 , . . . , cj . It is clear that dg(γ) ⊆ dg(δ (1) ) ⊆ · · · ⊆ dg(δ (r) ) because we are simply adding one cell each time. Since P ’s labeling satisfies rule (a), we know there is at most one cell labeled v in each row. This means the cells ci+1 , ci+2 , . . . , cr all occur in different rows, and since we labeled them in reverse reading order, this means yi+1 < yi+2 < · · · < yr . Thus, in order to establish that ν/γ is a γ-transposed skew column of size i and δ/ν is a ν-transposed skew row of size r − i, it remains to show that for each j = 1, 2, . . . , r, δ (j) is a σ-compatible weak composition and cj is a removable cell from δ (j) . To show that each δ (j) is a σ-compatible weak composition, we will prove a more general result. That is, suppose we have a σ-compatible weak composition δ (t) and we remove a cell ct = (xt , yt ) which is at the top of its column in δ (t) and the rightmost cell of dg(δ (t) /γ) in row yt . We will show that the shape that remains, δ (t−1) , is still a σ-compatible weak composition. First, note that since

57 we are removing a cell at the top of a column, and δ (t) is a weak composition shape, certainly δ (t−1) will be a weak composition shape. From the definition of σ-compatible, the only way that removing cell ct = (xt , yt ) creates a shape that is (t)

(t)

not σ-compatible is if there exists xt < j ≤ n such that σj < σxt and δxt = δj . Suppose for a contradiction that such a j exists, as in Figure 3.3. Then (j, yt ) is a cell in dg(δ (t) ). It cannot be that (j, yt ) ∈ dg(δ (t) /γ) because we assume that ct is the rightmost cell of dg(δ (t) /γ) in row yt . This means (j, yt ) ∈ dg(γ) so γj ≥ yt . Since ct = (xt , yt ) 6∈ dg(γ), we also have γxt < yt . Since γ is σ-compatible, the only way we can have j > xt and γxt < γj is if σxt < σj , which contradicts our assumption. Thus, δ (t−1) is a σ-compatible weak composition.

yt

cj

xt

xt

>

j

j

Figure 3.3: Proof of σ-compatability. Now, we use the more general result to show that each δ (j) is a σ-compatible weak composition. To start, we know that δ = δ (r) is an acceptable shape and thus a σ-compatible weak composition. We can apply the result sequentially to remove the cells cj for j = r, r −1, . . . , i+1, each of which is the rightmost cell of dg(δ (j) /γ) in its row, and at the top of a column. This is true because these were the cells labeled v in P and rule (a) says v labels can only occur for the rightmost cells in each row of dg(δ/γ). This means cr , . . . , ci+1 are all removable cells and the result of the previous paragraph says that δ (r) , δ (r−1) , . . . , δ (i+1) are all σ-compatible weak compositions. Next, we remove the cells cj for j = i, i − 1, . . . , 1. Since these cells were all labeled h in P and we enumerated them in reading order, each cell cj is the rightmost cell of dg(δ (j) /γ) in its row, and at the top of a column. This means that ci , . . . , c1 are all removable cells, and we can use the above result to conclude that δ (i) , δ (i−1) , . . . , δ (1) are all σ-compatible weak compositions, which is what we

58 wanted to show. (i,r−i)

(i,r−i)

So any diagram P ∈ Pδ/γ (i,r−i)

Dδ/γ

(i,r−i)

. Thus Dδ/γ

(i,r−i)

= Pδ/γ

is also in Dδ/γ

(i,r−i)

, which means Pδ/γ

⊆

, which proves the lemma.

b σ (Xn ) as a Lemma 3 and equation (3.2) give a way to express pr (Xn )E γ signed sum over Pieri-labeled diagrams, b σ (Xn ) = pr (Xn )E γ

r−1 X k X

(−1)i

k=0 i=0

X

X

δ:|δ/γ|=r

(i,r−i) D∈Pδ/γ

b σ (Xn ). E δ

(3.4)

Next, for each value of k, we can perform an involution on the set of diagrams of shape δ with Pieri labelings having at most k h’s. This will leave only the diagrams (k,r−k)

with satisfactory labelings with exactly k h’s that also satisfy rule (d). Let Sδ/γ

be the set of labeled diagrams of shape δ with a satisfactory labeling of dg(δ/γ) that has exactly k h’s and r − k v’s. Lemma 4. Fix r and fix k, 0 ≤ k ≤ r − 1. Then k X X (−1)i i=0

X

X

b σ (Xn ) = (−1)k E δ

δ:|δ/γ|=r D∈P(i,r−i)

X

b σ (Xn ) E δ

(3.5)

δ:|δ/γ|=r D∈S(k,r−k)

δ/γ

δ/γ

Proof. Let Pk (δ/γ) denote the set of all diagrams of shape δ with a Pieri labeling of dg(δ/γ) where the number of cells labeled h is at most k. That is, Pk (δ/γ) = (i,r−i)

∪ki=0 Pδ/γ

. (i,r−i)

Suppose D ∈ Pδ/γ

for some i between 0 and k. Define the sign of D by

sgn(D) = (−1)i = (−1)h(D) , where i = h(D) is the number of cells labeled h in D. b σ (Xn ). Thus, Define the weight of D by wt(D) = E δ

k X

(−1)i

i=0

X

X

δ:|δ/γ|=r D∈P(i,r−i) δ/γ

b σ (Xn ) = E δ

X

X

sgn(D)wt(D).

δ:|δ/γ|=r D∈Pk (δ/γ)

We define a weight-preserving sign-reversing involution J : Pk (δ/γ) → Pk (δ/γ) as follows. If D ∈ Pk (δ/γ), let last(D) denote the last cell of D in reading order. Then (i) if L(last(D)) = v and D has less than k h’s, we let J(D) be the diagram that results from D by replacing the v in cell last(D) by h, and

59 (ii) if L(last(D)) = h, then we let J(D) be the diagram that results from D by replacing the h in cell last(D) by v. Note that the last cell in reading order is never determined by rules (a) through (c). Thus in either case, the labeling of J(D) is a Pieri labeling, and the number of cells labeled h cannot exceed k so that J(D) ∈ Pk (δ/γ). If neither (i) or (ii) apply, then we let J(D) = D. For an example of the involution in the case where r = 6 and k = 5, see Figure 3.4. h

h

h v h v 6 2 1 4 3 5

J

h

h

h v h 6 2 1 4 3 5 h

Figure 3.4: The involution J, when r = 6, k = 5. We claim that J is a weight-preserving sign-reversing involution. There are two cases to consider. Case 1. The last cell is labeled h in D. In this case, 1 ≤ i ≤ k and J(D) has the same labeling as D except the last cell has a v. Thus, J(D) has i − 1 h’s, where 0 ≤ i − 1 ≤ k − 1. Then J(J(D)) = D because the number of h’s in J(D) is strictly less than k, so the last cell changes labels back to an h. Case 2. The last cell is labeled v in D. If 0 ≤ i ≤ k − 1, then J(D) has the same labeling as D except the last cell has an h. Thus, J(D) has i + 1 h’s, where 1 ≤ i + 1 ≤ k. Then J(J(D)) = D because the last cell is labeled h in J(D), so even if i + 1 = k, J will change the label of the last cell back to a v. Otherwise, i = k and J(D) has exactly the same labeling as D. Again, J(D) has i = k h’s and the last cell is a v, so applying J again leaves the labeling unchanged. In this case too, J(J(D)) = D. b σ (Xn ). The involution J is weight-preserving because wt(J(D)) = wt(D) = E δ It is sign-reversing because the number of h’s in D and J(D) differs by one, provided J changes the label of the last cell in reading order. The fixed points of the involution are the diagrams D for which neither (i) nor (ii) applies. That is,

60 the fixed points of J are the D ∈ Pk (δ/γ) with exactly k h’s and the last cell in reading ordered labeled with a v. These are precisely the diagrams of shape δ with satisfactory labelings of dg(δ/γ) that have exactly k h’s and r − k v’s. This set of (k,r−k)

diagrams is what we called Sδ/γ

(k,r−k)

. The sign of all such diagrams D ∈ Sδ/γ

is

(−1)k . Thus we have shown that

X

X

X

sgn(D)wt(D) = (−1)k

X

bδσ (Xn ). E

δ:|δ/γ|=r D∈S(k,r−k)

δ:|δ/γ|=r D∈Pk (δ/γ)

δ/γ

which is what we wanted to prove. We have from equation (3.4) and Lemma 4 that b σ (Xn ) = pr (Xn )E γ

k r−1 X X

X

(−1)i

k=0 i=0

X

b σ (Xn ) E δ

δ:|δ/γ|=r D∈P(i,r−i) δ/γ

=

r−1 X

(−1)k

k=0

X

X

δ:|δ/γ|=r

(k,r−k) D∈Sδ/γ

b σ (Xn ). E δ

(3.6)

Finally, we can apply another involution to prove Theorem 4. b σ in terms Proof. Proof of Theorem 4. We have from (3.6) an expression for pr E γ σ b of Eδ ’s. Suppose δ is an acceptable shape that avoids . By Lemma 1, dg(δ/γ) b σ appears on the right hand side of (3.6). has a satisfactory labeling so that E δ

Let Sr (δ/γ) denote the set of all diagrams of shape δ with a satisfactory labeling of dg(δ/γ) in which the number of cells labeled h is less than r. That is, (k,r−k)

Sr (δ/γ) = ∪r−1 k=0 Sδ/γ

. Note that every diagram with a satisfactory labeling is in

this set, since in all satisfactory labelings, the last cell in reading order must be labeled v, and there are a total of r labeled cells, so there must be strictly less than r cells labeled h. We will define an involution K : Sr (δ/γ) → Sr (δ/γ) to cancel diagrams with the same shape and different satisfactory labelings. Thus, we will show that the fixed points of this involution are the diagrams of shape δ for which dg(δ/γ) (r)

has a unique satisfactory labeling. The set of such shapes δ we called Uγ,σ .

61 Suppose D ∈ Sr (δ/γ). Define the sign of D by sgn(D) = (−1)h(D) , where h(D) is the number of cells labeled h in D. To define K(D), consider the first cell c in reading order of dg(δ/γ) whose label is not forced to be a v or an h by rules (a) through (d). Then we let K(D) be the diagram of the same shape that results from D by changing the entry in cell c from a v to an h or vice versa. Note K(D)’s labeling is still a satisfactory labeling because we have not changed the label of any cell to which rules (a) through (d) apply. The number of h’s in K(D) is either one fewer or one more than the number of h’s in D so that the sign of K(D) is the opposite of the sign of D. K is an involution because K applied to K(D) will change the label of the same cell c. The fixed points of K are the diagrams where every cell’s labeling is forced (r)

by rules (a) through (d), i.e., δ must be in Uγ,σ . Thus K shows that we can replace the sum on the right-hand side of (3.6) by X sgn(δ/γ)Eˆδσ (Xn ), (r)

δ∈Uγ,σ

which is what we wanted to prove. We conclude by noting that in the special case where σ = n and γ is bγσ = sµ (Xn ). In actually a partition shape corresponding to a partition µ, then E this case, we claim that our Murnaghan-Nakayama rule for generalized Demazure atoms reduces to the classical Murnaghan-Nakayama rule (1.1.1). First, if λ is a shape that appears on the right hand side of (1.1.1), then it also appears on the right hand side of (1.24) with the same coefficient. If dg(λ/γ) is a rim hook, then every cell of dg(λ/γ) either has another cell of dg(λ/γ) to its right or below it, with the exception of the last cell in reading order. This means that rule (d) labels the last cell in reading order, and rules (a) and (b) label all the other cells of dg(λ/γ) in a unique way, as in Figure 3.5. If this labeling has k cells labeled h, then this sλ appears as a term in the sum with coefficient (−1)k because λ will have k + 1 rows. Now, if λ does not appear on the right-hand side of (1.1.1), then its corresponding PBF shape λ will not appear on the right-hand side of (1.24). If dg(λ/γ) is not a connected skew shape, then consider the first connected component of

62 hhv v v hv 7 6 5 4 3 2 1

Figure 3.5: A rim hook and its satisfactory labeling dg(λ/γ) in reading order. The last cell in reading order of this component will not have its label forced by rules (a) through (d). It is neither above nor to the left of another cell. It is not the last cell in reading order of dg(λ/γ) because there is another component following this one, and the situation described by rule (c) does not apply to partition shapes. Thus, there is not a unique satisfactory labeling of dg(λ/γ). Similarly, if dg(λ/γ) contains a 2 × 2 square, Lemma 1 says that dg(λ/γ) has no satisfactory labeling. Thus the shapes λ for which dg(λ/γ) has a unique satisfactory labeling are exactly the shapes λ such that dg(λ/µ) is a rim hook. An abbreviated version of Chapter 3 also appeared in DMTCS Proceedings. LoBue, Janine; Remmel, Jeffrey. “A Murnaghan-Nakayama Rule for Generalized Demazure Atoms”, DMTCS Proceedings (Online), 2013. The dissertation author is an author of this paper. The complete version of Chapter 3 is currently being prepared for submission for publication. Remmel, Jeffrey; Tiefenbruck, Janine LoBue. The dissertation author is an author of this material.

Chapter 4 Murnaghan-Nakayama Rules for Quasisymmetric Schur Functions and Row-Strict Quasisymmetric Schur Functions

63

64 In this chapter, we shall prove our Murnaghan-Nakayama rules for quasiSchurs and row-strict quasi-Schurs, which were given in Theorems 5 and 6. These (r)

(r)

rules give combinatorial descriptions of the coefficients uα,β and vα,β in the expansions (1.22) and (1.23), pr (X)Sα (X) =

X

(r)

uα,β Sβ (X)

β

and pr (X)RS α (X) =

X

(r)

vα,β RS β (X).

β

4.1

Murnaghan-Nakayama Rule for Quasisymmetric Schur Functions This proof proceeds from the Pieri rules, following the method of Chapter

2. To begin, we use the symmetric function identity (2.2), pr (X) =

r−1 X

(−1)k s(r−k,1k ) (X).

(4.1)

(−1)k s(r−k,1k ) (X)Sα (X).

(4.2)

k=0

This allows us to write pr (X)Sα (X) =

r−1 X k=0

The symmetric function identity (2.3) allows us to replace the hook Schur function s(r−k,1k ) above to obtain an expression involving the homogeneous and elementary symmetric functions, from which we can apply the Pieri rules. Making this replacement gives r−1 k X X k pr (X)Sα (X) = (−1) (−1)k−i hr−i (X)ei (X)Sα (X)

=

k=0 r−1 X k X

i=0

(−1)i hr−i (X)ei (X)Sα (X).

(4.3)

k=0 i=0

Since we will be using the Pieri rules to multiply Sα (X) by some elementary symmetric function ei (X) followed by some homogeneous symmetric function

65 hr−i (X), let us consider the type of diagrams that arise from such a sequence of multiplications. Suppose, for example, that we wish to compute the product p2 (X)S(2,1) (X) using equation (4.3). As part of our calculation, when k = i = 1, we need to find the product h1 (X)e1 (X)S(2,1) (X). To multiply S(2,1) (X) by e1 (X), we must first add a single cell to dg(2, 1) to form a (2, 1)-row. If we color the cells of dg(2, 1) gray and place one white cell, marked with an h, to form a (2, 1)-row, the resulting colored diagrams are shown in Figure 4.1.

h h

h

Figure 4.1: The diagrams that result from the multiplication e1 (X)S(2,1) (X). Note that the two diagrams shown in Figure 4.2 will not appear, as the cell labeled h is not a removable cell.

h

h

Figure 4.2: The cell labeled h is not removable. Next, to multiply by h1 (X), we must add a white cell labeled with a v to one of these three valid diagrams so that the cells labeled v form an -column, where  = (3, 1), (2, 1, 1), or (2, 2). This results in a collection of twelve diagrams, which are shown on the right side of Figure 4.3, labeled G through R. The left side of this same figure depicts the diagrams that result from the multiplication h2 (X)e0 (X)S(2,1) (X) The type of diagrams that result from multiplying a quasisymmetric Schur function by an elementary symmetric function and then a homogeneous symmetric function are what we will call colored labeled diagrams. A colored labeled diagram around α is a diagram of a composition shape β such that 1 |α| cells are gray, 2 |β| − |α| cells are white,

66

p2S(2,1)(X) h2e0S(2,1)(X) v v

A:

B:

v v

D:

v

E: F:

hv

M:

H:

v h

N: v

h

O:

J:

v

v h

v v

v

v

h

G:

I:

v v

C:

h1e1S(2,1)(X)

K: L:

v

P:

v h

Q:

h

R:

v

v h

v

h

v h

h

v

h v

Figure 4.3: The colored labeled diagrams used to compute p2 (X)S(2,1) (X). 3 the gray cells form the diagram of a weak composition shape γ such that α(γ) = α, 4 all white cells are labeled with a v or an h, 5 the cells labeled h form an α-row, and 6 the cells labeled v form an -column, where  is the composition shape formed by the gray cells together with the white cells labeled h. As in the example for h1 (X)e1 (X)S(2,1) (X), the same shape β may appear multiple times under different colored labelings. See diagrams I and M. When we apply the Pieri rules in succession, as we have done in the above example, we see that we can express the product of a quasisymmetric Schur function, an elementary symmetric function, and a homogeneous symmetric function as a sum over colored labeled diagrams. Let D(i, j, α) be the set of all colored labeled diagrams around α with i cells labeled h and j cells labeled v. If D is a colored labeled diagram in D(i, j, α), let sh(D) denote the shape of the diagram,

67 which we called β above. Then by the Pieri rules, X

hj (X)ei (X)Sα (X) =

Ssh(D) (X).

(4.4)

D∈D(i,j,α)

Figure 4.3 shows the diagrams in D(0, 2, (2, 1)) on the left and D(1, 1, (2, 1)) on the right. To prove the Murnaghan-Nakayama rule for quasisymmetric Schur functions, we will first show that the colored labeled diagrams that arise from the Pieri rules can be characterized in another way, as diagrams with acceptable colorings and Pieri labelings. Then, we will use a sequence of involutions to cancel the contributions from many of these diagrams, so that our Murnaghan-Nakayama rule sums over only the much smaller fixed point set. First, we prove a lemma that gives a characterization for when dgC (β/α) has a satisfactory labeling. Lemma 2. Let α be a composition of m, and let β be an acceptable shape with acceptable coloring C. Then dgC (β/α) has a satisfactory labeling if and only if there is no cell (x, y) in dgC (β/α) such that (x + j, y) and (x, y − 1) are both cells in dgC (β/α) for some j > 0. In other words, dgC (β/α) can be labeled in accordance with rules (a) through (d) if and only if it avoids the configuration

.

Proof. Suppose dgC (β/α) has a white cell (x, y) such that (x + j, y) and (x, y − 1) are both white cells, for some j > 0. Then rule (a) says that an h must be assigned to cell (x, y) because it is not the rightmost cell in its row. Also, rule (b) says a v must be assigned to cell (x, y) because it is above another white cell. Since both of these rules cannot be satisfied simultaneously, it is not possible to label dgC (β/α) in a way that follows rules (a) through (d). Now suppose dgC (β/α) has no satisfactory labeling. This means that there is some cell (x, y) for which rules (a)-(d) disagree on how to label (x, y). Rules (a) and (c) cannot disagree on how to label a cell, since they both assign an h label. Similarly, rules (b) and (d) must always agree. Rule (c) can never conflict with rule (d) because rule (c) deals with cells for which there is a white cell in a lower row and rule (d) assigns labels to cells in the lowest row only. Rules (b) and

68 (c) can also never apply to the same cell, because rule (b) involves cells that are positioned above another white cell, whereas rule (c) involves only cells positioned above gray cells. Rules (a) and (d) can also never conflict, because rule (d) only applies to the last white cell in reading order, and rule (a) applies to cells that have another white cell to their right, or cells that are not the last cell in reading order. Rules (a) and (b), however, conflict exactly when there is a white cell with another white cell immediately below it and another white cell somewhere to its right. Therefore, if rules (a)-(d) disagree on how to label cell (x, y), it means that there is some j > 0 such that (x+j, y) and (x, y−1) are both cells in dgC (β/α). The next lemma establishes the relationship between the colored labeled diagrams that result from applying the Pieri rules and the labelings we called Pieri labelings. Consider the set of all diagrams of shape β with coloring C and labeling L, such that C is an acceptable coloring of β and L is a Pieri labeling of dgC (β/α) (i,r−i)

with i cells labeled h and r − i cells labeled v. Denote this set Pα,β

.

Lemma 5. Fix r, k, and i with, 0 ≤ k ≤ r − 1 and 0 ≤ i ≤ k. Then X

hr−i (X)ei (X)Sα (X) =

X

Sβ (X).

(4.5)

β:|β|−|α|=r P ∈P(i,r−i) α,β

Proof. Letting j = r − i in (4.4), we have shown that if we use the Pieri rules to multiply Sα (X) first by ei (X) and then by hr−i (X), then hr−i (X)ei (X)Sα (X) =

X

Ssh(D) (X),

(4.6)

D∈D(i,r−i,α)

where D(i, r − i, α) is the set of all colored labeled diagrams around α with i cells (i,r−i)

labeled h and r − i cells labeled v. If we let Dα,β

= {D ∈ D(i, r − i, α) : sh(D) =

β}, then we can collect terms corresponding to the same shape β and write hr−i (X)ei (X)Sα (X) =

X

X

Sβ (X).

(4.7)

β:|β|−|α|=r D∈D(i,r−i) α,β

(i,r−i)

We claim that Dα,β

(i,r−i)

and Pα,β

are actually the same sets. (i,r−i)

Given any colored labeled diagram around α, D ∈ Dα,β

, we show that

its coloring C is an acceptable coloring, and that its labeling L is a Pieri labeling

69 (i,r−i)

of dgC (β/α), and thus D ∈ Pα,β

. First, let us consider the coloring C of the

colored labeled diagram D. This coloring clearly meets the first three conditions of an acceptable coloring. It remains to show that there is no j < k such that column j has a white cell in row l and column k has a gray cell in row l with no cell above it. Suppose for a contradiction that in the diagram D, there is such a pair j, k. Let c1 = (x1 , y1 ), . . . , ci = (xi , yi ) be the cells of D labeled h listed in reading order. Let ci+1 = (xi+1 , yi+1 ), . . . , cr = (xr , yr ) be the cells of D labeled v listed in reverse reading order. Let dg(β (i,r−i) ) be the diagram of α plus the cells c1 , c2 , . . . , ci . Then for i = 1, 2, . . . , r, since the white cells of D are all either part of a row or a column, each ci is a removable cell from dg(β (i) ). Since the cell in row l and column j is white, it must be labeled in D with an h or a v, and so it is cell cb for some 1 ≤ b ≤ r, as in Figure 4.4. We know that dg(β (b) ) contains the cell (k, l) but not the cell (k, l + 1). This means that cell cb is not a removable cell from dg(β (b) ), since it is not on the rightmost column of its height. This is a contradiction, and so we can conclude that the coloring C of D is an acceptable coloring.

l+1 l cb j

k

Figure 4.4: Proof that C is an acceptable coloring. Next we show that the labeling L of D is a Pieri labeling. Let each ci and β

(i)

be as defined above. We must show that L satisfies rules (a), (b), and (c). It

may be helpful to refer to Figure 4.5. Suppose that (x, y) is a white cell in D and there is a cell to its right, (z, y) for z > x, which is also a white cell in D. If L labels (x, y) with a v and (z, y) with a v, this contradicts the fact that the cells labeled v form an α-column, with no two cells in the same row. If L labels (x, y) with a v and (z, y) with an h, then the cell (x, y) = cj for some i + 1 ≤ j ≤ r. In the diagram of shape β (j) , there must be no cell in position (z, y + 1). If there were, it could not be labeled with an h,

70

y+1 y y-1

cj u

x

z

Figure 4.5: Proof that L is a Pieri labeling. since the h’s form an α-row, and it could not be labeled with a v since the cells labeled v are assigned in reverse reading order, so cj is the highest cell labeled v in β (j) . Thus, the height of column z in β (j) equals the height of column x. This means cell cj = (x, y) is not removable, which is a contradiction. Thus, L does not label (x, y) with a v and (z, y) with an h. Thus, it must be the case that any white cell (x, y) that is not the rightmost white cell in its row is assigned an h by the labeling L, so L satisfies rule (a). Now suppose that (x, y) is a white cell in D and the cell below it, (x, y − 1), is also a white cell in D. If L labels (x, y) with an h and (x, y − 1) with an h, this contradicts the fact that the cells labeled h form an α-row, with no two cells in the same column. If L labels (x, y) with an h and (x, y − 1) with an v, then (x, y) = cj for some 1 ≤ j ≤ i and (x, y − 1) = ck for some i + 1 ≤ k ≤ r. Then β (j) includes (x, y) but not the cell below it, so β (j) is not a composition shape. Thus, it must be the case that any white cell (x, y) above another white cell is assigned a v by the labeling L, so L satisfies rules (b). Now suppose that (x, y) is a white cell in D, the cell below it, (x, y − 1), is a gray cell in D, and there exists some u < x for which (u, y − 1) is also white. That is, suppose (x, y) is a cell to which rule (c) would apply. In the labeling L, (u, y − 1) = c` for some 1 ≤ ` ≤ r and (x, y) = cj for some 1 ≤ j ≤ r. If L assigns (x, y) a v, then since the h’s are labeled first in reading order then the v’s in reverse order, it must be the case that j > `. Then, c` is not a removable cell in β (`) since column x is to the right of column u and both columns are of height y − 1 in β (`) . Thus, (x, y) must be labeled h, so L satisfies rule (c). We have shown that L is a (i,r−i)

Pieri labeling. This proves that Dα,β

(i,r−i)

⊆ Pα,β

.

Now, suppose that P is a diagram of shape β with coloring C and labeling

71 L, such that C is an acceptable coloring of β and L is a Pieri labeling of dgC (β/α) (i,r−i)

with i cells labeled h, so that P ∈ Pα,β (i,r−i)

labeled diagram and so P ∈ Dα,β

. We must show that P is a colored

. Conditions 1 through 4 in the definition of

a colored labeled diagram are immediate from the fact that C is an acceptable coloring and L labels the white cells of C. It remains to show that the cells labeled h by L form an α-row and the cells labeled v by L form an -column, where  is the composition shape formed by the gray cells together with the white cells labeled h. Consider the cells labeled h by L. Call these cells c1 = (x1 , y1 ), . . . , ci = (xi , yi ), listed in reading order. Next consider the cells labeled v by L. Call these cells ci+1 = (xi+1 , yi+1 ), . . . , cr = (xr , yr ), listed in reverse reading order. For j = 1, 2, . . . , r, let dg(β (j) ) be the diagram of α plus the cells c1 , c2 , . . . , cj , where we collapse any columns of height zero. It is clear that dg(β) @ dg(β (1) ) @ · · · @ dg(β (r) ) because we are simply adding one cell each time, always to the top of some column. Since L satisfies rule (a), we know there is at most one cell labeled v in each row. This means the cells ci+1 , ci+2 , . . . , cr all occur in different rows, and since we labeled them in reverse reading order, this means yi+1 < yi+2 < · · · < yr . Also, if we look at the white cells in any given column, there is at most one cell labeled h, and if such a cell exists, it must be the below any other white cells in the column by rule (b). Since the h cells are listed before the v cells and then the v cells are listed in reverse reading order, each β (j) is a composition shape because if dg(β (j) ) contains cell (x, y), it also contains all cells below it. In order to establish that the cells labeled h by L form an α-row of size i and the cells labeled v by L form an -column of size r − i, it remains to show that for each j = 1, 2, . . . , r, cj is a removable cell from β (j) . Fix j with 1 ≤ j ≤ r. By the definition of a removable cell, cj = (xj , yj ) will be a removable cell from dg(β (j) ) (j)

unless there exists xj < k ≤ n with βk = yj , so suppose for a contradiction that such a k exists, as pictured in Figure 4.6. If (k, yj ), is a white cell, then it is labeled in L with an h or a v. Either way, since h’s are enumerated first in reading order, followed by v’s, it must be that (k, yj ) = ct for t > j. So (k, yj ) 6∈ dg(β (j) ), which is a contradiction. If (k, yj ) is a gray cell and βk = yj , this violates condition 4 of

72 an acceptable coloring, and since we know P has an acceptable coloring C, this cannot happen. So it must be that (k, yj ) is a gray cell and βk > yj . Consider the (j)

cell (k, yj + 1) ∈ dg(β). If (k, yj + 1) is colored gray in P , then βk ≥ yj + 1 which is a contradiction. If (k, yj + 1) is colored white in P , then it is labeled in L with an h by rule (c). This means (k, yj + 1) = ct for t < j because the cells labeled h (j)

are listed in reading order. Thus, dg(β (j) ) includes this cell, so βk ≥ yj + 1, which is a contradiction. This proves that for every j, cj = (xj , yj ) is a removable cell from dg(β (j) ).

yj+1 yj c j xj

k

Figure 4.6: Proof that cj is a removable cell. (i,r−i)

We have shown that any diagram P ∈ Pβ/α (i,r−i)

diagram around α and so P ∈ Dα,β

is also a colored labeled (i,r−i)

, which means Pα,β

(i,r−i)

⊆ Dα,β

(i,r−i)

this together with what we have shown before, we conclude Dα,β

. Putting (i,r−i)

= Pα,β

,

which proves the lemma. Thus, we have from equation (4.3) and Lemma 5 that

pr (X)Sα (X) = =

r−1 X k X (−1)i hr−i (X)ei (X)Sα (X) k=0 i=0 r−1 X k X

(−1)i

k=0 i=0

X

X

Sβ (X).

(4.8)

β:|β|−|α|=r P ∈P(i,r−i) α,β

Next, we can perform an involution on the set of colored diagrams with Pieri labelings to leave only the diagrams whose labelings are satisfactory labelings that satisfy rules (a) through (d). Consider the set of diagrams of shape β with coloring C and labeling L, such that C is an acceptable coloring of β and L is a satisfactory (i,r−i)

labeling of dgC (β/α) with i h’s and r − i v’s. Call this set of diagrams Sα,β know

(i,r−i) Sα,β

⊆

(i,r−i) Pα,β ,

. We

since these diagrams must additionally satisfy rule (d).

73 Lemma 6. Fix r and fix k, 0 ≤ k ≤ r − 1. Then k X (−1) Sr−k,1k Sα (X) = (−1)i hr−i (X)ei (X)Sα (X) = (−1)k

X

k

i=0

X

Sβ (X)

β:|β|−|α|=r S∈S(k,r−k) α,β

Proof. Let Pα,β (k) denote the set of all diagrams with an acceptable coloring and a Pieri labeling in which the number of cells labeled h is at most k. That is, (i,r−i)

Pα,β (k) = ∪ki=0 Pα,β

(i,r−i)

. Suppose P ∈ Pα,β

. Define the sign of P by sgn(P ) =

(−1)i = (−1)h(P ) , where h(P ) is the number of cells labeled h in P . We know from equation (4.2) and Lemma 5 that k X (−1) Sr−k,1k Sα (X) = (−1)i hr−i (X)ei (X)Sα (X) k

=

i=0 k X

(−1)i

i=0

=

X

X

X

β:|β|−|α|=r

(i,r−i) P ∈Pα,β

X

Sβ (X)

sgn(P )Ssh(P ) (X).

β:|β|−|α|=r P ∈Pα,β (k)

We define a shape-preserving sign-reversing involution J : Pα,β (k) → Pα,β (k) as follows. If P ∈ Pα,β (k), then P is a diagram with an acceptable coloring C and a Pieri labeling L. Define J(P ) to be the diagram with the same coloring C and a different labeling J(L), defined as follows. J(L) is the labeling that results from changing the label of the last white cell in reading order if (i) the number of h’s, i, is strictly less than k or (ii) i = k and the last white cell in reading order is labeled with an h. First, note that J(L) is itself a Pieri labeling. The last white cell in reading order is never determined by rules (a) through (c), so J(L) will still satisfy rules (a) through (c). The coloring has not changed, and the new labeling J(L) is a Pieri labeling in which the number of cells labeled h cannot exceed k, so J(P ) ∈ Pα,β (k). Also, J is an involution on the set of Pieri labelings, and therefore, on the set of diagrams in Pα,β (k), because each Pieri labeling L must fall into one of the following two cases.

74 Case 1. The last white cell is labeled h in L. In this case, 1 ≤ i ≤ k and J(L) is the labeling that is the same as L except the last cell has a v. Thus, J(L) has i − 1 h’s, where 0 ≤ i − 1 ≤ k − 1. Then J(J(L)) = L because the number of h’s in J(L) is strictly less than k, so the last cell changes labels back to an h. Case 2. The last white cell is labeled v in L. If 0 ≤ i ≤ k − 1, then J(L) is the labeling that is the same as L except the last cell has an h. Thus, J(L) has i + 1 h’s, where 1 ≤ i + 1 ≤ k. Then J(J(L)) = L because the last cell is labeled h in J(L), so even if i + 1 = k, J will change the label of the last cell back to a v. Otherwise, i = k and J(L) is exactly the same labeling as L. Again, J(L) has i = k h’s and the last cell is a v, so applying J again leaves the labeling unchanged. In this case too, J(J(L)) = L. The involution J on Pα,β (k) is clearly shape-preserving, since only the labeling was changed. It is sign-reversing because the number of h’s in L and J(L) differs by one, provided J changes the label of the last white cell in reading order. The fixed points of the involution are the diagrams P that have a labeling L for which neither (i) nor (ii) applies. That is, the fixed points of J are the diagrams P ∈ Pα,β (k) whose Pieri labelings have i = k, and the last white cell in reading order is labeled with a v. This is exactly the set of diagrams whose satisfactory (k,r−k)

labelings have k h’s and r − k v’s, which we called Sα,β

. The sign of all such

k

diagrams is (−1) , so we have k X (−1)i hr−i (X)ei (X)Sα (X) = i=0

X

X

sgn(S)Ssh(S) (X)

β:|β|−|α|=r S∈Sα,β (k)

= (−1)k

X

X

β:|β|−|α|=r

(k,r−k) S∈Sα,β

Sβ (X).

For example, in calculating the product p2 (X)S(2,1) (X), we have r = 2 and α = (2, 1). Fix k = 0. We must consider all the Pieri-labeled diagrams with no h’s, that is, the six diagrams on the left side of Figure 4.3. The involution J changes the label of a cell of such a diagram if

75 (i) the number of h’s, i, is strictly less than k or (ii) i = k and the last white cell in reading order is labeled with an h. But the first case can never happen when k = 0 since it is impossible to have strictly less than zero cells labeled h. Similarly, the second case can also never happen because if i = k = 0, this means there are no cells labeled h, so the last white cell in reading order cannot be labeled with an h. Thus, in this case, the involution J does nothing, as diagrams A, B, C, D, E, and F all have satisfactory labelings. Now fix k = 1. We must consider all eighteen of the diagrams with at most one h, that is all the diagrams pictured in Figure 4.3. All the diagrams on the left side of this table have sign (−1)0 = 1, whereas all the diagrams on the right have sign (−1)1 = −1. The involution J cancels the following pairs of diagrams with opposite signs and the same shape: A with H, B with P , C with K, D with L, E with I, and F with J. The fixed points of J are therefore diagrams G, M , N , O, Q, and R, which are exactly the diagrams whose labelings are satisfactory, that is, they satisfy rules (a) through (d). We have from equations (4.3) and Lemma 6 that r−1 X (−1)k Sr−k,1k Sα (X) pr (X)Sα (X) =

=

k=0 r−1 X

(−1)k

k=0

X

X

β:|β|−|α|=r

(k,r−k) S∈Sα,β

(4.9) Sβ (X).

(4.10)

From here, we can apply two more involutions to leave only the uniquely colored diagrams, and then only the Murnaghan-Nakayama diagrams, which will complete the proof of Theorem 5. Let Uα,β (r) denote the set of uniquely colored diagrams, and for any diagram U ∈ Uα,β (r), let h(U ) denote the number of cells labeled h in U . The first of these two remaining involutions is as follows. Lemma 7. pr (X)Sα (X) =

X

X

(−1)h(U ) Sβ (X).

β:|β|−|α|=r U ∈Uα,β (r)

(4.11)

76 Proof. Let Sα,β (r) denote the set of all diagrams with an acceptable coloring and a satisfactory labeling in which the number of cells labeled h is less than r. That is, (k,r−k)

Sα,β (r) = ∪r−1 k=0 Sα,β

. Note that every diagram with an acceptable coloring and

satisfactory labeling is in this set, since in all satisfactory labelings, the last cell in reading order must be labeled v, and there are only r white cells to be labeled, so there must be strictly less than r cells labeled h. Suppose S ∈ Sα,β (r). Define the sign of S by sgn(S) = (−1)h(S) . We have from (4.9) that pr (X)Sα (X) =

r−1 X

(−1)k

k=0

=

X

X

X

β:|β|−|α|=r

(k,r−k) S∈Sα,β

X

Sβ (X)

(4.12)

sgn(S)Ssh(S) (X).

(4.13)

β:|β|−|α|=r S∈Sα,β (r)

We will do an involution to cancel the diagrams S ∈ Sα,β (r) with more than one way of coloring the cells so that the white cells have a satisfactory labeling. Thus, we will show that the fixed points of this involution are the diagrams in Uα,β (r). Suppose we have a diagram S ∈ Sα,β (r) with acceptable coloring C and b in satisfactory labeling L such that there exists another acceptable coloring C which the cells of dgCb (β/α) have a satisfactory labeling. We will call such a b a recoloring of S. We aim to classify what it means for a diagram to coloring C have a recoloring. For S to have a recoloring, it must be the case that the number of columns of S is greater than the number of parts of α, which we will call `. Since in any acceptable coloring, the gray cells collapse to form dg(α), we can only obtain a recoloring by changing which ` columns contain gray cells. Specifying a coloring is equivalent to specifying a set of ` columns which will contain gray cells, because the number of gray cells in the ith column of the set must be αi for all 1 ≤ i ≤ `. We cannot obtain a recoloring, then, by interchanging two columns containing gray cells, only by choosing a different set of ` columns to contain the gray cells. b it must be that there exists a column, say column If S has a recoloring C, i, which contains some number z of gray cells in C but contains only white cells b Similarly, there must also be a column, say column j, which contains only in C.

77 b We can think of a recoloring as white cells in C but contains z gray cells in C. having the effect of moving the z gray cells from column i to column j. Further, any of the columns between i and j must contain only white cells, because we are not allowed to reorder the gray columns in a recoloring. In any given recoloring, many such moves may take place at the same time, as any number of the ` columns containing gray cells can be relocated. Suppose S has a recoloring that moves z gray cells from column i to column j, among other moves, possibly. This means βi , βj ≥ z. We claim that if i < j, it must be that the height of column i, βi , is z and the height of column j, βj , is strictly greater than z. Additionally, if i > j, then it must be that βi > z and βj = z. In both cases, we also claim that any column between i and j must only contain one cell. These situations are pictured in Figure 4.7.

Case 2

Case 1

C

z+1 z

i

C

z+1 z

j

i

C

z+1 z

j

j

C

z+1 z

i

j

i

Figure 4.7: Any recoloring must move gray cells in one of these two ways. Case 1. Suppose i < j. Suppose for a contradiction that βi > z. If βi > z b will have white cells in positions (i, z), (i, z + 1), and (j, z + 1). and βj > z, then C This is an instance of the configuration and so by Lemma 2, dgCb (β/α) has no b is not a recoloring. If βi > z and βj = z, then C b will satisfactory labeling. Thus, C b will not meet condition 4 have a white cell (i, z) which is not removable, and so C of an acceptable coloring. Thus, we must have βi = z. Now suppose βi = βj = z. b which is not removable, so C b is not an Then again (i, z) will be a white cell in C acceptable coloring. Thus, it must be that in order for S to have a recoloring when i < j, βi = z and βj > z. Now consider the height of any column k with i < k < j. We have already established that column k can contain only white cells, since in a recoloring, we cannot reorder the gray columns. Suppose for a contradiction that βk > 1. Then (k, 1), (k, 2), and (j, 2) form the configuration

in S, so by

78 Lemma 2, dgC (β/α) has no satisfactory labeling, which is a contradiction. Further, b avoids , and so the white if βk = 1 for each i < k < j, then the recoloring C cells of dgCb (β/α) will have a satisfactory labeling. This is clear because S must avoid

, and moving a column of white cells to the left cannot create any new

instances of

.

Case 2. Suppose i > j. Suppose for a contradiction that βi = z. Since βj ≥ z, this means (j, z) is not a removable cell from S, so C is not an acceptable coloring, since it fails to meet condition 4 in the definition of an acceptable coloring. This is a contradiction, so it must be that βi > z. Now suppose βi > z and βj > z. Then the cells (j, z), (j, z + 1), and (i, z + 1) form the configuration

, which

contradicts the fact that dgC (β/α) has a satisfactory labeling. Thus, it must be that in order for S to have a recoloring when i > j, βi > z and βj = z. Now consider the height of any column k with j < k < i. We have already established that column k can contain only white cells, since in a recoloring, we cannot reorder the gray columns. Suppose for a contradiction that βk > 1. Then (k, 1), (k, 2), and (i, 2) will form the configuration

in the recoloring, so by Lemma 2, dgCb (β/α)

has no satisfactory labeling, which is a contradiction. Further, if βk = 1 for each b avoids i < k < j, then the recoloring C . This is true because S avoids , and the only possible new instance of

created by moving white cells from column j

to column i occurs in columns k and i, where j < k < i. This requires column k b avoids to have height at least 2, which is not the case here. Thus, C , and so the white cells of dgCb (β/α) will have a satisfactory labeling. This gives us the necessary ingredients to define our involution, I. Given an arbitrary diagram S ∈ Sα,β (r), find the smallest index i, if it exists, so that one of the following two statements about the coloring C of S is true. Case 1. Column i is a completely gray column of height z, there is a column j > i which is completely white and of height greater than z, and for all i < k < j, column k has only one cell, which is white. Case 2. Column i contains z gray cells and has height greater than z, there is a column j < i which is completely white and of height r, and for all j < k < i, column k has only one cell, which is white.

79 After choosing i to be minimal so that one of the above statements is true, choose j to be minimal among all possible columns meeting the required conditions. The involution I moves the z gray cells from column i to j, producing a b of I(S). The labeling L b of I(S) will also be different than the new coloring C labeling L of S, as as shown in Figure 4.8. In Case 1, we know L labels every cell in column j besides cell (j, 1) with a v, according to rule (b) of a satisfactory b the cells (j, x) for x ≤ z will be gray and therefore not labeled. labeling. In L, We must label (j, z + 1) with an h according to rule (c) of a satisfactory labeling, because it is above a gray cell and there is a white cell at (i, z). Cells (j, y) for y > z + 1 must be labeled v, according to rule (b). In column i, all cells besides b by rule (b). Now consider the white cells in row one (i, 1) must be labeled v in L of of S, which may be in any column. L must label them h, h, . . . , h, v from left to b must also right, according to rules (a) and (d) of a satisfactory labeling. Since L follow these rules, we know that the white cells in row one of I(S) must also be labeled left to right as h, h, . . . , h, v. This tells us how to relabel the white cells in columns i and j, as well as those in row 1. We claim that if we leave the labels b as they were in L, then L b will be a satisfactory of all other cells the same in L labeling.

Case 2

Case 1 z+1 z

L

v v v v hh v j i

L

v z+1 h z v v h h v j i

L

v z+1 h z v v h hv j i

L

v v v v h h v

z+1 z

j

i

Figure 4.8: Any recoloring creates a relabeling, in one of these two ways. Suppose cell (x, y) has x 6= i, j and y > 1. If (x, y) was labeled with an h in L by rule (a) of a satisfactory labeling, it is because there was a white cell to the right of it. If that white cell was in column j, it will still be to the right of (x, y) when it is moved to column i in I(S), because there is no cell between columns i and j that is not in the first row, so x < i. Otherwise, the white cell to the right of b by rule (a). Now, if (x, y) did not move, so either way (x, y) will be labeled h in L

80 (x, y) was labeled with a v in L by rule (b) of a satisfactory labeling, it is because there was a white cell below it. If x 6= i, j, since we did not change the coloring in column x, cell (x, y) will still have a white cell below it, and should be labeled v in b by rule (b). Now if in S, cell (x, y) is above a gray cell (x, y − 1) and there is a L white cell (u, y − 1) for u < x, then L labels cell (x, y) with an h according to rule (c). If u = j, then the white cell (u, y − 1) will be moved further left to (i, y − 1), b If u 6= j, so cell (x, y) will still meet the criteria to be labeled h by rule (c) in L. then nothing has changed regarding the status of cell (x, y) and so it will again be b Finally, no cell (x, y) with y > 1 will be labeled by rule labeled h by rule (c) in L. (d) of a satisfactory labeling, as that rule only applies to the last cell in reading order, and we know that the last cell in reading order occurs in row one. Thus, we have shown that any cell besides those in row one and columns i b is a satisfactory labeling and j must be labeled as before. So we conclude that L of I(S). Observe that there was only one cell above the first row, cell (j, z + 1), b In row one, the total number of whose label changed from a v in L to an h in L. b This says that in Case 1, h cells and v cells remained the same between L and L. h(I(S)) = h(S) + 1, so sgn(I(S)) = −sgn(S). Now in Case 2, we know L labels every cell in column j besides cell (j, 1) b the cells in column j with a v, according to rule (b) of a satisfactory labeling. In L, will be gray and therefore not labeled. In column i, we know L must label (i, z + 1) with an h according to rule (c) of a satisfactory labeling, because it is above a gray cell and there is a white cell at (j, z), which is one row below and to the left. Cells (i, y) for y > z + 1 must be labeled v in L by rule (b). In column i, all cells besides b by rule (b). Now consider the white cells in row one (i, 1) must be labeled v by L of of S, which may be in any column. L must label them h, h, . . . , h, v from left to b must also right, according to rules (a) and (d) of a satisfactory labeling. Since L follow these rules, we know that the white cells in row one of I(S) must also be labeled left to right as h, h, . . . , h, v. This tells us how to relabel the white cells in columns i and j, as well as those in row 1. We claim that if we leave the labels b as they were in L, then L b will be a satisfactory of all other cells the same in L labeling.

81 Suppose cell (x, y) has x 6= i, j and y > 1. If (x, y) was labeled with an h in L by rule (a) of a satisfactory labeling, it is because there was a white cell to the right of it. If that white cell was in column j, it will still be to the right of (x, y) when it is moved to column i in I(S), because i > j. Otherwise, the white b cell to the right of (x, y) did not move, so either way (x, y) will be labeled h in L by rule (a). Now, if (x, y) was labeled with a v in L by rule (b) of a satisfactory labeling, it is because there was a white cell below it. If x 6= i, j, since we did not change the coloring in column x, cell (x, y) will still have a white cell below it, and b by rule (b). Now if in S, cell (x, y) is above a gray cell should be labeled v in L (x, y − 1) and there is a white cell (u, y − 1) for u < x, then L labels cell (x, y) with an h according to rule (c). If u = j, then we know x > j. Actually, since y > 1 and there are no cells above row one in any columns between j and i, we know x > i. Then, if u = j, the white cell (u, y − 1) will be moved right to (i, y − 1), and since x > i, we know cell (x, y) will still have a white cell one row below and to b Otherwise, if u 6= j, nothing the left, and thus will be labeled h by rule (c) in L. has changed regarding the status of cell (x, y) and so it will again be labeled h by b Finally, no cell (x, y) with y > 1 will be labeled by rule (d) of a rule (c) in L. satisfactory labeling, as that rule only applies to the last cell in reading order, and we know that the last cell in reading order occurs in row one. In Case 2 as well, we have shown that any cell besides those in row one and b is a satisfactory columns i and j must be labeled as before. So we conclude that L labeling of I(S). Observe that there was only one cell above the first row, cell b In row one, the total (i, z + 1), whose label changed from an h in L to a v in L. b This says that number of h cells and v cells remained the same between L and L. in Case 2, h(I(S)) = h(S) − 1, so sgn(I(S)) = −sgn(S). Now that we have established that I maps Sα,β (r) into Sα,β (r), we must show that I is indeed an involution. Suppose that S with coloring C and labeling L falls into Case 1 so that the involution switches columns i and j. Then I(S), b and labeling L, b will fall into Case 2, switching the same which has coloring C columns back to create diagram S with coloring C and labeling L. Similarly, if S falls into Case 2, the involution will switch columns i and j, so that I(S) will fall

82 into Case 1, and I(I(S)) will have coloring C and labeling L. Thus, in both cases, I(I(S)) = S, so I is an involution. It is clear that I is shape-preserving, and we have shown that I is signreversing, so we will be able to cancel terms of opposite sign and leave only the fixed points. We have shown that the only way to recolor a diagram so that it still has a satisfactory labeling is to do one of the switches in Case 1 or 2. Thus, if S is a diagram for which it is not possible to do one of these switches, then S cannot be recolored. So the fixed points of the involution I are exactly the uniquely colored diagrams in Uα,β (r). Therefore, we have established the result that

pr (X)Sα (X) =

X

X

sgn(S)Ssh(S) (X)

β:|β|−|α|=r S∈Sα,β (r)

=

X

X

(−1)h(U ) Sβ (X).

β:|β|−|α|=r U ∈Uα,β (r)

In our example of computing the product p2 (X)S(2,1) (X), we had remaining after the involution of Lemma 6 the diagrams A, B, C, D, E, and F , associated with k = 0, and G, M , N , O, Q, and R, associated with k = 1. In Lemma 7, the sign of A, B, C, D, E, and F is (−1)0 = 1 because these diagrams have no h’s. The sign of diagrams G, M , N , O, Q, and R is (−1)1 = −1 since these diagrams have a single h cell. The involution I cancels diagrams that can be recolored. In this case, the only diagrams that can be recolored are A and O, and the involution has I(O) = A and I(A) = O so that the contributions from diagrams A and O cancel out. This leaves diagrams B, C, D, E, F , G, M , N , Q, and R, which are the uniquely colored diagrams. Now that we have written our desired product pr (X)Sα (X) in terms of diagrams that have only one coloring, we can do one last involution to leave only those diagrams with a unique satisfactory labeling as well. These are the MurnaghanNakayama diagrams of r and α, which will prove our main theorem. Proof. Proof of Theorem 5. Suppose U ∈ Uα,β (r), so that the only coloring of U is C. Suppose also that dgC (β/α) has more than one satisfactory labeling,

83 that is, there is more than one way to label its cells in accordance with rules (a) through (d). If U has coloring C and labeling L, let sgn(U ) = (−1)h(U ) , where h(U ) is the number of cells labeled h by L. Now define an involution K on the set of diagrams U ∈ Uα,β (r). K(U ) will be the diagram of the same shape as U with the same coloring C and a new labeling, K(L). To define K(L), take the first cell c in reading order of dgC (β/α) that was not forced to be a v or an h by rules (a) through (d). If L labeled cell c with an h, change it to a v. Otherwise, change it from a v to an h. This new labeling K(L) is still a satisfactory labeling because we have not changed the label of any cell to which rules (a) through (d) apply. The number of h’s in K(L) is either one fewer or one more than the number of h’s in L, so that sgn(U ) = −sgn(K(U )). K is an involution on Uα,β (r) because K applied to a diagram with coloring C and labeling K(L) will change the label of the same cell c back to its original label in L. Applying this involution to the diagrams U ∈ Uα,β (r) that have more than one satisfactory labeling gives a way to pair Sβ terms with opposite signs on the right hand side of X

pr (X)Sα (X) =

X

sgn(U )Sβ (X).

β:|β|−|α|=r U ∈Uα,β (r)

Since the fixed points of K are the diagrams for which rules (a)-(d) assign a v or h label to every cell of dgC (β/α), we can express the product pr (X)Sα (X) in terms of only these diagrams with one satisfactory labeling. These are exactly the Murnaghan-Nakayama diagrams of r and α, which proves the desired result, X pr (X)Sα (X) = sgn(D)Ssh(D) (X) D∈MN(r,α)

=

X

(−1)h(D) Ssh(D) (X),

D∈MN(r,α)

where MN(r, α) is the set of all Murnaghan-Nakayama diagrams of r and α.

To conclude our example, recall that the only terms left in the product p2 (X)S(2,1) (X) after Lemma 7 came from diagrams B, C, D, E, F , G, M , N , Q,

84 and R, the uniquely colored diagrams. Again, since the sign is determined by the number of h cells, diagrams B, C, D, E, and F have sign 1, while diagrams G, M , N , Q, and R have sign −1. The involution K cancels the following pairs of diagrams with the same shape and coloring, but more than one satisfactory labeling: M with E, Q with D, and R with F . This leaves as fixed points the MurnaghanNakayama diagrams B and C with sign 1, and G and N with sign −1. Thus, Theorem 5 says that p2 (X)S(2,1) (X) = S(4,1) (X) + S(2,3) (X) − S(2,1,1,1) (X) − S(1,2,2) (X), which, for a small number of variables, can be verified by direct computation.

4.2

Murnaghan-Nakayama Rule for Row-Strict Quasisymmetric Schur Functions Our goal for this section will be to express the product pr (X)RS α (X) as

a sum of row-strict quasisymmetric Schur functions. In particular, we aim to find (r)

the coefficients vα,β in the expansion (1.23), pr (X)RS α (X) =

X

(r)

vα,β RS β (X).

β

Notice that the Pieri rules for the quasi-Schurs (Theorem 2) and rowstrict quasi-Schurs (Theorem 3) are very similar, except the roles of α-rows and α-columns have been reversed. To use these rules to determine a MurnaghanNakayama rule using the same machinery, we will again write r−1 X k X pr (X)RS α (X) = (−1)i hr−i (X)ei (X)RS α (X). k=0 i=0

However, since multiplication is commutative, we will instead think of this as r−1 X k X pr (X)RS α (X) = (−1)i ei (X)hr−i (X)RS α (X).

(4.14)

k=0 i=0

Now we will use the Pieri rules to multiply RS α (X) first by some homogeneous symmetric function hr−i (X) followed by some elementary symmetric function ei (X). This allows us to place around the diagram of α an α-row, marked

85 with h’s, followed by an α-column, marked with v’s, just as in the Sα case. The type of diagrams that arise from such a sequence of multiplications are exactly the colored labeled diagrams of Section 4.1. The only difference is that now the number of cells labeled h and v have been interchanged. We can use Lemma 5 of the previous section to say that these colored labeled diagrams are equivalently characterized as diagrams with acceptable colorings and Pieri labelings. The involution J, however, must be modified since the number of cells labeled h and v have been interchanged. For a fixed k, consider the set of all diagrams with an acceptable coloring and a Pieri labeling in which the number of cells labeled v is at most k. For each diagram P in this set, associate P with sign (−1)v(P ) , where v(P ) is the number of cells labeled v in P . Then define a new involution J on the diagrams in this set by changing the label of the last white cell in reading order if (i) the number of v’s is strictly less than k or (ii) the number of v’s equals k and the last white cell in reading order is labeled with a v. This involution J is exactly the same as J, except the roles of v and h have been interchanged. Thus, its fixed points are the diagrams whose labelings satisfy rules (a) through (c) of a Pieri labeling plus one additional rule: (d) Assign an h to the last white cell in reading order. We will call such labelings row-strict satisfactory labelings. This means we can write k−1 X k X pr (X)RS α (X) = (−1)i hr−i (X)ei (X)RS α (X) r=0 i=0

=

(k,r−k)

where Sα,β

k−1 X

X

r=0

β:|β|−|α|=r

(−1)k

X

RS β (X)

(k,r−k)

S∈Sα,β

is the set of diagrams of shape β with coloring C and labeling L,

such that C is an acceptable coloring of β and L is a row-strict satisfactory labeling of dgC (β/α) with k v’s and r − k h’s.

86 Next, apply the same involution I to leave only the uniquely colored diagrams. This leaves X

pr (X)RS α (X) =

X

(−1)v(U ) RS β (X).

β:|β|−|α|=r U ∈Uα,β (r)

Finally, apply the same involution K to leave only the diagrams that have a unique coloring and a unique row-strict satisfactory labeling. That is, the fixed points of K are the uniquely colored diagrams for which rules (a), (b), (c), and (d) assign a v or h label to every cell of dgC (β/α). Call such diagrams row-strict Murnaghan-Nakayama diagrams of r and α. Then we can state the MurnaghanNakayama rule for row-strict quasi-Schurs as follows. Theorem 14. If α is a composition of m, then X (−1)v(D) RS sh(D) (X) pr (X)RS α (X) = D∈MN(r,α)

where MN(r, α) is the set of all row-strict Murnaghan-Nakayama diagrams of r and α. However, note that the condition of having all labels forced by rules (a), (b), (c), and (d) is exactly the same condition as having all labels forced by rules (a), (b), (c), and (d), since (d) and (d) determine the label of only the last cell in reading order. Moreover, D is a Murnaghan-Nakayama diagram of r and α if and only if D is a row-strict Murnaghan-Nakayama diagram of r and α, where D arises from D by changing the label of the last cell from a v to an h. That is, the map that sends D to D is a bijection from MN(r, α) to MN(r, α), and v(D) = v(D)+1. Applying this bijection means that Theorem 14 can be equivalently stated in terms of Murnaghan-Nakayama diagrams, which proves Theorem 6. We conclude this section with an example. Suppose we wish to compute the product p2 (X)RS (2,1) (X). Equation (4.14) says p2 (X)RS (2,1) (X) =

1 X k X

(−1)i ei (X)h2−i (X)RS (2,1) (X)

k=0 i=0

= e0 (X)h2 (X)RS (2,1) (X) + e0 (X)h2 (X)RS (2,1) (X) + e1 (X)h1 (X)RS (2,1) (X).

87 Then we can use the Pieri rules to express our desired product as a sum over colored labeled diagrams. These diagrams are shown in Figure 4.9, with the six diagrams arising from e0 (X)h2 (X)RS α (X) on the left, and the twelve diagrams arising from e1 (X)h1 (X)RS α (X) on the right.

p2RS(2,1)(X) h2e0RS(2,1)(X) A':

h

h

B': h

h1e1RS(2,1)(X) hv

M':

H':

v h

N': v

h

O':

h

C': D': E': h F':

h

h

v

I': h h

h

hh

h

G':

J':

v h

K': L':

v

P':

v h

Q':

h

R':

v h

v

h

v h

h

v

h v

Figure 4.9: The colored labeled diagrams used to compute p2 (X)RS (2,1) (X). Next, we perform the involution J for k = 0, 1. Fix k = 0, so the involution acts on the six diagrams on the left side of Figure 4.9, where the number of cells labeled v is at most 0. J changes the label of the last cell in reading order of a diagram if (i) the number of v’s is strictly less than k or (ii) the number of v’s equals k and the last white cell in reading order is labeled with a v. But when k = 0, it is impossible for a diagram to fall into case (i) because there cannot be strictly less than 0 v’s. Similarly, no diagram falls into case (ii) because it is impossible to have simultaneously 0 v’s and the last white cell labeled with a

88 v. Thus, all six diagrams A0 through G0 are fixed points of J when k = 0, because all have row-strict satisfactory labelings, that is, they satisfy rules (a), (b), (c), and (d). Now fix k = 1. We must consider all eighteen of the diagrams with at most one v, that is all the diagrams pictured in Figure 4.3. All the diagrams on the left side of this table have sign (−1)0 = 1, whereas all the diagrams on the right have sign (−1)1 = −1. By changing the label of the last cell in reading order, the involution J cancels the following pairs of diagrams with opposite signs and the same shape: A0 with Q0 , B 0 with R0 , C 0 with M 0 , D0 with O0 , E 0 with N 0 , and F 0 with G0 . The fixed points of J when k = 1 are therefore diagrams H 0 , I 0 , J 0 , K 0 , L0 , and P 0 , which are exactly the diagrams with row-strict satisfactory labelings. Thus, after applying J we are left with diagrams A0 , B 0 , C 0 , D0 , E 0 , and F 0 , all associated with sign (−1)0 = 1 because they have no v’s, and diagrams H 0 , I 0 , J 0 , K 0 , L0 , and P 0 , all associated with sign (−1)1 = −1 because they have one v. Next, the involution I cancels any remaining diagrams that can be recolored. In this case, the only diagrams that can be recolored are D0 and H 0 , which have opposite signs since they have a different number of v’s. Since I(D0 ) = H 0 and I(H 0 ) = D0 , the contributions from diagrams D0 and H 0 cancel out. This leaves diagrams A0 , B 0 , C 0 , E 0 , F 0 , I 0 , J 0 , K 0 , L0 , and P 0 , which are the uniquely colored diagrams. Finally, the involution K cancels the following pairs of diagrams with the same shape and coloring, but more than one row-strict satisfactory labeling: A0 with L0 , B 0 with J 0 , and C 0 with I 0 . This leaves as fixed points the row-strict Murnaghan-Nakayama diagrams E 0 and F 0 with sign 1, and K 0 and P 0 with sign −1. Thus, Theorem 14 says that p2 (X)RS (2,1) (X) = RS (1,2,2) (X)+RS (2,1,1,1) (X)− RS (2,3) (X) − RS (4,1) (X), which, for a fixed number of variables, can be verified by direct computation. To see the equivalence of Theorems 14 and 6, compare the set of MurnaghanNakayama diagrams used to compute p2 (X)S(2,1) (X) with the set of row-strict Murnaghan-Nakayama diagrams used to compute p2 (X)RS (2,1) (X). When r = 2 and α = (2, 1), we can see from Figure 4.10 that the Murnaghan-Nakayama

89 diagrams of r and α differ from the row-strict Murnaghan-Nakayama diagrams of r and α only in the labeling of the last cell. In particular, the shapes of the diagrams are the same but the row-strict Murnaghan-Nakayama diagrams have one more v. Theorem 14 says we can sum over all diagrams D on the right side of Figure 4.10 and take the sign as (−1)v(D) . Equivalently, Theorem 6 says we can sum over all diagrams D on the left side of Figure 4.10 and take the sign as (−1)v(D)+1 . MN(2, (2,1)) B:

C:

v v

G: v v

N: v

MN(2, (2,1)) E':

hv

F':

h

h

h

hh

v h

K':

P':

v h

Figure 4.10: The Murnaghan-Nakayama diagrams or the row-strict MurnaghanNakayama diagrams can be used to compute p2 (X)RS (2,1) (X). (r)

Note that Theorem 6 actually says how the coefficients uα,β in the Murnaghan(r)

Nakayama rule for quasi-Schurs (1.22) are related to the coefficients vα,β in the (r)

Murnaghan-Nakayama rule for row-strict quasi-Schurs (1.23). First, uα,β = 0 if (r)

and only if vα,β = 0, which happens when there is no Murnaghan-Nakayama dia(r)

gram of r and α that is of shape β. Otherwise, Theorem 5 says that uα,β = (−1)h(D) where h(D) is the number of cells labeled h in the Murnaghan-Nakayama diagram (r)

D of shape β. Likewise, Theorem 6 says that vα,β = (−1)v(D)+1 where v(D) is the number of cells labeled v in the Murnaghan-Nakayama diagram D of shape (r)

β. However, since D has r labeled cells total, we can write vα,β = (−1)v(D)+1 = (r)

(−1)r−h(D)+1 = (−1)h(D) (−1)r−1 = uα,β (−1)r−1 . This says that the coefficient of RS β in the Murnaghan-Nakayama rule for row-strict quasi-Schurs is (−1)r−1 times the coefficient of Sβ in the Murnaghan-Nakayama rule for quasi-Schurs. Thus, when r is odd, these coefficients are the same, and when r is even, these coefficients have opposite sign. This explains why in our example for r = 2, the shapes that appeared in the Murnaghan-Nakayama rule for row-strict quasi-Schurs had oppo-

90 site sign than in the Murnaghan-Nakayama rule for quasi-Schurs. Similarly, when (r)

(r)

r = 1, the coefficients uα,β and vα,β are the same because p1 (X) = h1 (X) = e1 (X) and the Pieri rules (1.17), (1.20) say that the coefficients are the same in the quasi-Schur expansion of hr (X)Sα (X) and the row-strict quasi-Schur expansion of er (X)RS α (X). While the relationship between the coefficients in the Murnaghan-Nakayama rule for quasi-Schurs and the coefficients in the Murnaghan-Nakayama rule for row-strict quasi-Schurs turns out to be a straightforward sign change, this simple connection is a nontrivial fact. In the proof, we express pr as a sum of hook-shaped Schur functions s(r−k,1k ) and we compute the product of pr with a hook Schur function. The set of colored labeled diagrams resulting from this hook Schur function multiplication is generally different in the Sα and RS α cases, as seen in Figures 4.3 and 4.9. These colored labeled diagrams have a prescribed number of h’s and v’s, and these two numbers are interchanged in going from the quasi-Schurs to the row-strict quasi-Schurs, which means the set of diagrams is different. So while the set of diagrams to which we apply our involutions is different, the ultimate fixed point sets are the same, because the characterizations of the fixed points as Murnaghan-Nakayama diagrams and row-strict Murnaghan-Nakayama diagrams do not restrict the number of v and h labels. The content of Chapter 4 is currently being prepared for submission for publication. Remmel, Jeffrey; Tiefenbruck, Janine LoBue. The dissertation author is an author of this material.

Chapter 5 Enumeration of Permuted Basement Fillings

91

92 In this chapter, we prove some enumerative results about the number of permuted basement fillings with a given shape γ and basement σ, in the case where σ is the identity n . In particular, we are able to count generalized rectangular shapes of the form ((s)t , 0)k , as well as some slight variations of these. In Section 5.1, we show how to count the number of r-tuples of non-crossing t-Dyck paths of length (t + 1)k. In Section 5.2, we give a bijection between the set of PBFs of shape ((s)t , 0)k and (s − 1)-tuples of non-crossing t-Dyck paths so that our results in Section 5.1 allow us to give explicit formulas for the number of PBFs of generalized rectangular shapes. In Section 5.3, we discuss a few other shapes that we are able to count and indicate directions for future research.

5.1

Lattice Paths We will count the number of PBFs of various shapes by finding bijections

between PBFs and certain lattice paths, so it is important to first determine how to count these lattice paths. Let the height of a lattice path L at any point x be the y-coordinate of the path at that point, written heightL (x) = y. In the introduction, we defined a t-path to be a path consisting of up-steps in the (1, 1) direction down-steps in the (1, −t) direction. A t-Dyck path, is a t-path of length (t + 1)k such that heightD (x) ≥ 0 for all x ≤ (t + 1)k. The number of t-Dyck paths of length (t + 1)k is given by the Fuss-Catalan
(t+1) (t+1) (t+1)k 1 , which we denote Ck . It is well-known that Ck counts number tk+1 k the number of rooted plane trees with (t + 1)k edges, where each vertex has degree 0 or t + 1. Such trees are called (t + 1)-ary trees. A simple bijection between t-Dyck paths of length (t + 1)k and (t + 1)-ary trees with (t + 1)k edges shows that (t+1)

the number of t-Dyck paths of length (t + 1)k is given by Ck

. As in [Cam02],

given a (t + 1)-ary tree, label each node’s children with U , except for the rightmost child, which should be labeled D. Now traverse the tree in preorder, that is, the order given by a depth-first search where the preference is to go left. This gives a sequence of U ’s and D’s, from which we can create a t-path by replacing each U

93 with an up-step (1, 1) and each D with a down-step (1, −t). There are a total of tk up-steps and k down-steps, so this path is a t-path of length (t + 1)k. Further, each down-step follows the t up-steps associated with its siblings so that the height of the path is never negative. This means the path is actually a t-Dyck path. This map can be reversed, meaning it is a bijection, so that the number of t-Dyck paths (t+1)

of length (t + 1)k is given by Ck

.

Given two t-paths A and B of the same length, we say that A and B are noncrossing t-paths if heightA (x)−heightB (x) ≥ 0 for all x or heightA (x)−heightB (x) ≤ 0 for all x. In the first case, we say A is above B, and in the second case, we say A is below B. Otherwise, we say A and B are crossing. Note that with these definitions, if A and B are the same path, it is correct to say both A is above B and A is below B. An r-tuple of non-crossing paths is a tuple (P1 , P2 , . . . , Pr ) of t-paths such that Pi is below Pi+1 for all 1 ≤ i < r. These tuples are very closely related to other combinatorial objects which have been well-studied. Take an r-tuple of non-crossing paths, (P1 , P2 , . . . , Pr ), and shift up each path Pi so that it begins at (0, (t + 1)(i − 1)). In the case that t = 1, such a configuration of now non-intersecting paths is called a watermelon and has been studied, for example, in [Fis84] and [Fei12]. We will call such configurations t-watermelons, in general. When the paths are bounded below by the x-axis, the watermelon is said to have a wall restriction. It was shown by Desainte-Catherine and Viennot that when t = 1 and the paths are ordinary Dyck paths, the number of r-tuples of non-crossing Dyck paths of length 2k is given by det(A) where Ai,j = Ck+i+j for i, j ∈ 0, . . . , r − 1 [DCV86]. One might hope to be able to generalize this formula by saying that the number of r-tuples of non-crossing t-paths of length (t + 1)k is (t)

(t+1)

equal det(A(t) ) where Ai,j = Ck+i+j for i, j ∈ 0, . . . , r − 1. Unfortunately, this is not true. For example, Figure 5.1 shows the three 2-paths of length 6. It is then easy to see that there are exactly 6 pairs of non-crossing 2-Dyck paths of length 6 while (3) (3) C 3 12 2 C3 det (3) = det = 21. (3) C 12 55 C4 3 We can, however, find another determinantal formula to count the number of non-crossing r-tuples of t-Dyck paths of length (t + 1)k. To this end, let Ai =

94

Figure 5.1: The three 2-paths of length 6. (0, (t + 1)(i − 1)) and Bi = ((t + 1)k, (t + 1)(i − 1)) for i = 1, . . . , r. For any 1 ≤ i, j ≤ r, let Pt (Ai , Bj ) denote the number of t-paths that start at Ai , end at Bj , and stay above the x-axis. Using the standard lattice path involution arguments of Lindstrom [Lin73] and Gessel and Viennot[GV85], we can prove the following theorem. Theorem 9. For any k, r, t ≥ 1, the number of r-tuples of non-crossing t-Dyck paths of length (t + 1)k is equal to det(M (k,r,t) ) where M (r,t,k) = (Pt (Ai , Bj ))i,j=1,...,r . To prove Theorem 9, we will count tuples of non-crossing t-Dyck paths by instead counting t-watermelons with a wall restriction, or families of non-intersecting t-paths. We claim that the number of r-tuples of non-crossing t-Dyck paths is the same as the number of families {P1 , . . . , Pr } of non-intersecting paths, where path Pi begins at Ai = (0, (t + 1)(i − 1)) and ends at Bi = ((t + 1)k, (t + 1)(i − 1)) for i = 1, 2, . . . , r. These can be counted by applying a famous lemma originally proven by Lindstrom [Lin73], but later applied by Gessel and Viennot[GV85], which we summarize here in the language of Aigner [Aig07]. (k,r,t)

Let M (k,r,t) = (mij

(k,r,t)

) be the r × r path matrix where mij

number of directed paths from Ai to Bj . In our case,

(k,r,t) mij

equals the

is the number of

t-paths from (0, (t + 1)(i − 1)) to ((t + 1)k, (t + 1)(j − 1)) not falling below the x-axis, which we denote Pt (Ai , Bj ). A path system P = (σ; P1 , . . . , Pr ) from A to B consists of a permutation σ and r paths Pi : Ai → Bσ(i) for i = 1, . . . , r. We define the sign of P, sign(P), to be the sign of σ. From the usual determinant formula and our definitions, we obtain det(M (k,r,t) ) =

X σ∈Sr

sign(σ)m1σ(1) . . . mrσ(r) =

X P

sign(P)

(5.1)

95 where the sum is over all path systems P = (σ; P1 , . . . , Pr ) from A to B. Our goal is to apply the Lindstrom-Gessel-Viennot method to prove Theorem 9, which gives an involution to cancel all path systems in which two or more paths have a vertex in common. Before we can do this, we need a lemma about t-paths in a path system. Lemma 8. Suppose path Pj is a t-path beginning at (0, (t + 1)(j − 1)), and Pk is a t-path beginning at (0, (t + 1)(k − 1)) where 0 < j < k. Then for all x, heightPk (x) − heightPj (x) is a multiple of t + 1. In particular, if it is ever the case that heightPk (x) − heightPj (x) < 0, it must be the case that there is an integer v < x such that heightPk (v) − heightPj (v) = 0 so that Pk and Pj intersect at vertex (v, heightPk (v)). Moreover, if Pj and Pk do not intersect, the height difference between the paths is always at least t + 1. Proof. Let x be arbitrary. If path Pj has exactly aj up-steps in the first x steps, then it must have x − aj down-steps, each of size t, so that heightPj (x) = (t + 1)(j − 1) + aj − t(x − aj ). Similarly, if path Pk has exactly ak up-steps in the first x steps, then heightPk (x) = (t + 1)(k − 1) + ak − t(x − ak ). Hence

heightPk (x) − heightPj (x) = ((t + 1)(k − 1) + ak − t(x − ak )) − ((t + 1)(j − 1) + aj − t(x − aj )) = (t + 1)(k − j) + ak − aj + ak t − aj t = (t + 1)(k − j + ak − aj ). Therefore, the difference in height between the paths, at any x, is a multiple of t + 1. In particular, if the two paths are non-intersecting, they stay at least t + 1 apart at all times. Also, if it is ever the case that heightPk (x) − heightPj (x) < 0, it must be the case that there is an integer v < x such that heightPk (v)−heightPj (v) = 0 so that Pk and Pj intersect at vertex (v, heightPk (v)). We can now prove Theorem 9 using the Lindstrom-Gessel-Viennot method. Proof. Proof of Theorem 9. Define an involution on our collection of path systems for M as follows. Given a path system P = (σ; P1 , . . . , Pr ) where Pi :

96 Ai → Bσ(i) for i = 1, . . . , r such that at least two paths in the system intersect, find the largest x and then largest y such that two different paths Pi and Pj go through (x, y). In fact, if we consider lattice paths with only up-steps in the (1, 1) direction and down-steps in the (1, −t) direction, this point (x, y) has exactly two paths intersecting. If not, suppose three or more paths meet at (x, y). By the pigeonhole principle, at least two paths must continue in the same direction from (x, y), and there is a point to the right of x at which two or more paths intersect, which contradicts the choice of x. Once we locate this (x, y), call the two paths that meet there Pk and Pl . The involution switches everything to the right of (x, y) in path Pk with everything to the right of (x, y) in the path Pl . This operation is called switching tails. Suppose this operation creates two new paths Pk0 and Pl0 . By switching tails, we have introduced a transposition into the permutation that dictates the endpoints of the paths. This makes the sign of the path system with Pk0 and Pl0 opposite the sign of the path system with Pk and Pl , so when we sum these two path systems in (5.1), they cancel. This tail-switching involution can be used to cancel all path systems except those where no two paths intersect. Now if P = (σ; P1 , . . . , Pr ) is a non-intersecting path system where Pi starts at (0, (t + 1)(i − 1)) for i = 1, . . . , r, then it follows from Lemma 8 that whenever 1 ≤ i < j ≤ r, then Pj must stay above Pi . It then follows that for all 1 ≤ j ≤ r, Pj must end at ((t + 1)k, (t + 1)(j − 1)) so that the permutation σ associated with P is the identity in Sr . Thus, det(M (k,r,t) ) actually gives the number of non-intersecting families {P1 , . . . , Pr } of t-Dyck paths where Pi goes from (0, (t + 1)(i − 1)) to ((t + 1)k, (t + 1)(i − 1)). Moreover, Lemma 8 ensures that for all 1 ≤ i < r and all 0 ≤ x ≤ (t + 1)k, heightPi+1 (x) − heightPi (x) ≥ (t + 1). It follows that that for any 1 ≤ i ≤ r, if Qi is the path that results from Pi by shifting all the vertices of Pi down by (t + 1)(i − 1), then (Q1 , . . . , Qr ) will be a tuple of non-crossing t-paths starting at (0, 0) and ending at ((t + 1)k, 0). This shifting operation is one-to-one because two different families of non-intersecting paths map to two different tuples of non-crossing paths. On the other hand, it is easy to take any tuple (Q1 , . . . , Qr ) of non-crossing t-Dyck paths and turn it into

97 a family of non-intersecting t-Dyck paths by by shifting all the vertices of Qi up by (t + 1)(i − 1) for each i. Thus the number of r-tuples of non-crossing t-Dyck paths of length (t + 1)k is given by det(M (k,r,t) ), which proves Theorem 9.

We should note that in the case t = 1 so that we are counting r-tuples of non-crossing Dyck paths, our matrix M (k,r,t) is not the same as the one given by Desainte-Catherine and Viennot [DCV86]. For example, suppose that we want to count the number of pairs of non-crossing Dyck paths of length 4. Then by Desainte-Catherine and Viennot [DCV86] result, this number is C C 2 5 3 2 det = det = 3. C3 C4 5 14 However, by Theorem 9, this number is given by P ((0, 0), (4, 0)) P ((0, 0), (4, 2)) 2 3 1 1 det(M (2,2,1) ) = det = det = 3. P1 ((0, 2), (4, 0)) P1 ((0, 2), (4, 2)) 3 6 That is, one can easily directly compute that P1 ((0, 0), (4, 0)) = 2, P1 ((0, 2), (4, 0)) = P1 ((0, 0), (4, 2)) = 3, and P1 ((0, 2), (4, 2)) = 6. For Theorem 9 to be useful, one must be able to count the number of tpaths from Ai to Bj not falling below the x-axis. First, for any x1 ≤ x2 , we can compute the numbers Pt ((x1 , y1 ), (x2 , y2 )) by recursion. We have some obvious initial conditions on the numbers Pt ((x1 , y1 ), (x2 , y2 )). That is, 1. Clearly, Pt ((x1 , y1 ), (x2 , y2 )) = 0 if either y1 < 0 or y2 < 0 since all of our t-paths must stay above the x-axis. 2. Since our up-steps are of the form (1, 1), it follows that Pt ((x1 , y1 ), (x2 , y2 )) = 0 if y2 − y1 > x2 − x1 and Pt ((x1 , y1 ), (x2 , y2 )) = 1 if y 2 − y 1 = x2 − x1 . 3. Similarly, since our down-steps are of the form (1, −t), it follows that Pt ((x1 , y1 ), (x2 , y2 )) = 0 if y2 − y1 < −t(x2 − x1 ) and Pt ((x1 , y1 ), (x2 , y2 )) = 1 if y2 − y1 = −t(x2 − x1 ).

98 4. Pt ((x1 , y1 ), (x1 , y2 )) = 0 if y1 6= y2 and Pt ((x1 , y1 ), (x1 , y2 )) = 1 if y1 = y2 . Given these initial conditions, we have the recursion that if x1 , y1 , x2 , y2 ≥ 0 and x1 < x2 , Pt ((x1 , y1 ), (x2 , y2 )) = Pt ((x1 + 1, y1 + 1), (x2 , y2 )) + Pt ((x1 + 1, y1 − t), (x2 , y2 )). One can obtain some general formulas for the numbers Pt (Ai , Bj ) that appear in Theorem 9 by using generating functions. That is, for any a, b ≥ 0, let St (a, b) be the set of t-paths that start at (0, a) and end at some (`, b) which do not go below the x-axis. If P is a t-path of length `, we let wt(P ) = x` and consider the generating function X

Pt,a,b (x) =

wt(P ).

(5.2)

P ∈St (a,b)

We can develop simple recursions for the generating function Pt,a,b (x). First, we must interpret Pt,a,b (x) to be 0 if either a < 0 or b < 0 since we are assuming that all our t-paths stay above the x-axis. Pt,0,0 (x) is the generating function for t-Dyck paths which are counted by the Fuss-Catalan numbers so that  ∞  X (t + 1)k 1 x(t+1)k . Pt,0,0 (x) = tk + 1 k k=0

(5.3)

Next we shall consider three cases. Case 1. 0 = b < a. In this case, find the least x for which the path goes through (x, 0). Then the part of P after x is simply a t-Dyck path. Also, there must be a down-step from (x − 1, t) to (x, 0). Since x was chosen to be minimal, the part of P before x − 1 is a path which does not fall below the line y = 1. By shifting it down one, it is equivalent to a path from (0, a − 1) to (x − 1, t − 1) that does not fall below the x-axis. Therefore,

Pt,a,0 (x) =

X

wt(P )

P ∈St (a,0)

=

X

wt(P1 ) · x · wt(P2 )

P1 ∈St (a−1,t−1),P2 ∈St (0,0)

= xPt,0,0 (x)Pt,a−1,t−1 (x).

(5.4)

99 Case 2. 0 = a < b. Suppose P ∈ St (0, b). Then we can find the greatest x1 for which P has height b − 1. After this point, there must be an up-step followed by a t-Dyck path that starts at (x1 + 1, b), ends at (`, b) for some ` ≥ x1 + 1, and never goes below the line y = b. Then we can find the greatest x2 for which P has height b − 2. This point must be followed by an up-step and a t-Dyck path which goes from (x2 + 1, b − 1) to (x1 , b − 1) without ever going below the line y = b − 1. Continuing on in this way allows us to decompose each path P into a sequence of b + 1 t-Dyck paths joined by b up-steps. This decomposition gives the following formula.

Pt,0,b (x) =

X

wt(P )

P ∈St (0,b)

X

=

wt(P1 ) · x · wt(P2 ) · x · . . . · x · wt(Pb+1 )

(P1 ,...,Pb+1 ),Pi ∈St (0,0) b+1 = xb Pt,0,0 (x).

(5.5)

Case 3. a > 0, b > 0. In this case, we can partition St (a, b) into two sets, one containing the paths that fall below the line y = b, and one containing the paths that do not. If a path P falls below the line y = b, it can be decomposed in the following way. Find the greatest x for which P goes through (x, b − 1). This point must be followed by an up-step and then a t-path which stays above y = b. In other words, P can be broken up into a path P1 from (0, a) to (x, b − 1) followed by an up-step and then a path P2 which is a t-Dyck path, shifted up to stay above y = b. If instead P is a path not falling below the line y = b, it is equivalent to a path from (0, a − b) to (l, 0) by shifting down b units. This allows us to write a recurrence for Pt,a,b .

Pt,a,b (x) =

X

wt(P )

P ∈St (a,b)

=

X

wt(P1 ) · x · wt(P2 ) +

P1 ∈St (a,b−1),P2 ∈St (0,0)

= xPt,0,0 (x)Pt,a,b−1 (x) + Pt,a−b,0 (x).

X

wt(P )

P ∈St (a−b,0)

(5.6)

100 If 0 < a < b, all paths P ∈ St (a, b) fall below the line y = b so that Pt,a,b (x) = xPt,0,0 (x)Pt,a,b−1 (x). In this case, a − b < 0 so that Pt,a−b,0 (x) = 0 and (5.6) still holds. One can verify that, when a < t, the following piecewise definition of Pt,a,b satisfies the recurrences (5.5), (5.4), and (5.6).  at+b at+b+1  Pt,0,0 ((xPt,0,0 )−t−1 + 1)a if b ≥ a   x b X Pt,a,b (x) = (a−j)t+b−j+1  x(a−j)t+b−j Pt,0,0 ((xPt,0,0 )−t−1 + 1)a−j−1 if b < a.   j=0

(5.7) Now for any t, a > 0, we have from (5.6) Pt,a,1 (x) = xPt,0,0 (x)Pt,a,0 (x) + Pt,a−1,0 (x) Pt,a,2 (x) = xPt,0,0 (x)Pt,a,1 (x) + Pt,a−2,0 (x) 2 = x2 Pt,0,0 (x)Pt,a,0 (x) + xPt,0,0 (x)Pt,a−1,0 (x) + Pt,a−2,0 (x)

.. . Pt,a,b (x) =

b X

b−j xb−j Pt,0,0 (x)Pt,a−j,0 (x).

(5.8)

j=0

Substituting a − 1 for a and t − 1 for b in (5.8) gives Pt,a−1,t−1 (x) =

t−1 X

t−1−j xt−1−j Pt,0,0 (x)Pt,a−j−1,0 (x).

(5.9)

j=0

When a ≥ t, we can replace (5.9) in (5.4), which gives an expression for Pt,a,0 . Pt,a,0 (x) = xPt,0,0 (x)Pt,a−1,t−1 (x) t−1 X t−j = xt−j Pt,0,0 (x)Pt,a−j−1,0 (x)

(5.10)

j=0

The formula for Pt,a,0 when a = 0, 1, . . . , t − 1 given by (5.7) allows us to start this recursive formula to calculate Pt,a,0 when a ≥ t. The result can then be used in (5.8) to compute any Pt,a,b (x) when a ≥ t. The number of t-paths from Ai = (0, (t + 1)(i − 1)) to Bj = ((t + 1)k, (t + 1)(j − 1)) not falling below the x-axis is given by the coefficient of x(t+1)k in Pt,(t+1)(i−1),(t+1)(j−1) . If we call this number

101 (k,r,t)

mij

(k,r,t)

, then the determinant of the matrix M (k,r,t) with i, j-entry mij

is the

number of families of r non-intersecting t-paths of length (t + 1)k. (k,r,t)

These matrix entries mij

will be sums of products of binomial coeffi-

cients. Below are listed a few of these entries for which we have calculated explicit formulas. (k,r,t)

=

(k,r,t)

=

(k,r,t)

=

m1,1

m1,2 m2,1

k(t+1) 1 tk+1 k
t+2 k(t+1) k−1 k−2 t   X




l=1

t fl (k) where l     0 fl (k) =

1

   (k,r,t) m2,2

=

k(t+1) 1 tk+1 k




if k < l if k = l

l(t+1)+1 k(t+1) k−l k−l−1




t   X t + gl (k) where l l=1

  0      1 gl (k) =  (k − 1)(t + 1) + 2      l(t+1)+1 k(t+1)
+ (l+1)(t+1)+1 k−l

(k,r,t)

The numbers mi,j

if k > l

k−l−1

k−l−1

if k < l if k = l if k = l + 1 k(t+1) k−l−2




if k > l + 1

, as we vary one of the variables, give rise to many

sequences of integers, some of which can be found in the On-line Encyclopedia of (k,3,2)

Integer Sequences (OEIS) [Slo]. For example, the sequence {m2,3

}k≥1 is A102594

in the OEIS, where it is given an interpretation in terms of non-crossing trees on a (k,3,2)

circle. The sequence {m2,2

}k≥1 is A006692 in the OEIS, which states that these

numbers arise from the self-convolution fourth power of the Fuss-Catalan numbers (3)

Ck that enumerate ternary trees. The sequence of determinants {det(M (2,r,2) )}r≥1 is A000217, the triangular numbers. Similarly, {det(M (2,r,3) )}r≥1 is A000292, the tetrahedral numbers, and as we increase t, we get other binomial coefficients which carry on the pattern.

102

5.2

Enumeration of Generalized Rectangular Shapes Now we are ready to use the results of Section 5.1 to enumerate PBFs.

We will use the notation [n] to denote the set of positive integers {1, 2, . . . , n}. We are interested in determining the number of PBFs with a given shape γ and basement σ, that is |P BF (γ, σ)|. We only consider the case where the basement is the identity permutation, n , and the shape is easily characterized. In particular, we will be interested in shapes that have some repeating pattern of column heights. The simplest result counts the number of PBFs with single-row shapes. Theorem 7. Let γ be any shape such that γi ≤ 1 for all i. Then |P BF (γ, n )| = 1. Proof. There is at least one element of P BF (γ, n ), the PBF P with constant columns. It is clear that this P is a valid PBF that comes from the shift map ρ−1 n , as no element could be placed further left while maintaining that the columns decrease reading up. Suppose there is some other PBF Q 6= P with Q ∈ P BF (γ, n ). Since Q does not have all constant columns, let a be the smallest entry in the first row of Q that is not above the basement entry a. We know a is above some basement entry greater than a, say b, otherwise the column condition for PBFs would be violated. See Figure 5.2. Now the position in the first row above the basement entry a must be empty, otherwise it contains some element (i) less than a, which contradicts the minimality of a, (ii) equal to a, which contradicts the fact that PBFs are row-strict, or (iii) greater than a, which violates the column condition. This means the a in the first row could have been placed further left, above the a in the basement. This means Q is not a result of the shift map ρ−1 n applied to some reverse row-strict tableau, because the shift map places elements in the leftmost position possible. Therefore, Q is not a valid PBF.

103

a ...

a b

Figure 5.2: Q as in the proof of Theorem 7. r r A r r Ar A r Ar

Figure 5.3: 2-path3 (P ) if row 3 of P contains 1, 2, 3, and 5 It is not hard to see that we have actually proven something about the first row of PBFs of all shapes with basement n . Corollary 1. If P is any PBF with basement n , we must have that P (j, 1) = j for all cells (j, 1) in P . Proof. The argument above shows that in the first row of a PBF with basement n , each entry must match the basement entry below in order to be a valid PBF. As PBFs come from shifting reverse row-strict tableaux one row at a time beginning with the bottom row, this result holds for any PBF with basement n . Let ((s)t , 0)k represent the shape with t columns of height s followed by a column of height 0, repeated in sequence k times. Thus, as s and t vary, ((s)t , 0)k ranges over the generalized rectangular shapes of any dimension. Our present goal is to count the number of PBFs of shape ((s)t , 0)k where s ≥ 2 and t, k ≥ 1. Let r be the row number of some row in a PBF P with basement permutation (t+1)k for some t, k. Recall from the introduction that t-pathr (P ) is the t-path of length (t + 1)k defined as follows: the ith step is an up-step if and only if the number i appears in row r of P . For example, if row 3 of a PBF P with basement 6 contains the numbers 1, 2, 3, and 5, then 2-path3 (P ) is the lattice path of length 6 which has its first, second, third, and fifth segments as up-steps, and its fourth and sixth segments as down-steps in the (1, −2) direction, as in Figure 5.3. Lemma 9. The elements a1 < · · · < atk are the elements of a row of a PBF P ∈ P BF (((s)t , 0)k , (t+1)k ) if and only if for all 1 ≤ v ≤ k, |{a1 , . . . , atk } ∩ [(t + 1)v − 1]| ≥ tv.

104 Proof. Let P ∈ P BF (((s)t , 0)k , (t+1)k ) and suppose a1 < · · · < atk are the elements of row r of P . Then for any 1 ≤ v ≤ k, there are tv elements of row r in the first (t + 1)v − 1 columns of P . Since the basement of P is the identity permutation and columns must weakly decrease reading up, |{a1 , . . . , atk } ∩ [(t + 1)v − 1]| ≥ tv. Vice versa, suppose for all 1 ≤ v ≤ k, |{a1 , . . . , atk } ∩ [(t + 1)v − 1]| ≥ tv. This means that we can form a reverse row-strict tableau T of shape (s)tk such that first row of S consists of the numbers equivalent to 1, 2, . . . , t (mod t + 1) that are less than (t + 1)k in decreasing order and the remaining rows of S are atk > atk−1 > · · · > a1 , reading from left to right. It is then easy to check that t k ρ−1 (t+1)k (T ) is a permuted basement filling of shape ((s) , 0) so that a1 < · · · < akt

is a row of a permuted basement filling of shape ((s)t , 0)k . Lemma 10. If row r with elements r1 < · · · < rtk is a row in a PBF P ∈ P BF (((s)t , 0)k , (t+1)k ), then t-pathr (P ) is a t-Dyck path. Proof. Let P ∈ P BF (((s)t , 0)k , (t+1)k ). If r1 < · · · < rtk are the elements in row r of P , then by Lemma 9, for all v ≥ 1, at least tv elements of [(t + 1)v − 1] are among {r1 , . . . , rtk }. Let L denote the lattice path t-pathr (P ). L has up-steps at positions {r1 , . . . , rtk }. Then for all v ≥ 1, at least tv of the first (t + 1)v − 1 steps of L are up-steps. Consider heightL ((t + 1)v − 1) for any v. There are at least tv up-steps and at most v − 1 down-steps in the first (t + 1)v − 1 steps. Since up-steps increase the height by 1 each and down-steps decrease the height by t each, this means heightL ((t + 1)v − 1) ≥ tv − t(v − 1) = t. Since v was arbitrary, heightL (m) ≥ t, where m ≡ t (mod t + 1). This implies, in particular, that the first t steps of L are up-steps. Suppose for a contradiction that there exists some q 6≡ t (mod t + 1) for which heightL (q) < 0. Let v be the integer such that (t + 1)v − 1 < q < (t + 1)(v + 1) − 1. Then q = (t + 1)v + j for some j ∈ {0, 1, . . . , t − 1}. Even if L had all up-steps between x = q and x = (t + 1)(v + 1) − 1, since heightL (q) < 0 and up-steps increase the height by one, we have heightL ((t + 1)(v + 1) − 1) < ((t + 1)(v + 1) − 1) − q = ((t + 1)(v + 1) − 1) − ((t + 1)v + j) = t − j. Since j ≥ 0, this says heightL ((t + 1)(v + 1) − 1) < t which is a contradiction because (t + 1)(v + 1) − 1 ≡ t (mod t + 1), so the height there should be at least t. Thus, it is impossible to have such a q, meaning L never falls below the x-axis. Hence L

3 3 3 3 4

2 4 5 5 6

6 7 7 7

8

(

=

,

(

1 1 1 1 2

105

Figure 5.4: An example of the map 1-Θ3 . is a t-Dyck path. Let t-N CDP r(t+1)k denote the set of all r-tuples (P1 , . . . , Pr ) of non-crossing t-Dyck paths which start at (0, 0) and end at ((t + 1)k, 0) such that for all 1 ≤ i < j ≤ r, Pi is below Pj . For s ≥ 2, given any Q ∈ P BF (((s)t , 0)k , (t+1)k ), let t-Θs (Q) = (t-path2 (Q), . . . , t-paths (Q)). An example of the map 1-Θ3 is given in Figure 5.4. Theorem 8. For all s ≥ 2 and t, k ≥ 1, t-Θs is a bijection from P BF (((s)t , 0)k , (t+1)k ) onto t-N CDP s−1 (t+1)k . Proof. For any Q ∈ P BF (((s)t , 0)k , (t+1)k ), let TQ be the reverse row-strict tableau of shape (s)tk given by ρ(t+1)k (Q) = TQ . First, we show t-Θs (Q) ∈ t-N CDP s−1 (t+1)k . Fix 2 ≤ r < s and let b1 < · · · < btk be the positions of the up-steps in t-pathr (Q) and a1 < · · · < atk be the positions of the up-steps in t-pathr+1 (Q). Then the rth row of TQ is btk > btk−1 > · · · > b1 , reading from left to right, and the (r + 1)st row of TQ is atk > atk−1 > · · · > a1 , reading from left to right. It follows that ai ≤ bi for i = 1, . . . , tk since TQ is weakly decreasing in columns, reading from bottom to top. We claim that this forces t-pathr+1 (Q) to lie above t-pathr (Q) so that (t-path2 (Q), . . . , t-paths (Q)) will be an (s − 1)-tuple of noncrossing t-Dyck paths. If t-pathr (Q) has i up-steps in the first m steps, this means b1 < · · · < bi ≤ m < bi+1 . But since ai ≤ bi , this also means a1 < · · · < ai ≤ m so that t-pathr+1 (Q) has at least i up-steps in the first m steps. So after the mth step,

106 the height of t-pathr+1 (Q) is at least the height of t-pathr (Q). Since m is arbitrary, this is true after each step so that t-pathr+1 (Q) lies above t-pathr (Q). Lemma 10 gives that for each r, t-pathr (Q) is a Dyck path, and we have just shown that these paths are non-crossing, so t-Θs maps to the correct output space t-N CDP s−1 (t+1)k . Next, it is easy to see that t-Θs is one-to-one. Given two PBFs P 6= Q ∈ P BF (((s)t , 0)k , (t+1)k ), there exists a row r such that P and Q have different elements in row r. Then t-pathr (P ) 6= t-pathr (Q)), which means t-Θs (P ) 6= t-Θs (Q). Now suppose that (P1 , . . . , Ps−1 ) ∈ t-N CDP s−1 (t+1)k . Then for any 1 ≤ r < s − 1, let b1 < · · · < btk be the positions of the up-steps in Pr and a1 < · · · < atk be the positions of up-steps in Pr+1 . Since Pr is a t-Dyck path, it must be the case that for all 1 ≤ v ≤ k, |{b1 , . . . , btk } ∩ [(t + 1)v − 1]| ≥ tv. If not, suppose there is some u for which |{b1 , . . . , btk } ∩ [(t + 1)u − 1]| < tu. Then there are at most tu − 1 up-steps and at least u down-steps in the first (t + 1)u − 1 steps of Pr . In other words, heightPr ((t + 1)u − 1)) ≤ (tu − 1) − t(u) = −1, which contradicts that Pr is a t-Dyck path. Then Lemma 9 says that {b1 , . . . , btk } can form the elements of a row of a PBF in P BF (((s)t , 0)k , (t+1)k ). The fact that Pr+1 lies above Pr implies that for any m, we must have at least as many up-steps in the first m steps of Pr+1 than we have up-steps in the first m steps of Pr . This implies that ai ≤ bi for all i. It follows that if T is the tableau of shape (s)k whose first row is (t + 1)s − 1, . . . , (t + 1)(s − 1), (t + 1)(s − 1) − 1, . . . , (t + 1)(s − 2), . . . , t, . . . , 1, reading from left to right, and whose (r + 1)st row consists of the positions of the up-steps of Pr for r = 1, . . . , s − 1, then T is a reverse row-strict tableau of shape (s)tk . But then Q = ρ−1 (t+1)k (T ) will be a permuted basement filling of shape (((s)t , 0))k such that t-Θs (Q) = (P1 , . . . , Ps−1 ). Hence t-Θs is surjective. Thus t-Θs : P BF (((s)t , 0)k , (t+1)k ) → t-N CDP s−1 (t+1)k is a bijection. In the case where t = 1, 1-Dyck paths are just Dyck paths so that we can apply Theorem 8 and the results of Desainte-Catherine and Viennot [DCV86] mentioned in Section 5.1 to prove the following theorem. Theorem 15. For all s ≥ 2 and k ≥ 1, |P BF ((s, 0)k , 2k )| = det(A) where A = (Ck+i+j )i,j=0,...,s−2 .

107 More generally, by Theorem 9 and equation (5.2), we have the following theorem. Theorem 16. If s ≥ 2 and t, k ≥ 1, then |P BF (((s)t , 0)k , (t+1)k )| = det(M (k,s−1,t) ) (k,s−1,t)

where mi,j

is the coefficient of x(t+1)k in Pt,(t+1)(i−1),(t+1)(j−1) for 1 ≤ i, j ≤

s − 1. It follows that |P BF (((2)t , 0)k , (t+1)k )| = use our formulas for

(k,2,t) mi,j

(t+1)k 1 tk+1 k




(t+1)

= Ck

. We can

described in Section 3 to find the initial values of

the sequence {|P BF (((3)t , 0)k , (t+1)k )|}k≥1 . For example, we have computed the following. 1. The initial values of the sequence {|P BF (((3)2 , 0)k , 3k )|}k≥1 are 1, 6, 66, 1001, 18564, 395352, 9315966, 237153345, . . .. 2. The initial values of the sequence {|P BF (((3)3 , 0)k , 4k )|}k≥1 are 1, 10, 200, 5700, 201894, 82888280, 378658800, 18761410800, . . .. 3. The initial values of the sequence {|P BF (((3)4 , 0)k , 5k )|}k≥1 are 1, 15, 475, 21850, 1260630, 84728952, 6357453960, 5184054333975, . . .. None of these sequences appear in the OEIS [Slo]. All of the results so far have shown how to count shapes that are made up of repeating rectangles, separated by single columns of height zero. It is easy to see, however, that shapes made up of repeating rectangles and separated by single columns of any height can be counted in an analogous way. Theorem 10. For all k, s, t, u ≥ 1, |P BF (((s + u)t , u)k , (t+1)k )| = |P BF (((s)t , 0)k , (t+1)k )|. Proof. We give a simple bijection between PBFs P ∈ P BF (((s + u)t , u)k , (t+1)k ) and PBFs Q ∈ P BF (((s)t , 0)k , (t+1)k ). Let Q = R(P ) be the PBF obtained by removing the first u rows of P and shifting all higher rows down, which we call the

108

1 1 1 1 1

2 4 3 2 4 5 2 3 4 5 6 2 3 4 5 6 2 3 4 5 6

→

1 2 4 3 1 2 4 5 1 2 3 4 5 6

Figure 5.5: An example of the map R : P BF (((s + u)t , u)k , (t+1)k ) → P BF (((s)t , 0)k , (t+1)k ). reduction of P . It should be clear that if P is a PBF of shape ((s + u)t , u)k , then rows 1 through u of P all contain (t + 1)k cells. Since the basement of P is (t+1)k , rows 1 through u of P all contain the numbers {1, 2, . . . , (t + 1)k} in increasing order. Removing any number of these rows results in a valid PBF. In particular, if Q = R(P ), then Q ∈ P BF (((s)t , 0)k , (t+1)k ). This shows that the map R sends PBFs of shape ((s+u)t , u)k to those of shape ((s)t , 0)k , as in Figure 5.5. It remains to show that the map is a bijection. Suppose P1 6= P2 ∈ P BF (((s + u)t , u)k , (t+1)k ). We have shown that rows 1 through u of P1 and P2 contain the same elements, {1, 2, . . . , (t + 1)k}, in the same order. They also both have the same basement, (t+1)k . But since P1 6= P2 , there must be some row r > u in which the elements of P1 and P2 differ. Since the elements of row r in P1 and P2 become the elements of row r − u in R(P1 ) and R(P2 ), it must be the case that R(P1 ) 6= R(P2 ). Also, R is onto. Let Q ∈ P BF (((s)t , 0)k , (t+1)k ). Then augment Q by shifting its rows up by u rows and adding, in rows 1 through u, the numbers {1, 2, . . . , (t + 1)k} in increasing order. This creates a PBF P of shape ((s + u)t , u)k that reduces to Q. The above theorem can be used in conjunction with Theorem 16 to find a determinantal formula for PBFs whose shapes are repeated rectangles separated by single columns of any height.

109

5.3

Enumeration of Other Shapes There is another simple class of weak composition shapes γ for which there

is a unique PBF P ∈ P BF (γ, n ). Let stairs = (0, 1, 2, . . . , s − 1) for s ≥ 2 and consider shapes of the form (stairs )k . Then we have the following theorem. Theorem 11. For any s ≥ 2 and k ≥ 1, there is a unique permuted basement filling of shape (stairs )k with basement ks . Proof. If γ = (stairs )k , we claim that the unique permuted basement fillingP ∈ b P BF (γ, ks ) has constant columns, or P (i, j) = i for all cells (i, j) ∈ dg(γ). For example, if s = 4, then the unique permuted basement filling of shape (stair4 )3 is pictured in Figure 5.6 4

1

3

4

2

3

4

2

3

4

8

5

12

7

8

11 12

6

7

8

10 11 12

6

7

8

9 10 11 12

Figure 5.6: The unique permuted basement filling of shape (stair4 )3 . Suppose P is a permuted basement filling of shape (stairs )k . By Corollary 1, we know that P (i, 1) = i for all columns i with a cell in row 1. Now let 1 < r < s − 1 and suppose by strong induction on r that if cell (i, `) is a cell in the diagram of (stairs )k , then P (i, `) = i for all 1 ≤ ` ≤ r. Now consider the elements in row r + 1 of P . In row r, the elements in columns r + 1, . . . , s k c are r + 1, . . . , s, respectively. Since (r + 1, r + 1) 6∈ dg((stair s ) ), it cannot be that an element x ≤ r + 1 lies in row r + 1 of P , since otherwise x could occupy cell (r + 1, r + 1) of P , which would mean that P did not arise from shifting a reverse row-strict tableau. But this means that the elements in row r + 1 of P in columns r + 2 . . . , s must be r + 2, . . . , s, respectively. That is, we must have r +1 < P (r +2, r +1) ≤ r +2 so that P (r +2, r +1) = r +2. But since the elements in each row of P are pairwise distinct, we know that r + 2 < P (r + 3, r + 1) ≤ r + 3 so that P (r+3, r+1) = r+3. Continuing on in this way, we see that P (j, r+1) = j for j = r + 2, . . . , s.

110 k c Next observe that since (s + r + 1, r + 1) 6∈ dg((stair s ) ), then it cannot be

that there is any element between s + 1 and s + r + 1 in row r + 1 of P , since otherwise such an element could occupy cell (s + r + 1, r + 1) of P , which would mean that P did not arise from shifting a reverse row-strict tableau. Thus the elements in cells (s + r + 2, r + 1), . . . , (2s, r + 1) must be greater than s + r + 1. Then we can argue as above that it must be the case that the elements in cells (s + r + 2, r + 1), . . . , (2s, r + 1) must be s + r + 2, . . . , 2s, respectively. The same argument can be applied to the cells (2s + r + 2, r + 1), . . . , (3s, r + 1). That is, since there is no cell (2s + r + 1, r + 1) in P , then it cannot be that there is any element between 2s + 1 and 2s + r + 1 in row r + 1 of P since otherwise such an element could lie on top of the 2s + r + 1 in row r of P which would mean that P did not arise by applying the shift map to a reverse row-strict tableau. This, in turn, will force that the elements in cells (2s + r + 2, r + 1), . . . , (3s, r + 1) must be 2s + r + 2, . . . , 3s, respectively. Continuing on in this way, we see that for all j, the elements in column j of P must be equal to j so that there is only one permuted basement filling of shape (stairs )k . There are a few other special cases in which we have been able to count the number of PBFs of a given shape. The following theorem shows that the number of PBFS of shape ((2)t , 0)k and shape ((2)t−1 , 3, 0)k are the same. Thus in this case, adding an extra cell on top of the right-most column of each rectangle does not change the number of PBFs. Theorem 12. For all t ≥ 2 and k ≥ 1, |P BF (((2)t−1 , 3, 0)k , (t+1)k )| = |P BF (((2)t , 0)k , (t+1)k )|. Proof. Consider any PBF Q ∈ P BF (((2)t−1 , 3, 0)k , 3k ). Since Q has basement (t+1)k , Corollary 1 says row 1 of Q contains all the elements x ≤ (t + 1)k such that x 6≡ 0 mod (t + 1). In particular, we have Q((t + 1)m + j, 1) = (t + 1)m + j for all 0 ≤ m ≤ k − 1 and 1 ≤ j ≤ t. Consider the entry Q((t + 1)m + t, 2) for some m. Call this entry e. For example, if t = 4, then we would have the situation pictured in Figure 5.7.

111

e d 5m+1 5m+2 5m+3 5m+4 5m

5m+1 5m+2 5m+3 5m+4

Figure 5.7: If t = 4 and e < 5m + 4, then d > e Since Q((t + 1)m + t, 1) = (t + 1)m + t and columns weakly decrease reading up, e = Q((t + 1)m + t, 2) ≤ (t + 1)m + t. Suppose for a contradiction that e < (t + 1)m + t, and let d be the number immediately left of e, which is Q((t + 1)m + t − 1, 2). Then d > e because if d < e then e would have been placed before d by the shift map, and would have ended up in column (t + 1)m + t − 1 because e ≤ (t + 1)m + t − 1 = Q((t + 1)m + t − 1, 1). But this did not happen, so we must have d > e, or Q((t + 1)m + t − 1, 2) > Q((t + 1)m + t, 2). Now consider the entry Q((t + 1)m + t, 3). Since Q((t + 1)m + t, 3) is above e, it must be the case that Q((t + 1)m + t, 3) ≤ e. But then Q((t + 1)m + t, 3) ≤ e < d so Q((t + 1)m + t, 3) could actually have been placed further left in column (t + 1)m + t − 1. Thus, if Q((t + 1)m + t, 2) < (t + 1)m + t, we conclude that Q is not a valid PBF, therefore it must be true that Q((t + 1)m + 2, t) = (t + 1)m + t. Since m was arbitrary, this is true for all 0 ≤ m ≤ k − 1. At this point, if P is a PBF of shape ((2)t−1 , 3, 0)k , there are tk elements that are not completely determined. That is, let FP denote the collection of elements occupying cells of the form ((t + 1)m + t, 3) plus the elements occupying cells of the form ((t + 1)m + j, 2) where 1 ≤ j ≤ t − 1. We claim that the elements of FP are pairwise distinct. That is, since P is a PBF, we know that all the elements in row 2 are pairwise distinct and all the element in row 3 are pairwise distinct so that the only possible way we could have repeated elements among the elements of FP is if there is an x = P ((t + 1)r + t, 3) = P ((t + 1)s + j, 2) for some 0 ≤ r, s ≤ k − 1 and 1 ≤ j ≤ t − 1. But the B-increasing condition on PBFs ensures P ((t + 1)s + j, 2) < P ((t + 1)r + t, 3) = x if s ≤ r and 1 ≤ j ≤ t − 1. Thus we must assume that s > r. In that case, we know that x 6= (t + 1)r + t since P (t + 1)r + t, 2) = (t + 1)r + t and the elements in row 2 of P must be pairwise distinct. But then x < (t + 1)r + t since x = P ((t + 1)r + t, 3) ≤ P ((t + 1)r + t, 2).

112 Also, x > P ((t + 1)r + t − 1, 2) because the shift map says that the x in row 3 sits in the leftmost position possible without violating the column condition and cell ((t + 1)r + t − 1, 3) is empty. However, this means that when the shift map was applied to row 2, x was placed before P ((t + 1)r + t − 1, 2). Since x ≤ P ((t + 1)r + t − 1, 1) = (t + 1)r + t − 1, x should have been placed in cell ((t + 1)r + t − 1, 2) or further left. Since this did not happen, P was not created from the shift map, and thus is not a valid PBF. This proves that the elements of FP are pairwise distinct. Since all the elements of FP lie above the first row of P , it follows that there is a reverse row-strict tableau TP of shape (2)kt whose first row consists of the elements of the form (t+1)m+j where 0 ≤ m ≤ k −1 and 1 ≤ j ≤ t and whose second row consists of the elements in FP . It then follows that QP = ρ−1 (t+1)k (TP ) is a PBF of shape ((2)t , 0)k . The correspondence P → QP defines a map Γt,k : P BF (((2)t−1 , 3, 0)k , (t+1)k ) → P BF (((2)t , 0)k , (t+1)k ). To show that Γt,k is a bijection, we must show that given any Q ∈ P BF (((2)t , 0)k , (t+1)k ), there is a unique P ∈ P BF (((2)t−1 , 3, 0)k , (t+1)k ) such that Γt,k (P ) = Q. We know that P must satisfy the following conditions. (1) P ((t + 1)m + j, 1) = (t + 1)m + j for all 0 ≤ m ≤ k − 1 and 1 ≤ j ≤ t. (2) P ((t + 1)m + t, 2) = (t + 1)m + t for all 0 ≤ m ≤ k − 1. (3) The elements in any column must be weakly decreasing, reading from bottom to top. (4) By the B-increasing condition for PBFs, it must be that for any 0 ≤ m ≤ k − 1, all the elements in columns of height 2 which lie to left of P ((t+1)m+t, 3) must be strictly less that P ((t + 1)m + t, 3). (5) Each row of P must have been created by the shift map. Suppose that the elements in row 2 of Q are a1 < · · · < atk . We will process the elements atk > · · · > a1 one at a time to construct P , showing that our decisions are completely forced by conditions (1) through (5). It may be helpful to refer to an example of this process, illustrated in Figure 5.8. First atk must go

113 10

9

8

6

5

4

3

2

1

11

10

9

7

6

5

3

2

1

Q=

6 (5) 1

2

5

3

5

6

3

5

4

7

9

3

5

6

7

9 10 11

3

5

4

7

9

3

5

6

7

9 10 11

10 3

(1) 1

2

3

5

6

7

11

7

9 10 11

7

9

1

2

3

5

6

7

9 10 11

7

9

7

9 10 11

11

1

2

3 (7) 1

2

10 3

(3) 1

2

3

5

6

6 3

(4) 1

2

3

5

6

8 11

9

8 11

7

9 10 11

9

7

9 10 11

8 11

10

6

8 11

10

3

5

4

7

9

1

2

3

5

6

7

9 10 11

1

2

3

5

4

7

9

1

2

3

5

6

7

9 10 11

(8)

3 (9)

10

6

3

8 11

2

10

7

7

6 (6)

10 3

(2)

10

3

6

8 11

10 8 11

Figure 5.8: An example of the Γ−1 3,3 map. in row 3 of P since if we placed it in a column of height 2 in P , we would get a violation of the B-increasing condition with the element in cell ((t + 1)(k − 1) + t, 3) of P . Since we know the elements below the cells in row 3 of P , we then must place atk as far left as possible in row 3 consistent with the fact that the columns of P must weakly decrease, reading from bottom to top. Now suppose by induction that we have placed atk > · · · > ai+1 consistent with (1) through (5) and that all our choices up to this point were forced. Now consider ai = (t + 1)m + j where 0 ≤ m ≤ k − 1 and 0 ≤ j ≤ t. We have three cases. Case 1. j = t. Then ai cannot go in row 2 of P since the elements in row 2 of P must be pairwise distinct and we know that P ((t + 1)m + t, 2) = (t + 1)m + t. Up to this point, there can be no elements in row 3 of the form (t + 1)r + t where r ≤ m since all these elements are less than or equal to ai and we have only placed elements which are bigger than or equal to ai . Thus to be consistent with the shift map, we must place ai in cell ((t + 1)m + t, 3). Case 2. 1 ≤ j < t. Note that if we were to place ai in row 2 of P , then the only place we could put it

114 is in cell ((t + 1)m + j, 2). That is, cell ((t + 1)m + j, 2) must be unoccupied since we have only processed elements larger than (t + 1)m + j and none of those can sit in cell ((t + 1)m + j, 2) since P ((t + 1)m + j, 1) = (t + 1)m + j. Additionally, ai is too large to occupy any cell left of ((t + 1)m + j, 2). Now if there is a currently unoccupied cell in row 3 that is to the right of cell ((t + 1)m + j, 2), then we must put ai in the left-most possible such cell of the form ((t + 1)s + t, 3) where s ≥ m. That is, ai cannot go in any of the cells ((t+1)r+t, 3) where r < m because placing ai in such a cell would violate the column condition. If we do not place ai in one of the unoccupied cells ((t + 1)s + t, 3) where s ≥ m, then we will be forced to have P ((t + 1)s + t, 3) < P ((t + 1)m + j, 2) = ai , which would violate the B-increasing condition. However, if all the cells in row 3 to the right of cell ((t + 1)m + j, 2) are currently occupied, then we must place ai in cell ((t + 1)m + j, 2). Thus our placement of ai is completely forced in this case. Case 3. j = 0. Consider two cases. First, suppose P ((t + 1)m + 1, 2) = (t + 1)m + 1. As we saw above in case 2, the only way (t + 1)m + 1 ends up in row 2 of P is if all the cells of row 3 to the right of column (t + 1)m + 1 are filled with elements larger than (t + 1)m + 1. If this is true, then we must place ai in row 2 because it is too large to be placed in the unfilled positions in row 3, which are all left of column (t + 1)m + 1. Since there is at least one available position where ai can be placed without violating the column condition and we know all the unfilled positions in row 3 would cause a column condition violation, then there is a unique position in row 2 where ai can be placed to be consistent with the shift map. Second, suppose that cell ((t + 1)m + 1, 2) is not currently filled. In this case, proceed as in case 2 above. That is, if all the cells in row 3 to the right of column (t + 1)m + 1 have been filled in, we must place ai in cell ((t + 1)m + 1, 2). Otherwise, we must place ai in the leftmost available position in row 3 to the right of column (t + 1)m + 1. Thus all of our choices are forced and we will construct a PBF P of shape ((2)t−1 , 3, 0)k such that Γt,k (P ) = Q. Experimental data suggests that for all s ≥ 2. |P BF ((s)t−1 , s + 1, 0)k , (t+1)k )| = |P BF ((s)t , 0)k , (t+1)k )|.

115 Theorem 12 handles the case s = 2. Theorem 12 strongly suggests that there is no analogue of the hook-content formula for PBFs. That is, it provides an example of two different shapes γ and δ whose diagrams consist of a different number of cells, and yet |P BF (γ, n )| = |P BF (δ, n )|. Similarly, |P BF (ν, n )| = 1 for any one-row shape ν by Theorem 7. This work leads to a large number of open problems. One could attempt to determine the cardinality of P BF (γ, n ) where γ is some other shape not considered here. Similarly, one could attempt to determine the cardinality of P BF (γ, σ) for other basements σ, which is a considerably more difficult problem. In some special cases, we have been able to count the number of PBFs of simple shapes with other basements. For example, for any s ≥ 2, k ≥ 1, consider the permutation in oneline-notation σs = (s)(s−1) . . . (1)(2s)(2s−1) . . . (s+1)(3s)(3s−1) . . . (2s+1) . . . ∈ Ssk . If γ = (1, (0)s−1 )k , then there are sk elements of P BF (γ, σ). This is easy to see, as there are k non-basement cells in the diagram of γ and each cell has s possible entries. That is, cell (js, 1) can be filled with any one of {(j − 1)s + 1, (j − 1)s + 2, . . . , js − 1, js}. In general, however, it still remains an open question to count the number of PBFs of a given shape and basement. It would also be useful to determine an interpretation for these enumerative results in terms of representation theory. The content of Chapter 5 is currently being prepared for submission for publication. Remmel, Jeffrey; Tiefenbruck, Janine LoBue. The dissertation author is an author of this material.

Chapter 6 Bruhat Order

116

117 In this chapter, we prove that the decomposition of a Schur function into generalized Demazure atoms with basement σ is a refinement of the decomposition into generalized Demazure atoms with basement τ exactly when σ < τ in the weak Bruhat order. The main theorem of this section follows. Theorem 13. Given two distinct permutations σ and τ in Sn , σ < τ in weak Bruhat order if and only if for all weak composition shapes γ = (γ1 , γ2 , ..., γn ), X bτ = bσ . there exists a set of shapes Sγ such that E E γ

δ

δ∈Sγ

The proof of this theorem uses a result of Haglund, Mason, and Remmel about the result of inserting words into empty fillings [HMR12]. Suppose we are given a word w = w1 . . . wk and σ ∈ Sn . Let E σ be the empty permuted basement filling with basement σ and let w → E σ = wk → (· · · (w2 → (w1 → E σ )) · · · ). Then, the following theorem comes from [HMR12]. Theorem 17. Suppose the letters of a word w are at most n and let σ ∈ Sn with σi < σi+1 . Let F σ = w → E σ and let γ = (γ1 , . . . , γn ) be the shape of F σ . Similarly, if si is the adjacent transposition (i, i + 1), let F σsi = w → E σsi and let δ = (δ1 , . . . , δn ) be the shape of F σsi . Then (i) {γi , γi+1 } = {δi , δi+1 } and δi ≥ δi+1 , (ii) F σsi (i, j) > F σsi (i + 1, j), for j ≤ δi where we let F σsi (i+1, j) = 0 if (i+1, j) is not a cell in F σsi , (iii) F σsi (j, k) = F σ (j, k) for j 6= i, i + 1 so that γj = δj for all j 6= i, i + 1, and (iv) for all j, {F σsi (i, j), F σsi (i + 1, j)} = {F σ (i, j), F σ (i + 1, j)}. In particular, Theorem 17 says that if we start with a word w and insert it separately into E σ and E σsi , the resulting PBFs have the same set of entries in each row. Note that every permutation can be obtained from the identity n by applying a sequence of adjacent transpositions, each of which increase the number

118 of inversions. Then one consequence of this theorem is that the PBFs w → E n and w → E σ have the same row sets for all σ ∈ Sn . Further, Theorem 17 says exactly how to change the entries of a PBF with each transposition that is applied to the basement, so that w → E σ is completely determined by w → E n . Thus, the row sets and the basement completely determine a PBF. If we are given any two permutations σ and τ , the PBFs w → E σ and w → E τ have the same row sets, since they both have the same row sets as w → E n . If we take τ to be n , then the row sets of any PBF w → E σ can be determined by computing w → E n . We know that any word inserted into E n yields a reverse row-strict tableau T . The rows of T can then be used to create the PBF w → E σ in a unique way, since the row sets and basement determine a permuted basement filling. The inverse shift map ρ−1 σ gives a way of creating this PBF w → E σ with basement σ whose entries in any row are the same as those in T . Since every permuted basement filling comes from inserting some word into an empty PBF E σ , all PBFs with basement σ can be determined by applying the map ρ−1 σ to some reverse row-strict tableau. The shift map ρσ associates a PBF that has basement σ with a reverse row-strict tableau that has the same set of entries in each row. The inverse shift associates a reverse row-strict tableau with a PBF of basement σ that map ρ−1 τ has the same set of entries in each row. Thus by composing them, we can create a map ρσ→τ = ρ−1 τ ◦ ρσ that takes a PBF with basement σ and shifts it to a PBF with basement τ . Theorem 17 deals with the case where σi < σi+1 and τ = σsi . The theorem says that in this case, when we take a PBF of shape δ = (δ1 , δ2 , . . . , δn ) with basement σ and shift it to basement τ , the resulting PBF is of shape (δ1 , . . . , δi−1 , max(δi , δi+1 ), min(δi , δi+1 ), δi+2 , . . . , δn ). That is, the height of all columns except possibly columns i and i + 1 are not changed, but δi and δi+1 must be arranged in decreasing order to create a shape that is compatible with basement τ . Now, we use these results to prove Theorem 13. Suppose that σ, τ ∈ Sn and σ < τ in the weak Bruhat order. It suffices to prove the result in the case where τ covers σ. That is, suppose τ = σsi where σi < σi+1 and si is the

119 transposition (i, i + 1). Fix any weak composition shape γ = (γ1 , γ2 , . . . , γn ). b τ to be nonzero, it must be true that γi ≥ γi+1 . Since τi > τi+1 , in order for E γ That is, γ must be τ -compatible. Define a weak composition shape γ˜ = γsi = (γ1 , . . . , γi−1 , γi+1 , γi , γi+2 , . . . , γn ). bτ = E bσ + E bσ . E γ

γ

First, we claim that Sγ = {γ, γ˜ }, so that

γ ˜

Given F ∈ P BF (γ, σ), we can apply the map ρσ→τ and obtain some F 0 ∈ P BF (γ, τ ), according to [HMR12]. Likewise, given G ∈ P BF (˜ γ , σ), applying ρσ→τ b σ +E bσ ≤ E bτ . yields some G0 ∈ P BF (γ, τ ). Since ρσ→τ preserves weights, this says E γ

0

γ ˜

γ

0

Conversely, given any PBF H ∈ P BF (γ, τ ), it must be that ρτ →σ (H ) is a PBF H of shape γ or γ˜ . If not, say H is of shape α 6= γ, γ˜ . Then since ρσ→τ (H) = H 0 , we know that H 0 has shape (α1 , ..., αi−1 , max(αi , αi+1 ), min(αi , αi+1 ), αi+2 , ..., αn ). But if α 6= γ, γ˜ , then this shape cannot equal γ, which is a contradiction because bγτ ≤ E bγσ + E bγ˜σ , H 0 ∈ P BF (γ, τ ). Thus, the shape of H is either γ or γ˜ . This says E so equality holds. Now, suppose σ ≮ τ in the weak Bruhat order. Either σ > τ or they are incomparable. Suppose first that σ > τ . It suffices to consider the case where σ covers τ . Assume by way of contradiction that for all weak composition shapes γ, X b σ . Since some elements of bτ = E there exists a set of shapes Sγ such that E δ γ δ∈Sγ

P BF (γ, τ ) map to elements of P BF (γ, σ) under ρτ →σ , one element of Sγ should bγσ = E bγτ + E bγ˜τ . be γ itself. From what was just shown, since σ > τ , we know, E Thus, bτ = E γ

X

bσ E δ

δ∈Sγ

bσ + = E γ

X

bσ E δ

δ∈Sγ ,δ6=γ

bτ + E bτ + = E γ γ ˜

X

bσ . E δ

δ∈Sγ ,δ6=γ

bτ + This leads to a contradiction because E γ ˜

X

b σ is nonzero. E δ

δ∈Sγ ,δ6=γ

Finally, suppose σ and τ are incomparable in the weak Bruhat order. Since σ, τ > n , for any shape γ, we know that there exists a set of shapes Mγ such that

120 bσ = E γ

X

bτ = b n and a set of shapes Tγ such that E E γ δ

δ∈Mγ

X

b n . We need to show E δ

δ∈Tγ

bτ = that there exists a shape γ for which there is no set Sγ such that E γ

X

bσ . E δ

δ∈Sγ

To do this, we will construct a particular γ for which Mγ − Tγ and Tγ − Mγ are nonempty. In particular, given two incomparable permutations σ and τ , we will construct two PBFs of basement σ having the same shape that map under ρσ→τ to PBFs of two different shapes with basement τ , as well as two PBFs of basement τ having the same shape that map under ρτ →σ to PBFs of two different shapes with basement σ. This will allow us to conclude that there is a γ such that for every X bγτ 6= bδσ . set Sγ , E E δ∈Sγ

Our goal is then to construct two PBFs, A and B, of basement σ having the same shape that map under ρσ→τ to two PBFs, A0 and B 0 , of different shapes with basement τ . Since τ and σ are incomparable, σ has some inversion that τ does not, and vice versa. Choose k > 0 to be minimal such that there exists an i with (i, i + k) ∈ Inv(σ) but (i, i + k) 6∈ Inv(σ). Then fix i to be minimal. Thus, each pair of incomparable permutations in Sn is associated with a unique i and k. For example, if τ = 23765481 and σ = 12763485, then the inversions of σ that are not inversions of τ are (3, 7), (3, 6), and (5, 8). Choosing k to be minimal means k = 3, and then choosing i to be minimal means i = 3, which corresponds to the inversion (3, 6) in σ. Now we will define a weak composition shape γ(τ, σ) = (γ1 , γ2 , . . . , γn ). Let R(τ, σ) be the sequence of elements in σ that are in the interval [i, i + k], in the same order as they appear in σ. For example, if τ = 23765481 and σ = 12763485 as before, then R(τ, σ) = (6, 3, 4, 5) since these are the elements in the interval [3, 6] in the order they appear in σ. Now for j = 1, . . . , n, define γj = σj if σj > i + k or σj < i. Otherwise, if σj ∈ [i, i + k], then let pj be the position of σj in R(τ, σ), indexed starting at 1, and define γj = i + k + 1 − pj . Consider the augmented diagram of shape γ, where we fill in the basement with the permutation σ. Then the height of column j matches the basement entry σj , so long as this basement entry is outside the interval [i, i + k]. For all basement entries inside the interval [i, i + k], the column heights are i + k, i + k − 1, . . . , i + 1, i, reading from left to

121

1 2 7 6 3 4 8 5

Figure 6.1: The shape γ(τ, σ) for τ = 23765481 and σ = 12763485. right. For example, when τ = 23765481 and σ = 12763485, the corresponding shape γ is pictured in Figure 6.1, where the columns arising from basement entries in [i, i + k] are shaded. First, observe some simple facts about the shapes γ(τ, σ) that result from this construction. Lemma 11. For all incomparable pairs of permutations τ and σ in Sn , λ(α(γ(τ, σ))) = (n, n − 1, . . . , 2, 1). Proof. This is apparent because γ(τ, σ) contains one part of size i, for all 1 ≤ i ≤ n. Thus, collapsing and rearranging γ(τ, σ) yields the staircase shape (n, n − 1, . . . , 1). Lemma 12. For all incomparable pairs of permutations τ and σ in Sn , γ(τ, σ) is a σ-compatible weak composition shape. Proof. We must show that if a < b and σa > σb , then γa ≥ γb . Suppose there is such a pair a, b. Then there are three cases. (1) σa , σb ∈ [i, i + k] In this case, both γa = i + k + 1 − pa and γb = i + k + 1 − pb . Then γa > γb because a < b means that pa < pb . (2) σa ∈ [i, i + k], σb < i In this case, γa = i + k + 1 − pa ∈ [i, i + k] and γb = σb < i. Then γa > γb .

122

7 7 A 77 7 2 7 1 2 7 σ= 1 2 7 7 7 B 77 7 2 7 1 2 7 σ= 1 2 7

6 6 6 6 6 6 6

3 6 6 6 6 6 6

3 3 3 3 3 3

3 3 3 3 3 3

4 4 4 4 4

8 8 8 8 8 8 8 8 8

4 4 4 4 4

8 8 8 8 8 8 8 8 8

5 5 5 5

5 5 5 5

3 3 3 2 3 2 3 2 3

7 7 7 7 7 7 7 7

3 3 3 3 2 3 2 3 2 3

7 7 7 7 7 7 7 7

6 6 6 6 6 6 6

6 6 6 6 6 6

4 5 5 5 5

4 5 5 5 5

4 4 4 4

8 8 8 8 8 8 8 8 1 8 1

4 4 4 4

8 8 8 8 8 8 8 8 1 8 1

A' =τ

B' =τ

Figure 6.2: The PBFs A and B in basement σ = 12763485, and when shifted to basement τ = 23765481. (3) σa > i + k, σb ∈ [i, i + k] In this case, γa = σa > i + k and γb = i + k + 1 − pb ∈ [i, i + k]. Then γa > γb . Thus, γ(τ, σ) is σ-compatible. Construct a filling of the augmented diagram of shape γ by placing σ as the basement permutation and filling the cells with entries that are constant in columns. That is, the basement permutation σ completely determines the rest of the filling, as in the top left of Figure 6.2. Call this filling A. We claim that A is a permuted basement filling. To see this, we prove a more general result. Theorem 18. Any filling F of a σ-compatible shape having constant columns and basement σ is a permuted basement filling. Proof. Clearly, the basement entries are σ1 , . . . , σn reading from left to right and the columns are weakly increasing from top to bottom. We must show that all type A triples are inversion triples and that F satisfies the B-increasing condition. Then the fact that all type B triples are inversion triples is automatic. To this end, b suppose x < y, γx ≥ γy , and dg(γ) contains the three cells a = (x, k), b = (y, k),

123 and c = (x, k − 1). Then F (c) = F (a) because F has constant columns. At the same time, F (b) 6= F (a) because F (a) = σx , F (b) = σy , and σa 6= σb . Then, either F (a) = F (c) < F (b) or F (b) < F (a) = F (c), which means a, b, and c form an inversion triple. Now suppose for a contradiction to the B-increasing condition that there is some x < y, γx < γy , and 1 ≤ k ≤ γx + 1 such that if a = (x, k − 1) and b = (y, k), then F (a) > F (b). Then since F has constant columns, F (a) = σx and F (b) = σy . Then F (a) > F (b) implies that σx > σy , but we have x < y and γx < γy , which means γ is not σ-compatible. This is a contradiction, so F satisfies the B-increasing condition. By Lemma 12 and Theorem 18, A is a PBF. Define another filling B by changing only one entry in A. A has a column that is filled entirely with i + k entries. Change the topmost entry in this column to an i in B, but leave all other entries the same, as in Figure 6.2. We claim that B is also a PBF. Lemma 13. The filling B is a permuted basement filling. Proof. Clearly, the basement entries are σ1 , . . . , σn reading from left to right and the columns are still weakly increasing from top to bottom, since we only made the highest entry in some column even smaller. Let u = (z, w) be the cell whose entry was changed from an i + k in A to an i in B. Any type A triple not involving u was an inversion triple in A, so is still an inversion triple in B. If u is involved in some type A triple, note that u cannot play the role of cell a, because u is the highest cell in its column, and no cell at the top of a column has some cell b which is both in the same row and part of a column that is further to the right and of strictly smaller height. If u is involved in a type A triple, it also cannot play the role of cell c because this requires it to have a cell above it, which is not the case. So if u is involved in some type A triple, it means that there exists some x < z with γx > γz . Then cells a = (x, w), u = (z, w), and c = (x, w − 1) form a type A triple. Since a and c are in the same column, B(a) = B(c), and we know B(u) = i. Further, B(a) 6= i because (i, i + k) ∈ Inv(σ), which means the column of B containing all i entries is to the right of column z, whereas column x is to the left of column z.

124 Thus either B(a) = B(c) < B(u) or B(u) < B(a) = B(c), which means that cells a, u, and c form an inversion triple. Any pair of cells satisfying the B-increasing condition in A still satisfies the condition in B, unless possibly if one of those cells was u = (z, w). If there is some y > z with γy > γz , then consider the relative sizes of the entries of u and some cell b = (y, w + 1). Since A(u) = i + k and we know A satisfies the B-increasing condition, this means i + k < A(b). But A(b) = B(b) because cell b was left unchanged, and B(u) = i, so B(u) < i + k < B(b). Thus, any such pair of cells u, b satisfies the B-increasing condition. Now if there is some x < z with γx < γz , consider the relative sizes of the entries of u and some cell a = (x, w − 1). Suppose for a contradiction that B(a) > B(u). Since column x is constant, we know B(a) = σx . If σx > i + k, this would mean the shape γ was not σ-compatible, but we know from Lemma 12 that this is not the case. This means B(a) < i + k. Also, since B(a) > B(u) = i, this means B(a) ∈ [i, i + k]. Since x < z and column heights in this interval are assigned to be decreasing from left to right, this means γx > γz , which is a contradiction. Thus, B satisfies the B-increasing condition. We have shown how to construct two PBFs A and B with the same shape γ and basement σ. Our next goal is to show that when we shift A and B to basement τ , the resulting PBFs A0 and B 0 have different shapes, as in Figure 6.2. First, we need some lemmas about the basement permutation σ. Lemma 14. Suppose τ, σ ∈ Sn are incomparable in the weak Bruhat order. Let k > 0 to be minimal such that there exists an i with (i, i + k) ∈ Inv(σ) but (i, i + k) 6∈ Inv(σ). Then choose i to be minimal. If 0 < ` < k, then it is not the case that σ −1 (i + k) < σ −1 (i + `) < σ −1 (i). That is, i + ` does not appear between i + k and i in σ. Proof. Suppose for a contradiction that for some 0 < ` < k, σ −1 (i + k) < σ −1 (i + `) < σ −1 (i). Then σ −1 (i + `) < σ −1 (i) means (i, i + `) ∈ Inv(σ). Since ` < k and k was chosen to be minimal, this means (i, i + `) ∈ Inv(τ ). Then in τ , i + ` appears before i which appears before i + k. In particular, (i + `, i + k) 6∈ Inv(τ ). Since σ −1 (i + k) < σ −1 (i + `), we have (i + `, i + k) ∈ Inv(σ). Since (i + `, i + k) is an inversion in σ but not τ and k − ` < k, this contradicts the minimality of k.

125 This means in particular that in R(τ, σ), i + k appears immediately before k. Thus, the height of the column with basement entry i + k is exactly one more than the height of the column with basement entry i, in both A and B. Lemma 15. Suppose τ, σ ∈ Sn are incomparable in the weak Bruhat order. Let k > 0 be minimal such that there exists an i with (i, i + k) ∈ Inv(σ) but (i, i + k) 6∈ Inv(σ). Then choose i to be minimal. If 0 < ` < k and σ −1 (i + `) < σ −1 (i + k), then τ −1 (i + `) < τ −1 (i). Proof. Suppose σ −1 (i + `) < σ −1 (i + k) so that i + ` appears before i + k in σ. Then since (i, i + k) ∈ Inv(σ), we know (i, i + l) ∈ Inv(σ). Then it must be the case that (i, i + l) ∈ Inv(τ ) otherwise k would not be minimal. This means τ −1 (i + `) < τ −1 (i). The next theorem shows what happens when we shift A and B to basement τ by applying the map ρσ→τ . Let A0 = ρσ→τ (A) and B 0 = ρσ→τ (B). Theorem 19. If sh(D) gives the shape of a diagram D, then sh(A0 ) 6= sh(B 0 ). Proof. First, notice that by Lemma 11, ρσ (A) = ρσ (B) = (n, n − 1, . . . , 2, 1). Further, if u = (z, w) is the cell with different entries in A and B, then in all rows j < w, A and B have exactly the same set of entries. Then the map ρ−1 τ , which shifts to basement τ one row at a time, produces the same result after shifting rows {1, . . . , w − 1} in A and B. Consider the set of elements in row w − 1 of A. First, i + k is one of these elements, since the height of the column with basement entry i + k is w. If i + k < a ≤ n, then the element a appears in row w − 1 because any column containing such an element a has height a > i+k ≥ w. Similarly, if 1 ≤ a < i, then the element a does not appear in row w − 1 because any column containing such an a has height a < i ≤ w − 1. Now, if a ∈ [i, i + k] and σ −1 (a) < σ −1 (i + k), this means the column containing a’s is left of column z, which we know has height w. Then since column heights for elements in the range [i, i + k] are decreasing from left to right, we know that any column containing such an element a has height greater than w. If a ∈ [i, i + k] and σ −1 (a) > σ −1 (i + k), this means the column

126 containing a’s is to the right of column z, and all such columns have height less than w. In order to have a height of w − 1, a must appear immediately after i + k in R(τ, σ). But we know from Lemma 14 that this means a = i. Thus, we have shown that row w − 1 of A contains exactly the elements of {i + k, . . . , n} plus the elements of {i, . . . , i + k − 1} that come before i + k in σ, plus the integer i. By the same argument, we can show that row w of A contains all the elements of {i + k, . . . , n} plus the elements of {i, . . . , i + k − 1} that come before i + k in σ. But row w of A does not contain i because the column containing i’s in A has height w − 1. Note that A and B have the same set of entries in row w − 1, but not in row w. In particular, row w of B contains all the elements of {i + k + 1, . . . , n} plus the elements of {i, . . . , i + k − 1} that come before i + k in σ, plus the integer i. This means that when the map ρσ→τ is applied to row w of A, the element i + k will be placed above the cell in row w − 1 containing i + k. This is true because the number of elements at least as big as i+k in row w is |{i+k, . . . , n}| = n − i − k + 1, and the number of elements at least as big as i + k in row w − 1 is also |{i + k, . . . , n}| = n − i − k + 1. Every such element in row w must be placed above another such element in row w − 1 since columns must weakly increase from top to bottom. Since i + k is the smallest of {i + k, . . . , n}, and therefore the last of those elements to be placed by the shift map, there is only one available position for i + k. We know this available position is directly above i + k because every other element in {i + k, . . . , n} is too large to be placed there while maintaining weakly increasing columns. We claim that when the map ρσ→τ is applied to row w of B, no element is placed above the cell in row w − 1 containing i + k. This will mean that A0 and B 0 have different shapes. In row w of B, the only elements that are small enough to be placed above a cell containing i + k are the elements of {i, . . . , i + k − 1} that come before i + k in σ, plus the integer i. However, by Lemma 15, any element of {i, . . . , i + k − 1} that comes before i + k in σ also comes before i + k in τ . Thus, in row w − 1 of B, the columns containing these elements are all left of the column containing i + k. So in row w of B, each element of {i, . . . , i + k − 1}

127 has some possible location where it could be placed to the left of the column with i + k. Since ρσ→τ places elements as far left as possible, and there is at least one available position left of the column with i + k, we know that no such element will be placed above i + k. Additionally, the element i will not be placed in column i + k because there is another position further to the left where it could be placed. That is, since (i, i + k) 6∈ Inv(τ ), the column containing i in row w − 1 is left of the column containing i + k. This proves that A and B have different shapes when shifted to basement τ . Since τ and σ are incomparable in the weak Bruhat order, we also know that τ has some inversion that σ does not. The same construction, with the roles of τ and σ interchanged, produces two PBFs of the same shape in basement τ that have different shapes when shifted to basement σ. This finishes the proof of Theorem 13. The content of Chapter 6 is currently being prepared for submission for publication. Remmel, Jeffrey; Tiefenbruck, Janine LoBue. The dissertation author is an author of this material.

Bibliography [Aig07]

M. Aigner. A Course in Enumeration, Graduate Texts in Mathematics 238. Springer, 2007.

[Cam02]

N.T. Cameron. Random Walks, Trees and Extensions of Riordan Group Techniques. PhD thesis, Howard University, 2002.

[CRK95]

J. Carbonara, J. Remmel, and A. Kulikauskas. A combinatorial proof of the equivalence of the classical and combinatorial definitions of Schur functions. J. Comb. Theory, Ser. A, 72:293–301, 1995.

[DCV86]

M. Desainte-Catherine and G. Viennot. Enumeration of certain young tableaux with bounded height. Technical Report LABRI-NI86-013, Universit´e de Bordeaux 1 (Talence), 1986.

[Fei12]

T. Feierl. The height of watermelons with wall. Journal of Physics A: Math. and Theor., 45(9), 2012.

[Fer11]

J. Ferreira. A Littlewood-Richardson type rule for row-strict quasisymmetric Schur functions. DMTCS Proceedings, 0(01), 2011.

[Fis84]

M. Fisher. Walks, walls, wetting, and melting. Joural of Statist. Phys., 34(5–6):667–729, 1984.

[Fun30]

H. Gray Funkhouser. A short account of the history of symmetric functions of roots of equations. The American Mathematical Monthly, 37(7):pp. 357–365, 1930.

[Ges84]

I. Gessel. Multipartite P-partitions and inner products of skew Schur functions. Contemporary Mathematics, 34:289–301, 1984.

[GV85]

I. Gessel and G. Viennot. Binomial determinants, paths, and hook length formulae. Advances Math., 58:300–321, 1985.

[HHL08]

J. Haglund, M. Haiman, and N. Loehr. A combinatorial formula for non-symmetric Macdonald polynomials. American Journal of Mathematics, 103(2):359–383, 2008.

128

129 [HLMvW11a] J. Haglund, K. Luoto, S. Mason, and S. van Willigenburg. Quasisymmetric Schur functions. J. Comb. Theory, Ser. A, 118(2):463– 490, 2011. [HLMvW11b] J. Haglund, K. Luoto, S. Mason, and S. van Willigenburg. Refinements of the Littlewood-Richardson Rule. Trans. Amer. Math. Soc., 363:1665–1686, 2011. [HMR12]

J. Haglund, S. Mason, and J. Remmel. Properties of the nonsymmetric Robinson-Schensted-Knuth algorithm. Journal of Algebraic Combinatorics, published online October, 2012.

[Knu70]

D. Knuth. Permutations, matrices, and generalized Young tableaux. Pacific Journal of Mathematics, 34(3):709–727, 1970.

[Lin73]

B. Lindstrom. On the vector representation of induced matroids. Bull. London Math. Soc., 5:85–90, 1973.

[LS90]

A. Lascoux and M.P. Sch¨ utzenberger. Keys and standard bases. IMA Volumes in Mathematics and its Applications, 19:125–144, 1990.

[Mac79]

I.G. Macdonald. Symmetric functions and Hall polynomials. Clarendon Press; Oxford University Press, Oxford: New York, 1979.

[Mac88]

I. G. MacDonald. A new class of symmetric functions. S´eminaire Lotharingien, 372/S20:131–171, 1988.

[Mac95]

I.G. MacDonald. Affine Hecke algebras and orthogonal polynomials. S´eminaire Bourbaki, 37:189–207, 1994-1995.

[Mas08]

S. Mason. A decomposition of Schur functions and an analogue of the Robinson–Schensted–Knuth algorithm. S´eminaire Lotharingien de Combinatoire, 57: B57e, 2008.

[Mas09]

S. Mason. An explicit construction of type A Demazure atoms. Journal of Algebraic Combinatorics, 29(3):295–313, 2009.

[MR14]

Sarah Mason and Jeffrey Remmel. Row-Strict Quasisymmetric Schur Functions. Annals of Combinatorics, 18(1):127–148, 2014.

[Pro89]

R. Proctor. Equivalence of the combinatorial and the classical definitions of Schur functions. J. Comb. Theory, Ser. A, 51:135–137, 1989.

[Sch61]

C. Schensted. Longest increasing and decreasing subsequences. Canadian Journal of Mathematics, 13:179–191, 1961.

130 [Slo]

N.J.A. Sloane. The online encyclopedia of integer sequences, published electronically at http://oeis.org.

[Sta71]

R. Stanley. Theory and applications of plane partitions: Part 2. Studies in Applied Math, 50:259–279, 1971.

[Sta99]

R.P. Stanley. Enumerative Combinatorics, Vol. 2 (Cambridge Studies in Advanced Mathematics 62). Cambridge University Press, 1999.