Combinatorial algorithms: generation, enumeration, and search [1st ed.] 9781003068006, 9781000102871, 1000102874, 9781000120042, 100012004X, 9781000141375, 1000141373, 1003068006

Citation preview

The CRC Press Series on

DISCRETE MATHEMATICS AND

ITS APPLICATIONS Series Editor

Kenneth H. Rosen, Ph.D. AT&T Bell Laboratories

Charles J. Colboum and Jeffrey H. Dinitz, The CRC Handbook of Combinatorial Designs

Steven Furino, Ying Miao, and Jianxing Yin, Frames and Resolvable Designs: Uses, Constructions, and Existence

Jacob E. Goodman and Joseph O’Rourke, Handbook of Discrete and Computational Geometry

Charles C. Lindner and Christopher A. Rodgers Design Theory

Daryl D. Harms, Miroslav Kraetzl, Charles J. Colboum, and John S. Devitt, Network Reliability: Experiments with A Symbolic Algebra Environment

Alfred J. Menezes, Paul C. van Oorschot, and Scott A. Vanstone, Handbook of Applied Cryptography

Richard A. Mollin, Quadratics Richard A. Mollin, Fundamental Number Theory with Applications

Douglas R. Stinson, Cryptography: Theory and Practice

COMBINATORIAL

ALGORITHMS

Generation, Enumeration, and Search

Donald L* Kreher Department of Mathematical Sciences Michigan Technological University

Douglas R- Stinson Department of Combinatorics and Optimization University of Waterloo

CRC Press Boca Raton London New York Washington, D.C.

Library of Congress Cataioging-in-Publication Data Kreher, Donald L, Combinatorial algorithms : generation, enumeration, and search / Donald L. Kreher, Douglas R. Stinson, p. cm. — (CRC Press series on discrete mathematics and its applications) , Includes bibliographical references and index, ISBN G-8493-3988-X (alk. paper) 1. Combinatorial analysis. 2. Algorithms. 1. Stinson, Douglas R. (Douglas Robert), 1956- . II. Title. III. Series. QA164.K73 1998 511'.6 — de21

98 -41243 CIP

This book contains information obtained from authentic and highly regarded sources. Reprinted material is quoted with permission, and sources are indicated. A wide variety of references are listed. Reasonable efforts have been made to publish reliable data and information, but the author and the publisher cannot assume responsibility for the validity of all materials or for the consequences of their use. Neither this book nor any part may be reproduced or transmitted in any form or by any means, electronic or mechanical, including photocopying, microfilming, and recording, or by any information storage or retrieval system, without prior permission in writing from the publisher. The consent of CRC Press LLC does not extend to copying for general distribution, for promotion, for creating new works, or for resale. Specific permission must be obtained in writing from CRC Press LLC for such copying. Direct all inquiries to CRC Press LLC, 2000 N.W. Corporate Blvd., Boca Raton, Florida 33431. Trademark Notice: Product or corporate names may be trademarks or registered trademarks, and are used only for identification and explanation, without intent to infringe.

Visit the CRC Press Web site at wwwxrcpressxom © 1999 by CRC Press LLC No claim to original U.S. Government works International Standard Book Number 0-8493 -398S -X Library of Congress Card Number 98-41243 Printed in the United States of America 7 8 9 0 Printed on acid-free paper

Preface

Our objective in writing this book was to produce a general, introductory textbook on the subject of combinatorial algorithms. Several textbooks on combinatorial algorithms were written in the 1970s, and are now out-of-date. More recent books on algorithms have either been general textbooks, or books on specialized topics, such as graph algorithms to name one example. We felt that a new textbook on combinatorial algorithms, that emphasizes the basic techniques of generation, enumeration and search, would be very timely. We have both taught courses on this subject, to undergraduate and graduate students in mathematics and computer science, at Michigan Technological Uni­ versity and the University of Nebraska-Lincoln. We tried to design the book to be flexible enough to be useful in a wide variety of approaches to the subject. We have provided a reasonable amount of mathematical background where it is needed, since an understanding of the algorithms is not possible without an understanding of the underlying mathematics. We give informal descriptions of the many algorithms in this book, along with more precise pseudo-code that can easily be converted to working programs. C implementations of all the algorithms are available for free downloading from the website http ://www.math.mtu .edu/ ~ kreher/cages .html

There are also many examples in the book to illustrate the workings of the algo­ rithms. The book is organized into eight chapters. Chapter 1 provides some back­ ground and notation for fundamental concepts that are used throughout the book. Chapters 2 and 3 are concerned with the generation of elementary combinato­ rial objects such as subsets and permutations, to name two examples. Chapter 4 presents the important combinatorial search technique called backtracking. It in­ cludes a discussion of pruning methods, and the maximum clique problem is stud­ ied in detail. Chapter 5 gives an overview of the relatively new area of heuristic search algorithms, including hill-climbing, simulated annealing, tabu search and genetic algorithms. In Chapter 6, we study several basic algorithms for permuta­ tion groups, and how they are applied in solving certain combinatorial enumera­ tion and search problems. Chapter 7 uses techniques from the previous chapter

PREFACE

to develop algorithms for testing isomorphism of combinatorial objects. Finally, Chapter 8 discusses the technique of basis reduction, which is an important tech ­ nique in solving certain combinatorial search problems. There is probably more material in this book than can be covered in one semester. We hope that it is possible to base several different types of courses on this book. An introductory course suitable for undergraduate students could cover most of the material in Chapters 1-5. A second or graduate course could concentrate on the more advanced material in Chapters 6 - 8. We hope that, aside from its primary purpose as a textbook, researchers and practitioners in all areas of combinatorial computing will find this book useful as a source of algorithms for practical use. We would like to thank the many people who provided encouragement while we wrote this book, pointed out typos and errors, and gave us useful suggestions on material to include and how various topics should be treated. In particular, we would like to convey our thanks to Mark Chateauneuf, Charlie Colbourn, Bill Kocay, Francois Margot, Wendy Myrvold, David Olson, Partic Ostergard, Jamie Radcliffe, Stanislaw Radziszowski and Mimi Williams. Donald L. Kreher Douglas R. Stinson

To our wives, Carol and Janet.

Contents

1

Structures and Algorithms 1.1 What are combinatorial alg o rith m s?.......................... 1.2 What are combinatorial structures?............................. 1.2.1 Sets and l i s t s ....................................................... 1.2.2 G r a p h s ................................................................ 1.2.3 Set sy stem s.......................................................... 1.3What are combinatorial problem s?.................................... 1.4 O - N o ta tio n .................................................................... 1.5 Analysis of a lg o rith m s ................................................. 1.5.1 Average-case com plexity.................................... 1.6 Complexity c la s s e s ........................................................ 1.6.1 Reductions between p ro b le m s ........................... 1.7 Data structures .............................................................. 1.7.1 Data structures for s e t s ....................................... 1.7.2 Data structures for lis ts ....................................... 1.7.3 Data structures for graphs and set systems . . . 1.8 Algorithm design techniques........................................ 1.8.1 Greedy a lg o rith m s............................................. 1.8.2 Dynamic programming....................................... 1.8.3 D ivide-and-conquer.......................................... 1.9 N o t e s ................................................................................ Exercises .................................................................................

1 1 2 2 4 5 7 9 10 12 13 16 17 17 22 22 23 23 24 25 26 27

2

Generating Elementary Combinatorial Objects 2.1 Combinatorial generation.............................................. 2.2 Subsets .......................................................................... 2.2.1 Lexicographic ordering....................................... 2.2.2 Gray c o d e s .......................................................... 2.3 A;-Element subsets.......................................................... 2.3.1 Lexicographic ordering....................................... 2.3.2 Co-lex ordering................................................... 2.3.3 Minimal change ordering...................................

31 31 32 32 35 43 43 45 48

C O N TE N TS

2.4

Permutations .................................................................. 2.4.1 Lexicographic ordering....................................... 2.4.2 Minimal change ordering.................................... 2.5 N o t e s ............................................................................... Exercises ...................................................................................

52 52 57 64 64

More Topics in Combinatorial Generation 3.1 Integer p a r titio n s ............................................................ 3.1.1 Lexicographic ordering....................................... 3.2 Set partitions, Bell and Stirling n u m b e rs...................... 3.2.1 Restricted growth functions ............................. 3.2.2 Stirling numbers of the first k in d ....................... 3.3 Labeled t r e e s .................................................................... 3.4 Catalan fam ilies................................................................. 3.4.1 Ranking and unranking....................................... 3.4.2 Other Catalan fa m ilie s ....................................... 3.5 N o t e s ............................................................................... Exercises ...................................................................................

67 67 74 78 81 87 91 95 98 101 103 103

4 Backtracking Algorithms 4.1 Introduction..................................................................... 4.2 A general backtrack a lg o rith m ...................................... 4.3 Generating all cliq u es...................................................... 4.3.1 Average-case analysis ....................................... 4.4 Estimating the size of a backtrack t r e e ......................... 4.5 Exact c o v e r ...................................................................... 4.6 Bounding fu n ctio n s......................................................... 4.6.1 The knapsack p ro b le m ....................................... 4.6.2 The traveling salesmanp ro b le m ........................ 4.6.3 The maximum clique p ro b le m .......................... 4.7 Branch and b o u n d ............................................................ 4.8 N o t e s ............................................................................... Exercises ...................................................................................

105 105 107 109 112 115 118 122 123 127 135 141 144 145

5 Heuristic Search 5.1 Introduction to heuristic algorithms................................ 5.1.1 Uniform graph p a r titio n .................................... 5.2 Design strategies for heuristic algorithms .................... 5.2.1 H ill - clim bing....................................................... 5.2.2 Simulated a n n e a lin g ........................................... 5.2.3 Tabu s e a rc h .......................................................... 5.2.4 Genetic alg o rith m s.............................................. 5.3 A steepest ascent algorithm for uniform graph partition 5.4 A hill-climbing algorithm for Steiner triple systems . .

151 151 155 156 157 158 160 161 165 167

3

C O N TE N TS

5.4.1 Implementation d e ta ils ....................................... 5.4.2 Computational results ....................................... 5.5 Two heuristic algorithms for the knapsack problem . . 5.5.1 A simulated annealing algorithm ....................... 5.5.2 A tabu search a lg o rith m .................................... 5.6 A genetic algorithm for the traveling salesman problem 5.7 N o t e s ............................................................................... Exercises .................................................................................

170 174 175 175 178 181 186 189

6 Groups and Symmetry 6.1 G roups............................................................................... 6.2 Permutation g ro u p s ......................................................... 6.2.1 Basic alg o rith m s................................................ 6.2.2 How to store a g ro u p .......................................... 6.2.3 Schreier-Sims a lg o rith m .................................... 6.2.4 Changing the b a s e ............................................. 6.3 Orbits of subsets ............................................................. 6.3.1 Burnside’s lemma ............................................. 6.3.2 Computing orbit representatives....................... 6.4 Coset representatives...................................................... 6.5 Orbits of k - tu p le s ............................................................ 6.6 Generating objects having autom orphism s.................... 6.6.1 Incidence m a tric e s ............................................. 6.7 N o t e s ............................................................................... Exercises .................................................................................

191 191 195 199 201 203 211 213 214 217 223 224 226 227 232 232

7 Computing Isomorphism 7.1 Introduction...................................................................... 7.2 In v a ria n ts......................................................................... 7.3 Computing ce rtific a te s................................................... 7.3.1 T r e e s ................................................................... 7.3.2 G r a p h s ................................................................ 7.3.3 Pruning with automorphisms............................. 7.4 Isomorphism of other structures ................................... 7.4.1 Using known automorphisms .......................... 7.4.2 Set sy stem s......................................................... 7.5 N o t e s ............................................................................... Exercises .................................................................................

237 237 238 245 245 253 264 272 272 272 275 275

8

277 277 281 291 294

Basis Reduction 8.1 Introduction...................................................................... 8.2 Theoretical developm ent................................................ 8.3 A reduced basis algorithm ............................................. 8.4 Solving systems of integer equations.............................

CONTENTS

8.5 The Merkle -Hellman knapsack sy ste m ......................... 8.6 N o t e s ............................................................................... Exercises ...................................................................................

300 306 307

Bibliography

311

Algorithm Index

319

Problem Index

323

Index

325

1 ___________________________________________________

Structures and Algorithms

1.1

What are combinatorial algorithms?

In this book, we are primarily interested in the study of algorithms to investigate combinatorial structures. We will call such algorithms combinatorial algorithms, and informally classify them according to their desired purpose, as follows. generation Construct all the combinatorial structures o f a particular type. Examples of the combinatorial structures we might wish to generate include subsets, permutations, partitions, trees and Catalan families. A generation algorithm will list all the objects under consideration in a certain order, such as a lexicographic order. It may be desirable to predetermine the position of a given object in the generated list without generating the whole list. This leads to the discussion of ranking, which is studied in Chapters 2 and 3. The inverse operation of ranking is unranking and is also studied in these two chapters. enumeration Compute the number o f different structures o f a particular type. Every generation algorithm is also an enumeration algorithm, since each object can be counted as it is generated. The converse is not true, however. It is often easier to enumerate the number of combinatorial structures of a particular type than it is to actually list them. For example, the number of fc-subsets of an n -element set is

which is easily computed. On the other hand, listing all of the k-subsets is more difficult. There are many situations when two objects are different representations of the “same” structure. This is formalized in the idea of isomorphism of struc­ tures. For example, if we permute the names of the vertices of a graph, the

1

2

Structures and Algorithms

resulting two graphs are isomorphic. Enumeration of the number of non ­ isomorphic structures of a given type often involves algorithms for isomor­ phism testing, which is the main topic studied in Chapter 7. Algorithms for testing isomorphism depend on various group-theoretic algorithms, which are studied in Chapter 6. search Find at least one example o f a structure o f a particular type (if it exists). A typical example of a search problem is to find a clique of a specified size in a given graph. Generating algorithms can sometimes be used to search for a particular structure, but for many problems, this may not be an efficient approach. Often, it is easier to find one example of a structure than it is to enumerate or generate all the structures of a specified type. A variation of a search problem is an optimization problem, where we want to find the optimal structure of a given type. Optimality will be defined for a particular structure according to some specified measure of “profit” or “cost”. For example, the maximum clique problem requires finding a clique of largest size in a given graph. (The size of a clique is the number of vertices it contains.) Many interesting and important search and optimization problems belong to the class of NP-hard problems, for which no efficient (i.e., polynomial­ time) algorithms are known to exist. The maximum clique problem men ­ tioned above falls into this class of problems. For NP-hard problems, we will often use algorithms based on the idea of backtracking, which is the topic of Chapter 4. An alternative approach is to try various types of heuris­ tic algorithms. This topic is discussed in Chapters 5 and 8.

1.2

What are combinatorial structures?

The structures we study in this book are those that can be described as collections of ^ -element subsets, A;-tuples, or permutations from a parent set. Given such a structure, we may be interested in all of the substructures contained within it of a particular or optimal type. On the other hand, we may wish to study all structures of a given form. We introduce some of the types of combinatorial structures in this section that will be used in later parts of the book. 1.2.1 Sets and lists The basic building blocks of combinatorial structures are finite sets and lists. We review some basic terminology and notation now. A (finite) set is a finite collection of objects called the elements of the set. We write the elements of a set in brace brackets. For example, if we write X = {1 ,3 ,7 ,9 }, then we mean that X is a set that contains the elements 1,3, 7 and 9.

What are combinatorial structures?

3

A set is an unordered structure, so {1,3, 7,9} = {7,1,9,3}, for example. Also, the elements of a set are distinct. We write x E X to indicate that x is an element of the set X . The cardinality (or size) of a set A , denoted |X |, is the number of elements in X . For example, |{ 1 ,3 ,7 ,9}| = 4. For a nonnegative integer k , a k-set is a set of cardinality k. The empty set is the (unique) set that contains no elements. It is a 0 -set and is denoted by 0. If X and Y are sets, then we say that X is a subset of Y if every element of X is an element of Y. This is equivalent to the following condition: x e x => x e Y. If X is a subset of Y, then we write X C Y . A k-subset of Y is a subset of Y that has cardinality k. A (finite) list is an ordered collection of objects which are called the items of the list. We write the items of a list in order between square brackets. For example, if we write X = [1,3,1,9], then we mean that X is the list that contains the items 1,3,1 and 9 in that order. Since a set is an ordered structure, it follows that [1,3,1,9] ^ [1,1,9,3], for example. Note that the items of a list need not be distinct. The length of a list X is the number of items (not necessarily distinct) in X . For example, [1,3,1,9] is a list of length 4. For a nonnegative integer n, an n-tuple is a list of length n. The empty list is the (unique) list that contains no elements; it is written as [ ]. If X is a list of length n, then the items in X are denoted X [ 0 ] ,X [ l ] ,...,X [ n — 1], in order. We usually denote the first item in the list X as X[0], as is done in the C programming language. Thus, if X = [1,3,1,9], then X[0] = 1, X[ l \ = 3, X[2] — 1 and X[3] = 9. However, in some situations, we may list the elements of X as X[l\, X[2] , . . . , X[n]. An alternative notation for a list, that we will sometimes use, is to write the items in the list X in subscripted form, as X o , X \ , . . . , X n- i . The Cartesian product (or cross product) of the sets X and Y, denoted by 1 x 7 , is the set of all ordered pairs whose first item is in X and whose second item is in Y. Thus 1 x 7 - {[x, y] : x e X and y E Y}. For example, if Al = {1,3, 7,9} and Y = {0,2,4}, then {1,3,7,9} x {0,2} = {[1,0], [1,2], [3,0], [3,2], [7,0], [7,2], [9,0], [9,2]}. If X is a finite set of cardinality n, then a permutation of X is a list ir of length n such that every element of X occurs exactly once in the list 7r. There are exactly n! = n (n — 1) • • • 1 permutations of an n-set. For a positive integer k < n, a kpermutation of X is a list 7r of length k such that every element of X occurs at most once in the list 7r. There are exactly

4

Structures and Algorithms

^ -permutations of an n-set. 1.2.2

Graphs

We begin by defining the concept of a graph. Definition 1.1: A graph consists of a finite set V of vertices and a finite set £ of edges, such that each edge is a two element subset of vertices. It is customary to write a graph as an ordered pair, (V, £). A complete graph is a graph in which £ consists of all two element subsets of V. If | V| = n, then the complete graph is denoted by K n. We will usually represent the vertices of a graph (V, £) by dots, and join two vertices x and y by a line whenever {x, y} G £. A vertex x is incident with an edge e if x G e. The degree of a vertex x G V, denoted by deg(x), is the number of edges that are incident with the vertex x. A graph is regular of degree d if every vertex has degree d. In Example 1.1 we present a graph that is regular of degree three.

Example 1.1 A graph Let V = { 0 ,1 ,2 ,3 ,4 ,5 ,6 ,7 } , and let £ = {{0,1}, {0,2}, {2,3}, {1,3}, {0,4}, {1,5}, {2,6}, {3,7}, {4,5}, {4,6}, {6,7}, {5,7}}. This graph is called the cube D and can be represented by the diagram in Figure 1.1.

0

2

FIGURE 1.1 The cube and a Hamiltonian circuit.

One of the many interesting substructures that can occur in a graph is a Hamil­ tonian circuit. This is a closed path that passes through every vertex exactly once. The list [0,2,3, 7 ,6 ,4 ,5 ,1 ] describes a Hamiltonian circuit in the graph in Fig ­ ure 1.1. It is indicated by thick lines in the diagram. Note that different lists can

5

What are combinatorial structures?

represent the same Hamiltonian circuit. In fact, there are 2n such lists, where n is the number of vertices in the graph, since we can pick any of the n vertices as the “starting point” and then traverse the circuit in two possible directions. A graph (V, 8) is a weighted graph if there is a weight function w : 8 - » R associated with it. The weight of a substructure, such as a Hamiltonian circuit, is defined to be the sum of the weights of its edges. Finding a smallest weight Hamiltonian circuit in a weighted graph is called the Traveling Salesman prob­ lem and is discussed in Chapter 4. Another example of a substructure in a graph is a clique. A clique in a graph Q = (V ,£) is a subset S C V such that { x, y} € 8 for all x , y £ S, x ^ y. A clique that has the maximum size among all cliques of Q is called a maximum clique. In Figure 1.1, every edge {x, y} determines a maximum clique of the graph, since there are no cliques of size 3. Finding a maximum clique in a graph is called the Maximum Clique problem and is discussed in Chapter 4. 1.2.3

Set systems

We next define a generalization of a graph called a set system.

Observe that a graph is just a set system in which every block has cardinality two. Another simple example of a set system is a partition of a set X . A partition is a set system (X , B) in which A fl B = 0 for all A, B £ B with A ^ B , and UA e sA = X . We now define another, more complicated, combinatorial structure, and then formulate it as a special type of set system. Definition 1.3: A Latin square of order n is an n by n array A , whose entries are chosen from an n -element set y , such that each symbol in y occurs in exactly one cell in each row and in each column of A.

Example 1.2 A Latin square o f order four Let = {1,2,3,4} and let

y

CO

4

2 3 4 1 4 4 1 2 3 2 1 CO

1 2

6

Structures and Algorithms

Suppose that A is a Latin square of order n on a symbol set y . Label the n rows of A with the n elements of y t and likewise label the n columns of A with the n elements of y . Now define a set system (X, B) as follows: X = y x {1,2,3}, and

# = {{(yi.l),(y 2 ,2 ),(i4 [y 1,y2],3)} : y i , y 2 € >>}.

If we start with the Latin square of order four that was presented in Example 1 .2 , then we obtain the following set system (X , B):

* = 11 ,2 ,3 ,4 } x {1 ,2 ,3 } {[1,1], [1,2], [1,3]}, {[1,1], [3,2], [3,3]}, {[2,1], [1,2], [2,3]}, {[2,1], [3,2], [4,3]}, B = < {[3,1], [1,2], [3,3]}, {[3,1], [3,2], [1,3]}, {[4,1], [1,2], [4,3]}, {[4,!], [3,2],[2,3]},

{[1,1], {[1,1], {[2,1], {[2,1], {[3,1], {[3,1], {[4,1], {[4, i],

[2,2], [2,3]}, [4,2], [4,3]}, [2,2], [1,3]}, [4,2], [3,3]}, [2,2], [4,3]}, [4,2], [2,3]}, [2,2], [3,3]}, [4,:2], [i, 3]}

If we look at the blocks in this set system we see that every pair of the form {(yii (2/2 ?j)}» where i ^ j, occurs in exactly one block in B. This motivates the following definition. Definition 1.4: A transversal design TD(n) is a set system (X ,B ) for which there exists a partition {X i, X 2, X 3j of X such that the following prop ­ erties are satisfied: 1.

\X\ — 3n, and \X{\ = n for 1 < i


0 and no > 0 such that 0 < c x g(n) < f ( n ) < c' x g(n) for all n > no. If /( n ) is 0(