Theory of Formal Languages with Applications 9810237294, 9789810237295

Citation preview

THEORY OF FORMAL LANGUAGES WITH APPLICATIONS

This page is intentionally left blank

THEORY OF FORMAL LANGUAGES WITH APPLICATIONS

Dan A Simovici Richard L Tenney Department of Mathematics and Computer Science, University of Massachusetts at Boston

V f e World Scientific wb

Singapore • New Jersey • London • Hong Kong

Published by World Scientific Publishing Co. Pte. Ltd. P O Box 128, Farrer Road, Singapore 912805 USA office: Suite IB, 1060 Main Street, River Edge, NJ 07661 UK office: 57 Shelton Street, Covent Garden, London WC2H 9HE

British Library Cataloguing-in-Publication Data A catalogue record for this book is availablefromthe British Library.

THEORY OF FORMAL LANGUAGES WITH APPLICATIONS Copyright © 1999 by World Scientific Publishing Co. Pte. Ltd. All rights reserved. This book, or parts thereof, may not be reproduced in any form or by any means, electronic or mechanical, including photocopying, recording or any information storage and retrieval system now known or to be invented, without written permissionfromthe Publisher.

For photocopying of material in this volume, please pay a copying fee through the Copyright Clearance Center, Inc., 222 Rosewood Drive, Danvers, MA 01923, USA. In this case permission to photocopy is not required from the publisher.

ISBN 981-02-3729-4

Printed in Singapore by Uto-Print

Contents Preface

1X

Introduction

X1

I

Introductory Notions

1

1 Preliminaries 1.1 Introduction 1.2 Sets, Relations, and Functions 1.2.1 Sets 1.2.2 Ordered Pairs and Cartesian Products . . . 1.2.3 Relations 1.2.4 Equivalence Relations 1.2.5 Partial Orders 1.2.6 Functions 1.3 Operations and Algebras 1.3.1 Operations 1.3.2 Algebras, Semigroups, and Monoids 1.3.3 Morphisms and Subalgebras 1.3.4 Congruences 1.4 Sequences 1.4.1 The Monoid of Sequences 1.4.2 Arithmetic Progressions 1.5 Graphs 1.6 Cardinality 1.7 Exercises 1.8 Bibliographical Comments

3 3 3 3 4 6 9 11 12 16 17 19 21 22 24 26 29 30 37 45 55

2 Words and Languages 2.1 Introduction 2.2 Words 2.3 Languages 2.4 Substitutions and Morphisms 2.5 Matrices and Languages 2.6 Polynomial Functions

57 57 57 60 65 67 71

vi

Contents 2.7 2.8

II

Exercises Bibliographical Comments

Regular and Context-Free Languages

82 92

95

3 Regular Languages 97 3.1 Introduction 97 3.2 Finite Automata 98 3.2.1 Deterministic Automata 98 3.2.2 Nondeterministic Automata 107 3.2.3 Configurations 114 3.3 Transition Systems 116 3.4 Closure Properties 122 3.5 The Pumping Lemma 128 3.6 Minimal Automata 132 3.7 Syntactic Monoids 136 3.7.1 Automata and Monoids 137 3.7.2 The Syntactic Monoid of a Language 139 3.8 Fixed Points and Regular Languages 141 3.9 Regular Expressions 147 3.9.1 The Unique Readability of Regular Expressions 147 3.9.2 Regular Expressions as Notations for Regular Languages . 150 3.9.3 Closure Properties and Regular Expressions 152 3.9.4 A Formal System for Regular Expressions 158 3.10 Transducers 165 3.11 Automata and String Patterns 171 3.12 Applications of Regular Expressions 184 3.12.1 Regular Expressions and UNIX 184 3.12.2 The grep Utility and Its Relatives 186 3.12.3 The aux Text Processing Program 187 3.12.4 The lex Lexical Analyzer Generator 189 3.13 Exercises 191 3.14 Bibliographical Comments 221 4 Rewriting Systems and Grammars 4.1 Introduction 4.2 Semi-Thue and Thue Systems 4.3 Grammars and Chomsky Hierarchy 4.3.1 Equivalent Grammars 4.4 Regular Operations 4.5 Properties of Type-2 Grammars 4.6 Regular Languages and Type-3 Grammars 4.7 Exercises 4.8 Bibliographical Comments

223 223 223 228 233 237 242 254 258 267

Contents 5 Context-Free Languages 5.1 Introduction 5.2 Derivations and Derivation Trees 5.3 Fixed-Points and Context-Free Languages 5.4 Normal Forms 5.4.1 Chomsky Normal Form 5.4.2 Greibach Normal Form 5.5 The Pumping Lemmas 5.6 Closure Properties 5.7 Regular and Context-Free Languages 5.8 Ambiguity 5.9 Parikh Theorem 5.10 The Chomsky-Schiitzenberger Theorem 5.11 Exercises 5.12 Bibliographical Comments

_

• •.

XH 269 269 • 270 281 286 286 289 298 302 306 308 314 319 322 335

6 Pushdown Automata 6.1 Introduction 6.2 Nondeterministic Pushdown Automata 6.3 Deterministic Context-Free Languages 6.4 Exercises 6.5 Bibliographical Comments

337 337 337 352 370 376

III

377

Algorithmic Aspects

7 Partial Recursive Functions 7.1 Computable Functions 7.2 Primitive Recursive Functions 7.3 Primitive Recursive Predicates 7.4 Bounded Minimalization 7.5 Extensions 7.6 Numerical Primitive Recursive Functions 7.7 Transformations between Alphabets 7.8 Primitive Recursive Languages 7.9 Partial Recursive Functions 7.10 Exercises 7.11 Bibliographical Comments

379 379 380 385 391 393 395 401 411 414 422 430

8 Recursively Enumerable Languages 8.1 Introduction 8.2 Labeled Markov Algorithms 8.3 Turing Machines 8.4 Systems of Deterministic Turing Machines 8.5 Church's Thesis 8.5.1 Functions Computable by Turing Machines 8.5.2 Closing the Circle 8.5.3 Recursive Languages

431 431 432 439 444 448 451 458 463

viii

Contents

8.6 8.7 8.8 8.9 8.10 8.11 8.12

8.5.4 Universality Recursive Enumerable Languages Rice's Theorem Post Correspondence Problem Multitape Turing Machines Nondeterministic Turing Machines Exercises Bibliographical Comments

9 Context-Sensitive Languages 9.1 Introduction 9.2 Linear Bounded Automata 9.3 Closure Properties 9.4 Normal Forms for Context-Sensitive Grammars . 9.5 Exercises 9.6 Bibliographical Comments

IV

Applications

464 471 489 491 497 503 506 521 523 523 523 531 546 548 549

551

10 Codes 10.1 Introduction 10.2 Unique Decipherability 10.3 The Kraft-McMillan Inequality 10.4 Huffman Codes and Data Compression 10.5 Exercises 10.6 Bibliographical Comments

553 553 554 561 567 571 573

11 Biological Applications 11.1 Introduction 11.2 ^-Systems 11.3 Nucleic Acids 11.4 Exercises 11.5 Bibliographical Comments

575 575 575 589 601 606

Bibliography

607

Notation Index

613

Topic Index

619

Preface The theory of formal languages has a long and dignified history. A major influence on the nascent theory, around 1960, were the attempts of the linguist Noam Chomsky to formulate a general theory of the syntax of natural languages. Chomsky's intellectual itinerary greatly influenced the field at a time when computers were starting to cope with increasingly complex tasks. A melting pot of ideas then developed, with a surprising convergence of thought between linguists, mathematicians, logicians, and newly born computer scientists. At present, formal languages are part of the basic training of most computer scientists. They are everywhere to be found in the design and in the very operation of computer systems. A modem, like an interface manager, will have to respond to various external stimuli. Its design and its behavior are then best understood when it is viewed as a device reacting to external events while being governed by a finite set of rules—in short, a finite automaton. Next, the syntax of programming languages is best described by context-free grammars, themselves recognized by pushdown automata. We then have available one of the fundamental building blocks of the design of parsers and compilers. Finally, the last steps of the complexity ladder take us to languages of a higher structural complexity, which swiftly lead to (un)decidability questions. This brings us to the humbling realization that mathematically well-posed problems are far from being all decidable! The theory of formal languages and their companion automata thus provides a powerful approach to the design of systems and to a variety of problems in computer science. Dan Simovici and Richard Tenney develop the core theory in a lucid manner. Their self-contained presentation combines mathematical rigor and intellectually stimulating applications. For instance, the reader will find in the book a perspective on algorithms for the processing of text files, lexical analysis, and parsing. A notably innovative aspect is the last part that offers two chapters on coding theory, data compression, as well as biological applications. It should be a pleasure for most to discover there formal models that describe the development of simple organisms or the splicing of nucleic acids. To make a long story short, we have here a new book that offers new per­ spectives on an old subject. It contains a thorough treatment of a theory that is fundamental not only in computer science but in many other scientific en­ deavors. The authors have done a great job of exposition. I hope you will enjoy reading the book as much as I did. Philippe Flajolet Rocquencourt, February 28, 1999

This page is intentionally left blank

Introduction The theory of formal languages is an important part of the fundamental edu­ cation of computer scientists and linguists. It is also becoming significant for biologists. This discipline blends algebraic techniques with abstract models of computing devices. Its origins can be traced to the work of Chomsky, Rabin, Scott, Nerode, Ginsburg, and Schutzenberger, and this beautiful area of theoret­ ical computer science remains active today. Along the way are such milestones as the theory of abstract families of languages and various applications of the theory of complexity in the study of formal languages. This book combines algebraic and algorithmic methods with decidability results and explores applications both within and outside computer science. Formal languages provide the theoretical underpinnings for the study of programming languages. They are also the foundation for compiler design, and they are important in such areas as data compression, computer networks, etc. Recently, formal languages have been applied in biology and economics. The first part of the book presents mathematical preliminaries. It begins with a chapter that elucidates the mathematical background expected of the reader—elementary notions about sets, algebras, and graphs—as well as the notation that we use. It is intended to make this book as self-contained as prac­ tical. The second chapter deals with words and languages viewed as collections of words. These are basic ingredients in the discipline of formal languages, so this chapter presents the most important algebraic and combinatorial properties of words and languages in order to make later chapters more readable. The second part is centered on regular and context-free languages. The class of regular languages is studied in the third chapter, starting with deterministic finite automata. We then consider various extensions of these devices, includ­ ing nondeterministic automata and transition systems, as alternative ways of defining the same class of languages. We introduce regular expressions as nota­ tions for regular languages, and we conclude the chapter by examining several applications of the notions developed in the chapter. The fourth chapter introduces the notions of semi-Thue system, and espe­ cially important, the notion of grammar. We study Chomsky's hierarchy, and we show the closure of each Chomsky class with respect to the regular operations. We place particular emphasis on context-free languages due to their role in compiler design. This class of languages is introduced using the class of context-free grammars; the devices that provide an alternative characterization of this class, pushdown automata are discussed in the next chapter. To allow

xii

Introduction

us to include topics that are usually not found in text books, we minimize our discussion of applications to compilers. There are excellent texts that cover this area. The third part of the book focuses on the algorithmic aspects of formal languages. We use labeled Markov algorithms and Turing machines as general abstract models of computation. We cover undecidability results starting with the halt­ ing problem for Turing machines and the Post correspondence problem and continuing with undecidability results for various classes of languages. Particu­ lar attention is paid in to the class of context-sensitive languages. The last part of the book presents applications of formal language theory. We have chosen two distinct and representative areas: coding theory over free monoids, important for computer communications and applications of formal languages in biology. We believe that the presence of these applications moti­ vates students to study formal languages. The book is intended as a textbook for an upper-level undergraduate or a graduate course is formal languages. Each chapter ends with suggestions for further reading. The book contains more than 600 exercises; they form an integral part of the material. Some of the exercises are in reality supplemental material. For these, we include solutions. The authors are grateful for support received from the University of Mas­ sachusetts at Boston and extend special thanks to Professors Tatsuo Higuchi, Michitaka Kameyama, and Akira Maruoka from Tohoku University, who hosted Dan Simovici in 1998.

Copyright © 1999. World Scientific Publishing Company. All rights reserved.

Part I

Introductory Notions

Copyright © 1999. World Scientific Publishing Company. All rights reserved.

This page is intentionally left blank

Chapter 1

Preliminaries

Copyright © 1999. World Scientific Publishing Company. All rights reserved.

1.1 1.2 1.3 1.4 1.5 1.6 1.7 1.8

1.1

Introduction Sets, Relations, and Functions Operations and Algebras Sequences Graphs Cardinality Exercises Bibliographical Comments

Introduction

In this chapter we present some mathematical preliminaries intended to provide a referrence for the reader. We discuss basic facts of set theory, universal algebra and the notion of cardinality of a set.

1.2

Sets, Relations, and Functions

In this section we review briefly some concepts from set theory.

1.2.1

Sets

Informally, a set is a collection of objects, called elements. For any set S and object a, either a is one of the objects in S or it is not. In the former case, we use any of the following phrases: "a is a member of 5," "a is an element of 5," or "a is in S," and we write a € S. The elements of a set C may be themselves sets; in such cases, we refer to C as a collection of sets. We use the following notations for various subsets of the set E of real num-

4

Preliminaries

bers: N for the set of P for the set of Q for the set of Z for the set of

natural numbers positive integers rational numbers integers

Let S and T be two sets. The set S is included in T (or 5 is a subset of T) if every element of S is an element of T. In this case, we write 5 C T. If S is not a subset of T, we write S p. Conversely, suppose that P = |J D P and let x G P and (ar, y) G p. Since P is a union of blocks of VP it follows that [x]p C P , so 2/ G P because 2/ G [rr]p. I Theorem 1.2.29 Lei M be a set and let p,p' be equivalence relations on M. The following statements are equivalent:

1.2 Sets, Relations, and Functions

11

(i) P C p'; (ii) every p' -saturated set is p-saturated; (hi) every block of the quotient set M/p' is p-saturated. Proof. (1) implies ( 2 ) : Suppose that p C p' and let P be a subset of M that is p'-saturated. Let x G M and assume that (x,y) G p. Then (x,y) G p' and this implies y G P. Therefore, P is p-saturated. (2) implies (3) : Suppose that every p'-saturated set is p-saturated. Since every block of M/p' is p'-saturated it follows that each such block is p-saturated. (3) implies (1): Assume that every block of the quotient set M/p' is psaturated and let (x, y ) G p , s o y 6 [x]p. Note that [x]p> is p-saturated; therefore, [x]p C [x]p> and this implies y G [x]p>. Therefore, y G [x]p>, so (x,y) G p'. ■ Corollary 1.2.30 Let M be a set and let p,p' be equivalence relations on M such that p C p'. If M/p is a finite set, then so is M/p'. Proof. Since every block of the quotient set M/p' is the union of blocks of the quotient set M/p it is immediate that M/p' is finite. I Theorem 1.2.31 Let p be an equivalence on a set M. If L,L' C M are two p-saturated sets, then LUL', LC\L' and M - L are p-saturated languages. Proof. We leave this straightforward argument to the reader.

Copyright © 1999. World Scientific Publishing Company. All rights reserved.

1.2.5

I

Partial Orders

Definition 1.2.32 A partial order on a set M is a relation p C M x M that is reflexive, antisymmetric and transitive. U Example 1.2.33 Define the relation p C N x N by p = {(m, n) G N x N | n = mp for some p G N}. It is easy to see that p is reflexive and transitive. Suppose that (ra,n) G p and (n, m) G p. Then, there are p, q G N such that n = mp and m = nq. Note that n = 0 if and only if m = 0; otherwise, we have n = nqp, so pq = 1 because n # 0. This implies p = q = l,so m = n. 0 Definition 1.2.34 A partially ordered set (or, in short a pose£) is a pair (M, p), where M is a set and p is a partial order on M. U Example 1.2.35 The pair (N,p) is a poset, where p is the partial order intro­ duced in Example 1.2.33. If S is a set, it is easy to verify that the pair (0>(S), C) is a poset. □ In general, partial ordered relations are denoted using the symbol " 0, then m]CHILDREN(v) is a bijection between the sets CHILDREN(v) and the set {(0, a i o ) , . . . , ( n - 1 , a i n _ J } , for some aio,... ,ain_x e A. Also, m(v0) = (0, a) for some a € A. fj Assume that T = (G, v 0 , M, m) is a labeled ordered tree, and let v 6 V. If the vertex v has n children, then we refer to the child u of v for which m(u) = (0, aio) as the leftmost child ofv; the child w of v for which m(w) = (n - 1, a i n _ J is the rightmost child of v. Example 1.5.25 Consider the labeled ordered tree T=(G,v0,Nx{a,6,c},m) represented in Figure 1.7, where V = {Vi\0 given by T = (V,p,v 0 ,N x A,m). Here V = \J{Vt | 0 < i < n - 1}, E = [){Ei | 0 < i < n - 1} U {(v0,t>oi) I 0 < i < n - 1}, and A = |J{^i I 0 < i < n - 1}. The marking function is

Copyright © 1999. World Scientific Publishing Company. All rights reserved.

m(v)

tiveVi{voi} if v = voi and m(v) = (0,ai) if v = ^o,

for v € V. Example 1.5.26 Let T0,Ti and T2 be the labeled ordered trees shown in Fig­ ure 1.8(a),(b) and (c), respectively. The tree T = (T0,Ti,T2;6> is shown in U Figure 1.8(d). Definition 1.5.27 Let T = (G, v 0 , M, m) be a labeled ordered tree, where G = (V, £ , s, d), and let u be a vertex of this tree. Denote by Vu the set of all vertices in V such that there exists a path whose origin is u and whose end is v. The v-labeled ordered subtree of T is the labeled ordered tree Tv = (Gu, u, Af, m | K ) , where Gu is the subgraph generated by the set Vu. U Example 1.5.28 The vi-labeled ordered subtree of the labeled ordered tree shown in Figure 1.7 is the labeled ordered tree shown in Figure 1.9. □ To simplify the representation of labeled ordered trees, from now on, we will show only the second component of the value of the labelling function for each vertex of the tree. Also, the names of the vertices will be omitted. For example, the tree represented in Figure 1.7 will be drawn as shown in Figure 1.10.

1.6

Cardinality

The purpose of this section is to present a formalization of the notion of size of a set. Definition 1.6.1 Two sets, A,B have the same cardinality, written A ~ B, if U there exists a bijection / : A —> B.

38

Preliminaries

(O.o)

(l.c)

(!,«») (a)

(b)

(c)

(0,6)

t0, a)

Copyright © 1999. World Scientific Publishing Company. All rights reserved.

((0,6)

MO, a)

fc

K2,6)

V(l,c)

1(0, c)

^(l,o)

(1,6) (d)

Figure 1.8: Construction of the tree (T0, Ti, T2; b)

(°' a )

(l,o)

(2,c)

(3,6)

Figure 1.9: A Labeled Ordered Subtree of a Labeled Ordered Tree

39

1.6 Cardinality

Copyright © 1999. World Scientific Publishing Company. All rights reserved.

Figure 1.10: Labeled Ordered Tree Example 1.6.2 The set of even numbers, E = {n \ n = 2k, for some k e N} and the set N have the same cardinality, because / : N —> E defined by f(n) = 2n is a bijection. Theorem 1.6.3 The relation ~ is an equivalence relation. Proof. For every set A, 1A : A —» A is a bijection. Therefore, A ~ A for every A, so - is reflexive. If / : A —> B is a bijection, then / x : A —> A is a bijection, so A ~ B implies B ~ A, which shows that ~ is symmetric. Transitivity follows from the fact that the composition of two bijections is a bijection. Theorem 1.6.4 IfA~B, then 9(A) ~ 9(B). Proof. Let / : A —► B be a bijection between A and B. Define the mapping F . y(^) _ > y(B) by F(L) = {be B \ b = f(a) for some a G L} for every L G 3>(A). It is easy to verify that F is a bijection. Thus, 9(A) ~ 9(B). I Definition 1.6.5 A set A is countable if it has the same cardinality as a subset of N. A is finite if there is an integer k G N such that A has the same cardinality as a subset of { 0 , 1 , . . . , k}. Note that any finite set is countable. The following theorem will help us enumerate finite sets. Theorem 1.6.6 If A is finite, then there is a unique k G N for which A ~ { 0 , 1 , . . . , f c - l } . In this case, we write \A\ = k and say that "A has k elements." Proof. Assume A is finite. Let M = {m G N | A has the same cardinality as some subset of { 0 , 1 , . . . ,m - 1}}- Since A is finite, M # 0, so M has a least element, k, which clearly satisfies the requirements of the theorem. ■ Often it is desirable to be able explicitly to enumerate the elements of A. If A is finite, with \A\ = k, then there is a bijection / : { 0 , 1 , . . . k - 1} —► A, and we can enumerate A = { a 0 , a u . . . , a k - i ] , where a* = f(i). If A is infinite but countable, we write |A| = N0 and say "A is countably infinite." x The following theorem permits us to enumerate countably infinite sets. i T h e symbol N (pronounced "aleph") is the first letter of the Hebrew alphabet. This notation is standard in set theory.

40

Preliminaries

Theorem 1.6.7 l'f\A\ = N0, then there is a bisection f : N —> A. Proof. Since A is countable, there is a bijection g : A —> S C N. To define / : N —► A inductively, we simultaneously define both / and a sub­ set of S. Let /(Q) = 0 - 1 (so), where s 0 is the smallest element in S. As­ sume { / ( 0 ) , / ( l ) , . . . f(k - 1)} and {s 0 , s i , . . . sk-i} have been defined. Then define f(k) = g ^.s*), where sk is the smallest element in S - {s 0 , * i , . . . s*-i}Since A is infinite, S is also infinite, so S - {*o,*i,...**-i} # 0, and a smallest element always exists. By construction, if m 0 < mx then / ( m i ) £ {/(0)>/(l),--./(ra o )}, s i n c e P is a bijection (and hence g'1 is, too.) So, if /(m 0 ) = / ( m i ) then clearly m 0 = mi. We have to check that / is also onto. An easy induction shows that sk > k, for all k € N. Let a € A, with g(a) = m. Then, m = s, for some j < m, so f(sj) = a. |

Copyright © 1999. World Scientific Publishing Company. All rights reserved.

Corollary 1.6.8 / / | A | = ^ 0 , then there i$ a bijection g : A —► N. Proof. This follows from the fact that the inverse of a bijection is again a bijection. ■ If A is countably infinite, then we can "enumerate" A using the the bijection of Theorem 1.6.7. Thus, we have A = { a 0 , a i , a 2 , . . . } , where, just as in the finite case, a{ = /(»). On the other hand, Corollary 1.6.8 allows us to associate a natural number with each element of a countably infinite set. Next we give a useful characterization of countable sets. Theorem 1.6.9 A set A is countable if and only if there exists an injection f :A —► N. Proof. The necessity of the condition is immediate since every bijection is also an injection. Suppose, therefore, that / : A —► N is an injection. Then, the function g : A —► Ran(/i) is obviously a bijection between A and Ran(/i), a subset of N, so A is indeed countable. ' | Theorem 1.6.10 Let A,B be two countable sets. Then, AUB is countable. Proof. Assume A, B are two countable sets, and let / : A —► N and g : B N be injections. Define h: AUB —> N by n

W-{

2p(x) + l

►

if*€*.

The function h : A U B —► N is easily seen to be an injection; hence, A U B is countable. ■ Corollary 1.6.11 The union of any finite collection of countable sets is count­ able. Proof. Use proof by induction and Theorem 1.6.10. | A stronger version of Corollary 1.6.11 is shown below (cf. Theorem 1.6.7). The following Theorem and its Corollary simplify the proofs of several theorems. Theorem 1.6.12 Let A,B,C be sets, where A is countable. (i) / / there is a surjection f :A —► B, then B is countable. (ii) / / there is an injection £ : C —► A, then C is countable.

41

1.6 Cardinality

Proof. For the first part of the theorem assume A is countable and / : A —> B is a surjection. Since A is countable, there is an injection g : A —> N. Define h : B —► N by h(b) = mm{g(a) \ f(a) = b}. We need to verify that h is an injection. Let 60, h e B such that h(b0) = h(h). Let Oi € A be the element such that /i(M = gfa) for i = 0,1. Then, g(a0) = g(ai), and since g is an injection, a0 = au so f(a0) = / ( a i ) , and thus 60 = &iFor the second part note that the function gl: C —-► N is an injection; this implies immediately the countability of C. ■ Corollary 1.6.13 Let A,B be two sets. If f : A —> B is a bijection, then A is countable if and only if B is countable. Proof. The statement is an immediate consequence of Theorem 1.6.12. I Corollary 1.6.14 Let A,B be two sets. If f : A —► B is an injection and A is not countable, then B is not countable. Proof. The statement is an immediate consequence of Theorem 1.6.12. I

Copyright © 1999. World Scientific Publishing Company. All rights reserved.

Corollary 1.6.15 Any subset of a countable set is countable. Proof. Assume B C A, where A is countable. If B = 0, then it is clearly countable. If B ^ 0, pick b G B, and define / : A —■> B by / x

ifxGJS

The function / is clearly a surjection, so £ is countable.

■

Theorem 1.6.16 LetA0,... ,A„-i 6e n cotmfo&Ze sets. The Cartesian product AQ x ■ • • x A n _i is countable. Proof. Since A 0 , . . . , A n _i are countable sets, by Corollary 1.6.8, there exist injections ft : Ai —> N for 0 < i < n-1. For (a 0 ,... ,a„_i) € A0 x • • • x A„_i, define Mao,...,an_1) = 2 ^ o ) . 3 ^ ( - )

p-llS

where pi_i is the i t h prime number for 0 < i < n - 1. By the Fundamental Theorem of Arithmetic2, h : A0 x • • • x A n _i —> N is an injection, so A0 x ■ • • x A„_i is countable. Example 1.6.17 Let D be a countable set. The set Seqn(£>) = £>n is a countable set for every n € N. For the purposes of defining bijections, it is sometimes convenient to be able to force sets in a collection that might not be pairwise disjoint to be disjoint. We do this by making a copies of the sets, each "marked" with its index. Once they are marked, we may combine the sets and show the result is countable. Then we 2 The Fundamental Theorem of Arithmetic states that each natural number larger than one can be written uniquely as a product powers of primes.

42

Preliminaries

remove the markings with a projection function to obtain the desired, final set. Since the projection is a surjection from the set built with the marked sets onto this resultant set, Theorem 1.6.12 guarantees that the final set is countable. Corollary 1.6.19 gives an example of this technique. Let P02 and P? be the usual projection functions, so Pf(xQ,xi) = xu for i = 0,1. Theorem 1.6.18 The union of a countable collection of countable sets that are pairwise disjoint, is a countable set. Proof. Let K be a countable set, and let each {Ak \ k € K} be countable. Then there are injections / : K —> N and gk : Ak —y N for each k N. Let P = {p 0 ,Pi,... } be an enumeration of the prime numbers. Since the sets Ak are pairwise disjoint, given any a e A, there is a unique k with a e Ak. We use this fact to define

It follows from the Fundamental Theorem of Arithmetic that h is an injection, and thus A is countable. | Corollary 1.6.19 The union of a countable collection of countable sets is a countable set. Proof. Let K be a countable set, and let each {Ak \ k e K} be countable. Form sets A'k = Ak x {k}. These are clearly pairwise disjoint, so A' = |^J A'k is countable. k£K

Let

A = |J Ak. keK

The projection P02 : A' —> A is a surjection, and thus by Theorem 1.6.12 A is countable. ■ Example 1.6.20 We proved that if D is a countable set, Seqn(£>) is countable (cf. Example 1.6.17). Therefore, Seq(D) = U P " | n € N} is countable as a union of a countable collection of sets. □ Proofs related to that of Theorem 1.6.18 are important in the area of the theory of computation, so let us revisit it. The set N x N is countably infinite, so, by Corollary 1.6.8, there exists a bijection p : N x N > N. Definition 1.6.21 A pairing function is a bijection p : N x N —> N. D

1.6 Cardinality

43

Example 1.6.22 There are many possible pairing functions, but consider the one suggested by the following picture: 3 p(*»i) 3 0 i 1 3 6 10 0 0 4 7 11 ... 2 1 12 5 2 13 9 3 14 4 Let Dm be the diagonal that contains all pairs (i, j) such that t + j = „„. m. It is clear that Dm contains ra + 1 pairs. Note that the pair (ij) is located on the diagonal Di+j and that this diagonal is preceded by the diagonals Do, • • , Di+j-i that have a total of 1 + 2 + • • • + (t + j) = (* + j)(» + j + l)/2 elements. Thus, the pair (*, j) is enumerated on the place (i + j)(i + j + l)/2 + t and this shows that the mapping h : N x N —► N given by

Copyright © 1999. World Scientific Publishing Company. All rights reserved.

p(*,i) = |[(« + i) 2 + ^ + il

(L8)

is a bijection. This is a way of counting a countably infinite collection of copies of N. We can turn it into a way of counting many similar things, such as the sets Ak of Theorem 1.6.18. For our purposes here, we will assume that K = N and each of the sets Ak is infinite (that was not the case in the theorem). In position (t, j ) , we place the j t h element of the z th set. Technically, this is the element g^l(j) of the set A{. If we tried to count the first set, A0, all the way to the end, we would never get to the second set, so instead, we count one element from A0. Then we count a second element from A0 and one from the second set, Ax. Then we count a third from the first set, a second from the second, and one from the third, and so forth. This is called "dovetailing." In this way, we eventually count all possible elements in the set [){Ai | i € N}. So, the proof relies on using such a counting as a way to navigate through a countably infinite collection of sets, each of which is countable. Dovetailing is an important proof technique in computability and we shall use it in Chapter 8. It is important to realize that not all sets are countable. Consider 3>(N), the power set of N. This certainly has at least as many elements as N, since {k} is in 3>(N) for each k £ N. However, it has so many more sets that it is not possible to count them all. Theorem 1.6.23 The set ?(N) is not countable. Proof. Assume that 3>(N) were countable. Then there would be an bijection f : N ► ?(N); i.e., for each n G N, we would have a distinct subset f(n) C N. We show that the existence of this bijection leads to a contradiction. Define the set D = {n | n , a / | a

floo ^01 ^02 ao3 ao4 • aio a n &12 ai3 014 . ^20 ^21 0 2 2 0 2 3 &24 • ^30 ^31 0>32 « 3 3 ^34 • ei4o a 4 i 0 4 2 0 4 3 0 4 4 . ^ 5 0 ^51 0>52 0>53 « 5 4 •

a>ko a>ki Q>k2 a>kz a k 4

...Q>kk

where

Copyright © 1999. World Scientific Publishing Company. All rights reserved.

{:

if j € / ( t ) .

In other words, the ai>;s correspond to the characteristic function of the set /(»). The set 23 is formed by "going down the diagonal" and spoiling the possibility that D = f(k), for each A;. At row A:, we look at akk in column k. If this is 1, i.e., if k G f(k), then we make sure that the corresponding position for the set D has a 0 in it by saying that k