Unimodal and Multimodal Biometric Data Indexing 9781614516293, 9781614517450

Citation preview

Somnath Dey, Debasis Samanta Unimodal and Multimodal Biometric Data Indexing

Somnath Dey, Debasis Samanta

Unimodal and Multimodal Biometric Data Indexing

ISBN 978-1-61451-745-0 e-ISBN 978-1-61451-629-3 Library of Congress Cataloging-in-Publication data A CIP catalog record for this book has been applied for at the Library of Congress. Bibliographic information published by the Deutsche Nationalbibliothek The Deutsche Nationalbibliothek lists this publication in the Deutsche Nationalbibliografie; detailed bibliographic data are available in the Internet at http://dnb.dnb.de. © 2014 Walter de Gruyter, Inc., Boston/Berlin Cover image: Bram Janssens/Hemera/Thinkstock Printing: CPI buch bücher.de GmbH, Birkach ♾ Printed on acid-free paper Printed in Germany www.degruyter.com

Preface Biometric-based person identification continues to gain in importance as institutions of our society come to rely on biometrics to assist with many everyday functions in addition to using biometrics in law enforcement, military and other commercial pursuits. As the uses for biometrics expand, we need to be able to process ever larger amounts of biometric data in the order of millions of data points in order to provide reliable responses to queries. Several large scale applications use multiple biometric traits, such as combination of iris and fingerprint, fingerprint and face, fingerprint and hand geometry for authentication purpose. Consequently, many of the traditional approaches for authentication where the identity of a query template is decided by matching the query template with all stored templates are impractical. This work is on biometric data indexing for large-scale identification systems with a focus on different biometrics data indexing methods. It provides the state-of-the-art coverage including different biometric traits, together with the pros and cons for each. Discussion of different multimodal fusion strategies are also included. We introduce the reader to unimodal and multimodal biometric traits with large scale biometric data so matching processes can be accomplished in real-time without compromising the accuracy of the identification system. Summary background information on basic concepts, biometric technologies, and advanced research, is also provided. Some of the unique contributions of this work are: showing how to generate index i

Preface keys from iris, fingerprint and face biometric; generating index keys for multimodal system; proposing new indexing mechanisms; providing iris texture-based indexing technique; minutiae-based fingerprint indexing technique and SURF feature-based face indexing technique; introducing new combining and ranking methods for multimodal biometrics; and, applying hash functions on index keys to extract key points of a query image, and others. This book would not have been possible without the support and encouragement of many people. We would like to take this opportunity to express our sincere thanks and gratitude to everyone who assisted us along the way. We would like to thank Prof. D. Roy Chowdhury, Prof. M. K. Tiwari, Prof. I. Sengupta, Prof. J. Mukhopadhyay, Dr. K. S. Rao, Dr. S. Misra and Dr. Monalisa Sarma. We wish to convey our special thanks to Sayani, Chandan, Bodhi, Aparna, Debasish, Sayan, Manoj, Pradipta, Soumalya, Santa for their constant support. We also thank all the members of our family. We would like to thank the Indian Institute of Technology Kharagpur and the Indian Institute of Technology Indore. Finally, we express our sincere thanks to de Gruyter publishing house for the production of the book. Somnath Dey Debasis Samanta June 2014

ii

Contents Preface

i

Contents

iii

List of Figures

ix

List of Tables

xix

1 Fundamentals of Biometric Technology

1

1.1

Biometric Authentication Technology . . . . . . . . . .

1

1.2

Some Major Biometric Applications

. . . . . . . . . .

2

1.3

Operational Process of Biometric Technology . . . . .

4

1.4

Biometric Data Indexing . . . . . . . . . . . . . . . . .

7

1.5

Metrics for Performance Measure . . . . . . . . . . . .

7

1.6

Biometric Modalities . . . . . . . . . . . . . . . . . . .

8

1.6.1

Iris Biometric . . . . . . . . . . . . . . . . . . .

9

1.6.2

Fingerprint Biometric . . . . . . . . . . . . . .

9

1.6.3

Face Biometric . . . . . . . . . . . . . . . . . .

9

1.6.4

Palmprint Biometric . . . . . . . . . . . . . . .

10

1.6.5

Hand Geometry Biometric . . . . . . . . . . . .

10

1.6.6

Voice Biometric . . . . . . . . . . . . . . . . . .

11

1.6.7

Gait Biometric . . . . . . . . . . . . . . . . . .

12

1.6.8

Signature Biometric . . . . . . . . . . . . . . .

12

iii

Contents 1.7

1.8

Comparative Study of Different Biometric Modalities .

13

1.7.1

Identification of Parameters . . . . . . . . . . .

13

1.7.2

Estimation of Values of Parameters . . . . . . .

14

1.7.3

Estimation of Impact Value . . . . . . . . . . .

15

1.7.4

Quantitative Comparison . . . . . . . . . . . .

21

Summary . . . . . . . . . . . . . . . . . . . . . . . . .

21

2 Multimodal Biometric and Fusion Technology

33

2.1

Multimodal Biometric Authentication Technology . . .

33

2.2

Fusion of Multimodalities . . . . . . . . . . . . . . . .

34

2.3

Fusion Levels . . . . . . . . . . . . . . . . . . . . . . .

36

2.3.1

Sensor Level Fusion . . . . . . . . . . . . . . .

36

2.3.2

Feature Level Fusion . . . . . . . . . . . . . . .

38

2.3.3

Match-score Level Fusion . . . . . . . . . . . .

40

2.3.4

Decision Level Fusion . . . . . . . . . . . . . .

41

Different Fusion Rules . . . . . . . . . . . . . . . . . .

42

2.4.1

Fixed fusion rules . . . . . . . . . . . . . . . . .

42

2.4.2

Trained Fusion Rules . . . . . . . . . . . . . . .

45

2.5

Comparative Study of Fusion Rule . . . . . . . . . . .

62

2.6

Summary . . . . . . . . . . . . . . . . . . . . . . . . .

68

2.4

3 Biometric Indexing: State-of-the-Art 3.1

79

Survey on Iris Biometric Data Indexing . . . . . . . .

79

3.1.1

Iris Texture-Based Indexing . . . . . . . . . . .

80

3.1.2

Iris Color-Based Indexing . . . . . . . . . . . .

82

Survey on Fingerprint Biometric Data Indexing . . . .

83

3.2.1

Minutiae-Based Indexing . . . . . . . . . . . . .

85

3.2.2

Ridge Orientation-Based Indexing . . . . . . .

88

3.2.3

Other Feature-Based Indexing Techniques . . .

92

3.3

Survey on Face Biometric Data Indexing . . . . . . . .

94

3.4

Survey on Multimodal Biometric Data Indexing . . . .

95

3.5

Summary . . . . . . . . . . . . . . . . . . . . . . . . .

97

3.2

iv

Contents 4 Iris Biometric Data Indexing

111

4.1

Preliminaries of Gabor Filter . . . . . . . . . . . . . . 112

4.2

Preprocessing . . . . . . . . . . . . . . . . . . . . . . . 115

4.3

Feature Extraction . . . . . . . . . . . . . . . . . . . . 117

4.4

Index Key Generation . . . . . . . . . . . . . . . . . . 118

4.5

Storing . . . . . . . . . . . . . . . . . . . . . . . . . . . 119 4.5.1

Index Space Creation . . . . . . . . . . . . . . . 119

4.5.2

Storing Iris Data . . . . . . . . . . . . . . . . . 120

4.6

Retrieving . . . . . . . . . . . . . . . . . . . . . . . . . 124

4.7

Performance Evaluation . . . . . . . . . . . . . . . . . 127 4.7.1

Performance Metrics . . . . . . . . . . . . . . . 128

4.7.2

Databases . . . . . . . . . . . . . . . . . . . . . 130

4.7.3

Evaluation Setup . . . . . . . . . . . . . . . . . 131

4.7.4

Validation of the Parameter Values . . . . . . . 132

4.7.5

Evaluation . . . . . . . . . . . . . . . . . . . . . 134

4.8

Comparison with Existing Work . . . . . . . . . . . . . 141

4.9

Summary . . . . . . . . . . . . . . . . . . . . . . . . . 143

5 Fingerprint Biometric Data Indexing 5.1

5.2

149

Preprocessing . . . . . . . . . . . . . . . . . . . . . . . 150 5.1.1

Normalization . . . . . . . . . . . . . . . . . . . 150

5.1.2

Segmentation . . . . . . . . . . . . . . . . . . . 151

5.1.3

Local Orientation Estimation . . . . . . . . . . 152

5.1.4

Local Frequency Image Representation . . . . . 152

5.1.5

Ridge Filtering . . . . . . . . . . . . . . . . . . 153

5.1.6

Binarization and Thinning . . . . . . . . . . . . 154

5.1.7

Minutiae Point Extraction . . . . . . . . . . . . 154

Feature Extraction . . . . . . . . . . . . . . . . . . . . 157 5.2.1

Two Closest Points Triangulation . . . . . . . . 157

5.2.2

Triplet Generation . . . . . . . . . . . . . . . . 158

5.3

Index Key Generation . . . . . . . . . . . . . . . . . . 160

5.4

Storing . . . . . . . . . . . . . . . . . . . . . . . . . . . 163 v

Contents

5.5

5.6

5.4.1

Linear Index Space . . . . . . . . . . . . . . . . 164

5.4.2

Clustered Index Space . . . . . . . . . . . . . . 165

5.4.3

Clustered kd-tree Index Space . . . . . . . . . . 168

Retrieving . . . . . . . . . . . . . . . . . . . . . . . . . 173 5.5.1

Linear Search (LS) . . . . . . . . . . . . . . . . 174

5.5.2

Clustered Search (CS) . . . . . . . . . . . . . . 175

5.5.3

Clustered kd-tree Search (CKS) . . . . . . . . . 177

Performance Evaluation . . . . . . . . . . . . . . . . . 178 5.6.1

Databases . . . . . . . . . . . . . . . . . . . . . 178

5.6.2

Evaluation Setup . . . . . . . . . . . . . . . . . 181

5.6.3

Evaluation . . . . . . . . . . . . . . . . . . . . . 182

5.6.4

Searching Time . . . . . . . . . . . . . . . . . . 192

5.6.5

Memory Requirements . . . . . . . . . . . . . . 195

5.7

Comparison with Existing Work . . . . . . . . . . . . . 195

5.8

Summary . . . . . . . . . . . . . . . . . . . . . . . . . 199

6 Face Biometric Data Indexing 6.1

6.2

205

Preprocessing . . . . . . . . . . . . . . . . . . . . . . . 206 6.1.1

Geometric Normalization . . . . . . . . . . . . 206

6.1.2

Face Masking . . . . . . . . . . . . . . . . . . . 208

6.1.3

Intensity Enhancement . . . . . . . . . . . . . . 209

Feature Extraction . . . . . . . . . . . . . . . . . . . . 210 6.2.1

Key Point Detection . . . . . . . . . . . . . . . 211

6.2.2

Orientation Assignment . . . . . . . . . . . . . 215

6.2.3

Key Point Descriptor Extraction . . . . . . . . 217

6.3

Index Key Generation . . . . . . . . . . . . . . . . . . 218

6.4

Storing . . . . . . . . . . . . . . . . . . . . . . . . . . . 220

6.5

6.4.1

Index Space Creation . . . . . . . . . . . . . . . 220

6.4.2

Linear Storing Structure . . . . . . . . . . . . . 223

6.4.3

Kd-tree Storing Structure . . . . . . . . . . . . 223

Retrieving . . . . . . . . . . . . . . . . . . . . . . . . . 225 6.5.1

Linear Search . . . . . . . . . . . . . . . . . . . 228 vi

Contents 6.5.2 6.6

Kd-tree Search . . . . . . . . . . . . . . . . . . 230

Performance Evaluation . . . . . . . . . . . . . . . . . 232 6.6.1

Database . . . . . . . . . . . . . . . . . . . . . 232

6.6.2

Evaluation Setup . . . . . . . . . . . . . . . . . 234

6.6.3

Validation of the Parameter Value . . . . . . . 237

6.6.4

Evaluation . . . . . . . . . . . . . . . . . . . . . 239

6.7

Comparison with Existing Work . . . . . . . . . . . . . 251

6.8

Summary . . . . . . . . . . . . . . . . . . . . . . . . . 252

7 Multimodal Biometric Data Indexing

257

7.1

Feature Extraction . . . . . . . . . . . . . . . . . . . . 259

7.2

Score Calculation . . . . . . . . . . . . . . . . . . . . . 261

7.3

Reference Subject Selection . . . . . . . . . . . . . . . 262 7.3.1

Sample Selection . . . . . . . . . . . . . . . . . 262

7.3.2

Subject Selection . . . . . . . . . . . . . . . . . 264

7.4

Reference Score Calculation . . . . . . . . . . . . . . . 266

7.5

Score Level Fusion . . . . . . . . . . . . . . . . . . . . 267 7.5.1

Score Normalization . . . . . . . . . . . . . . . 267

7.5.2

Score Fusion . . . . . . . . . . . . . . . . . . . 268

7.6

Index Key Generation . . . . . . . . . . . . . . . . . . 270

7.7

Storing . . . . . . . . . . . . . . . . . . . . . . . . . . . 272 7.7.1

Index Space Creation . . . . . . . . . . . . . . . 272

7.7.2

Storing Multimodal Biometric Data . . . . . . 273

7.8

Retrieving . . . . . . . . . . . . . . . . . . . . . . . . . 275

7.9

Rank Level Fusion . . . . . . . . . . . . . . . . . . . . 278 7.9.1

Creating Feature Vector for Ranking . . . . . . 279

7.9.2

SVM Ranking . . . . . . . . . . . . . . . . . . . 279

7.10 Performance Evaluation . . . . . . . . . . . . . . . . . 282 7.10.1 Database . . . . . . . . . . . . . . . . . . . . . 282 7.10.2 Evaluation Setup . . . . . . . . . . . . . . . . . 285 7.10.3 Training of SVM-based Score Fusion Module . 286 7.10.4 Training of SVM-based Ranking Module . . . . 286 vii

Contents 7.10.5 Validation of the Parameter Values . . . . . . . 287 7.10.6 Evaluation . . . . . . . . . . . . . . . . . . . . . 290 7.11 Comparison with Existing Work . . . . . . . . . . . . . 296 7.12 Summary . . . . . . . . . . . . . . . . . . . . . . . . . 298 8 Conclusions and Future Research

305

8.1

Dimensionality of Index Key Vector . . . . . . . . . . . 305

8.2

Storing and Retrieving . . . . . . . . . . . . . . . . . . 308

8.3

Performance of Indexing Techniques . . . . . . . . . . 310

8.4

Threats to Validity . . . . . . . . . . . . . . . . . . . . 312 8.4.1

Internal Validity . . . . . . . . . . . . . . . . . 312

8.4.2

External Validity . . . . . . . . . . . . . . . . . 314

8.4.3

Construct Validity . . . . . . . . . . . . . . . . 314

8.5

Future Scope of Work . . . . . . . . . . . . . . . . . . 315

Index

323

viii

List of Figures 1.1

Examples of biometric traits that can be used for authenticating an individual. Physical traits include fingerprint, iris, face and hand geometry while behavioral traits include signature, keystroke dynamics and gait [43].

1.2

Different government and commercial applications which use biometric authentication system [43, 26]. . . . . . .

5

(a)

Iris recognition at Amsterdam Schiphol airports .

5

(b)

Fingerprint verification system between local banks and the Department of Home Affairs (DHA) . .

(c) (d) 1.3

3

5

Iris recognition system in the e-airport at Tokyo Narita Airport . . . . . . . . . . . . . . . . . . .

5

Contactless palm-vein systems in ATMs in Japan.

5

Different modes of operations of a biometric authentication system. . . . . . . . . . . . . . . . . . . . . . . .

6

(a)

Enrollment steps of an authentication system. . .

6

(b)

Steps of an authentication system in verification mode . . . . . . . . . . . . . . . . . . . . . . . .

(c)

6

Steps of an authentication system in identification mode . . . . . . . . . . . . . . . . . . . . . . . .

6

1.4

Major steps in identification system with indexing. . .

7

2.1

Block diagram of the different levels of fusion. . . . . .

35

2.2

Multilayer perceptron network. . . . . . . . . . . . . .

50

ix

List of Figures 2.3

Block diagram of Mixture-of-Experts. . . . . . . . . . .

4.1

An overview of the proposed iris biometric data index-

52

ing approach. . . . . . . . . . . . . . . . . . . . . . . . 112 4.2

Preprocessing result of a sample iris image [12, 14]. . . 116 (a)

Original eye image . . . . . . . . . . . . . . . . . 116

(b)

Pupil boundary detected eye image . . . . . . . . 116

(c)

Iris boundary detected eye image . . . . . . . . . 116

(d)

Localized iris image . . . . . . . . . . . . . . . . 116

(e)

Normalized iris image . . . . . . . . . . . . . . . 116

(f)

Enhanced iris image . . . . . . . . . . . . . . . . 116

4.3

Index space organization. . . . . . . . . . . . . . . . . 120

4.4

Index keys of the 6th subject with 5 samples. . . . . . 122

4.5

Retrieving a match for a query iris image from a database.127

4.6

HR and P R with different scales and orientations. . . 133

4.7

(a)

HR with different scales and orientations . . . . 133

(b)

P R with different scales and orientations . . . . 133

HR and P R for different values of δ. . . . . . . . . . . 135 (a)

HR for different values of δ . . . . . . . . . . . . 135

(b)

P R for different values of δ . . . . . . . . . . . . 135

4.8

CM C curves with reference to different databases. . . 136

4.9

F P IR vs. F N IR curves with and without indexing for CASIA-V3-Interval database. . . . . . . . . . . . . . . 137

4.10 Average table size and memory requirements to enroll the iris data for different iris image databases. . . . . . 140 (a)

Average table size to enroll iris data for different iris image databases . . . . . . . . . . . . . . . . 140

(b)

Memory requirements to enroll iris data for different iris image databases . . . . . . . . . . . . . 140

5.1

An overview of the proposed fingerprint biometric data indexing approach. . . . . . . . . . . . . . . . . . . . . 150 x

List of Figures 5.2

Preprocessing of fingerprint image. . . . . . . . . . . . 151

5.3

Pixel position and crossing number for two different types of minutiae. . . . . . . . . . . . . . . . . . . . . . 155

5.4

5.5

(a)

Pixel position in a 3 × 3 window . . . . . . . . . 155

(b)

CN=1 (ridge ending) . . . . . . . . . . . . . . . . 155

(c)

CN=3 (bifurcation) . . . . . . . . . . . . . . . . 155

Minutiae position and minutiae orientation. . . . . . . 156 (a)

Ridge ending . . . . . . . . . . . . . . . . . . . . 156

(b)

Bifurcation . . . . . . . . . . . . . . . . . . . . . 156

Two closest point triangulation with sample fingerprint. 159 (a)

Two closest point triangulation . . . . . . . . . . 159

(b)

Two closest point triangle with sample minutiae points of a fingerprint image

. . . . . . . . . . . 159

(c)

Duplicate two closest point triangles . . . . . . . 159

(d)

Duplicate triplets . . . . . . . . . . . . . . . . . . 159

5.6

Components of a minutiae triplet. . . . . . . . . . . . . 161

5.7

Linear index space. . . . . . . . . . . . . . . . . . . . . 166

5.8

Clustered index space. . . . . . . . . . . . . . . . . . . 167

5.9

Structure of a kd-tree and an example of kd-tree with three dimensional points. . . . . . . . . . . . . . . . . . 170 (a)

Components of a kd-tree node . . . . . . . . . . 170

(b)

3D points . . . . . . . . . . . . . . . . . . . . . . 170

(c)

Kd-tree example . . . . . . . . . . . . . . . . . . 170

5.10 Clustered kd-tree index space. . . . . . . . . . . . . . . 171 5.11 CMC curves with one enrolled sample per fingerprint in LS, CS and CKS for the different databases. . . . . 184 (a)

CMC curves with NIST DB4 database . . . . . . 184

(b)

CMC curves with NIST DB4 Natural database . 184

(c)

CMC curves with FVC 2004 DB1 database . . . 184

(d)

CMC curves with FVC 2004 DB2 database . . . 185

(e)

CMC curves with FVC 2004 DB3 database . . . 185 xi

List of Figures (f)

CMC curves with FVC 2004 DB4 database . . . 185

5.12 CMC curves of LS with the different number of enrolled samples for the different databases. . . . . . . . . . . . 187 (a)

CMC curves of LS with gallery G1, G3 and G5 for FVC 2004 DB1 database . . . . . . . . . . . . 187

(b)

CMC curves of LS with gallery G1, G3 and G5 for FVC 2004 DB2 database . . . . . . . . . . . . 187

(c)

CMC curves of LS with gallery G1, G3 and G5 for FVC 2004 DB3 database . . . . . . . . . . . . 187

(d)

CMC curves of LS with gallery G1, G3 and G5 for FVC 2004 DB4 database . . . . . . . . . . . . 188

5.13 CMC curves of CS with the different number of enrolled samples for the different databases. . . . . . . . . . . . 189 (a)

CMC curves of CS with gallery G1, G3 and G5 for FVC 2004 DB1 database . . . . . . . . . . . . 189

(b)

CMC curves of CS with gallery G1, G3 and G5 for FVC 2004 DB2 database . . . . . . . . . . . . 189

(c)

CMC curves of CS with gallery G1, G3 and G5 for FVC 2004 DB3 database . . . . . . . . . . . . 189

(d)

CMC curves of CS with gallery G1, G3 and G5 for FVC 2004 DB4 database . . . . . . . . . . . . 190

5.14 CMC curve of CKS with the different number of enrolled samples for the different databases. . . . . . . . 191 (a)

CMC curves of CKS with G1, G3 and G5 for FVC 2004 DB1 . . . . . . . . . . . . . . . . . . . 191

(b)

CMC curves of CKS with G1, G3 and G5 for FVC 2004 DB2 . . . . . . . . . . . . . . . . . . . 191

(c)

CMC curves of CKS with G1, G3 and G5 for FVC 2004 DB3 . . . . . . . . . . . . . . . . . . . 191

(d)

CMC curves of CKS with G1, G3 and G5 for FVC 2004 DB4 . . . . . . . . . . . . . . . . . . . 192 xii

List of Figures 5.15 Comparison with existing work for NIST DB4 database. 197 (a)

NIST DB4 . . . . . . . . . . . . . . . . . . . . . 197

(b)

NIST DB4 . . . . . . . . . . . . . . . . . . . . . 197

5.16 Comparison with existing work for NIST DB4 Natural database. . . . . . . . . . . . . . . . . . . . . . . . . . 198 (a)

NIST DB4 Natural . . . . . . . . . . . . . . . . . 198

(b)

NIST DB4 Natural . . . . . . . . . . . . . . . . . 198

5.17 Comparison with existing work for FVC2004 DB1 database.198 6.1

An overview of the proposed face biometric data indexing approach. . . . . . . . . . . . . . . . . . . . . . . . 206

6.2

6.3

6.4

Face images after different preprocessing tasks. . . . . 208 (a)

Face image before align . . . . . . . . . . . . . . 208

(b)

Face image after align . . . . . . . . . . . . . . . 208

(c)

Detected face boundary . . . . . . . . . . . . . . 208

(d)

Fixed size face image . . . . . . . . . . . . . . . . 208

(e)

Masked face image . . . . . . . . . . . . . . . . . 208

(f)

Enhanced face image . . . . . . . . . . . . . . . . 208

Approaches to scale space creation. . . . . . . . . . . . 212 (a)

Scale space creation by reducing image size . . . 212

(b)

Scale space creation by increasing filter size . . . 212

Gaussian partial 2nd order derivative and their approximation. . . . . . . . . . . . . . . . . . . . . . . . . . . 215

6.5

(a)

Derivative in x- direction (Lxx ) . . . . . . . . . . 215

(b)

Derivative in y- direction (Lyy ) . . . . . . . . . . 215

(c)

Derivative in xy- direction (Lxy ) . . . . . . . . . 215

(d)

Approximated derivative in x- direction (Dxx ) . . 215

(e)

Approximated derivative in y- direction (Dyy ) . . 215

(f)

Approximated derivative in xy- direction (Dxy ) . 215

Non-maximal suppression by checking the nearest neighbor in subsequent scale spaces. . . . . . . . . . . . . . 216 xiii

List of Figures 6.6

Haar filters in x- and y- directions and orientation of a key point. . . . . . . . . . . . . . . . . . . . . . . . . . 216 (a)

Haar filter in x-direction . . . . . . . . . . . . . . 216

(b)

Haar filter in y-direction . . . . . . . . . . . . . . 216

(c)

Orientation assignment to a key point . . . . . . 216

6.7

SURF descriptor extraction at a key point.

6.8

Proposed index space to store all index keys of all face images.

6.9

. . . . . . 217

. . . . . . . . . . . . . . . . . . . . . . . . . . 221

Linear index space to store all index keys of the ith cell. 224

6.10 Kd-tree to store the index keys of the ith cell of the index space. . . . . . . . . . . . . . . . . . . . . . . . . 226 6.11 Sample images from FERET, FRGC and CalTech256 databases. . . . . . . . . . . . . . . . . . . . . . . . . . 234 (a)

Frontal face image samples without expression from FERET database . . . . . . . . . . . . . . . . . . 234

(b)

Frontal face image samples with expression from FERET database . . . . . . . . . . . . . . . . . . 234

(c)

Frontal face image samples without expression from FRGC database . . . . . . . . . . . . . . . . . . 234

(d)

Frontal face image samples with expression from FRGC database . . . . . . . . . . . . . . . . . . 234

(e)

Sample face images in outdoor environment from FRGC database . . . . . . . . . . . . . . . . . . 234

(f)

Sample face images from CalTech database . . . 234

6.12 HR and P R with FERET, FRGC and CalTech256 databases for different values of δ. . . . . . . . . . . . . . . . . . 238 (a)

HR and P R with Gallery11 and P robe11 for different values of δ . . . . . . . . . . . . . . . . . . 238

(b)

HR and P R with Gallery21 and P robe21 for different values of δ . . . . . . . . . . . . . . . . . . 238 xiv

List of Figures (c)

HR and P R with Gallery41 and P robe41 for different values of δ . . . . . . . . . . . . . . . . . . 238

6.13 CMC curve with and without indexing for FERET, FRGC and CalTech256 databases. . . . . . . . . . . . 240 (a)

CMC curve with and without indexing for FERET Gallery11 and P robe11 sets . . . . . . . . . . . . 240

(b)

CMC curve with and without indexing for FRGC Gallery21 and P robe21 sets . . . . . . . . . . . . 240

(c)

CMC curve with and without indexing for CalTech256 Gallery41 and P robe41 sets . . . . . . . 240

6.14 CMC curves for different P robe sets with single enrolment of a subject with FERET, FRGC and CalTech256 databases. . . . . . . . . . . . . . . . . . . . . . . . . . 242 (a)

CMC curves for different P robe sets with FERET Gallery11 . . . . . . . . . . . . . . . . . . . . . . 242

(b)

CMC curves for different P robe sets with FRGC Gallery21 . . . . . . . . . . . . . . . . . . . . . . 242

(c)

CMC curves for different P robe sets with CalTech256 Gallery41 . . . . . . . . . . . . . . . . . 242

6.15 CMC curves for different P robe sets with multiple enrolment of a subject with FERET, FRGC and CalTech256 databases. . . . . . . . . . . . . . . . . . . . . 244 (a)

CMC curves for different P robe sets with FERET Gallery12 . . . . . . . . . . . . . . . . . . . . . . 244

(b)

CMC curves for different P robe sets with FRGC Gallery22 . . . . . . . . . . . . . . . . . . . . . . 244

(c)

CMC curves for different P robe sets with CalTech256 Gallery42 . . . . . . . . . . . . . . . . . 244

6.16 FPIR vs. FNIR curve with and without indexing for FERET, FRGC and CalTech256 databases. . . . . . . 246 (a)

FPIR vs. FNIR curve for FERET database . . . 246 xv

List of Figures (b)

FPIR vs. FNIR curve for FRGC database . . . . 246

(c)

FPIR vs. FNIR curve for FRGC database . . . . 246

6.17 Average searching time with different sizes of databases for FERET, FRGC and CalTech256 databases. . . . . 248 (a)

Average searching time for FERET database . . 248

(b)

Average searching time for FRGC database . . . 248

(c)

Average searching time for CalTech256 database

248

6.18 Average number of comparisons with different sizes of databases for FERET, FRGC and CalTech256 databases.249 (a)

Average number of comparisons for FERET database249

(b)

Average number of comparisons for FRGC database249

(c)

Average number of comparisons for CalTech256 database . . . . . . . . . . . . . . . . . . . . . . . 249

6.19 Memory requirements with different sizes of databases for FERET, FRGC and CalTech256 databases. . . . . 250

7.1

(a)

Memory requirements for FERET database . . . 250

(b)

Memory requirements for FRGC database . . . . 250

(c)

Memory requirements for CalTech256 database . 250

Overview of the proposed approach. . . . . . . . . . . 258 (a)

Reference subject selection . . . . . . . . . . . . 258

(b)

Identification system with indexing . . . . . . . . 258

7.2

Representation of multimodal biometric samples. . . . 260

7.3

Index space for the mth reference subject. . . . . . . . 273

7.4

Example of storing the 3rd sample of the 5th subject into the 1st index space of the database. . . . . . . . . 275

7.5

(a)

Index key of the 3rd sample of the 5th subject . . 275

(b)

Database of the 1st index space . . . . . . . . . . 275

Example of retrieving candidate sets for a given query. (a)

278

Tables of the biometric trait (B1) from all index spaces of the database . . . . . . . . . . . . . . . 278

(b)

Index keys for query subject . . . . . . . . . . . . 278 xvi

List of Figures (c) 7.6

Candidate sets for all biometric traits . . . . . . 278

An example of SVM-based rank to each retrieved subject for a query. . . . . . . . . . . . . . . . . . . . . . . 282

7.7

(a)

Candidate sets with retrieved data . . . . . . . . 282

(b)

Feature vectors for all subjects in candidate sets

(c)

Subject with rank after SVM . . . . . . . . . . . 282

282

Sample images of CASIAV3I iris, WVU fingerprint and FRGC face databases. . . . . . . . . . . . . . . . . . . 285 (a)

Sample iris images . . . . . . . . . . . . . . . . . 285

(b)

Sample fin-gerprint images . . . . . . . . . . . . 285

(c)

Sample face images . . . . . . . . . . . . . . . . . 285

7.8

HR and P R with different number of reference subjects. 288

7.9

HR and P R with different number of enrolled samples and different table sizes. . . . . . . . . . . . . . . . . . 290 (a)

HR with different number of enrolled samples and different table sizes . . . . . . . . . . . . . . . . . 290

(b)

P R with different number of enrolled samples and different table sizes . . . . . . . . . . . . . . . . . 290

7.10 HR and P R for different values of δ with the proposed indexing technique. . . . . . . . . . . . . . . . . . . . . 291 7.11 CMC curves of the proposed indexing technique with different combination of biometric traits. . . . . . . . . 292 7.12 F P IR versus F N IR curves of the proposed identification system without any indexing and with iris, fingerprint, face and multimodal based indexing. . . . . . . . 293 7.13 Memory requirements for different number of enrolled samples in the proposed indexing technique. . . . . . . 296 8.1

CMC curves of different indexing approaches. . . . . . 311

xvii

List of Tables 1.1

Rank assigned to each parameter for iris biometric . .

15

1.2

Rank assigned to each parameter for fingerprint biometric 16

1.3

Rank assigned to each parameter for face biometric . .

1.4

Rank assigned to each parameter for palmprint biometric 17

1.5

Rank assigned to each parameter for hand geometry

17

biometric . . . . . . . . . . . . . . . . . . . . . . . . .

18

1.6

Rank assigned to each parameter for voice biometric .

19

1.7

Rank assigned to each parameter for gait biometric . .

19

1.8

Rank assigned to each parameter for signature biometric 20

1.9

Impact value of parameters . . . . . . . . . . . . . . .

20

1.10 Cost of each biometric trait . . . . . . . . . . . . . . .

22

2.1

Fusion rules used at various fusion levels. . . . . . . . .

63

2.2

Summary of state of the art fusion rules and fusion levels 64

3.1

Summary of different iris data indexing techniques . .

3.2

Summary of different fingerprint data indexing techniques 93

3.3

Summary of different face data indexing techniques . .

3.4

Summary of different multimodal biometric data indexing techniques . . . . . . . . . . . . . . . . . . . . . . .

84 96 97

4.1

Characteristics of databases used in our experiments . 132

4.2

HR and P R for different iris image databases . . . . . 136 xix

List of Tables 4.3

Time required (in second) to retrieve best match using indexing and without indexing

. . . . . . . . . . . . . 139

4.4

Comparison of HR and P R with existing work . . . . 142

4.5

Retrieval time complexity and execution time of different indexing mechanisms . . . . . . . . . . . . . . . . . 143

5.1

HR, P R and searching time (ST ) for the different fingerprint databases with single enrollment (gallery G1) into the database . . . . . . . . . . . . . . . . . . . . . 183

5.2

HR, PR and searching time (ST) in LS with the different number of enrollments (Gallery G1, G3 and G5) into the database . . . . . . . . . . . . . . . . . . . . . 186

5.3

HR, P R and searching time (ST ) in CS with the different number of enrollments (gallery G1, G3 and G5) into the database . . . . . . . . . . . . . . . . . . . . . 188

5.4

HR, P R and searching time (ST ) in CKS with the different number of enrollments (gallery G1, G3 and G5) into the database . . . . . . . . . . . . . . . . . . 190

5.5

Execution time and average number of comparisons required in LS, CS and CKS with the different number of enrolled samples (ES) . . . . . . . . . . . . . . . . . . 194

5.6

Memory requirements (in KB) of linear, cluster and clustered kd-tree index spaces . . . . . . . . . . . . . . 196

5.7

Comparison of search complexity and execution time of the existing work . . . . . . . . . . . . . . . . . . . . . 199

6.1

Description of Gallery and P robe sets of FERET, FRGC and CalTech256 face databases. . . . . . . . . . . . . . 235

6.2

Performance of the proposed approach with and without indexing using linear and kd-tree search . . . . . . 241

6.3

Performance of different P robe sets with single enrolment of a subject in linear and kd-tree search . . . . . 243 xx

List of Tables 6.4

Performance of different P robe sets for multiple enrolments of a subject in linear and kd-tree search . . . . . 245

6.5

Comparison of the proposed approach with existing approaches . . . . . . . . . . . . . . . . . . . . . . . . . . 251

7.1

Feature summary of different biometric traits . . . . . 261

7.2

Characteristics of the different score normalization methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 267

7.3

Characteristics of the different score fusion methods . 269

7.4

Retrieved subjects and their votes in each candidate set 279

7.5

Summary of biometric databases and virtual users . . 284

7.6

HR and P R of the proposed indexing technique with unimodal and multimodal traits . . . . . . . . . . . . . 292

7.7

Average retrieving time for a query with different database sizes. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 295

7.8

Comparison with existing multimodal indexing techniques [11] . . . . . . . . . . . . . . . . . . . . . . . . . 297

8.1

Feature representations of the different indexing approaches . . . . . . . . . . . . . . . . . . . . . . . . . . 307

8.2

Performances of the different indexing approaches . . . 310

xxi

Chapter 1

Fundamentals of Biometric Technology This chapter presents the basic concept of the biometric authentication technology. This chapter is organized as follows. Sections 1.1 and 1.2 will give the definition of biometric technology and some major applications. The different operation modes of a biometric system will be discussed in Section 1.3. The concept of biometric data indexing will be given in Section 1.4. The different metrics for a biometric system are defined in Section 1.5. A brief overview of the different biometric modalities will be presented in Section 1.6. Section 1.7 will present a comparative study of the different biometric modalities. Finally, Section 1.8 will summarize the chapter.

1.1

Biometric Authentication Technology

Authentication plays the crucial role in any security related applications such as e-commerce, defense, etc. In this regard, the biometric is considered in recent times. Biometrics (ancient Greek: bios =“life", metron =“measure") is the study of methods for uniquely recognizing humans based on one or more intrinsic physiological or behavioral 1

1. Fundamentals of Biometric Technology characteristics of a human [39, 42, 55]. Automatic biometric processing systems work by first capturing a sample of a biometric feature, such as recording a digital sound signal for voice recognition, or taking a digital color image for face recognition. The sample is then transformed using some sort of mathematical function onto a biometric template. The biometric template provides a normalized and highly discriminating representation of the feature, which can then be objectively compared with other templates in order to determine the identity of a human. Fingerprint, iris, face, hand-geometry, palmprint and ear are the most commonly used physiological biometric traits, whereas voice, signature, keystroke dynamics, gait are the some example of behavioral biometric traits. Figure 1.1 shows the different biometric traits for a person. A good biometric is characterized by 1) use of a feature which should be highly unique so that the chance of any two people having the same feature set is zero, 2) stable so that the features do not change over time, 3) easily capturable so that it is convenient to the users and 4) prevents tempering or copying of features. When a single biometric trait is used for authentication purpose, then the authentication system is called unimodal authentication system. Unimodal biometric systems have several limitations such as noisy data, intra-class variations, restricted degrees of freedom, nonuniversality, spoof attacks and unacceptable error rates. Some of these limitations can be overcome by using multiple biometric traits or multiple source of information. Such system is called multimodal authentication system. In the different applications, biometrics are used in the different modes which are described in the following.

1.2

Some Major Biometric Applications

Several commercial and government organizations use biometric based authentication in many critical applications [34, 14, 11, 5, 26] like 2

1.2. Some Major Biometric Applications Physiological Biometric Trait

Fingerprint

Iris

Face

Ear

Behavioral Biometric Trait

Palmprint

Voice

Key-stroke dynamics

Gait Signature

Vein pattern Hand geometry

Figure 1.1: Examples of biometric traits that can be used for authenticating an individual. Physical traits include fingerprint, iris, face and hand geometry while behavioral traits include signature, keystroke dynamics and gait [43].

forensic science, surveillance, national border controls, national identity card, defense security, attendance management, payroll management, driving licenses, secure financial transactions, ticket less travel, etc. In the US, Federal Bureau of Investigation (FBI) manages a fingerprint identification system and database called the Integrated Automated Fingerprint Identification System (IAFIS) which currently consists of the fingerprint records of over 51 million criminals and over 1.5 million civilians (non-criminals). The US Transportation Security Administration (TSA) and Department of Homeland Security (DHS) [10] deployed biometric-based passenger departure system to make security procedure more efficient at the different airports [26, 5]. In UK, Iris Recognition Immigration System (IRIS) [7] is deployed to replace the passport presentation with biometric-based authentication system [6]. An automated iris recognition-based immigration clearing system is used at Amsterdam Schiphol Airport under Privium Program [8, 1]. In Canada, biometric based Canadian Passenger Accelerated Service System (CANPASS) [26, 2, 3] operates in eight international airports. Though, the most of the current biometric based authentication systems rely on the unimodal biometric trait, there are 3

1. Fundamentals of Biometric Technology several applications [14, 11, 13, 9, 4, 32, 5, 6] which are operated with multimodal biometric in large-scale. For example, US Visit [14] has a repository of two fingerprints for more than 50 million people. In Orlando, LaGuardia, Newark, Dulles, Regan, Denver and San Francisco international airports, CLEAR program [4, 32] use iris and fingerprint biometrics for passenger departure. The program is operated by Verified Identity Pass (VIP) and it enrolls 1,75,000 members till July 2008 [26]. The Privium Program [8] has registered about 40,000 frequent travelers [26] with multiple biometric in Schiphol Airport (Amsterdam NL). Unique Identification Authority of India (UIDAI) [11, 12] has planned to register 600 million users with multiple biometric traits in India in the next few years where the number of accesses per day (in the different public and private domains) are expected to be around 1 to 5 millions. Snapshots of few biometric based applications are shown in Fig. 1.2.

1.3

Operational Process of Biometric Technology

Depending on the context of applications, a biometric system can operate either in verification or identification mode. In both modes, users have to enroll via enrollment process. The system validates an individual’s identity by comparing the captured biometric data with his/her own biometric template(s) stored in the system database in the verification mode. Here, an individual submits a claimed identity to the system, such as personal identification number (PIN), a user name or a smart card. The system performs one-to-one comparisons to determine whether the identity claimed by the individual is genuine or false. In the identification mode, an individual is recognized by searching the templates of all users in the database. This system performs one-to-many comparison to establish the identity of 4

1.3. Operational Process of Biometric Technology 



(a) Iris recognition at Amsterdam Schiphol airports

(b) Fingerprint verification system between local banks and the Department of Home Affairs (DHA)

(c) Iris recognition system in the e-airport at Tokyo Narita Airport

(d) Contactless palm-vein systems in ATMs in Japan.

Figure 1.2: Different government and commercial applications which use biometric authentication system [43, 26].

the individual. In identification mode, there is no need to provide any identity of the individual. Figure 1.3(a) shows the enrollment process of a biometric authentication system, and Fig. 1.3(b) and (c) show the block diagrams of verification and identification systems, respectively. In these block diagrams, sensor module captures the raw biometric information from human bodies and feature extraction module processes the raw biometric information and extracts the features which is also referred as biometric template. The matching module 5

1. Fundamentals of Biometric Technology Biographic information

Sensor module

Image

Biometric template

Feature extraction module

Database

Users

(a) Enrollment steps of an authentication system. Claimed identity

Sensor module

Image

Feature extraction module

Biometric template

1:1

Matching module

Claimed template

Database

User

Decision module

Genuine / Imposter

(b) Steps of an authentication system in verification mode

Sensor module

Image

Feature extraction module

Biometric template

1:N

All

Matching templates module

Database

User

Decision module

Identity / Reject

(c) Steps of an authentication system in identification mode

Figure 1.3: Different modes of operations of a biometric authentication system.

compares the templates of two persons and generates a score value which represents the similarity (dissimilarity) of two persons. Finally, the decision module determines the genuineness or the identity of a person in verification or identification mode, respectively. Moreover, in identification mode, the matching module compares the biometric template of a person with all stored templates in the database. This process is computationally expensive when the number of stored templates in the database is huge. Identification of a person can be done faster if we filter out some templates which are not similar to the captured template. 6

1.4. Biometric Data Indexing Index key generation module

Sensor module

Image Feature extraction

Biometric template

module

Matching module

Index key

A set of templates

Database

User

Decision module

Identity / Reject

Figure 1.4: Major steps in identification system with indexing.

1.4

Biometric Data Indexing

Biometric data indexing is a scheme to generate an index key from the biometric data and assign the key to the corresponding template. Based on the index keys the biometric templates can be categorized into different groups in the database. The indexing technique is used to declare a person’s identity with lesser number of comparisons rather than searching the entire database. An identification system retrieves a small set of similar templates from the database based on the index key and performs detailed matching with the retrieved templates to determine the identity of an individual. An overview of an identification system with indexing is shown in Fig. 1.4.

1.5

Metrics for Performance Measure

There are a number of performance evaluation metrics [59] followed in biometric authentication system to evaluate the performance. Commonly adopted performance evaluation metrics are stated in the following. • Genuine Accept Rate (GAR): It is the ratio of the number of input samples correctly classified as genuine to the total number of genuine input samples. A higher value of GAR indicates better performance. 7

1. Fundamentals of Biometric Technology • Genuine Reject Rate (GRR): It is the ratio of the number of input samples correctly classified as impostor to the total number of impostor input samples. A higher value of GRR indicates better performance. • False Accept Rate (FAR): It is the proportion of impostor input samples falsely classified as genuine samples. It may be noted that F AR = 1 − GRR. A lower value of FAR indicates a better performance of a system. • False Reject Rate (FRR): It is the proportion of number of genuine input samples falsely classified as impostor samples. Please note that F RR = 1 − GAR. A lower value of F RR indicates a better performance of a biometric authentication system. • Equal Error Rate (ERR): When F RR becomes equal to F AR, the ratio is called as equal error rate. A lower value of ERR indicates a better performance. • Failure To Acquire (FTA) or Failure To Capture (FTC): It is the ratio of the number of times a biometric system fails to capture the biometric sample presented to it. A lower value of F T A indicates better acquisition performance. • Failure To Enroll (FTE): It is the ratio of the number of users that cannot be successfully enrolled into a biometric system to a total number of users presented to the biometric system. A lower value of F T E indicates better population coverage.

1.6

Biometric Modalities

In this section, a brief discussion on some important biometric traits has been presented. 8

1.6. Biometric Modalities

1.6.1

Iris Biometric

Iris is one of the most accurate biometric traits available today [51]. The pattern on the iris and the color information give highly distinguishing characteristics which are used to recognize a human with the help of iris [44, 27]. Experiments [35] performed by International Biometric Group (IBG) have shown F AR smaller than 0.01% with F RR below 10% and EER of less than 1%. Since, iris is a part of human eye which is highly sensitive, the acquisition time may take from several seconds to a couple of minutes. The testing of iris technology by IBG [35] shows that the enrollment time in iris recognition varies from as low as 13 seconds up to 5 minutes. The template size however is not more than 512 bytes [27] for leading vendors.

1.6.2

Fingerprint Biometric

Fingerprint is one of the oldest and the most popular biometric trait used for person identification [57, 62, 18, 54, 22, 37, 15]. Fingerprint consists of a pattern of ridges and valley. These ridges form various special patterns like ridge endings, ridge bifurcations, loops, delta [56]. These special patterns are called as minutiae points. Fingerprint authentication consists of a comparison of fingerprints based on these minutiae points [44]. There are many types of sensors available for capturing a fingerprint like tactile sensor, sweep sensor, infrared sensor etc. Results of the international fingerprint verification competition (FVC) 2006 [29] show an EER as low as 0.02% with a F AR of around 0.02%. It also shows that the enrollment time is less than one second. Top vendors provide template sizes as small as 250 bytes [35].

1.6.3

Face Biometric

Humans remember and recognize each other based on face than any other modality [16, 67, 17, 47, 19, 21, 58]. Facial features like eyes, 9

1. Fundamentals of Biometric Technology nose, mouth along with the facial geometry and bone structure are used in person identification based on the face biometric [68]. Experiment conducted in Facial Recognition Vendor Test (FRVT) 2006 [53] shows F ARs of 0.001% and F RRs of 0.13%. The facial image sizes are about 400 pixels between centers of eyes for controlled images and 190 pixels for uncontrolled images in very high resolution. The enrollment time given by IBG [35] is around 20 to 30 seconds for a controlled image.

1.6.4

Palmprint Biometric

The palmprint system is a hand-based biometric technology which considers the inner surface of a hand. The system captures the pattern of skin as the fingertips but it is larger than a fingertip in size. To uniquely identify a person, palmprint based biometric technology generally uses many features [59, 38, 25, 42, 60] like a) geometry features (width, length and area), b) principal line features (both location and form of the principal lines in a palmprint), c) wrinkle features (different from the principal lines in that they are thinner and more irregular) and d) delta point features (the delta point is defined as the center of a delta-like region in the palmprint). Typically, palmprint technology extracts more than 100 features from a palmprint image [33]. The performance reported on the literature shows that a system can operate at a 0.096% F AR and 1.05% F RR. However, the performance may degrade due to the damage on palmprint.

1.6.5

Hand Geometry Biometric

Hand geometry recognition is the longest implemented biometric type, debuting in the market in the late 1980s. The hand geometry biometric technology relies on the physical dimensions of a human hand which contains information that is capable to authenticate the identity of an individual. Typically, hand geometry based biometric sys10

1.6. Biometric Modalities tem [59, 38, 42, 60, 43, 69] captures the contour information of the hand geometry in the form of 2D grayscale image and binarizes the captured grayscale image into a black-and-white silhouette. From the silhouette image, finger lengths, widths, surface areas, angles between landmarks, and ratios of those quantities [41, 31, 69, 60] are calculated for authentication. Some hand geometry systems use fingerpositioning guides to aid in repeatable placement of the hand. In recent years, some commercial systems have begun to capture a full 3D model of the hand [69, 60]. Such systems use multiple cameras, structured light, or range-finder cameras. The size of the template for a hand geometry is small typically, less than 100 bytes string of data. Existing research work shows 1% F AR and 2.42% F RR can be achieved using hand geometry biometric. Though, the performance of the hand geometry shows interesting results, it has several constraints like data size, image quality, testing environment, number of users, etc.

1.6.6

Voice Biometric

Voice is a behavioral biometric modality used by humans to distinguish each other [23, 36, 17, 24, 37]. It is not as unique as the human face but we still remember and can differentiate between human voices. Voice is easily affected by medical conditions like cough, cold, throat infections etc. and external noise. Voice templates are larger in size than the most of the other modalities with sizes between 2000-10000 bytes. For proper feature extraction, the input must consist of several paraphrases or sequence of numbers repeated a number of times. A single paraphrase must be at least 1-1.5 seconds long [35]. Kung et al. [48] show a system using only voice biometric with an EER of about 3%.

11

1. Fundamentals of Biometric Technology

1.6.7

Gait Biometric

Gait is another behavioral biometric which captures the pattern of a walking person. The gait of a person is determined by their underlying musculo-skeletal structure [20], so it has some ability to discriminate between persons. Generally gait recognition approaches are divided into three types [20, 43, 52]: i) temporal alignment based, ii) silhouette shape based and iii) static parameter based approaches. The temporal alignment based approach [43, 61, 20] is the most common used method. It uses both shape and dynamics of walking pattern. This silhouette shape based approach [49, 50, 43] use the similarity of the silhouette shape. This approach gives high performance. The third approach selects the parameters that can be used to characterize gait dynamics, such as stride length, cadence and stride speed along with static body parameters such as the ratio of the sizes of various body parts [45, 64]. It has been observed that the gait biometric gives above 60% identification rate with a moderate database size.

1.6.8

Signature Biometric

For many years people have used their signatures to get verified. The main difference is that earlier one had to be ’in place’ to write the signature but today it can be done electronically [66, 28]. Signature verification measures several factors, like the pressure, rhythm, acceleration rates and stroke length of the signature. It is based on the dynamics of making a signature. The signature is a reflex action and not influenced by deliberate muscular control [66]. There are different ways to capture data for analysis. A special pen can be used to recognize and analyze the different movements when writing a signature. In this case, the data are captured within the pen. Information can also be captured within a special tablet that measures time, pressure, acceleration and the duration the pen touches it [63, 28]. As the user writes on the tablet, the movement of the pen generates acoustic 12

1.7. Comparative Study of Different Biometric Modalities emissions that are transmitted like stress waves in the material. The sound generated when the user writes the signature is also used for verification.

1.7

Comparative Study of Different Biometric Modalities

This section provides a qualitative comparison of the different biometric traits. For this purpose a number of parameters have been identified from the literature and each biometric trait has been analyzed with respect to these parameters. This is followed by the methodology to estimate the value of each parameter and subsequently a notation of impact value for each parameter has been introduced.

1.7.1

Identification of Parameters

Twelve parameters have been listed out from the different literatures [46, 69, 43, 40]. The definition of each parameter is given in the following. Universality: It denotes the universality of a biometric to measure how much population hold the biometric. Distinctiveness: This parameter measures how a biometric differs from person to person. People can be differentiated based on this characteristic. Stability: It measures the persistence of a biometric. Or in other words, how much a characteristic varies considerably over time. Capturability: This parameter signifies how much a biometric is easy to extract, easy to represent as template and easy to enroll. Multiplicity: How many unique features are measurable in a biometric is denoted by this parameter. Accuracy: Unlike password or token-based system, an ideal biometric system does not make a perfect match with the stored template. It makes decision depending on the some defined threshold. For taking 13

1. Fundamentals of Biometric Technology decision it can make two types of errors: False Acceptance and False Rejection. This parameter measures how accurately an input pattern matches with the stored pattern. Performance: It refers to achieve the speed of system using biometric characteristics. It considers the following estimations: enrollment time, searching time and matching time. Acceptability: It refers user’s eagerness to use a biometric system in daily life. It depends upon the accuracy of the system, easy to usefulness and cost of the system. Privacy: How much a biometric is inaccessible to retrieve other information for malicious use. Cost: This parameter measures the deployment cost of a biometric system. Storage: The storage efficiency with respect to memory requirement is measured by this parameter. Circumviability: It refers how much easy to copy or duplicate a biometric feature.

1.7.2

Estimation of Values of Parameters

Each biometric trait is analyzed with reference to these twelve parameters as identified in Section 1.7.1. The result of the analysis is ranked on a scale 0 to 10. The rank 10 is given when the parameter has the highest impact on a biometric trait and rank 0 is given when the parameter has no impact on the biometric trait. It may be noted that, for the parameters cost and storage, the rank is given in reverse convention than the others. The cost and storage of a biometric system are ranked as excellent if it is with lower cost and needs minimum storage, respectively. On the contrary, it ranks as very poor if the cost is high and large storage space is required to store features. Table 1.11.8 give analysis of the parameters of the different biometric traits. 14

1.7. Comparative Study of Different Biometric Modalities Table 1.1: Rank assigned to each parameter for iris biometric Parameters Universality

Observation Most of the people possess the iris biometric.

The Iris texture is distinctive for each person, even Distinctiveness identical twins have unique irises and even the right and the left iris of a person are different.

Rank 9 10

Stability

Iris features are not affected by growth, wear and tear, aging.

9

Capturability

The iris feature is not easy to capture due to small target (1 cm) to acquire from a distance (1 m) and reflection of light.

4

Multiplicity

There are many feature can be measure collectively.

8

Accuracy

Accuracy of the iris biometric systems are very high due to its distinctiveness features.

8

Performance

The performance of an iris biometric is very high.

9

Acceptability

The method is expensive and people might be reluctant using it due to fears of eye damages.

4

Privacy

This biometric keep good user privacy.

8

Cost

The cost of the camera, which is used in biometric system is high. So this system is expensive.

4

Storage

This biometric needs less memory for the data to be stored.

8

Circumviability

It is almost impossible to copy or create duplicate the iris feature.

9

1.7.3

Estimation of Impact Value

It is observed that some parameters may be compromised without loosing the objective of an application. To include this, the notion of impact value has been introduced. The impact value can be measured on a scale 0 to 1, with 0 value indicating the lowest impact and 1 indicating the highest impact. A number of applications [43, 42, 30, 39, 40, 65] have been investigated in varieties of field and based on the investigation the results are summarized in the Table 1.9. The result in the Table 1.9 can be further justified as follows. The maximum impact is given to the universality factor because if the person does not possess the biometric feature the biometric system has no functionality. If the person is not differentiable based on a particular biometric there is a high chance of duplicity and the biometric 15

1. Fundamentals of Biometric Technology Table 1.2: Rank assigned to each parameter for fingerprint biometric Parameters

Observation

Rank

Universality

Most of the people have the fingers. But Some people cannot use fingerprint methods because they have too ’weak’ ridge patterns, or loose their fingers in some injuries. So, universality of fingerprint is good.

8

Distinctiveness

Fingerprint of each person is unique. No two individuals have identical ridge patterns.

10

Stability

Finger pattern (ridge and valley on the surface) is also stable after the first seven months of fetal development. So, it is not affected by growth and aging. But it may change by reason of injuries, wear and tear, burns, diseases or surgery other unnatural causes. So, its stability is average.

6

Capturability

Difficulty of fingerprint feature extraction is positioning of finger in sensor and some fingerprint feature are not capturable due to ’weak’ ridge patterns, which means that the sensors can not ’read’ their fingerprints or dirty or sticky fingers

7

Multiplicity

The multiplicity of fingerprint is average.

6

Accuracy

Considering the distinctiveness and stability parameter the fingerprint biometric system’s accuracy is good.

8

Performance

The performance of fingerprint system is better than average as dryness and cleanness of skin slightly change the attainable speed of the biometric system.

7

Acceptability

Though, there is some overhead of positioning in fingerprint in sensor, but considering its size, accuracy and performance the user’s preferability is better than average.

7

Privacy

Except the physical distortion of finger the fingerprint system keep the users privacy.

8

Cost

The cost of iris, fingerprint, hand geometry, gait and DNA biometric system is high due to cost of the extra hardware. Respective to those biometric systems the cost of fingerprint, signature and face recognition system is low because the cost of extra hardware is less than other.

4

Storage

Fingerprint technology is required small storage space required for the biometric template, reducing the size of the database memory required.

8

Fingerprints can be copied in the form of data sets, Circumviability paper prints, wax molds, etc. But there is many sensors, which detect live ness of fingerprint.

7

system provide high security. So distinctiveness’s impact value is also high. If biometric features change with time, a biometric system gives the false rejection, and hence accuracy is degraded. Its importance 16

1.7. Comparative Study of Different Biometric Modalities Table 1.3: Rank assigned to each parameter for face biometric Observation

Parameters Universality

All people have the face biometric.

Rank 10

The face structures are not distinctive. There are many Distinctiveness persons, who have the almost same face structure.

4

Stability

Face structure changes with growth and aging.

5

Capturability

The face picture is easy to capture.

8

Multiplicity

There are so many features, which is measured collectively.

8

Accuracy

The accuracy of a face recognition system is not good.

5

Performance

The performance is also not good of a face biometric system.

5

Acceptability

Face recognition could reach high user acceptance, because people are more or less familiar with having their pictures taken. The method does not require any more of the user than the fact that s/he is positive to having her/his picture taken. For the user it is a passive procedure where s/he only has to look into the camera.

8

Privacy

This biometric can be abused.

2

Cost

Te cost of face recognition system is moderate.

5

Storage

The system requires higher storage to store its features.

3

It can be copied or duplicate easily but with expensive Circumviability arrangement.

5

Table 1.4: Rank assigned to each parameter for palmprint biometric Observation

Parameters Universality

All most all the people have palmprint.

Distinctiveness Many persons have the same palmprint.

Rank 9 5

Stability

Palmprint biometric may not remain invariant for long period of time, major injuries.

4

Capturability

It is easy to capture the palmprint image by a camera.

7

Multiplicity

A large amount of features can be measured.

8

Accuracy

The accuracy is moderate due to the false rejection and false acceptance.

5

Performance

Performance is moderate.

6

Acceptability

It is user friendly.

8

Privacy

It can lead to a number of information than the purpose of biometric system.

3

Cost

The cost of deployment is moderate.

6

Storage

Need very small storage.

7

Circumviability It is easy to duplicate one’s palmprint.

17

3

1. Fundamentals of Biometric Technology Table 1.5: Rank assigned to each parameter for hand geometry biometric Parameters

Observation

Rank

Universality

Most of the people have their hands. However, some people may loose their hand on some major injuries. So, universality of hand geometry is good.

8

The basic shape and size of our hands are not distinctive. Distinctiveness The system is not well suited for performing one-to-many identification matches.

5

Stability

Hand geometry is not stable with growth, wear and tear, aging and grime injury.

6

Capturability

Hand geometry features are easy to capture because they are not affect by dirtiness or environmental factor.

9

Multiplicity

There are many features can be measure collectively.

9

Accuracy

The accuracy of hand geometry system is not good because it is not distinctive and injured of hand affect the accuracy of the system. But hand geometry sensors do not affected by extreme temperatures and are not impacted by dirty hands.

6

Performance

The performance of hand geometry system is moderate.

5

Acceptability

The hand geometry is easy to use. Because of the guidance pins, the hand will always be in the right position. This means that users will probably make fewer errors. Since hand geometry devices are easy to use, they are considered user friendly. But size of hand geometry system is quite large.

7

Privacy

Except the physical distortion of hand, the hand geometry system keeps the users privacy.

6

Cost

The hand geometry system needs extra hardware, which is not so cost effective.

5

Storage

It takes small storage place to store the template.

7

Circumviability Easy to copy or duplicate.

4

is also high. If the biometric features are not captured properly a biometric system may produce uncertain results so capturability gets high priority on a biometric system. The multiplicity factor refers how collectively measure all features of a biometric trait. Its priority is 0.7. Accuracy refers that a biometric system is how much able to produce an accurate result without giving any false acceptance and false match. So it has a high impact on a biometric system. The performance of a biometric system depends on the different time constraint. If it takes more time to enroll or match a biometric feature 18

1.7. Comparative Study of Different Biometric Modalities Table 1.6: Rank assigned to each parameter for voice biometric Observation

Parameters Universality

Rank

Most of the people able to speak with microphone.

7

Distinctiveness There are so many voices that are not distinguishable.

6

Stability

The voice of person change with age, weather and illness of a person or intentionally.

4

Capturability

The voice biometric is easy to capture but to feature extraction is difficult due to channel noise.

6

Multiplicity

The multiplicity is high.

8

Accuracy

It is sensitive to background and channel noise, hoarseness, vocal stress etc.

4

Performance

The performance of the system is average.

5

Acceptability

Voice biometric is user friendly because talking is something we do every day, voice biometrics is really not anything new. No special effort is needed from the user, apart from talking into the device.

8

Privacy

It has some limitations so far privacy is concerned.

7

Cost

Voice biometric is cost effective because there is no need of special hardware. Only extra noise cancellation software is required.

8

Storage

It takes average storage space.

2

Circumviability

It is easy to record a person’s voice and use it in a fraudulent way.

2

Table 1.7: Rank assigned to each parameter for gait biometric Observation

Parameters Universality

Majority of the people can walk.

Distinctiveness Many persons have the same walking style.

Rank 7 4

Stability

Gait biometric may not remain invariant for long period of time, due to variation of body weight, major injuries.

2

Capturability

It is not very difficult to capture the walking style by a video camera.

7

Multiplicity

A large amount of features can be measured.

8

Accuracy

The accuracy is bad due to the false rejection and false acceptance.

4

Performance

Performance is moderate.

5

Acceptability

It is user friendly.

8

Privacy

It can lead to a number of information than the purpose of biometric system.

3

Cost

The cost of deployment is moderate.

5

Storage

Store as video hence it takes adequate storage.

2

Circumviability It is easy to duplicate one’s walking style.

19

3

1. Fundamentals of Biometric Technology Table 1.8: Rank assigned to each parameter for signature biometric Observation

Parameters Universality

Rank

There are many people who does not make signature.

Sometimes signature is impossible to tell apart from Distinctiveness large number of person.

6 4

Stability

Signature is affected by hand/finger injuries. Another is that the signature often changes in time.

4

Capturability

Good to capture this biometric.

8

Multiplicity

A large amount of data present in a signature verification template.

8

Accuracy

The accuracy of signature verification is bad. Because there are so many person whose signature are same for this reason false match rate is high on the other hand a persons signature is quickly change with time for this reason false rejection rate is also high.

4

Performance

The performance of the signature verification is average.

6

Acceptability

People are familiar with writing their signatures and this fact leads to a very high grade of acceptance for signature verification.

9

Privacy

It cannot be manipulated to get other private information.

9

Cost

It does not require as such special arrangement apart from using a special pen and/or tablet.

7

Storage

It takes little storage space.

7

Circumviability It is very easy to duplicate once signature with less effect.

2

Table 1.9: Impact value of parameters Parameters Universality (α1 ) Distinctiveness (α2 )

Impact Value

Parameters

Impact Value

0.9

Multiplicity (α5 )

0.7

Privacy (α9 )

0.5

0.9

Accuracy (α6 )

0.9

Cost (α10 )

0.6

0.8

Storage (α11 )

0.4

0.7

Circumviability (α12 )

0.9

Stability (α3 )

0.9

Capturability (α4 )

0.9

Performance (α7 ) Acceptability (α8 )

Parameters

Impact Value

then it loses the overall acceptance. This metric is considered with 0.8 impact value. If users are not eager to use the biometric system then a biometric system does not establish in practical application. Acceptability metric is also considered with 0.7 impact value on a biometric system. If a biometric system is unable to keep the privacy 20

1.8. Summary of users the user may not use biometric system. The privacy metric is considered as a medium impact metric. The cost of the system is another issue. If one biometric system cost is too high and it provides very high security and other biometric system cost is medium and it provides usable security, then many commercial applications use the second one, but the high security application (e.g. defense) uses the first one. So, 0.6 impact value is assigned to it. The storage metric is assigned with 0.4 impact value because this time storage cost is low. If anyone can able to make duplicate or copy the biometric features the system does not provide higher security and they may abuse the system. So, the circumviability has the high impact on a biometric system.

1.7.4

Quantitative Comparison

In order to compare the different biometric system, an overall cost has been measured for each biometric system. Based on the rank of each parameter on a biometric system and the impact of the each parameter, a heuristic has been proposed to measure the cost. The cost is calculated using Eq. 1.1.

Costb =

12 ∑

αi W αi

(1.1)

i=1

Where, αi and Wαi represent the rank of the ith (i = 1, . . . , 12) parameter for a biometric, which is in the scale of 0 to 10 and the impact value of the αi in the scale of 0 to 1. The quantitative performance of each biometric is measured using Eq. 1.1 and the result has been reported in Table 1.10.

1.8

Summary

An overview of a biometric system has been presented in this chapter. The different applications of biometric have been discussed. How 21

1. Fundamentals of Biometric Technology Table 1.10: Cost of each biometric trait Parameters Universality Distinctiveness Stability Capturability Multiplicity Accuracy Performance Acceptability Privacy Cost Storage Circumviability

Impact Value 0.9

Ir

Fp

Fa

Pp

Hg

Vo

Ga

Si

9

8

10

9

8

7

7

6

0.9

10

10

4

5

5

6

4

4

0.9

9

6

5

4

6

4

2

4

0.9

4

7

8

7

9

6

7

8

0.7

8

6

8

8

9

8

8

8

0.9

8

8

5

5

6

4

4

4

0.8

9

7

5

6

5

5

5

6

0.7

4

7

8

8

7

8

8

9

0.5 0.6 0.4

8 4 8

8 4 8

2 5 3

3 6 7

6 5 7

7 8 2

3 5 2

9 7 7

0.9

9

7

5

3

4

2

3

2

Costb

69.3

65.7

53.7

53.6

58.2

50.4

44.8

53.4

Ir→Iris, Fp→Fingerprint, Fa→Face, Pp→Palmprint, Hg→Hand geometry, Vo→Voice, Ga→Gait, Si→Signature

does a biometric system work also discussed in this chapter. Twelve parameters are proposed to quantify a biometric system. Twelve parameters have been chosen from the exhaustive survey of existing literatures on several biometrics and their applications. In order to analyze the performance of a biometric system a heuristic has been proposed. Based on the proposed parameterization technique, all biometric systems have been compared and result reveals that all physiological biometrics are comparable to behavioral biometrics. Further, the iris biometric system outperforms all other biometric systems.

22

BIBLIOGRAPHY

Bibliography [1] The Eyes Have It. URL http://www.accessexcellence.org/ WN/SU/irisscan.php. (Accessed on September, 2010). [2] Canadian Passenger Accelerated Service System (CANPASS). Canada Border Services Agency, . URL http://www.cbsa-asfc. gc.ca/prog/canpass/menu-eng.html. (Accessed on September, 2012). [3] CANPASS - Air, . URL http://www.cbsa-asfc.gc.ca/prog/ canpass/canpassair-eng.html. (Accessed on November, 2010). [4] CLEAR Program. CLEAR Airport Security and Automate Security. URL http://clearme.com/index. (Accessed on September, 2012). [5] Crossing

U.S.

Borders.

URL

http://www.dhs.gov/

crossing-us-borders/. (Accessed on September, 2011). [6] Entering the UK.

Home Office UK Border Agency.

URL http://www.ukba.homeoffice.gov.uk/customs-travel/ Enteringtheuk/. (Accessed on September, 2012). [7] IRIS.

Using the Iris Recognition Immigration System

(IRIS).

URL

http://www.ukba.homeoffice.gov.uk/

travellingtotheuk/Enteringtheuk/usingiris/. (Accessed on November, 2010). [8] Iris

Scans

at

Amsterdam

Airport

Schiphol.

URL

http://www.schiphol.nl/Travellers/AtSchiphol/Privium/ Privium/IrisScans.htm. (Accessed on October, 2012). [9] Personal Identity Verification (PIV) of Federal Employees and Contractors.

URL http://www.nist.gov/itl/csd/set/piv.

cfm. (Accessed on August, 2012). 23

1. Fundamentals of Biometric Technology [10] TSA. Transportation Security Administration. URL http:// www.tsa.gov/. (Accessed on September, 2012). [11] AADHAAR. Unique Identification Authority of India, . URL http://uidai.gov.in/. (Accessed on July, 2012). [12] UIDAI tity

Strategy

Number

for

Overview. Every

Creating

Resident

in

a

Unique

India,

.

IdenURL

http://uidai.gov.in/UID_PDF/Front_Page_Articles/ Documents/Strategy_Overveiw-001.pdf.

(Accessed on May,

2012). [13] UIDAI. Unique Identification Authority of India from Wikipedia, .

URL

http://en.wikipedia.org/wiki/Multipurpose_

National_Identity_Card. (Accessed on January, 2012). [14] US-VISIT Program.

URL http://ipv6.dhs.gov/files/

programs/usv.shtm. (Accessed on September, 2012). [15] F. Alonso-Fernandez, J. Fierrez, J. Ortega-Garcia, J. GonzalezRodriguez, H. Fronthaler, K. Kollreider, and J. Bigun. A comparative study of fingerprint image-quality estimation methods. Information Forensics and Security, IEEE Transactions on, 2(4): 734–743, Dec. 2007. ISSN 1556-6013. doi: 10.1109/TIFS.2007. 908228. [16] S. Ben-Yacoub, Y. Abdeljaoued, and E. Mayoraz. Fusion of face and speech data for person identity verification. IEEE Transactions on Neural Networks, 20(5):1065–1074, September 1999. doi: 10.1109/72.788647. [17] S. Ben-Yacoub, J. Luttin, K. Jonsson, J. Matas, and J. Kittler. Audio-visual person verification. In IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 1999, volume 1, pages –585 Vol. 1, 1999. doi: 10.1109/CVPR.1999.786997. 24

BIBLIOGRAPHY [18] F. Besbes, H. Trichili, and B. Solaiman. Multimodal biometric system based on fingerprint identification and iris recognition. In 3rd International Conference on Information and Communication Technologies: From Theory to Applications (ICTTA 2008), pages 1–5, April 2008. doi: 10.1109/ICTTA.2008.4530129. [19] K.W. Bowyer, K.I. Chang, P.J. Flynn, and Xin Chen. Face recognition using 2-d, 3-d, and infrared: Is multimodal better than multisample? In Proceedings of the IEEE, volume 94, pages 2000– 2012, Nov. 2006. doi: 10.1109/JPROC.2006.885134. [20] J. E. Boyd. Synchronization of oscillations for machine perception of gaits. Computer Vision and Image Understanding, 96(1):35–59, 2004. [21] Kyong I. Chang, Kevin W. Bowyer, and Patrick J. Flynn. An evaluation of multimodal 2d+3d face biometrics. IEEE Transaction on Pattern Analysis Machine Intelligence, 27(4):619–624, 2005. ISSN 0162-8828. doi: http://dx.doi.org/10.1109/TPAMI. 2005.70. [22] Yi Chen, Sarat Dass, and Anil Jain. Fingerprint quality indices for predicting authentication performance. In In: Proc. AVBPA, Springer LNCS-3546, pages 160–170, 2005. [23] Girija Chetty and Michael Wagner. Audio-visual multimodal fusion for biometric person authentication and liveness verification. In Fang Chen and Julien Epps, editors, NICTA-HCS Net Multimodal User Interaction Workshop (MMUI 2005), volume 57 of CRPIT, pages 17–24, Sydney, Australia, 2005. ACS. [24] Ming-Cheung Cheung, Man-Wai Mak, and Sun-Yuan Kung. Multi-sample data-dependent fusion of sorted score sequences for biometric verification. In International Conference on Acoustics, 25

1. Fundamentals of Biometric Technology Speech, and Signal Processing, 2004., volume 5, pages 681–684, 2004. [25] T. Connie, A. T. B. Jin, M. G. K. Ong, and D. N. C. Ling. An automated palmprint recognition system. Image and Vision Computing, 2005(5):501–515, 23. [26] J. Daugman. Encyclopedia of Biometrics, chapter Iris Recognition at Airports and Border-Crossings, pages 819–825. Springer, New York, 2009. [27] John Daugman. The importance of being random: Statistical principles of iris recognition. Pattern Recognition, 36(2):279–291, 2003. [28] J. Fierrez-Aguilar, J. Ortega-Garcia, and J. Gonzalez-Rodriguez. Target dependent score normalization techniques and their application to signature verification. IEEE Transactions on SMC-C, 35(3):418–425, 2005. [29] FVC.

International fingerprint verification competition 2006.

http://bias.csr.unibo.it/fvc2006/, 2006. [30] M. Gamassi, M. Lazzaroni, M. Misino, V Piuri, D. Sana, and F. Scotti. Accuracy and performance of biometric systems. In Proceedings of the 21st IEEE Instrumentation and Measurement Technology Conference, pages 510–515, 2004. [31] S. Gonzalez, C. M. Travieso, J. B. Alonso, and M. A. Ferrer. Automatic biometric identification system by hand geometry. In In Proceedings of the 37th Annual International Carnahan Conference on Security Technology, pages 281–284, 2003. [32] J. ing

Hammond. Introduced

Biometric in

Orlando, 26

Traveler Denver

ScreenNext.

BIBLIOGRAPHY http://www.examiner.com/article/biometric-traveler-screeningintroduced-orlando-denver-next/, November 2010. (Accessed on July, 2012). [33] C.C. Han, H.L. Cheng, K.C. Fan, and C.L. Lin. Personal authentication using palmprint features. Pattern Recognition, 36 (2):371–381, 2003. [34] T. Huang, Z. Xiong, and Z. Zhang. Handbook of Face Recognition, chapter Face Recognition Applications, pages 617–638. Springer, 2nd edition, 2011. [35] IBG.

IBG

reports

and

research.

http://www.biometricgroup.com/reports/public/reports_and_ research.html, 2006. [36] M. Ichino, H. Sakano, and N. Komatsu. Multimodal biometrics of lip movements and voice using kernel fisher discriminant analysis. In 9th International Conference on Control, Automation, Robotics and Vision (ICARCV ’06), pages 1–6, Dec. 2006. doi: 10.1109/ ICARCV.2006.345473. [37] A. Jain and A. Ross. Fingerprint mosaicking. In IEEE International Conference on Acoustics, Speech, and Signal Processing, 2002. Proceedings. (ICASSP ’02), volume 4, pages IV–4064–IV– 4067 vol.4, 2002. doi: 10.1109/ICASSP.2002.1004811. [38] A. Jain, L. Hong, and R. Bolle. On-line fingerprint verification. IEEE Transactions on Pattern Analysis and Machine Intelligence, 19(4):302–314, 1997. [39] A. K. Jain and S. Pravakar. Biometric authentication. INSIGHT, 2:29–52, 2002. 27

1. Fundamentals of Biometric Technology [40] A. K. Jain, R. Bolle, and S. Pankanti, editors. Personal Identification in Networkwed Society.

BiometricsMA: Kluwer,

1999. [41] A. K. Jain, A. Ross, and S. Pankanti. A prototype hand geometry based verification system. In In Proceedings of the 2nd International Conference on Audio Video based Biometric Personal Authentication (AVBPA), pages 166–171, 1999. [42] A. K. Jain, A. Ross, and S. Prabhakar. An introduction to biometric recognition. IEEE Transactions on Circuits and Systems for Video Technology, 14(1):4–20, January 2004. [43] A. K. Jain, P. Flynn, and A. Ross, editors. Handbook of Biometrics. Springer-Verlag, New York, 1st edition, 2007. [44] A.K. Jain, A. Ross, and S. Pankanti. Biometrics: a tool for information security. Information Forensics and Security, IEEE Transactions on, 1(2):125–143, June 2006. ISSN 1556-6013. doi: 10.1109/TIFS.2006.873653. [45] A. Johnson and A. Bobick. A multi-view method for gait recognition using static body parameters. In In International Conference on Audio- and Video-Based Biometric Person Authentication, pages 301–311, 2001. [46] A. Klokova. Comparison of various biometric methods. In In proc. of Multimedia Conference, 2011. [47] K. Kollreider, H. Fronthaler, and J. Bigun. Verifying liveness by multiple experts in face biometrics. In Biometrics, pages 1–6, 2008. [48] S. Y. Kung and Man-Wai Mak. On consistent fusion of multimodal biometrics. In IEEE International Conference on Acous28

BIBLIOGRAPHY tics, Speech and Signal Processing, 2006. (ICASSP 2006), volume 5, pages V–1085 – V 1088, 2006. [49] Z. Liu and S. Sarkar. Simplest representation yet for gait recognition: Averaged silhouette. In In Proc. of International Conference on Pattern Recognition, volume 4, pages 211–214, 2004. [50] Z. Liu and S. Sarkar. Improved gait recognition by gait dynamics normalization. IEEE Transactions on Pattern Analysis and Machine Intelligence, 28(6):863–876, 2006. [51] Tony Mansfield, Gavin Kelly, David Chandler, and Jan Kane. Biometric product testing final report. Technical report, Centre for Mathematics and Scientific Computing, National Physical Laboratory, UK., 2001. [52] M. S. Nixon and J. N. Carter. Advances in automatic gait recognition. In In Proc. of International Conference on Automatic Face and Gesture Recognition, pages 139–146, 2004. [53] P. Jonathon Phillips, W. Todd Scruggs, Alice J. O Toole, Patrick J. Flynn, Kevin W. Bowyer, Cathy L. Schott, and Matthew Sharpe. sults, 2006.

Frvt 2006 and ice 2006 large-scale re-

URL http://www.frvt.org/FRVT2006/docs/

FRVT2006andICE2006LargeScaleReport.pdf. [54] S. Prabhakar and A.K. Jain. Decision-level fusion in fingerprint verification. Pattern Recognition, 35(4):861–874, April 2002. [55] S. Prabhakar, S. Pankanti, and A. K. Jain. Biometric recognition: Security and privacy concerns. IEEE Security & Privacy, pages 33–42, March/April 2003. [56] Salil Prabhakar, Anil K. Jain, and Sharath Pankanti. Learning fingerprint minutiae location and type. Pattern Recognition, 36: 1847–1857, 2003. 29

1. Fundamentals of Biometric Technology [57] A. Rattani, D. R. Kisku, M. Bicego, and M. Tistarelli. Feature level fusion of face and fingerprint biometrics. In First IEEE International Conference on Biometrics: Theory, Applications, and Systems, 2007 (BTAS 2007)., pages 1–6, September 2007. doi: 10.1109/BTAS.2007.4401919. [58] Arun Ross and Rohin Govindarajan. Feature level fusion using hand and face biometrics. In Proceedings of SPIE Conference on Biometric Technology for Human Identification II, volume 5779, pages 196–204, 2005. [59] Arun A. Ross, Karthik Nandakumar, and Anil K. Jain. Handbook of Multibiometrics. Springer-Verlag New York, Inc. Secaucus, NJ, USA, 2006. [60] R. Sanchez-Reillo, C. Sanchez-Avilla, and A. Gonzalez-Marcos. Biometric identification through hand geometry measurements. IEEE Transactions on Pattern Analysis and Machine Intelligence, 22(10):1168–1171, 2000. [61] S. Sarkar, P. Jonathon Phillips, Z. Liu, I. Robledo-Vega, P. Grother, and K. W. Bowyer. The human id gait challenge problem: Data sets, performance, and analysis. IEEE Transactions on Pattern Analysis and Machine Intelligence, 27(2):162–177, 2005. [62] R. Snelick, U. Uludag, A. Mink, M. Indovina, and A. Jain. Largescale evaluation of multimodal biometric authentication using state-of-the-art systems. IEEE Transactions on Pattern Analysis and Machine Intelligence, 27(3):450–455, March 2005. ISSN 0162-8828. doi: 10.1109/TPAMI.2005.57. [63] J. A. Sullivan and N. A Schmid. Performance analysis of physical signature authentication. IEEE Transactions on Information Theory, 47(7):3034–3039, 2001. 30

BIBLIOGRAPHY [64] R. Tanawongsuwan and A. Bobick. Gait recognition from timenormalized jointangle trajectories in the walking plane. In In Proc. of IEEE Conference on Computer Vision and Pattern Recognition, pages 726–731, 2001. [65] J. L. Wayman. Report on the evaluation of the inpass hand geometry system. Technical report, National Biometric Test Center Collected Works 1997-2000, San Jose State University, 2000. [66] J. Yi, C. Lee, and J. Kim. Online signature verification using temporal shift estimated by the phase of gabor filter. IEEE Transactions on Signal Processing, 53(2):776–783, 2005. [67] Wenchao Zhang, Shiguang Shan, Wen Gao, Yizheng Chang, Bo Cao, and Peng Yang. Information fusion in face identification. In Proceedings of the 17th International Conference on Pattern Recognition (ICPR 2004), volume 3, pages 950–953 Vol.3, Aug. 2004. doi: 10.1109/ICPR.2004.1334686. [68] W. ZHAO, R. CHELLAPPA, P. J. PHILLIPS, and A. ROSENFELD. Face recognition: A literature survey. ACM Computing Surveys, December 2003. Volume 35, pages 399-458. [69] R. Zunkel. Biometrics: Personal Identification in Networked Society, chapter Hand Geometry Based Authentication. Kluwer Academic Publishers, 1999.

31

Chapter 2

Multimodal Biometric and Fusion Technology This chapter presents the basic concept of multimodal biometric and the different fusion rules in the different fusion levels. The chapter is organized as follows. Section 2.1 will give the idea of multimodal biometric and advantages of using multiple biometrics. The concept of fusion and the different levels of fusion will be discussed in Section 2.2 and 2.3. The different fusion rules will be described in Section 2.4. A comparison of the different fusion rules at the different fusion levels will be presented in Section 2.5. Section 2.6 will summarize the chapter.

2.1

Multimodal Biometric Authentication Technology

In multimodal biometric authentication, multiple modalities are used for person identification. Two or more biometric traits are gathered from a person and are used to identify him/her uniquely. The user is asked to present multiple samples for identification and the final recognition decision is made on all or a combination of them. 33

2. Multimodal Biometric and Fusion Technology Use of more than one modality has many advantages as well as challenges associated with it. Modalities such as iris and fingerprint are unique and accurate and play an important role in the overall decision making process. Use of multiple modalities allows more users to enroll in the system because they can present alternate modalities if the ones asked for are not presentable at that moment due to an illness, an injury or any other difficulties. Multimodality also makes it increasingly difficult to fake multiple modalities [41]. The accuracy of unimodal systems is adversely affected by the input sample quality [9, 8]. Use of multiple modalities provides robustness against sample quality degradation. It also helps to improve the reliability of a biometric system. While designing a multimodal biometric system, one must consider the type of data to be acquired (e.g.. 2 Dimensional or 3 Dimensional), the type of recognition algorithm to be used on each data element (e.g. Principal Component Analysis (PCA) or Independent Component Analysis (ICA)), the output of an algorithm (the distance or error metric), how to combine them and the level at which it should be performed. All these are the major challenges in multimodal biometric authentication system.

2.2

Fusion of Multimodalities

Compared to a unimodal biometric authentication system, a multimodal system requires more storage space to store the different types of samples. The benefits of a multimodal system may get overshadowed if it takes inordinate amounts of storage space and consequently increasing times for sample collection, templates matching and decision making. To overcome these limitations, a possible approach is to combine multiple modalities. This approach is called as fusion. The fusion can be performed at various stages in a biometric authentica34

2.2. Fusion of Multimodalities tion system. These stages are called fusion levels and are shown in Fig. 2.1. • Sensor Level Fusion: In sensor level fusion, outputs of the different sensors are fused together to form a new input which is used in further stages [41]. • Feature Level Fusion: Features extracted from the different biometric modalities are combined to form a single feature vector in this level of fusion [41]. • Match-score Level Fusion: After comparing the stored template and query sample the matcher produces a measure of their similarity called as a match score. The match score produced by the different matchers are combined in this level of fusion [41]. • Decision Level Fusion: The decisions made by the different matchers are combined in this level of fusion to arrive at the final recognition decision [41]. A detail discussion of the above mentioned fusion techniques are presented in the next section. The fusion techniques help in reducing the total number of feature vectors from the different modalities which otherwise would have

Stored user templates

Live input

Sensor module

Feature extraction module

Sensor level fusion

Matcher module

Feature level fusion

Decision module

Match-score level fusion

Accept/ Reject

Decision level fusion

Figure 2.1: Block diagram of the different levels of fusion.

35

2. Multimodal Biometric and Fusion Technology increased the storage space requirements manifolds. Taking the specialist capabilities of each classifier, a combined classifier may be build up to provide better results than a single classifier. In other words, combining the different experts results in a system can outperform the experts when taken individually. This is especially true if the different experts are not correlated [3]. It also increases the degrees of freedom [52]. The concern of time spent in processing of multiple modalities can be avoided by use of parallel mode of operations.

2.3

Fusion Levels

This section will talk about various levels of fusion. A typical biometric system is shown in Fig. 2.1 with the sensor, feature extraction, matcher and decision making modules. Each modality requires a separate set of these four modules. Information available at the output of each of these modules can then be combined and passed onto the next module. This process of combining information is known as fusion. Fusion can be carried out at four different places or levels as shown in Fig. 2.1. A brief description about each level of fusion, the issues that are required to be handled in order to fuse information at these levels, the limitations of information fusion at those levels and their advantages and disadvantages are presented in the following.

2.3.1

Sensor Level Fusion

Sensor level fusion is fusion performed at the earliest stage and is done before the feature extraction phase. The sensors capture a biometric trait like facial image, iris image, fingerprint etc. The main objective of the sensor level fusion is to fuse these captured samples as they contain the richest amount of information about that biometric trait. According to Zhang et al [58], fusion should be carried as early as possible. 36

2.3. Fusion Levels Issues in Sensor Level Fusion: Typically, sensor level fusion can be done if the samples represent the same biometric trait [41]. For instance, multiple snapshots of a face can be taken by cameras positioned at the different locations and are then combined to form a composite 3D image. Sensor level fusion is more beneficial and useful in multi-sample systems [42, 6, 11] where multiple samples of a same biometric trait are taken and combined to form a composite sample. The samples to be fused must be compatible. Existing Approaches to Sensor Level Fusion: Sensor level fusion involves the formation of a composite image or signal [41]. A 3D face image can be generated from multiple views of the same face in two dimensions [26]. Mosaicing [18] of fingerprints is performed in order to form a composite fingerprint from multiple dab prints. The user provides multiple dab prints of his fingerprint and they are combined to form a fingerprint mosaic. Bowyer et al. [6, 7] used multiple 2D images of human face along with infrared and 3D images of the same. These images are used in seven different combinations ranging from the individual images to all three types of images and the experimental results show the 2D+3D+infrared combination produces the smallest EER of less than 0.002%. In other words, the multi-sample fusion of facial images provides the most accurate recognition performance as compared that of the individual images. Advantages: As the sensor outputs contain raw data, the information content is the richest at this level of fusion. Such a rich amount of data can contain noise which may be due to inappropriate lighting conditions, background noise, and presence of dirt and sweat on the sensor surface. Noise reduces the quality of a biometric sample and will degrade the accuracy of the authentication system [9, 8] but the richness of information available at this level allows for application of sample quality improvement techniques without the loss of any information. Raw data is also the largest in size and fusion of such data 37

2. Multimodal Biometric and Fusion Technology reduces the total required storage space. Limitations: Sensor level fusion is applicable only to multi-instance and multi-sample biometric systems which contain multiple instances or samples of the same biometric trait. Multimodal biometric systems require samples from the different modalities to be combined. Samples from the different modalities may not be compatible [6] and hence cannot be used for sensor level fusion. Most of the commercial-offthe-shelf (COTS) products do not provide sensor outputs. Performing sensor level fusion in that case is not possible.

2.3.2

Feature Level Fusion

This is the second level of information fusion in a biometric authentication system. The feature extraction phase produces feature vectors which contain the second richest level of information as compared to the raw data captured by biometric sensors. Storing feature vectors instead of raw data requires comparatively less storage space. But still, these feature vectors are high dimensional and to store such high dimensional feature vectors from the different modalities more space is required than that for a single modality. Simple techniques like averaging or weighted averaging, feature level fusion can be used to reduce the total number of feature vectors [41] if they originate from the same feature extraction algorithm. Issues in feature level fusion: Feature sets of the different modalities are generally not compatible as each modality has its own unique features (e.g. fingerprints contain minutia points which are defined by their type and orientation whereas irises are identified by their pattern and color information which are scalar entities). This type of fusion techniques can be applied only when the feature sets are compatible and/or closely synchronized [57]. Modalities like hand geometry and palmprint or voice and corresponding lip and facial movements are closely coupled modalities and can be synchronized by fusion at this 38

2.3. Fusion Levels level [56, 10]. Biometric systems that use feature level fusion may suffer from the curse of dimensionality if they use simple concatenation of feature vectors as a fusion technique [39]. Existing approaches to feature level fusion: Feature level fusion is generally achieved by concatenation of feature vectors [36, 24, 39]. Feature vectors obtained from the feature extraction stages of the different modalities are simply concatenated to form the resultant feature vector which may be in higher dimensions than any of its constituent vectors. Xu et al. [57] propose a novel approach to feature level fusion of ear and face profile using Kernel Canonical Correlation Analysis (KCCA) first map the feature vectors into a higher dimensional space before applying correlation analysis on them. Advantages: Feature vectors contain second richest level of information as compared to the raw biometric data. Feature level fusion allows for synchronization of closely coupled modalities [56, 10] such as face and lip movements while talking or modalities like hand geometry and palmprint. This synchronization can avert spoof attacks as faking facial movements as well as voice accurately at the same time is not easy. It can also be used as a liveliness detection technique. Limitations: Methods like concatenation are generally used for combining feature vectors [57, 10]. Such a concatenation results in an increase in the total number of vector dimensions and the system may suffer from curse of dimensionality and thus, degraded recognition performance. Storing a lager dimensional feature vector also takes up valuable storage space. Feature concatenation also results in the noise from the fused modalities getting added up as well. Noise reduces the quality of a biometric sample captured and will degrade the accuracy of the authentication system [9, 8]. Most of the commercial-off-theshelf () products do not provide feature vectors. Performing a feature level fusion in that case is not possible. 39

2. Multimodal Biometric and Fusion Technology

2.3.3

Match-score Level Fusion

Match score level fusion is level three of fusion in a typical biometric authentication system. The biometric matcher module produces match scores which are an indicator of similarity or dissimilarity between the input sample and the one stored in a database. Match scores constitute the third richest level of information after raw data and feature vectors and match score level fusion aims at combining these match scores and use the resultant score to make a final recognition decision. As these scores are readily available, match score level fusion is the most effective level of fusion [53]. Issues in match-score level fusion: Not all biometric matchers output their scores in a fixed format. Some biometric matchers may output similarity scores while others may provide dissimilarity scores. The scores may even follow the different numerical ranges and distributions. These scores are required to fall in a common numerical range and normalization techniques are used to achieve that [20, 45, 46, 47, 33]. A detailed study of normalization techniques is done by Jain et al. [20]. Existing approaches for match-score level fusion: Score level fusion is the most popular level of fusion and many research works are devoted to it [30, 37, 45, 47, 46, 17, 13, 33, 3, 2, 58, 4, 23, 1, 34]. Once the scores are normalized, simple arithmetic rules like sum [47, 51, 34], weighted sum [31, 34], product [51], weighted product, product of probabilities, sum of probabilities, min-max score [47] can be used. Techniques like matcher weighting and user weighting are explored by Snelick et al. [46]. Matcher weighting techniques assign weights to the matchers based on the accuracy of that biometric trait. While the user weighting techniques assign user-specific weights [50] to each user. These weights are determined empirically. Advantages: Commercial-off-the-shelf (COTS) products give access to the match scores hence, they are readily available for fusion [41]. 40

2.3. Fusion Levels Match scores are simple real numbers within a certain numeric range and thus can go through any numerical transformations. They also contain the lowest level of data complexity. This makes them both easier to understand and combine. Match scores are the smallest in size compared to the raw data and feature vector. Limitations: Match scores are required to be normalized before fusion. The robustness of normalization techniques plays an important role in the robustness of the biometric system that uses it [20].

2.3.4

Decision Level Fusion

This is the fourth level of information fusion in a biometric authentication system. Sample for each biometric trait goes through the separate sensor, feature extraction and matching modules. The matchers make an independent local decision about the identity of a user. Decision level fusion aims at fusing these local decisions to form a final/ global recognition decision that classifies the user as genuine or an impostor. Issues in decision level fusion: This is the least rich level of information as the amount of information available for fusion can be a simple binary 0s and 1s (1 indicating a match and 0 indicating a non-match) or match/non-match drawstring.. Since the final decision is made on a certain combination of the decisions output by the different matchers, their configurations; sequential or parallel, plays an important role in the formation of a final decision as well as the level of security provided by the system [4, 51, 1]. Existing approaches to decision level fusion: Existing approaches consist of simple rules like the AND rule [4, 51, 1], the OR rule [51, 1], majority and weighted majority voting [41]. Bayesian fusion rules [37, 2] are also used which aim at transforming the decision labels output by individual matchers into probability values. Behavior knowledge space method [38] is a method of decision level fusion which makes use of a look-up table to decide the final label based on the labels output by individual matchers. 41

2. Multimodal Biometric and Fusion Technology Advantages: Fusion of decisions allows for the use of independent, unimodal biometric authentication products off the shelf. Products from the different vendors can also be used without worrying about their compatibility. Also the security level of the entire system can be adjusted as per the requirement by changing the configuration in which the matchers are connected [4, 51, 1]. Decision level fusion makes use of the smallest and the most unambiguous piece of information for fusion i.e. the matcher decision. As the most commercial-off-the-shelf products (COTS) provide access to only the final decision, make this as the only possible and feasible level of fusion. Limitations: Relative performances of the matchers must be taken into consideration while making the final decision. Weighted fusion rules must assign matcher weights based on the relative accuracy of the matchers, so that the most accurate matcher plays an important role in determining the final decision.

2.4

Different Fusion Rules

Fusion levels make use of fusion rules for combining the data available. A survey of the different types of fusion rules and the issues related to them is performed in this section. Fusion rules are broadly classified into two categories: 1) Fixed fusion rules and 2) Trained fusion rules. The rules falling in these two categories are described in the next subsections.

2.4.1

Fixed fusion rules

These rules are said to be fixed rules because when fusion is performed using these rules, no training takes place. It is a linear process and no feedback is taken from the outputs in order to better or optimize the results. 42

2.4. Different Fusion Rules Following terms are used in the mathematical expressions in the fusion rules. f = fused score d

=

final decision

M

=

number of matchers

xi

=

output score for the ith matcher

di

=

decision of the ith matcher

2.4.1.1

AND Rule

In AND rule [4, 51, 1], the final decision is a conjunction of the individual matcher decisions. AND rule is explained in Eq. (2.1). The matchers can be imagined to form a serial combination. It results in a match if and only if all the inputs result into a match. Systems using AND rule for fusion exhibit a high FRR. Because of it, this type of fusion gives the highest level of security. These systems, however, have a very low, even as low as 0% FAR. d = d1 ∧ d2 ∧ d3 ∧ . . . ∧ dM

2.4.1.2

(2.1)

OR Rule

In OR rule [51, 1], the final decision is a disjunction of the individual matcher decisions. OR rule is explained in Eq. (2.2). The matchers can be imagined to form a parallel combination. It results in a match if any one of the inputs results in a match. Systems using OR rule for fusion exhibit a higher FAR. Because of it, this type of fusion gives the lowest level of security. These systems, however, have very low, even as low as 0% FRR. d = d1 ∨ d2 ∨ d3 ∨ . . . ∨ dM 43

(2.2)

2. Multimodal Biometric and Fusion Technology 2.4.1.3

Majority Voting

In this rule, the final decision is the one that is in the majority among all the matchers as shown in Eq. (2.3). If majority of the matchers are reliable and accurate, then the final decision can be trusted otherwise, an accurate matcher may fail to affect the final output decision in the presence of majority of less accurate matchers. d = majority(d1 , d2 , d3 , . . . , dM ) 2.4.1.4

(2.3)

Maximum Rule

The maximum rule [41] gives the final output score as the maximum of all the matcher scores. A mathematical formula is given in Eq. (2.4). The accuracy of the matcher with the maximum match-score will determine the accuracy of the final decision. d = max(d1 , d2 , d3 , . . . , dM ) 2.4.1.5

(2.4)

Minimum Rule

In minimum rule [41], the final output score is the minimum of all the matcher scores shown in Eq. (2.5). The accuracy of the matcher with the minimum match-score will determine the accuracy of the final decision. d = min(d1 , d2 , d3 , . . . , dM ) 2.4.1.6

(2.5)

Sum Rule

Sum rule [34, 51] is similar to the OR rule, but for match scores. The final score is the sum of the individual match scores (Eq. (2.6)). This rule has been shown to perform better than product rule and Linear Discriminant Analysis (LDA) [40, 12]. The best combination results are obtained by applying simple sum rule [34]. Sum rule is 44

2.4. Different Fusion Rules not significantly affected by the probability estimation errors and this explains its superiority [23]. The ROC performances of sum rule are always better than that of the product rule [49]. M ∑

d=

xi

(2.6)

i=1

2.4.1.7

Product Rule

Product rule [51] is similar to the AND rule, but for match scores. The final score is the product of all the input scores. This rule performs as if the matchers are arranged in a series combination (Eq. (2.7)). This rule assumes independence between individual modalities. If the scores are normalized in the range [0, 1] then the final match score will always be less than the smallest of the match scores. d=

M ∏

xi

(2.7)

i=1

2.4.1.8

Arithmetic Mean Rule

In arithmetic mean rule [51], final fused score is the arithmetic mean of the match scores of all the matchers (Eq. (2.8)). The match scores should be in the same numerical range i.e. normalized so that they can be averaged out properly. M ∑

f=

2.4.2

xi

i=1

M

(2.8)

Trained Fusion Rules

The trained rules, unlike the fixed rules, contain parameters that are obtained, refined and optimized over a training period. Some systems have learning phases which are used to learn specific weights. 45

2. Multimodal Biometric and Fusion Technology Following terms are used in the mathematical expressions for the fusion rules. f = fused score d

=

final decision

M

=

number of matchers

xi

=

output score for the ith matcher

di

=

decision of the ith matcher

2.4.2.1

Weighted Sum Rule

Weighted sum rule [31, 34] is a modification of the simple sum rule. In this rule, a weighted factor is multiplied by the score and then sum is calculated as given in Eq. (2.9). Each matcher carries a weight, Wi . Matcher weights are calculated heuristically and empirically. Generally, the matcher weights are directly proportional to their accuracy. The more accurate a matcher is, the more weight it carries.

f=

M ∑

Wi xi

(2.9)

i=1

where

M ∑

Wi = 1.

i=1

2.4.2.2

Weighted Product Rule

This rule is a modification of the simple product rule. In this rule, a weighted factor is multiplied by the score and then the product is calculated as given in Eq. (2.10). Each matcher carries some weight, Wi . Matcher weights are calculated using some heuristic methods. Matcher weights are directly proportional to their accuracy. The more accurate a matcher is, the more weight it carries.

f=

M ∏ i=1

46

Wi xi

(2.10)

2.4. Different Fusion Rules where

M ∑

Wi = 1.

i=1

2.4.2.3

User Weighting

In user weighting [19, 41], each user is assigned a specific weight into each modality. The final score is calculated by multiplying the assigned weights with each modality’s score as given in Eq. (2.11). User weights are calculated heuristically and through empirical measurements. This method provides an improvement over the matcher weighing method. Performance improvement is seen with reduced FRR as the user convenience and user’s habituation with the system increases. Automatic learning and update of system parameters help in reducing the errors associated with an individual, thus improving system performance [19]. Not all users have all of their traits up to an acceptable level/standard and so assigning equal weights to such traits will not exploit their full potential and may dampen their uniqueness. This will also cause their false rejection. User specific weighing and userspecific thresholds will accommodate more people into the system, thus reducing the system’s FRR. f inal_score_af ter_userweighting = Wu f 2.4.2.4

(2.11)

Fisher Linear Discriminant (FLD)

Fisher’s linear discriminant [2, 58] is a classification method that projects high-dimensional data onto a line and performs classification in this one-dimensional space. The projection maximizes the distance between the means of the two classes while minimizing the variance within each class. The Fisher Linear Discriminant (FLD) gives a projection matrix W that reshapes the scatter of a data set to maximize class separability, defined as the ratio of the between-class scatter matrix to the within-class scatter matrix. This projection defines features 47

2. Multimodal Biometric and Fusion Technology that are optimally discriminating. For that purpose FLD finds such a W to maximize the criterion function defined in Eq. (2.12). J(W ) =

W T Sb W W T Sw W

(2.12)

where, Sw and Sb are the within-class scatter matrix and the between-class scatter matrix, respectively. These two matrices are defined in Eq. (2.13) and Eq. (2.14).

Sw =

Nj c ∑ ∑

(xji − µj )(xji − µj )T

(2.13)

j=1 i=1 c ∑ Sb = (µj − µ)(µj − µ)T

(2.14)

j=1

In Eq. (2.13) and (2.14), xji is the ith sample of class j, µj is the mean of class j, c is the number of classes, Nj is the number of samples in class j, and µ is the mean of all classes.

2.4.2.5

Support Vector Machine (SVM)

SVM [25, 34, 3, 2, 14] is a binary classifier that maps the input vector xi ∈ Rd to output labels yi = −1, +1 for i = 1, 2, 3, . . . , l, where l is the number of patterns to be classified. A binary classifier is used to map the n-dimensional data points into the label space of two classes (genuine and imposter). The main objective of SVM is to find an optimal separating hyperplane that separates the dataset into two groups and maximizes the separation between them. The separating Hyperplane could be linear or nonlinear [14]. A general form of SVM can be represented as, f (x) =

∑

αi yi K(x, xi ) + b

i=Ω

48

(2.15)

2.4. Different Fusion Rules Where, αi is Lagrange multipliers, Ω is indexed to support vectors for which αi ̸= 0, yi is output labels, b is bias term, K(x, xi ) is kernel function and x is the input vector to be classified. The kernel function decides the nature of the separating surface. This function can be polynomial (see Eq. (2.16)) or a Radial Bias Function (RBF) (see Eq. (2.17)). K(x, xi ) = (xxi + 1)d ,

d>0

K(x, xi ) = e−g|x−y|

2

(2.16)

(2.17)

The choice of kernel function depends upon the satisfaction of Mercer’s conditions. It has been shown that the polynomial classifier performs better than the RBF classifier kernel function [2]. The final decision is made depending on whether f (x) is above or below a threshold. The experimental comparison of many fusion techniques along with SVM is made in [2] and it shows an FAR of [1.07, 2.09]% and an FRR of [0.0, 1.5]%. This is the best classifier among the classifiers studied in [2]. These results are well within acceptable limits and as it can be seen an FRR of 0% has been achieved. 2.4.2.6

Multi Layer Perceptron (MLP)

An MLP [55, 2] is a supervised learning neural network consisting of many layers of nodes or neurons as shown in Fig. 2.2. The input layer is denoted by i and accepts inputs like a particular pattern or a feature vector from the biometric system. The next layer is called as a hidden layer since it exists only between the input layer and the output layer. There can be more than one hidden layer and they are denoted by j. The final layer is the output layer, denoted by k, and provides the outputs to the outside world. 49

2. Multimodal Biometric and Fusion Technology wij

Wjk

i1

j1 Output(s)

Inputs

k1 i2

Input layer (i)

j2 Hidden layer (s)

Output layer (k)

Figure 2.2: Multilayer perceptron network.

All the layers are fully connected; meaning a node in one layer is connected to every node in the next layer. Each of these connections or branches carries a specific weight. The weight matrices are wij for weights between the input and the hidden layer and Wjk for the weights between the hidden layer and the output layer. These weights are obtained during the training phase. Input to each node in the input layer undergoes identity transformation, Ii = Oi where Ii is the input and Oi is the output of the ith node. Input Ij to the hidden layer is given by the following transformation. Ij =

∑

wij Oi

(2.18)

j

where wij is the weight of branch between i to j and Oi is the output of the ith node from the input layer. The output of the hidden layer is given by, O j = f Ij

(2.19)

where Ij and Oj are the input and output of the j th node, respectively. f () is an activation function of the node. Ik is the input to the output layer is given by the following transformation. Ik =

∑

Wjk Oj

k

50

(2.20)

2.4. Different Fusion Rules where, Wjk is the weight of branch between j node to k node and Oj is the output of the j th node from the hidden layer. Once these inputs are available at the output layer, they undergo similar transformation as given by Eq. (2.19). MLP is a supervised learning network. That means the desired output is available and the MLP is trained to deliver an output close to the desired output (e.g. for learning the AND rule, when the input is two 0s, the desired output is a 0). During the training phase, many input patterns are presented to the MLP along with the desired output and the weights wij and Wjk are adjusted in such a way that actual output is as close to the desired output as possible. The learning phase is completed when the MLP can deliver actual output very close to the desired output. For continuous adjustment of the weights wij and Wjk , MLP uses a back-propagation algorithm [15, 44, 28]. This algorithm calculates the error which is the difference between actual and desired output. This error is propagated backwards from the output layer until all the weights up to the input layer are adjusted. The weights are adjusted as follows, wkj (t + 1) = wkj (t) + η(δk Ok )

(2.21)

where, wkj (t + 1) and wkj (t) are the weights connecting nodes k and j at iteration (t + 1) and t, respectively and η is the learning rate. The difference between the actual output (δk ) and the desired output (dk ) is calculated as follows. ′

δk = f (Ik )(dk − Ok )

(2.22)

This error is then back propagated through the entire network and the weights are adjusted as follows. ′

δj = f (Ik )

∑ k

51

δk wkj

(2.23)

2. Multimodal Biometric and Fusion Technology 2.4.2.7

Mixture-of-Experts (MOE)

Mixture-of-Expert rules [25] consists of two layers as shown in the Fig. 2.3. Layer of local experts: This layer consists of a number of local experts. Each local expert contains some classifiers. These classifiers work on a single modality. Each classifier uses its own matching algorithms and comes up with local score. These local scores are then passed along to the second layer. Layer of gating network: This layer consists of a gating network [25]. This network combines the local scores into a final fused score using hard switching fusion network, sum rule and product rule in rulebased fusion, support vector machines, multilayer perceptrons, and binary decision trees in learning-based fusion. This core is then used to classify the user as genuine or an imposter. The gating network may be one of the following. 1) Adaptive Hard-Switching Networks: Final fused score is given by Eq. (2.24). f=

M ∑

(2.24)

Wi xi

i=1

In this mode of operation, Wi = 1 for all i and Wj = 0 for all j and j ̸= i. Effectively, the final score is the score of any one of the matcher.

Fused score Adaptive Gating Network 11

12

1st Local Expert

2nd Local Expert

1N

Nth Local Expert

Figure 2.3: Block diagram of Mixture-of-Experts.

52

2.4. Different Fusion Rules 2) Adaptive Weighted Combination Networks: Final fused score is given by Eq. (2.25). f=

M ∑

Wi xi

(2.25)

i=1

In this mode of operation, the classifier weights are nonzero. They M ∑ fall in the range [0, 1] such that Wi = 1. i=1

It is a weighted sum of all the classifier rules. This is termed as a soft fusion method. Matcher learning techniques are used to determine an optimum value for constants Wi . 3) Adaptive Nonlinear Fusion Networks: The previous two type of gating networks produce a linear separation between the imposter and genuine distributions. For more information on experimental results of the same please refer [25]. These types of gating networks make use of non-linear classifiers such as SVM or decision-based neural networks. It gives more flexible decision boundaries. The experimental results in [25] show this type of fusion to be the best in terms of performance. αj are Lagrange multipliers and S contains indexes to the support vectors sj . B is a bias multiplier and K(s, sj ) is a kernel function. The experimental results in [25] show this type of fusion to be the best in terms of performance. 2.4.2.8

Bimodal Fusion (BMF)

Bimodal fusion [10] fuses features from two modalities, hence the name. Both the modalities undergo with the different feature extraction phases. The features obtained after feature extraction phases are then fused to form a combined feature vector. In this approach, multimodal fusion is based on late fusion and its variants. Detailed discussion of this method can be found in [22, 35]. Late fusion or fusion at the score level involves combining the scores of the different 53

2. Multimodal Biometric and Fusion Technology classifiers, each of which has made an independent decision. The distinguishing factor here is the separation of the different modalities in the feature extraction phase. 2.4.2.9

Cross-Modal Fusion

In cross-modal fusion [10], the modalities go through a feature extraction phase and the obtained features are then passed through another feature extraction technique in order to obtain cross-modal features. The cross-modal features are based on optimizing cross-correlations in a rotated cross-modal subspace. The features are fused in a crossmodal subspace. The individual combination methods which are used to extract features for the cross-modal fusion, try to combine the synchrony between the features fused. Therefore, features like facial movements and voice that have close synchronization between them have generally benefited from such fusion techniques. Chetty et al. [10] explain a cross modal technique using Latent Semantic Analysis (LSA) and Canonical Correlation Analysis (CCA). The LSA technique tries to extract underlying semantic relationship between the face and the audio vectors while the CCA technique tries to find a linear mapping that maximizes the cross-correlation between the feature sets combined. 2.4.2.10

3-D Multimodal Fusion

3-D multimodal fusion [10] technique requires 3D features. The 2D samples that are captured by the sensor are used to make a 3D model of that modality. Modalities such as face, fingerprint are prominently used in their 2D as well as 3D forms. This requires appropriate supplementary information to construct 3D features from many 2D features. These features include frontal and side views for 3D face generation, sweep sensors outputs in the form of strips from the front and side views of a fingerprint. Various internal anatomical relationships be54

2.4. Different Fusion Rules tween the modalities make it easy to find external and visible relationship between them from the sensor outputs. These include head movement during pronunciation of a particular letter or word. This fusion technique can also help the system to perform liveliness checks by capturing and checking for such movements and hence provide protection against spoof attacks, either manual or computer generated.

2.4.2.11

Canonical Correlation Analysis (CCA), and Kernel Canonical Correlation Analysis (KCCA)

In these methods, association between two feature vectors are measured. CCA is a classical multivariate method dealt with linear dependencies of the feature vectors and KCCA is a method that generalizes the classical linear CCA to nonlinear setting. These methods are described in the following. Canonical Correlation Analysis (CCA): In this approach [57, 56], only two feature vectors are considered for fusion. A correlation function is required to be defined between these two feature vectors. CCA finds a pair of linear transformations one for each of the vectors to maximize the correlation coefficient between extracted features. Then CCA finds additional pairs of variables that are maximally correlated subject to the constraint that they are uncorrelated with previous variables. This process is repeated until all correlation features of two vectors are extracted. Based on this idea, the canonical correlation features of two vectors are extracted. These features are used as discriminant information between the groups of feature vectors. CCA finds a pair of a linear transformation between the two vectors such that it maximizes the correlation coefficient between them. This simplicity of having linear transformation leads to the disadvantage of this method not being able to identify all or most of the useful descriptors possible. Also, the CCA suffers from small sample size problem.

55

2. Multimodal Biometric and Fusion Technology Kernel Canonical Correlation Analysis (KCCA): Kernels are functions or methods of mapping data into higher dimensional feature spaces. The KCCA [57, 56] projects the features from lower dimensions into higher dimensional Hilbert spaces [43]. This higher dimensional feature maximized the correlation coefficient between two vectors. It is used to extract the nonlinear canonical correlation features associated with the modalities.

2.4.2.12

Simple and Weighted Average

Simple averaging [52] of output is also called as Basic Ensemble Method (BEM). It is used when the individual classifiers produce a continuous output. It is assumed that classifiers are mutually independent and that there is no bias toward any of the classifiers means the accuracy and the efficiency of each classifier are same. Therefore, they all have the same weights. The final score is calculated in Eq. (2.26). M ∑

f=

xi

i=1

M

(2.26)

Weighted averaging [51] of output is called as Generalized Ensemble Method (GEM). The assumption in BEM about the mutual independence of classifiers and their unbiasness is not a practical one. It is very natural to have classifiers having the different accuracy and thus should carry the different weights. The final score is given by Eq. (2.27)

f=

M ∏ i=1

where

M ∑

Wi = 1.

i=1

56

Wi xi

(2.27)

2.4. Different Fusion Rules 2.4.2.13

Optimal Weighting Method (OWM)

OWM [52, 51] is an effective way of combining the classifiers linearly. But, there might exist nonlinear effects of classifier combination and interactions among the classifiers. Ignoring these interactions may result in inaccurate results. 2.4.2.14

Likelihood Ratio-Based Biometric Score Fusion

This method of fusion [33, 32] uses the likelihood ratio test [33, 41]. It is a form of density-based score level fusion. It requires the explicit estimation of genuine and imposter score densities [41]. Accuracy of the density-based fusion technique depends upon the accuracy with which the densities are calculated. The likelihood ratio test is as shown in the following. According to the Neyman-Pearson theorem [33] P (ψ(X) = 1|H0 ) = α ψ(X) = 1 iffgen (x)/fimp (x) ≥ η = 0 otherwise

(2.28) (2.29)

where, X = [X1 , X2 , . . . , XM ]

:

match sores of M different matchers.

fgen (x) = conditional joint densities of the M matchers given the genuine class fimp (x) = conditional joint densities of the M matchers given the imposter class α

= a given level, to be kept fixed, generally it is the FAR

η

= decision threshold

H0

= hypothesis that the user is imposter

H1

= hypothesis that the user is genuine

ψ

= statistical test for testing the hypotheses.

57

2. Multimodal Biometric and Fusion Technology According to the Neyman-Pearson theorem, theorem, the optimal test to determine whether the user is a genuine user or an imposter is the likelihood ratio test given by Eq. (2.29), at a given FAR, α. The threshold η is decided such that the GAR is maximized. When the underlying densities are known, the rule guarantees that there is no other rule with a higher GAR than this. The densities fgen (x) and fimp (x) are calculated from the training set.

2.4.2.15

Borda Count Method

Borda count method [30, 41] is used in rank level fusion [30]. In rank level fusion, each user is assigned a rank by all the matchers based on their relative similarity with the input query sample. This is useful in the identification scenario rather than in verification scenario. Ranks give a relative ordering among the users, starting from the user with the highest rank to the one with the lowest rank. Unlike match scores, ranks do not give any details about the measure of similarity or dissimilarity between the input sample and the one stored in the database. But the ranking gives more details than just a match or non-match decision. In the borda count method, the ranks assigned to the users by biometric matchers are summed up to form a statistic sk for user k (see Eq. (2.30)). sk =

R ∑

rj,k

(2.30)

j

where, rj,k is the rank assigned to the k th user by the j th matcher for j = 1, . . . , R and k = 1, . . . , M . The ranks are revised based on this statistic sk for each user. The user with the least value for sk gets the highest rank. Borda count assumes that all the matchers have equal accuracies, this cannot always be true. Also a simple summation of ranks will put users with 58

2.4. Different Fusion Rules average ranks on the same levels with those that have scored the best rank for one matcher but worst for the other (e.g. a user with a rank of 3 from two matchers will have a final statistic, sk as 6). A user with rank of 1 for the first matcher and 5 for the other matcher will also have the same statistic of 6. Such a high discrepancy in a user’s ranks provided by the different matchers clearly indicates the difference in accuracies of those matchers, something that is not considered by the Borda count method. 2.4.2.16

Logistic Regression Method

Logistic Regression [41] is a generalization of the Borda count method [30]. A weighted sum of ranks assigned to a user is performed for each user to form a statistic sk (see Eq. (2.31)). sk =

R ∑

wj rj,k

(2.31)

j

where rj,k is the rank assigned to the k th user by the the j th matcher and wj is the weight assigned to j th matcher for j = 1, . . . , R and k = 1, . . . , M . The weights are determined in a training phase by logistic regression method [48]. A matcher with higher accuracy is assigned a higher weight. This overcomes the drawback of the Borda count method related to the assumption of all matchers being of equal accuracy. However, Logistic regression method does require a training phase to calculate the weights accurately. 2.4.2.17

Kernel Fischer Discriminant Analysis (KFDA)

KFDA [17] is used to solve the problem of nonlinear decision boundary between genuine and imposter distributions. Conventional methods degrade the fusion accuracy when decision boundary is linear. It can 59

2. Multimodal Biometric and Fusion Technology control the security level (FAR, FRR) on a threshold as it does not make a unique decision boundary like SVM and other methods. The KFDA works as follows. The input feature vectors are first projected into a very large, possibly infinite functional space via nonlinear mapping. Then Linear Discriminant Analysis (LDA) is applied there. The LDA projects 2- dimensional data onto a line. This makes it easier to discriminate it linearly and maximize the ratio of inter-class scatter to intra-class scatter.

2.4.2.18

Minimum Cost Bayesian Classifier

Minimum cost Bayesian classifier [2, 54] is used in decision level fusion. This works on the principle of minimization of the expected cost function called Bayes Risk. The mathematical formulation of the problem is shown in Eq. (2.32). B = E{Cij }

(2.32)

where, i, j ∈ {0, 1}, Cij is the cost of deciding a = i when w = j is present and w = 10 denotes either the presence or absence of the claimed identity. Considering these four conditions of a and w, Eq. (2.32) becomes Eq. (2.33). B = C00 P (a = 0, w = 0) + C01 P (a = 0, w = 1) +C10 P (a = 1, w = 0) + C11 P (a = 1, w = 1)

(2.33)

The posteriori authentication probability is given in Eq. (2.34) using Bayes’ theorem. p = P (w = 1|x) =

f (x|w = 1)P (w = 1) ξ(x)

(2.34)

In Eq. (2.34), f (x|w) is a likelihood function. The joint density of local authentication probabilities is given in Eq. (2.35). 60

2.4. Different Fusion Rules

ξ(x) = f (x|w = 0)P (w = 0) + f (x|w = 1)P (w = 1)

(2.35)

If P (w = 1) is replaced by g then Eq. (2.35) becomes Eq. (2.36). ξ(x) = f (x|w = 0(1 − g) + f (x|w = 1)g

(2.36)

Therefore, p=

f (x|w = 1)g f (x|w = 0(1 − g) + f (x|w = 1)g

(2.37)

For n sensors the total likelihood function is given in Eq. (2.38). f (x|w) = Πni=1 fi (xi |w)

(2.38)

Where fi (xi |w) is a likelihood function for sensor i. The sensors are assumed to be independent and thus the total likelihood function is a product of likelihood functions of all the sensors. Value of ’a’ is decided with the help of standard 0-1 cost function. { a=

i |w=1) 1 if Πni=1 ffii (x (xi |w=1) >

(1−g)(C10 −C00 ) g(C01 −C11 )

0 otherwise

(2.39)

The most important part in all these calculations is the accurate modeling of the likelihood function. The choice of a proper function is pivotal in deciding the quality of the probability fusion. The experimental comparison of many fusion techniques along with Minimum risk Bayesian classifier is made in [2] and it shows an FAR of [0.63, 1.5]% and an FRR of [0.0, 1.75]%. This is amongst the best classifier among the classifiers studied in [2]. These results are well within acceptable limits and as it can be seen an FRR of 0% has been achieved. 61

2. Multimodal Biometric and Fusion Technology 2.4.2.19

Decision Tree

A Decision tree [2, 27] consists of nodes and branches. The nodes are tests that are performed on a particular attribute of the data and the leaf corresponds to a particular label. The path from the root node to the leaf node then consists of a series of tests that classify the input data into a particular class. Alternatively, a decision tree gives a graphical representation of the classification process. They recursively partition the data points in an axis-parallel manner. At each node, the test splits the input on some discriminating feature and thus provides a natural way of feature selection.

2.5

Comparative Study of Fusion Rule

Comparative Study of Fusion Rules in the Different Fusion Levels In previous sections, multimodal biometric fusion levels and fusion rules are reviewed. The observations of the different fusion rules and fusion rules are summarized in this section. Table 2.1 shows the different fusion rules and their applications in the different fusion levels. Summary of observations on the state of the art in fusion levels and modalities and fusion rules is shown in Table 2.2. A comment on each work is also given in column 5 of Table 2.2. The survey reveals that fusion at match-score levels is a favored choice for information fusion among the researchers (refer Table 2.2). This popularity could be attributed to the easy availability of match scores, reduced complexity while fusing and speedup in the fusion process. Another benefit of using MLF is that biometric matchers can be chosen commercial-off-the-shelf (COTS), without any modifications, which in turn, reduces system development cost. Selecting an appropriate fusion rule depends upon many factors. As far as the level of security is concerned, the AND rule of fusion provides the highest level of security while OR rule provides the least. 62

2.5. Comparative Study of Fusion Rule Table 2.1: Fusion rules used at various fusion levels. Fusion Rule

SLF

FLF

MLF

RLF

DLF

AND Rule

Y

OR Rule

Y

Majority Voting

Y

Maximum Rule

Y

Minimum Rule

Y

Sum Rule

Y

Product Rule

Y

Arithmetic Mean Rule

Y

Weighted Sum Rule

Y

Weighted Product Rule

Y

User Weighting

Y

Fisher Linear Discriminant(FLD)

Y

Support Vector Machine (SVM)

Y

Multi Layer Perceptron (MLP)

Y

Mixture-of-Experts (MOE)

Y

Y

Y

Bimodal Feature Fusion (BMF)

Y

Cross-Modal Fusion

Y

3-D Multi-Modal Fusion

Y

CCA and KCCA

Y

Simple and Weighted Average

Y

Optimal Weighting Method(OWM)

Y

Likelihood Ratio-Based Score Fusion

Y

Borda Count Method

Y

Logistic Regression Method

Y

KFDA

Y

Minimum Cost Bayesian Classifier

Y

Decision trees

Y

When a large number of training samples are available, use of SVMs is potentially time consuming. On the other hand, SVMs are unaffected by the dimensionality of feature vectors which makes them an 63

2. Multimodal Biometric and Fusion Technology attractive choice for fusing high-dimensional feature vectors. SVMs and minimum cost Bayesian classifier are some of the most accurate classifiers as compared to decision trees, FLD and MLP.

Table 2.2: Summary of state of the art fusion rules and fusion levels Level

SLF

FLF

Work

Jain & Ross [18]

Modalities Fusion rules

Fingerprint

Remark

Mosaicking

Shows improved levels of recognition performance by extracting minutiae from the newly constructed mosaicked fingerprint template.

Sum, Product, Mean, Median, Max rule

Shows a performance prediction model for sensor fusion.

KCCA

Shows novel approach to extract the non linear canonical correlation features present in ear and profile face.

Wang & Bhanu [55]

Face, Speech

Xu & Mu [57]

Ear, Profile Face

Rattani et al. [36]

Face, Fingerprint

Concatenation

Shows a novel approach for feature reduction using k-means clustering algorithm.

Wang et al. [56]

Face, Ear

GPCA (Generalized PCA), KCCA

Shows novel approach to pose problem using pose transformation. Continued to next page

64

2.5. Comparative Study of Fusion Rule Table 2.2 – continued from previous page Level

Work

Chetty & Wagner [10]

Kumar & Zhang [24]

Ross & Govindarajan [39]

Humm et al. [16]

MLF

Nandakumar et al. [33]

Modalities Fusion rules

Remark

BMF, CMF, 3MMF

Shows improved performance against imposter attacks, type 1 (still photo and prerecorded voice) and type 2 (CG animated video and prerecorded voice) replay attacks and robustness to pose and illumination variations.

Naive Bayes classifier, decision trees, SVM, k-Nearest Neighbour

Shows effectiveness of feature selection and usefulness of shape properties of human hand.

PCA, LDA

Shows the advantages of fusion at this level by feature selection and removal of redundant/correlated features.

Signature, Speech

Weighted Sum

Shows a novel user authentication system based on a combined acquisition of online pen and speech signals and also spoken signatures.

Face, Fingerprint

Likelihood Ratio test, Sum rule, SVM

Shows higher levels of recognition performance without the need of parameter tuning as required in Sum rule and SVM.

Face, Speech

Hand shape, Palmprint

Face, Hand

Continued to next page

65

2. Multimodal Biometric and Fusion Technology Table 2.2 – continued from previous page Level

Work

Remark

Samad et al. [42]

Speech spectrogram

Mean rule

Shows the use of spectrographic features from speech for biometric identification.

Bowyer et al. [6]

Face 2D, 3D, Infrared

Weighted sum

Shows multi-sample and multimodal approaches for fusion of facial images.

Shu & Ding [45]

Face, Iris, online and offline Signature

Sum, Product, Weighted sum rule

Shows the effectiveness and robustness of Adaptive Confidence Transform (ACT) normalization technique.

Face, Fingerprint

Sum, Minimum, Maximum rules, Matcher weighing, User weighting

Gives a large scale evaluation of multimodal system using state-of-the-art COTS.

Face 2D, 3D

Weighted sum, Product, Minimum rule

Shows one of the largest experimental study for 2D + 3D Face recognition.

Face, Ear, Signature

Modified Borda Count, modified Logistic Regression

Shows improvement over original Borda Count method and Logistic Regression method.

Bhatnagar Palmprint et al. [5]

Maximum of Sum of ranks

Shows a novel approach by making use of K-template partitions.

Snelick et al. [46]

Chang et al. [7]

RLF

Modalities Fusion rules

Monwar & Gavrilova [30]

Continued to next page

66

2.5. Comparative Study of Fusion Rule Table 2.2 – continued from previous page Level

Work Besbes et al. [4]

Modalities Fusion rules Fingerprint, Iris

Remark

AND rule

Shows the effectiveness of AND rule in maximizing security system.

Shows a state-of-the-art system to combat drawbacks of unimodal biometric systems.

Monwar & Gavrilova [29]

Face, Ear, Signature

AND ,OR rule, Majority Voting, Weighted majority voting, behavior knowledge space

Ailisto et al. [1]

Weight, Height, Body fat

AND rule, OR rule

Shows an innovative identification system using light biometrics.

Roli et al. [38]

Face, Video (head rotation), Speech

Majority voting, Behavior knowledge space

Shows an experimental comparison between fixed and trained fusion rules.

Toh & Yau [49]

Sum, Product, AND, OR rule, Optimal Weighing, Fingerprint, Simple Voice average, Weighted average, combination of hyperbolic functions

DLF

67

Shows a simple approach to model parameter computation as compared to complex neural networks.

2. Multimodal Biometric and Fusion Technology From the existing literature, it can be observed that there is neither a fix level of fusion nor a certain fusion rule that always gives the most accurate and reliable results along with an optimum level of computational, storage and temporal complexity. Five levels of fusion, a variety of biometric traits and more than two dozen fusion techniques provide lots of opportunities for customization of multimodal biometric authentication system. However, for all practical purposes, considering the fusion complexity, reusability and efficiency, fusion at match-score levels would be the preferred choice of many system developers.

2.6

Summary

To meet the increasing demand of reliable, accurate and robust biometricbased authentication systems, multimodal biometrics have been advocated. A number of researches addressing the issues and challenges of multimodal biometrics have been reported in the recent literature. This chapter is an attempt to survey the latest and state-of-the- art techniques in multimodal biometrics and this work has been published in [21]. The literatures across five fusion levels and more than two dozen fusion rules have been studied. This study will give a user enough details for making an appropriate choice of fusion level and fusion rules in building an accurate, robust and effective multimodal authentication system.

68

BIBLIOGRAPHY

Bibliography [1] Heikki Ailisto, Mikko Lindholm, Satu-Marja Makela, and Elena Vildjiounaite. metrics.

Unobtrusive user identification with light bio-

In Roope Raisamo, editor, Nordic Conference on

Human-Computer Interaction 2004, pages 327–330. ACM, 2004. ISBN 1-58113-857-1.

URL http://dblp.uni-trier.de/db/

conf/nordichi/nordichi2004.html#AilistoLMV04. [2] S. Ben-Yacoub, Y. Abdeljaoued, and E. Mayoraz. Fusion of face and speech data for person identity verification. IEEE Transactions on Neural Networks, 20(5):1065–1074, September 1999. doi: 10.1109/72.788647. [3] S. Ben-Yacoub, J. Luttin, K. Jonsson, J. Matas, and J. Kittler. Audio-visual person verification. In IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 1999, volume 1, pages –585 Vol. 1, 1999. doi: 10.1109/CVPR.1999.786997. [4] F. Besbes, H. Trichili, and B. Solaiman. Multimodal biometric system based on fingerprint identification and iris recognition. In 3rd International Conference on Information and Communication Technologies: From Theory to Applications (ICTTA 2008), pages 1–5, April 2008. doi: 10.1109/ICTTA.2008.4530129. [5] J. Bhatnagar, A. Kumar, and N. Saggar. A novel approach to improve biometric recognition using rank level fusion. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR’ 07), pages 1–6, 2007. [6] K.W. Bowyer, K.I. Chang, P.J. Flynn, and Xin Chen. Face recognition using 2-d, 3-d, and infrared: Is multimodal better than multisample? In Proceedings of the IEEE, volume 94, pages 2000– 2012, Nov. 2006. doi: 10.1109/JPROC.2006.885134. 69

2. Multimodal Biometric and Fusion Technology [7] Kyong I. Chang, Kevin W. Bowyer, and Patrick J. Flynn. An evaluation of multimodal 2d+3d face biometrics. IEEE Transaction on Pattern Analysis Machine Intelligence, 27(4):619–624, 2005. ISSN 0162-8828. doi: http://dx.doi.org/10.1109/TPAMI. 2005.70. [8] Y. Chen, S.C. Dass, and A.K. Jain. Localized iris image quality using 2-d wavelets. In International Conference on Biometric Authentication, pages 373–381, 2006. [9] Yi Chen, Sarat Dass, and Anil Jain. Fingerprint quality indices for predicting authentication performance. In In: Proc. AVBPA, Springer LNCS-3546, pages 160–170, 2005. [10] Girija Chetty and Michael Wagner. Audio-visual multimodal fusion for biometric person authentication and liveness verification. In Fang Chen and Julien Epps, editors, NICTA-HCS Net Multimodal User Interaction Workshop (MMUI 2005), volume 57 of CRPIT, pages 17–24, Sydney, Australia, 2005. ACS. [11] Ming-Cheung Cheung, Man-Wai Mak, and Sun-Yuan Kung. Multi-sample data-dependent fusion of sorted score sequences for biometric verification. In International Conference on Acoustics, Speech, and Signal Processing, 2004., volume 5, pages 681–684, 2004. [12] S. K. Dahel and Q. Xiao.

Accuracy performance analysis of

multimodal biometrics. In Proceedings of the 2003 IEEE Workshop on Information Assurance United States Military Academy, pages 170– 173. Information Assurance Workshop, 2003. IEEE Systems, Man and Cybernetics Society, 18-20 June 2003. doi: 10.1109/SMCSIA.2003.1232417. [13] Sarat C. Dass, Karthik N, and Anil K. Jain. A principled approach to score level fusion in multimodal biometric systems. In 70

BIBLIOGRAPHY In Proceedings 5th International Conference Audio- and VideoBased Biometric Person Authentication, pages 1049–1058, 2005. [14] B. Gutschoven and P. Verlinde. Multi-modal identity verification using support vector machines (svm). In Proceedings of the Third International Conference on Information Fusion, 2000 (FUSION 2000), volume 2, pages THB3/3–THB3/8 vol.2, July 2000. [15] Tzung-Pei Hong and Jyh-Jong Lee. Parallel neural learning by iteratively adjusting error thresholds. In Proceedings of International Conference on Parallel and Distributed Systems, 1998., pages 107–112, Dec 1998. doi: 10.1109/ICPADS.1998.741026. [16] A. Humm, J. Hennebert, and R. Ingold. Combined handwriting and speech modalities for user authentication. IEEE Transactions on Systems, Man and Cybernetics, Part A: Systems and Humans, 39(1):25–35, 2009. [17] M. Ichino, H. Sakano, and N. Komatsu. Multimodal biometrics of lip movements and voice using kernel fisher discriminant analysis. In 9th International Conference on Control, Automation, Robotics and Vision (ICARCV ’06), pages 1–6, Dec. 2006. doi: 10.1109/ ICARCV.2006.345473. [18] A. Jain and A. Ross. Fingerprint mosaicking. In IEEE International Conference on Acoustics, Speech, and Signal Processing, 2002. Proceedings. (ICASSP ’02), volume 4, pages IV–4064–IV– 4067 vol.4, 2002. doi: 10.1109/ICASSP.2002.1004811. [19] A. K. Jain and A. Ross. Learning user-specific parameters in a multibiometric system. In Proceedings of International Conference on Image Processing (ICIP’2002), volume 1, pages I: 57–60, Rochester, New York, September 22-25 2002. 71

2. Multimodal Biometric and Fusion Technology [20] Anil Jain, Karthik Nandakumar, and Arun Ross. Score normalization in multimodal biometric systems. Pattern Recognition, 38 (12):2270–2285, December 2005. [21] T. Joshi, S. Dey, and D. Samanta. Multimodal Biometrics: State of the Art in Fusion Techniques. International Journal of Biometrics (IJBM), 1(4):393–417, 2009. [22] J. Kittler, G. Matas, K. Jonsson, and M. Sanchez. Combining evidence in personal identity verification systems. Pattern Recognition Letters, 18(9):845–852, 1997. [23] Josef Kittler, Mohamad Hatef, Robert P. W. Duin, and Jiri Matas. On combining classifiers. IEEE Trans. Pattern Anal. Mach. Intell., 20(3):226–239, 1998. ISSN 0162-8828. doi: http: //dx.doi.org/10.1109/34.667881. [24] A. Kumar and D. Zhang. Personal recognition using hand shape and texture. Image Processing, IEEE Transactions on, 15(8): 2454–2461, Aug. 2006. ISSN 1057-7149. [25] S. Y. Kung and Man-Wai Mak. On consistent fusion of multimodal biometrics. In IEEE International Conference on Acoustics, Speech and Signal Processing, 2006. (ICASSP 2006), volume 5, pages V–1085 – V 1088, 2006. [26] Hyun Cheol Lee, Eun Seok Kim, Gi Taek Hur, and Hee Young Choi. Generation of 3d facial expressions using 2d facial image. In ICIS ’05: Proceedings of the Fourth Annual ACIS International Conference on Computer and Information Science, pages 228– 232, Washington, DC, USA, 2005. IEEE Computer Society. ISBN 0-7695-2296-3. doi: http://dx.doi.org/10.1109/ICIS.2005.68. [27] Carsten Maple and Vitaly Schetinin. Using a bayesian averaging model for estimating the reliability of decisions in multimodal 72

BIBLIOGRAPHY biometrics. In The First International Conference on Availability, Reliability and Security (ARES 2006), pages 929 – 935. IEEE Computer Society, April 2006. doi: http://dx.doi.org/10.1109/ ARES.2006.141. [28] MLP.

The

multilayer

perceptron

classifier.

http://europa.eu.int/en/comm/eurostat/research/supcom.95/16/ result/node7.html, 1995. [29] M. M. Monwar and M. Gavrilova. A robust authentication system using multiple biometrics. In Computer and Information Science, 2008. [30] Md. Maruf Monwar and Marina Gavrilova. Fes: A system for combining face, ear and signature biometrics using rank level fusion. In Fifth International Conference on Information Technology: New Generations (ITNG 2008), pages 922–927, April 2008. doi: 10.1109/ITNG.2008.254. [31] T. Nakagawa, I. Nakanishi, Y. Itoh, and Y. Fukui. Multi-modal biometrics authentication using on-line signature and voice pitch. In International Symposium on Intelligent Signal Processing and Communications (ISPACS 2006), pages 399–402, Dec. 2006. doi: 10.1109/ISPACS.2006.364913. [32] Karthik Nandakumar, Yi Chen, A. K. Jain, and Sarat C. Dass. Quality-based score level fusion in multibiometric systems. In 18th International Conference on Pattern Recognition (ICPR 2006), volume 4, pages 473–476, 0-0 2006. doi: 10.1109/ICPR. 2006.951. [33] Karthik Nandakumar, Yi Chen, Sarat C. Dass, and Anil Jain. Likelihood ratio-based biometric score fusion. IEEE Transactions on Pattern Analysis and Machine Intelligence, 30(2):342– 73

2. Multimodal Biometric and Fusion Technology 347, 2008.

ISSN 0162-8828.

doi: http://dx.doi.org/10.1109/

TPAMI.2007.70796. [34] Michael Goh Kah Ong, Tee Connie, Andrew Teoh Beng Jin, and David Ngo Chek Ling. A single-sensor hand geometry and palmprint verification system. In WBMA ’03: Proceedings of the 2003 ACM SIGMM workshop on Biometrics methods and applications, pages 100–106, New York, NY, USA, 2003. ACM. ISBN 1-58113779-6. doi: http://doi.acm.org/10.1145/982507.982526. [35] N. Poh and J. Korczak. Hybrid biometric person authentication using face and voice features. In Proc. of Int. Conf. on Audio and Video-Based Biometric Person Authentication, pages 348– 353, Halmstad, Sweden, June 2001. [36] A. Rattani, D. R. Kisku, M. Bicego, and M. Tistarelli. Feature level fusion of face and fingerprint biometrics. In First IEEE International Conference on Biometrics: Theory, Applications, and Systems, 2007 (BTAS 2007)., pages 1–6, September 2007. doi: 10.1109/BTAS.2007.4401919. [37] G. Richard, Y. Mengay, I. Guis, N. Suaudeau, J. Boudy, P. Lockwood, C. Fernandez, F. Fernandez, C. Kotropoulos, A. Tefas, Pitas, R. Heimgartner, P. Ryser, C. Beumier, P. Verlinde, S. Pigeon, G. Matas, J. Kittler, J. Biglin, Y. Abdeljaoued, E. Meurville, L. Besacier, M. Ansorge, G. Maitre, J. Luettin, S. Ben-Yacoub, B. Ruiz, K. Aldama, and J. Cortes. Multi modal verification for teleservices and security applications (m2vts). In IEEE International Conference on Multimedia Computing and Systems, 1999, volume 2, pages 1061–1064 vol.2, Jul 1999. doi: 10.1109/MMCS.1999.778659. [38] F. Roli, G. Fumera, and J. Kittler. Fixed and trained combiners for fusion of imbalanced pattern classifiers. In Information 74

BIBLIOGRAPHY Fusion, 2002. Proceedings of the Fifth International Conference on, volume 1, pages 278–284 vol.1, 2002. doi: 10.1109/ICIF.2002. 1021162. [39] Arun Ross and Rohin Govindarajan. Feature level fusion using hand and face biometrics. In Proceedings of SPIE Conference on Biometric Technology for Human Identification II, volume 5779, pages 196–204, 2005. [40] Arun Ross and Anil Jain. Information fusion in biometrics. Pattern Recogn. Lett., 24(13):2115–2125, 2003. ISSN 0167-8655. doi: http://dx.doi.org/10.1016/S0167-8655(03)00079-5. [41] Arun A. Ross, Karthik Nandakumar, and Anil K. Jain. Handbook of Multibiometrics. Springer-Verlag New York, Inc. Secaucus, NJ, USA, 2006. [42] S.A. Samad, D.A. Ramli, and A. Hussain. A multi-sample singlesource model using spectrographic features for biometric authentication. In 6th International Conference on Information Communications & Signal Processing, 2007, pages 1–5, Dec. 2007. doi: 10.1109/ICICS.2007.4449710. [43] Giovanni Sansone. Orthogonal Functions. Dover Publications, 1991. [44] S. Seung. Multilayer perceptrons and backpropagation learning. http://hebb.mit.edu/courses/9.641/2002/lectures/lecture04.pdf, 2002. [45] Chang Shu and Xiaoqing Ding. Multi-biometrics fusion for identity verification. In 18th International Conference on Pattern Recognition (ICPR 2006), volume 4, pages 493–496, 2006. doi: 10.1109/ICPR.2006.821. 75

2. Multimodal Biometric and Fusion Technology [46] R. Snelick, U. Uludag, A. Mink, M. Indovina, and A. Jain. Largescale evaluation of multimodal biometric authentication using state-of-the-art systems. IEEE Transactions on Pattern Analysis and Machine Intelligence, 27(3):450–455, March 2005. ISSN 0162-8828. doi: 10.1109/TPAMI.2005.57. [47] Robert Snelick, Mike Indovina, James Yen, and Alan Mink. Multimodal biometrics: issues in design and testing. In Sharon L. Oviatt, Trevor Darrell, Mark T. Maybury, and Wolfgang Wahlster, editors, Proceedings of the 5th international conference on Multimodal interfaces (ICMI ’03), pages 68–72. ACM, 2003. ISBN 158113-621-8. URL http://dblp.uni-trier.de/db/conf/icmi/ icmi2003.html#SnelickIYM03. [48] Ying

So.

A

tutorial

on

logistic

regression.

http://www.ats.ucla.edu/stat/ sas/library/logistic.pdf, 1995. [49] K.-A. Toh, Xudong Jiang, and Wei-Yun Yau. Exploiting global and local decisions for multimodal biometrics verification. IEEE Transactions on Signal Processing, 52(10):3059–3072, Oct. 2004. ISSN 1053-587X. doi: 10.1109/TSP.2004.833862. [50] Kar-Ann Toh and Wei-Yun Yau. Some learning issues in userspecific multimodal biometrics.

In 8th Control, Automation,

Robotics and Vision Conference, 2004 (ICARCV 2004), volume 2, pages 1268– 1273, December 2004. doi: 10.1109/ICARCV.2004. 1469028. [51] Kar-Ann Toh and Wei-Yun Yau. Combination of hyperbolic functions for multimodal biometrics data fusion. IEEE Transactions on Systems, Man, and Cybernetics, Part B, 34(2):1196–1209, April 2004. ISSN 1083-4419. doi: 10.1109/TSMCB.2003.821868. [52] Kar-Ann Toh, Wei-Yun Yau, and Xudong Jian. A reduced multivariate polynomial model for multimodal biometrics and clas76

BIBLIOGRAPHY sifiers fusion. IEEE Transactions on Circuits and Systems for Video Technology, 14(2):224–233, February 2004. [53] Brad Ulery, William Fellner, Peter Hallinan, Austin Hicklin, and Craig Watson. Studies of biometric fusion. Technical report, National Institute of Standards and Technology, 2006. [54] K. Veeramachaneni, L. A. Osadciw, and P. K. Varshney. Adaptive multimodal biometric fusion algorithm. In SPIE Aerosense, 2003. [55] Rong Wang and Bir Bhanu. Performance prediction for multimodal biometrics. In 18th International Conference on Pattern Recognition (ICPR’06), 2006. [56] Yu Wang, Zhi-Chun Mu, Ke Liu, and Jun Feng. Multimodal recognition based on pose transformation of ear and face images. In Proceedings of the 2007 International Conference on Wavelet Analysis and Pattern Recognition, Beijing, China, November 2007. IEEE. [57] Xiaona Xu and Zhichun Mu. Feature fusion method based on kcca for ear and profile face based multimodal recognition. In IEEE International Conference on Automation and Logistics, 2007, pages 620–623, Aug. 2007. doi: 10.1109/ICAL.2007.4338638. [58] Wenchao Zhang, Shiguang Shan, Wen Gao, Yizheng Chang, Bo Cao, and Peng Yang. Information fusion in face identification. In Proceedings of the 17th International Conference on Pattern Recognition (ICPR 2004), volume 3, pages 950–953 Vol.3, Aug. 2004. doi: 10.1109/ICPR.2004.1334686.

77

Chapter 3

Biometric Indexing: State-of-the-Art In this chapter, a survey of literature related to biometric indexing is reported. The chapter is organized as follows. Various iris data indexing techniques will be reviewed in Section 3.1. Section 3.2 will cover an in-depth discussion about the fingerprint biometric data indexing. Work related to face biometric data indexing will be presented in Section 3.3. Section 3.4 will review the work on multimodal biometric data indexing. Finally, the chapter will be summarized in Section 3.5.

3.1

Survey on Iris Biometric Data Indexing

In the last few decades, iris recognition algorithms for verification [19, 21, 20, 58, 59, 45, 88, 22, 74, 46, 33] have been extensively investigated. However, work on iris data indexing have been scarcely reported. All the existing work on iris data indexing techniques [83, 66, 67, 61, 62, 26, 78, 69, 71, 40] can be divided into two categories: iris texture-based indexing and iris color-based indexing. In iris texture-based indexing, the index keys are generated from the iris texture whereas color of iris is used in iris color-based indexing. Detailed descriptions of these techniques are given in Section 3.1.1 and 3.1.2 . 79

3. Biometric Indexing: State-of-the-Art

3.1.1

Iris Texture-Based Indexing

A coarse iris classification technique using fractals has been proposed by Yu et. al. [83]. This technique classifies iris images into four categories. In this method, an iris image is segmented into sixteen blocks. Among these blocks, eight blocks belong to the upper part and remaining blocks to the lower part of the iris. Then, the fractal dimension values are calculated from the image blocks and the mean values of the fractal dimension of the upper and the lower parts are used to classify iris images into four categories. Mukherjee and Ross [67, 66] develop two methods for iris data indexing. In first method, IrisCode feature [21] is calculated using Gabor wavelet. IrisCode features according to their method are as large as of 2048 dimensions. To reduce the dimensionality of the feature vector, row/column/block averaging and Principal Component Analysis (PCA) methods are used. Then k-means clustering approach is applied to partition the reduced dimension iris code into multiple groups. In other technique, an iris texture is divided into a number of blocks. For each block, the histogram of signed differences pixel intensities (of similar positioned pixels in the block and adjacent blocks) is calculated. This method is known as Signed Pixel Level Difference Histogram (SPLDH) analysis [67, 66]. Like the IrisCode method, this method also deals with features of around 256 dimensions. Local Binary Pattern (LBP) [68] is used in another iris data indexing method proposed by Mukherjee [66]. Author divides the normalized iris into a number of blocks and computes block-based LBP values. Finally, k-means clustering is applied to group the iris data. Mehrotra et al. [62] use Scale Invariant Feature Transform (SIFT) [55] method to extract texture features for iris data indexing. This method finds a number of key points. Then high dimensional (124 dimensions) SIFT features are extracted for each key point. The key points are indexed using geometric hashing [81]. This indexing method also deals 80

3.1. Survey on Iris Biometric Data Indexing with a large number of key points (≈ 100 key points) with high dimensional feature vector. In another work, Mehrotra et al. [61] propose an iris data indexing technique based on Discrete Cosine Transform (DCT). In this method, the normalized iris image [61] is divided into sub-bands using multiresolution DCT transformation [82] and the DCT energy based histogram is calculated for each sub-band. Then, each histogram is divided into fixed size bins to group the iris images. To generate a global key for each iris image, they obtain the bin number for each sub-band and traverse all sub-bands in Morton order [65]. B-tree structure is used to store the iris data. At the time of probing, a key from query iris image is generated and the tree is traversed using the query key. Finally, the retrieved templates are compared with the query template to find the best match. An iris indexing mechanism based on the context clustering property of the Burrows-Wheeler Transform (BWT) [8] is developed by Gadde et al. [26]. This method converts a normalized gray scale iris image to a binary image and chooses a horizontal n-bit pattern. Then the locations of these patterns are found in the iris image using BWT. An indexing scheme is defined based on the count of occurrence of the n-bit binary patterns in the normalized binary iris image. For this purpose, the image is divided into K segments vertically. After that the iris image is assigned an index with the segment number where the segment number has the maximum number of occurrences for the n-bit pattern. At the time of probing, the test image is assigned four indices corresponding to the top four segments with maximum counts of occurrence of n-bit pattern. Finally, the test image is compared only against those images in the database that have these 4 indices as their index, starting with the first index. In another work, Rathgeb and Uhl [71] utilize biometric hash generation (HG) [76] for iris indexing. The basic idea of this method is 81

3. Biometric Indexing: State-of-the-Art to generate hash value from the different parts of an iris image. In this approach, the normalized iris image [71] is divided into several blocks and an n-bit binary hash is computed using HG [76]. Then an n-bit Karnaugh Map (KM) [79] is constructed to store the iris data. A pointer to each iris template is stored at the according node of the KM. At the time of querying, the extracted hash is mapped onto the stored KM. If a node points at one or more iris templates, the extracted iris-code is matched against those. This procedure is repeated recursively for neighboring nodes until identification is yielded or a certain search depth is reached. Apart from the above texture-based iris data indexing techniques, Vatsa et al. [78] develop another indexing scheme which consists of two phases. In the first phase, the method uses Euler code [7, 77] to generate a small subset of possible matches. In the next phase, it uses 2ν-SVM [17] match score fusion algorithm to find the best matches from the list of possible matches obtained in first phase using textural and topological features of the iris images.

3.1.2

Iris Color-Based Indexing

Few iris color-based indexing schemes have been proposed in the existing literature. Fu et al. [24] make the first attempt to classify iris images using color. This technique is based on artificial color filtering. All irises images are divided into nine classes and an artificial color filter [15] is designed corresponding to each class. For an iris image, all nine filters are applied to the image and the class is assigned corresponding to the filter which gives the maximum response. The artificial color filter trains with a discriminator that assigns a pixel either to the class of interest or to some other classes. Another attempt to index iris images using color has been made by Puhan and Sudha [69]. This method also refers group based color indexing scheme which relies on the natural iris color. In this technique, 82

3.2. Survey on Fingerprint Biometric Data Indexing the iris color image is converted from RGB space to YCbCr color space and two types of color indices, namely blue and red indices, are computed using Cb and Cr components, respectively. The range of values of red and blue color indices are partitioned into a number of groups. Depending upon the values of red and blue color indices, an image is assigned to one of these groups. During searching, for a query, a few groups from blue and red color indices are selected based on the blue and red color indices of the query image. The nearest identities are declared based on the intersection between these groups. Jayaraman et al. [40] develop a hierarchical approach to efficiently retrieve the iris data from a large iris database. This approach uses iris color to index the database and iris texture to retrieve the iris images from the indexed iris database. First, a set of iris images is selected which are similar to the query by applying indexing technique. The index value is computed by averaging the intensity values of all red and blue color pixels. To store the iris data the kd-tree structure is used. Then, iris texture features of the selected images are used to determine the identity of the query image. The iris texture features are extracted through Speed Up Robust Features (SURF) [2] algorithm. A summary of the different iris biometric-based data indexing techniques is given in Table 3.1.

3.2

Survey on Fingerprint Biometric Data Indexing

Fingerprint based biometric data indexing has been extensively studied. Several techniques [6, 10, 11, 14, 13, 27, 30, 41, 47, 48, 49, 54, 51, 52, 57, 66, 73, 4, 86, 16, 18, 23, 37, 43, 53, 60, 80] have been proposed in literature for fingerprint data indexing. The existing fingerprint indexing techniques can be divided into two main categories based on the feature extraction methods. The first category is Minutiae-based 83

3. Biometric Indexing: State-of-the-Art Table 3.1: Summary of different iris data indexing techniques Feature used

Dimension

Indexing technique

Yu et al. [83]

Fractal dimensions

16

Classification technique

Mukherjee and Ross [67]

Colomn/Block Average and PCA of Iris Code

64 to 256

k-mean clustering

Mukherjee and Ross [67]

Signed Pixel Level Difference Histogram of non overlapping blocks

256

Tree structure

Mukherjee [66]

Histogram of block-wise LBP

>128

k-mean clustering

Mehrotra et al. [62]

SIFT features at different scale spaces

256

Geometric hashing

Mehrotra et al. [61]

DCT energy

10

B-tree

Gadde et al. [26]

Burrows-Wheeler Transform

>50

Group based method

Rathgeb and Uhl [71]

Hash Code from block wise gray scale value

16 to 32

Karnaugh map

Euler code

4

Linear

Artificial color filter

>9

Pattern recognition method

Puhan and Sudha [69]

Color indices

2

Group based method

Jayaraman et al. [40]

Color indices

2

Kd-tree

Iris texture-based indexing

Method

Iris color-based indexing

Vatsa et al. [78] Fu et al. [24]

indexing which uses minutiae points of a fingerprint to extract the indexing features and the second category is Ridge orientation-based indexing which uses orientation of the ridges and valleys of a fingerprint for index key generation. A detailed descriptions of some important fingerprint data indexing methods are given in the following. 84

3.2. Survey on Fingerprint Biometric Data Indexing

3.2.1

Minutiae-Based Indexing

In this category of fingerprint indexing techniques, the index feature vectors are extracted from either the minutiae description or the triplet structure form by the minutiae points. Germain et al. [27] present an algorithm to generate a list of candidate set using minutiae triplets from the database. This method uses two structures, called map and multimap [9], to compute all index score values. In enrollment time, all possible minutiae triplets from a fingerprint are extracted and the set of index keys from these triplets is created. The length of each side, ridge counts between each pair of sides, angles measures with respect to the sides for a triplet are used as index key features. In this method, a label is given to each index key and an entry to the multimap structure is added for the index key. In the querying process, a set of index keys is generated for a given query and used to retrieve the items from the multimap which are stored under the same index. Each retrieved item is represented by a hypothesized match between subsets of features in the query instance and the reference model instance that creates the item stored in the multimap. This hypothesized match is labeled by the reference model identifier. Further, a map structure is used to calculate the votes for these hypothesized matches and generate a candidate list. Bebis et al. [4] propose a fingerprint indexing technique based on the Delaunay triangulations [5]. This method, first, generates all Delaunay triangles from the extracted minutiae points. The ratios of largest side with the two smallest sides and cosine angle between the two smallest sides of the triangle are defined as index feature vector. The method uses fingerprint retrieving method similar to that described by Germain et al. [27] to generate candidate set. Bhanu and Tan [6] propose fingerprint indexing technique based on minutiae triplets. First, all possible triplets are created from the extracted minutiae points. Geometric features like angles, handedness, 85

3. Biometric Indexing: State-of-the-Art type, direction and maximum side are used for indexing purpose. In the querying process, this method searches for a number of matching triplets between query and all stored templates by applying some constraints on the geometric features of the triplets. The templates has been selected based on the number of matched triplets. Further, posterior probability of the query and the selected templates is calculated as index score and the index scores are sorted in descending order. Finally, candidate set is generated by putting a threshold on the index score. Another Delaunay triangulations based fingerprint indexing approach is proposed by Liang et al. [49]. However, this approach uses low-order Delaunay triangulation [1, 29]. In this approach, first, authors find all minutiae points from a fingerprint and generate a set of minutiae triplet using low-order Delaunay triangulation [1]. For each triplet, minutia details of each vertex of the triangle, minimum and median angles in the triangle, triangle handedness, length of the longest edge in the triangle and the difference between angles of two edges are computed. These information are used as indexing features. In this way, a set of indexing features are generated from the all triplets. At the time of querying, first the matched triplets between the query and the stored templates are found by applying individual threshold on each feature. Finally, indexing scores are calculated between matched triplets and query triplet and based on a threshold value a candidate list is generated. Ross and Mukherjee [73] suggest ridge associated features for calculating index vectors. This approach is also based on Delaunay triangulation [29, 1]. A set of triplets are derived using Delaunay triangulation and a feature vector is generated from each triplet. Then features corresponds to the geometry of the triangles formed by the triplets are computed. Further, the shape of the ridges associated with the three minutiae points constituting the triplet are estimated. The 86

3.2. Survey on Fingerprint Biometric Data Indexing shape features are computed by fitting a quadratic curve to the ridges associated with each triplet. Once all feature vectors of all triplets are generated, these vectors are partitioned into a number of clusters using k-means clustering algorithm for indexing purpose. Querying performs by k-mean clustered search. The score value is computed from the correspondence between the fingerprint templates of the retrieved clusters and the query template. Zhang et al. [86] introduce an indexing technique based on minutiae structure. The direction, coherence and curvature are extracted from each minutiae point as indexing features. In addition to this, an orientation vector is defined to characterize the each minutiae. To estimate the similarity between two minutiae points, an orderly sequence of these features are maintained. The similar minutiae points from two fingerprints are determined by comparing the direction, coherence, curvature and orientation features at the time of querying. An index score is computed which represents the possibility that two minutiae are correspondent based on not only their similarity but the similarities to other minutiae points. Finally, the fingerprint are sorted in descending order of the index scores and retrieve top fingerprints to generate candidate list. In a recent work, Minutia Cylinder-Code (MCC) [12] based fingerprint indexing technique has been proposed by Cappelli et al. [14]. They represent spatial and directional relationships between a minutiae and its neighborhood structure with a minutiae cylinder [12]. Then each minutiae cylinder is encoded into a fixed-length binary feature vector. Hence, a fingerprint image is represented by a set of bit vectors. The bit vectors are indexed by means of Locality Sensitive Hashing (LSH) [36, 28]. In LSH, each vector is projected into a subspace with low dimension. For this purpose, they define a set of hash functions. These hash functions map a binary vector to a natural number in low dimension. They maintain a hash table correspond87

3. Biometric Indexing: State-of-the-Art ing to a hash function to store the feature vectors into the database. The feature vectors are stored into the hash tables based on the hash values. To retrieve the candidates from the database, the same hash functions are applied to each binary vector of the query template and the number of collisions with database vectors is counted using an accumulator matrix. The candidates are finally ranked according to the similarity distances of the query and the retrieved templates. The similarity between two MCC codes is measured by Hamming distance.

3.2.2

Ridge Orientation-Based Indexing

In this category, fingerprint indexing techniques use the ridge orientation to compute the index feature vectors. Lumini et al. [57] present an indexing scheme based on directional image which represents the local ridge and valley directions from a fingerprint image. They generate a multidimensional feature vectors from the directional image. Further, they apply Karhunen-Lo`eve (KL) transform to reduce the dimensionality of the directional feature vectors. At the time of retrieving, they apply linear search with these low-dimensional features and generate the candidate list based on a threshold value. Cappelli et al. [11] presents an fingerprint indexing method which uses dynamic masks for directional image partitioning. Directional image represents local average directions of a fingerprint ridge lines and is obtained by Chong et al. [18] method. Authors perform “guided” partitioning of the directional image according to the fingerprint topology. For this purpose they define a set of dynamic masks which are directly derived from the most common fingerprint classes (Arch, Tented Arch, Left Loop, Right Loop, Whorl, etc.). Each mask is characterized by a set of vertices which defines the borders of the regions and use to segment the directional image. Further, the application of these masks produces a numerical feature vector which represents each fingerprint as a multidimensional point. At the time of retrieval, the numeral 88

3.2. Survey on Fingerprint Biometric Data Indexing feature vector is generated from a query image and used as an access key for similarity search. The searching is performed in the multidimensional space. Lee et al. [47] aim for a fingerprint classification method based on a feature map. They use ridge features consisting of orientations and inter-ridge spacing within a relatively small fingerprint area to build the feature map. They divide the fingerprint image into a number of non-overlap blocks. The local ridge orientation and inter-ridge spacing are calculated for a block using the method proposed by Hong et al. [35] and each block is represented by a vector. They define feature map as a matrix, where each element represents a distribution of a feature vector. Each row and column of the matrix represents orientation and inter-ridge spacing values, respectively. The dimension of the feature map is reduced using well-known Principal Component Analysis [25] dimensionality reduction method. Finally, they compute the distances between query and all stored templates and generate a candidate list by putting threshold on the distance. Another fingerprint retrieval framework has been reported by Jiang et al. [41]. Authors use orientation features as the main retrieval feature and the dominant ridge distance as an auxiliary retrieval feature. To compute orientation features, first, they estimate orientation field [35] of a fingerprint image which consists of a local dominant orientations in different local neighborhoods of a fingerprint image. Then they select a reference point and find its location and direction. Finally, they align the local orientations with respect to the reference point. The orientation feature vector is constructed by concatenating the aligned local orientations within a circle. Additional to this feature vector, they define the dominant ridge distance as the second mean of the local ridge distances. The local ridge distance is computed by measuring the distance between the center points of two adjacent ridges along a line perpendicular to the local ridge orientation using Hong et 89

3. Biometric Indexing: State-of-the-Art al. [35] method. They perform indexing using both continuous classification and clustering approach. In continuous classification, they sort the fingerprints in the database according to the dominant ridge distance. At the time of retrieval, first, they narrow the search space by selecting a subset of fingerprints whose dominant ridge distances are within a range centered at the query dominant ridge distance. Then, they use orientation vector to generate the final list from the reduced search space. In clustering technique, they use k-mean clustering algorithm to cluster the fingerprints into different groups. Liu et al. [51] introduce a fingerprint retrieval based on two complex filter responses. In their approach, first, they compute the orientation field of the fingerprint image and find the location and orientation of some singular points (e.g. core or delta) in the fingerprint image. They apply two complex filters on the orientation field at the singular point and calculate the magnitude of the filter responses. Further, they align the filter response with the direction of the singular point and construct a numerical feature vector by concatenating the aligned magnitudes of the two filter responses. This feature vector is used as the global feature for fingerprint indexing. For a query fingerprint, the retrieval is done by measuring the Euclidean distances between the query and all stored fingerprints. Based on a threshold value they generate the final candidate list. In another approach, Liu et al. [52] propose clustering of fingerprints using multi-scale orientation features and average ridge distance feature. First, they compute the orientation field by uniformly dividing the fingerprint into a number of blocks [42]. To compute the multi-scale orientation field, they construct a circular tessellation with non-uniform blocks and place the circular tessellation at some reference point (core or delta) of the fingerprint. They compute the orientation feature vectors from the each block of circular tessellation. In addition to this, they compute the average ridge distance of the finger90

3.2. Survey on Fingerprint Biometric Data Indexing print. They partition all fingerprints by applying k-means clustering with the orientation feature vectors. Further, they divide each partition into a number of bins based on the average ridge distance. At the time of retrieval, first they select the cluster using orientation feature vectors of query fingerprint. Further, they use average ridge distance to retrieve the fingerprint from the bins of the selected cluster. In a recent work, Cappelli [10] proposes a fingerprint indexing approach based on vector and scalar fingerprint features. The both types of features are obtained from local ridge-line orientations and frequencies. To compute local orientations, they divide the image into a number of non-overlapping blocks and use traditional gradient-based technique [70] to estimate the local orientations. Once the orientations are available, they estimate local ridge frequencies at the same location. They also find the fingerprint core points in order to correctly align the fingerprint [43, 72] using local orientations. Then both the orientation and frequency images are aligned with respect to each core point and are downsampled. They compute the downsampled local orientations, corresponding strength of the orientations and downsampled local frequencies as feature vector for indexing. In addition to this feature vector, two additional scalar features are used. These two scalar values are the average frequency and the average vertical orientation difference of the non-downsampled orientation and frequency images. To compare two fingerprints, they calculate four score values. First score compares local orientations and strength, second score compares local frequencies, third score compares average frequencies and fourth score compares average vertical orientation differences of two fingerprints. The weighted combination of these four scores is used to retrieve the candidate set. At the time of retrieval, the combined scores are calculated between query template and all stored templates. The candidates similar to the query fingerprint are retrieved from the database by putting a threshold on the combined score. 91

3. Biometric Indexing: State-of-the-Art

3.2.3

Other Feature-Based Indexing Techniques

Other than the above two categories, some approaches use different features for fingerprint indexing. For example, Li et al. [48] use three symmetric filters to measure different orientation structures of a fingerprint. Based on three symmetrical measurements they propose a fingerprint indexing approach. They design three kinds of symmetrical filters, which are the core type filter, the delta type filter and the parallel type filter, on the orientation image to map the different structures of fingerprints into three different feature spaces. They obtain three index vectors from these feature vectors. These vectors are reduced to a fixed length and rotated to set the direction of the cores as a vertical line. Authors generate three lists of candidates at the time of querying. The Euclidean distances between the index vectors of the query template and all stored templates are used to measure the similarity. Finally, they combine these three resulting lists using highest rank method to generate the final candidate set. In another work, Gyaourova and Ross [30] develop a method to index and retrieve fingerprint images by utilizing match scores. Their method relies on comparing a fingerprint with a small set of reference images. First, they select a set of reference subjects and compute match scores against each reference subject. Then discretization function is applied on the match scores to convert them to a discrete domain. These discrete score values are used to generate an index key for a fingerprint. At the time of retrieval, they compute Hamming distance between query index key and all stored index keys. A candidate is retrieved by applying a threshold on the calculated Hamming distance. The existing fingerprint data indexing techniques are summarized in Table 3.2.

92

3.2. Survey on Fingerprint Biometric Data Indexing Table 3.2: Summary of different fingerprint data indexing techniques

Ridge orientation-based indexing

Minutiae-based indexing

Method

Feature used

Dimension

Indexing technique

Germain et al. [27]

Length of each side, ridge counts between each pair of sides and angles of minutiae triplets

9

FLASH

Bebis et al. [4]

Ratio of the largest side with minimum and median sides and cosine angle between the two smallest sides of the Delaunay triangles

3

FLASH

Bhanu and Tan [6]

Angles, handedness, type of minutiae, direction and length of the maximum side of all minutiae triplets

6

Continuous classification

Liang et al. [49]

Orientation, minimum and median angles, length of the longest edge, type of minutiae of low-order Delaunay triangles

>6

Continuous classification

Ross and Mukherjee [73]

Geometry of the Delaunay triangle and shape of ridges associated with the minutiae points of the triangle

9

k-means clustering

Zhang et al. [86]

Direction, coherence, curvature and orientation vector of minutiae point

28

Continuous classification

Cappelli et al. [14]

MCC of neighborhood structure of minutiae points

>15

LSH

Lumini et al. [57]

Reduced dimensional directional image

5

Continuous classification

Cappelli et al. [11]

Average ridge directions and dynamic masks

>30

Continuous classification

Lee et al. [47]

Feature map of local orientations and inter-ridge spacing

35

Continuous classification

193

Continuous classification and k-mean clustering

Jiang et al. [41]

Dominant ridge distance and local ridge orientations

Continued to next page

93

3. Biometric Indexing: State-of-the-Art Table 3.2 – continued from previous page

Others

Method

3.3

Feature used

Dimension

Indexing technique

Liu et al. [51]

Gabor response at singular points

338

Continuous classification

Liu et al. [52]

Average ridge distance and local orientations

157

k-means clustering

Cappelli [10]

Downsampled local ridge orientations and frequencies

17

Continuous classification

Li et al. [48]

Core orientation pattern, delta orientation pattern and parallel orientation pattern

300

Continuous classification

Gyaourova and Ross [30]

Variance of query image and reference score

250

Continuous classification

Survey on Face Biometric Data Indexing

In last few decades, extensive studies have been done in the field of face recognition algorithms. Several feature extraction methods are known for face-based authentication system. Nevertheless, retrieval of face biometric data with a large number of features stored in a huge pool of memory are scarcely reported. In the existing literature, very few work have been reported for face indexing to reduce the search space. We describe these techniques [50, 64, 44] in the following. Lin et al. [50] propose an indexing structure to search the face from a large database. They compute a set of Eigenfaces based on the faces in the database. Then, they assign a rank to each face in the database according to its projection onto each of the Eigenface [75]. Similarly, they compute the Eigenfaces for a query and rank a query face. Finally, they select a set of faces from the database corresponding to the nearest faces in the ranked position with respect to each Eigenface of the query face. These selected faces are used for recognition. 94

3.4. Survey on Multimodal Biometric Data Indexing A linear subspace approximation method for face indexing has been developed by Mohanty et al. [64]. They build a linear model to create a subspace-based on the match scores. A linear transformation is applied to project face images into the linear subspace. For this purpose, first, they apply a rigid transformation obtained through Principal Component Analysis and then a non-rigid affine transformation. An iterative stress minimization algorithm is used to obtain a distance matrix in a low-dimensional space and propose a linear out-of-sample projection scheme for test images. Any new face image is projected into this embedded space using an affine transformation. Kaushik et al. [44] introduce a modified geometric hashing technique to index the face database. They extract features from a face image using SURF [3, 2] operator. They apply mean centering, principal component analysis, rotation and normalization to preprocess the SURF features. Finally, they use geometric hashing to hash these features to index each facial image in the database. Other than the above indexing techniques, several dimensionality reduction techniques [75, 85, 56, 63, 87, 84, 56, 64] have been proposed for face identification system. But these techniques are not utilizing any indexing technique. Hence, the discussion of these techniques has been skipped. We have given a summary of the above mentioned face biometric based indexing techniques in Table 3.3.

3.4

Survey on Multimodal Biometric Data Indexing

Multimodal biometric data indexing is newly explored area in the field of biometric identification. Very few work have been reported in the literature. We give an overview of these techniques in the following. Jayaraman et al. [38, 39] propose an indexing technique for multimodal biometric system using iris, ear, face and signature. They 95

3. Biometric Indexing: State-of-the-Art Table 3.3: Summary of different face data indexing techniques Method

Feature used

Dimension

Indexing technique

Lin et al. [50]

PCA and Eigenface

>83

Condensed database

Mohanty et al. [64]

Subspace approximation on pair-wise distances of training face data

>125

k-nearest neighbor search

Kaushik et al. [44]

Speeded-Up Robust Features (SURF)

>128

Geometric hashing

used kd-tree to store the feature vector of these traits. Before storing the data into kd-tree they normalized the features of each biometric trait and projected to a lower dimensional feature space. They applied PCA for dimensionality reduction. In this technique, more than 100 dimension feature vectors are used. Cascading technique for multimodal biometric system has been developed by Hong et. al. [34]. They use face and fingerprint in their multimodal system. In this technique, exhaustive searching is performed with a single biometric trait to reduce the search space and final matching is performed with multiple traits into the reduced search space. Gyaourova and Ross [32, 31] developed an indexing technique to reduce the search space for multimodal biometric system. They select a fixed set of images of each biometric trait from the database. They compute scores between an image and the fixed set of images for each biometric trait. Based on these scores they generate a fixed-length index codes for each biometric trait and store these index code into the database. To retrieve a set of candidates, they compute similarity between the probe index code and all enrolled index codes and put a threshold to select the candidates. They propose two fusion techniques which use the information from multiple modalities. In one 96

3.5. Summary fusion technique, they concatenate the index codes of different modalities and retrieve the candidate set. In another technique [32, 31], they retrieve the candidate set corresponding to the index code of each modality and then take the union of all retrieved candidate set. We summarize the above mentioned multimodal biometric indexing techniques in Table 3.4.

3.5

Summary

In this chapter, a survey on biometric data indexing techniques with iris, fingerprint, face and multimodal traits has been presented. From the survey of iris data indexing, it can be observed that the existing indexing techniques use either texture or color features for indexing. The texture feature-based techniques use high dimensions of features whereas the color features are with very low dimension. Further, the Table 3.4: Summary of different multimodal biometric data indexing techniques

Fusion

Indexing technique

>100

Iris, ear, face and signature

FL and SL

Kd-tree

PCA on Eigenface for retrieval. Minutiae features of fingerprint for matching

>100

Face and fingerprint

DL

Cascading

Index code based on combined score value

256

Face and fingerprint

SL

Method

Feature used

Jayaraman et al. [39]

Harr features, PCA to reduce the dimensionality

Hong and Jain [34] Gyaourova and Ross [32]

DimenModality sion

FL → Feature level fusion; SL → Score level fusion, DL → Decision level fusion

97

Linear search

3. Biometric Indexing: State-of-the-Art iris color is not stable in all times. Several indexing techniques are reported for fingerprint data indexing. However, the majority of these approaches use continuous classification scheme. In continuous classification, the fingerprint search is performed by either comparing the query fingerprint with all database templates based on index key values or comparing the query fingerprint with the templates until the match value satisfies some criteria like less than some threshold value. Although, the comparison between the query fingerprint and template is much faster than the fine matching. Moreover, the continuous classification only ranks the database templates according to their similarities to the query fingerprint. Some techniques use a large number of low dimensional index key for searching. Face-based indexing techniques are not well studied in the literature. However, there are several methods for dimensionality reduction of face features are available in the literature. But these methods are suitable when the database size is very large. Further, reported work for multimodal biometric data indexing deal with the high dimensional feature vectors of different unimodal traits. The dimensionality of the feature vector affects the searching time. In addition to this, cascading technique uses single trait for searching the database which does not take the advantages of other biometric traits at the time of searching. Finally, the multimodal data indexing approaches use different level of fusions which, in fact, affect the performance of a multimodal biometric identification system.

98

BIBLIOGRAPHY

Bibliography [1] M. Abellanas, P. Bose, J. Garc´ia, F. Hurtado, M. Nicol´as, and P. A. Ramos. On Properties of Higher Order Delaunay Graphs with Applications. In Proceedings of the 21st European Workshop on Computational Geometry, pages 119–122, Eindhoven, The Netherlands, March 2005. [2] H. Bay, T. Tuytelaars, and L. V. Gool. SURF: Speeded Up Robust Features. In Proceedings of the 9th European Conference on Computer Vision, volume LNCS-3951, pages 404–417, Graz, Austria, May 2006. doi: http://dx.doi.org/10.1007/11744023_32. [3] H. Bay, A. Ess, T. Tuytelaars, and L. V. Gool. SURF: Speeded Up Robust Features. Computer Vision and Image Understanding, 110(3):346–359, 2008. doi: http://dx.doi.org/10.1016/j.cviu. 2007.09.014. [4] G. Bebis, T. Deaconu, and M. Georgiopoulos. Fingerprint Identification using Delaunay Triangulation. In Proceedings of the IEEE International Conference on Intelligence, Information, and Systems, pages 452–459, Bethesda, USA, October-November 1999. doi: http://dx.doi.org/10.1109/ICIIS.1999.810315. [5] M. D. Berg, O. Cheong, M. V. Kreveld, and M. Overmars. Computational Geometry: Algorithms and Applications.

Springer-

Verlag New York Inc., 3rd edition, 2008. ISBN 3540779736. [6] B. Bhanu and X. Tan. Fingerprint Indexing Bsed on Novel Features of Minutiae Triplets. IEEE Transactions on Pattern Analysis and Machine Intelligence, 25(5):616–622, 2003. ISSN 01628828. [7] A. Bishnu, B. B. Bhattacharya, M. K. Kundu, C. A. Murthy, and T. Acharya. Euler Vector for Search and Retrieval of Gray-tone 99

3. Biometric Indexing: State-of-the-Art Images. IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics, 35(4):801–812, 2005. [8] M. Burrows and D. J. Wheeler. SRC Research Report: A Blocksorting Lossless Data Compression Algorithm. Technical report, Systems Research Center, Digital Equipment Corporation, Palo Alto, CA, USA, 1994. URL http://www.eecs.harvard.edu/ ~michaelm/CS222/burrows-wheeler.pdf. [9] A. Califano and I. Rigoutsos. FLASH: A Fast Look-up Algorithm for String Homology. In Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pages 353–359, NY , USA, June 1993. ISBN 081863880X. [10] R. Cappelli. Fast and Accurate Fingerprint Indexing Based on Ridge Orientation and Frequency. IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics, 41(6):1511– 1521, 2011. [11] R. Cappelli, A. Lumini, D. Maio, and D. Maltoni. Fingerprint Classification by Directional Image Partitioning. IEEE Transactions on Pattern Analysis and Machine Intelligence, 21(5):402– 421, 1999. [12] R. Cappelli, M. Ferrara, and D. Maltoni. Minutia Cylinder-Code: A New Representation and Matching Technique for Fingerprint Recognition. IEEE Transactions Pattern Analysis and Machine Intelligence, 32(12):2128–2141, 2010. [13] R. Cappelli, M. Ferrara, and D. Maio. Candidate List Reduction based on the Analysis of Fingerprint Indexing Scores. IEEE Transactions on Information Forensics and Security, 6(3):1160– 1164, 2011. 100

BIBLIOGRAPHY [14] R. Cappelli, M. Ferrara, and D. Maltoni. Fingerprint Indexing Based on Minutia Cylinder-Code. IEEE Transactions on Pattern Analysis and Machine Intelligence, 33(5):1051–1057, 2011. [15] H.J. Caulfield, J. Fu, and S.-M. Yoo. Artificial Color Image Logic. Information Sciences, 167(1–4):1–7, 2004. [16] A. Chatterjee, S. Mandal, G. M. A. Rahaman, and A. S. M. Arif. Fingerprint Identification and Verification System by Minutiae Extraction using Artificial Neural Network. International Journal of Computer and Information Technology (IJCIT), 1(1):12–16, 2010. [17] P.-H. Chen, C.-J. Lin, and B. Schlkopf. A Tutorial on ν-Support Vector Machines. Applied Stochastic Models in Business and Industry, 21(2):111–136, 2005. [18] M. M. S. Chong, H. N. Tan, L. Jun, and R. K. L. Gay. Geometric Framework for Fingerprint Image Classification. Pattern Recognition, 30(9):1475–1488, 1997. [19] J. Daugman. The Importance of being Random: Statistical Principles of Iris Recognition. Pattern Recognition, 36(2):279–291, 2003. [20] J. Daugman. How Iris Recognition Works. IEEE Transactions on Circuits and Systems for Video Technology, 14(1):21–30, 2004. [21] J. G. Daugman. High Confidence Visual Recognition of Persons by a Test of Statistical Independence. IEEE Transactions on Pattern Analysis and Machine Intelligence, 15(11):1148–1161, November 1993. [22] S. Dey and D. Samanta. Fast and Accurate Personal identification based on Iris Biometric. International Journal of Biometrics, 2 (3):250–281, June 2010. 101

3. Biometric Indexing: State-of-the-Art [23] J. Feng and A. Cai. Fingerprint Indexing using Ridge Invariants. In Proceedings of the 18th International Conference on Pattern Recognition, volume 4, pages 433–436, Hong Kong, August 2006. [24] J. Fu, H. J. Caulfield, S.-M. Yoo, and V. Atluri. Use of Artificial Color Filtering to Improve Iris Recognition and Searching. Pattern Recognition Letter, 26(14):2244–2251, 2005. [25] K. Fukunaga.

Introduction to Statistical Pattern Recognition.

Academic Press, 2nd edition, 1990. [26] R. B. Gadde, D. Adjeroh, and A. Ross. Indexing Iris Images using the Burrows-Wheeler Transform. In Proceedings of the IEEE International Workshop on Information Forensics and Security, pages 1–6, Seattle, USA, December 2010. [27] R. S. Germain, A. Califano, and S. Colville. Fingerprint Matching using Transformation Parameter Clustering. IEEE Computational Science and Engineering, 4(4):42–49, 1997. ISSN 10709924. [28] A. Gionis, P. Indyk, and R. Motwani. Similarity Search in High Dimensions via Hashing. In Proceedings of the 25th International Conference on Very Large Data Bases, pages 518–529, Edinburgh, Scotland, UK, September 1999. [29] J. Gudmundsson, M. Hammar, and M. V. Kreveld. Higher Order Delaunay Triangulations. Computational Geometry, 23(1):85–98, 2002. [30] A. Gyaourova and A. Ross. A Novel Coding Scheme for Indexing Fingerprint Patterns. In Proceedings of the 7th International Workshop Statistical Pattern Recognition, pages 755–764, Orlando, USA, December 2008. 102

BIBLIOGRAPHY [31] A. Gyaourova and A. Ross. A Coding Scheme for Indexing Multimodal Biometric Databases. In Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops, pages 93–98, Miami, FL, June 2009. [32] Aglika Gyaourova and Arun Ross. Index Codes for Multibiometric Pattern Retrieval. IEEE Transactions on Information Forensics and Security, 7(2):518–529, 2012. [33] K. Hollingsworth, T. Peters, K. W. Bowyer, and P.J. Flynn. Iris Recognition using Signal-Level Fusion of Frames From Video. IEEE Transactions on Information Forensics and Security, 4(4): 837–848, 2009. [34] L. Hong and A. Jain. Integrating Faces and Fingerprints for Personal Identification. IEEE Transactions on Pattern Analysis and Machine Intelligence, 20(12):1295–1307, 1998. [35] L. Hong, Y. Wan, and A. Jain. Fingerprint Image Enhancement: Algorithm and Performance Evaluation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 20(8):777–789, 1998. ISSN 0162-8828. [36] P. Indyk and R. Motwani. Approximate Nearest Neighbors: Towards Removing the Curse of Dimensionality. In Proceedings of the 30th Annual ACM Symposium on Theory of Computing, pages 604–613, Dallas, Texas, United States, May 1998. [37] A. K. Jain, S. Prabhakar, and L. Hong. A Multichannel Approach to Fingerprint Classification.

IEEE Transactions on Pattern

Analysis and Machine Intelligence, 21(4):348–359, 1999. ISSN 0162-8828. [38] U. Jayaraman, S. Prakash, Devdatt, and P. Gupta. An Indexing Technique for Biometric Database. In Proceedings of the Interna103

3. Biometric Indexing: State-of-the-Art tional Conference on Wavelet Analysis and Pattern Recognition, volume 2, pages 758–763, Hong Kong, China, August 2008. [39] U. Jayaraman, S. Prakash, and P. Gupta. An Efficient Technique for Indexing Multimodal Biometric Databases. International Journal of Biometrics (IJBM), 1(4):418–441, 2009. [40] U. Jayaraman, S. Prakash, and P. Gupta. An Efficient Color and Texture based Iris Image Retrieval Technique. Expert Systems with Applications, 39(5):4915–4926, 2012. [41] X. Jiang, M. Liu, and A. C. Kot. Fingerprint Retrieval for Identification. IEEE Transactions on Information Forensics and Security, 1(4):532–542, 2006. [42] X. D. Jiang. Extracting Image Orientation Feature by using Integration Operator. Pattern Recognition, 40(2):705–717, 2007. [43] K. Karu and A. K. Jain. Fingerprint Classification. Pattern Recognition, 29(3):389–404, 1996. [44] V. D. Kaushik, A. K. Gupta, U. Jayaraman, and P. Gupta. Modified Geometric Hashing for Face Database Indexing. In Proceedings of the 7th international conference on Advanced Intelligent Computing Theories and Applications: with Aspects of Artificial Intelligence, pages 608–613, Zhengzhou, China, August 2011. [45] J. Kim, S. Cho, J. Choi, and R. J. Marks-II. Iris Recognition using Wavelet Features. Journal of VLSI Signal Processing Systems, 38 (2):147–156, 2004. ISSN 0922-5773. [46] E. Krichen, S. Garcia-Salicetti, and B. Dorizzi. A New PhaseCorrelation-Based Iris Matching for Degraded Images.

IEEE

Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics, 39(4):924–934, 2009. 104

BIBLIOGRAPHY [47] S. Lee, Y. Kim, and G. Park. A Feature Map Consisting of Orientation and Inter-Ridge Spacing for Fingerprint Retrieval. In Proceedings of the Audio and Video-Based Biometric Person Authentication 9, pages 184–190, New York, USA, July 2005. [48] J. Li, W. Y. Yau, and H. Wang. Fingerprint Indexing Based on Symmetrical Measurement. In Proceedings of the 18th International Conference on Pattern Recognition, volume 1, pages 1038–1041, Hong Kong, China, August 2006. [49] X. Liang, A. Bishnu, and T. Asano. A Robust Fingerprint Indexing Scheme using Minutia Neighborhood Structure and LowOrder Delaunay Triangles. IEEE Transactions on Information Forensics and Security, 2(4):721–733, 2007. [50] K. H. Lin, K. M. Lam, X. Xie, and W. C. Siu. An Efficient Human Face Indexing Scheme using Eigenfaces. In Proceedings of the IEEE International Conference on Neural Networks and Signal Processing, pages 920–923, Nanjing, China, December 2003. [51] M. Liu, X. Jiang, and A. C. Kot. Fingerprint Retrieval by Complex Filter Responses. In Proceedings of the 18th International Conference on Pattern Recognition, volume 1, pages 1042–1045, Hong Kong, China, August 2006. [52] M. Liu, X. Jiang, and A.C. Kot. Efficient Fingerprint Search Based on Database Clustering. Pattern Recognition, 40(6):1793– 1803, June 2007. [53] M. Liu, , and P.-T. Yap. Invariant Representation of Orientation Fields for Fingerprint Indexing. Pattern Recognition, 45(7):2532– 2542, 2012. [54] T. Liu, G. Zhu, C. Zhang, and P. Hao. Fingerprint Indexing based on Singular Point Correlation. In Proceedings of the Inter105

3. Biometric Indexing: State-of-the-Art national Conference on Image Processing, volume 2, pages 293– 296, Genoa, Italy, September 2005. doi: http://dx.doi.org/10. 1109/ICIP.2005.1530386. [55] D. G. Lowe. Distinctive Image Features from Scale-invariant Keypoints. International Journal on Computer Vision, 60(2):91–110, 2004. [56] J. Lu, K.N. Plataniotis, and A.N. Venetsanopoulos. Face Recognition using LDA-Based Algorithms. IEEE Transactions on Neural Networks, 14(1):195–200, 2003. [57] A. Lumini, D. Maio, and D. Maltoni. Continuous versus Exclusive Classification for Fingerprint Retrieval. Pattern Recognition Letters, 18:1027–1034, 1997. [58] L. Ma, T. Tan, Y. Wang, and D. Zhang. Personal Identification based on Iris Texture Analysis. IEEE Transaction on Pattern Analysis and Machine Intelligence, 25(12):1519–1533, 2003. [59] L. Ma, T. Tan, Y. Wang, and D. Zhang. Efficient Iris Recognition by Characterizing Key Local Variations. IEEE Transactions on Image Processing, 13(6):739–750, 2004. [60] P. Mansukhani, S. Tulyakov, and V. Govindaraju. A Framework for Efficient Fingerprint Identification using a Minutiae Tree. IEEE Systems Journal, 4(2):126–137, 2010. ISSN 1932-8184. [61] H. Mehrotra, G. S. Badrinath, B. Majhi, and P. Gupta. Indexing Iris Biometric Database using Energy Histogram of DCT Subbands.

In Proceedings of the International Confer-

ence on Contemporary Computing, volume CCIS-40, pages 194– 204, Noida, India, August 2009. doi: http://dx.doi.org/10.1007/ 978-3-642-03547-0_19. 106

BIBLIOGRAPHY [62] H. Mehrotraa, B. Majhia, and P. Gupta. Robust Iris Indexing Scheme using Geometric Hashing of SIFT Keypoints. Journal of Network and Computer Applications, 33(3):300–313, 2010. doi: http://dx.doi.org/10.1016/j.jnca.2009.12.005. [63] N. Mittal and E. Walia. Face Recognition using Improved Fast PCA Algorithm. In Proceedings of the Congress on Image and Signal Processing, pages 554–558, Sanya, Hainan, May 2008. doi: http://dx.doi.org/10.1109/CISP.2008.144. [64] P. Mohanty, S. Sarkar, R. Kasturi, and P. J. Phillips. Subspace Approximation of Face Recognition Algorithms: An Empirical Study. IEEE Transactions on Information Forensics and Security, 3(4):734–748, 2008. [65] G. M. Morton. A Computer Oriented Geodetic Data Base and a New Technique in File Sequencing. Technical report, International Business Machines Company, Ottawa, Canada, 1966. URL http://books.google.co.in/books?id=9FFdHAAACAAJ. [66] R. Mukherjee.

Indexing Techniques for Fingerprint and Iris

Databases. Master’s thesis, College of Engineering and Mineral Resources at West Virginia University, 2007. [67] R. Mukherjee and A. Ross.

Indexing Iris Images.

In Pro-

ceedings of the 19th International Conference on Pattern Recognition, pages 1–4, Tampa, Florida, USA, December 2008. ISBN 9781424421756.

doi:

http://dx.doi.org/10.1109/ICPR.

2008.4761880. [68] T. Ojala, M. Pietik¨ainen, and T. M¨aenp¨a¨a. Multiresolution GrayScale and Rotation Invariant Texture Classification with Local Binary Patterns. IEEE Transaction on Pattern Analysis and Machine Intelligence, 24(7):971–987, 2002. 107

3. Biometric Indexing: State-of-the-Art [69] N. B. Puhan and N. Sudha. A Novel Iris Database Indexing Method using the Iris Color. In Proceedings of the 3rd IEEE Conference on Industrial Electronics and Applications (ICIEA 2008), pages 1886–1891, Singapore, June 2008. doi: http://dx.doi.org/ 10.1109/ICIEA.2008.4582847. [70] N. K. Ratha, S. Y. Chen, and A. K. Jain.

Adaptive Flow

Orientation-based Feature Extraction in Fingerprint Images. Pattern Recognition, 28(11):1657–1672, 1995. [71] C. Rathgeb and A. Uhl. Iris-Biometric Hash Generation for Biometric Database Indexing. In Proceedings of the 20th International Conference on Pattern Recognition, pages 2848–2851, Istanbul, August 2010. [72] K. Rerkrai and V. Areekul. A New Reference Point for Fingerprint Recognition. In Proceedings of the International Conference on Image Processing, pages 499–502, Vancouver, Canada, September 2000. [73] A. Ross and R. Mukherjee. Augmenting Ridge Curves with Minutiae Triplets for Fingerprint Indexing. In Proceedings of the SPIE Conference on Biometric Technology for Human Identification IV, volume 6539, pages 65390C–65390C–12, Orlando, USA, April 2007. [74] C. Sanchez-Avila, R. Sanchez-Reillo, and D. de Martin-Roche. Iris-based Biometric Recognition using Dyadic Wavelet Transform. IEEE Aerospace and Electronic Systems Magazine, 17(10): 3–6, 2002. [75] M. Turk and A. Pentland. Eigenfaces for Recognition. Journal of Cognitive Neurosicence, 3(1):71–86, 1991. 108

BIBLIOGRAPHY [76] U. Uludag, S. Pankanti, S. Prabhakar, and A. K. Jain. Biometric Cryptosystems: Issues and Challenges. Proceedings of the IEEE, 92(6):948–960, 2004. [77] M. Vatsa, R. Singh, and A. Noore. Reducing the False Rejection Rate of Iris Recognition using Textural and Topological Features. International Journal of Signal Processing, 2(1):66–72, 2005. [78] M. Vatsa, R. Singh, and A. Noore. Improving Iris Recognition Performance using Segmentation, Quality Enhancement, Match Score Fusion, and Indexing. IEEE Transactions on Systems, Man, and Cybernetics, Part B, 38(4):1021–1035, 2008. [79] S. P. Vingron. Switching Theory: Insight Through Predicate Logic, chapter Karnaugh Maps, pages 57–76. Springer-Verlag, 2004. [80] Y. Wang, J. Hu, and D. Phillips. A Fingerprint Orientation Model based on Fourier Expansion (FOMFE) and its Application to Singular-point Detection and Fingerprint Indexing. IEEE Transactions on Pattern Analysis and Machine Intelligence, 29 (4):573–585, 2007. [81] H. Wolfson and I. Rigoutsos. Geometric Hashing: An Overview. IEEE Computational Science and Engineering,, 4(4):10–21, 1997. [82] D. Wu and L. Wu. Image Retrieval based on Subband Energy Histograms of Reordered DCT Coefficients. In Proceedings of the 6th International Conference on Signal Processing, volume 1, pages 26–30, Australia, December 2002. [83] L. Yu, D. Zhang, K. Wang, and W. Yang. Coarse Iris Classification using Box-counting to Estimate Fractal Dimensions. Pattern Recognition, 38(11):1791–1798, 2005. [84] W. Yu, X. Teng, and C. Liu. Face Recognition using Discriminant Locality Preserving Projections. Image and Vision Computing, 24 109

3. Biometric Indexing: State-of-the-Art (3):239–248, 2006. doi: http://dx.doi.org/10.1016/j.imavis.2005. 11.006. [85] P.C. Yuen and J.H. Lai. Face Representation using Independent Component Analysis. Pattern Recognition, 35(6):1247–1257, 2002. [86] Y. Zhang, J. Tian, K. Cao, P. Li, and X. Yang. Improving Efficiency of Fingerprint Matching by Minutiae Indexing. In Proceedings of the 19th International Conference on Pattern Recognition, pages 1–4, Tampa, FL, December 2008. [87] W. Zhao, R. Chellappa, A. Rosenfeld, and P. J. Phillips. Face Recognition: A Literature Survey. ACM Computing Surveys, 35 (4):399–458, 2003. [88] Y. Zhu, T. Tan, and Y. Wang. Biometric Personal Identification based on Iris Patterns. In Proceedings of the 15th International Conference on Pattern Recognition, volume 2, pages 801– 804, Barcelona , Spain, September 2000. doi: http://dx.doi.org/ 10.1109/ICPR.2000.906197.

110

Chapter 4

Iris Biometric Data Indexing Iris biometric-based identification system deals with high dimensional complex features. Hence, the searching with these features makes the identification process extremely slow as well as increases the false acceptance rate beyond an acceptable range. To improve the efficiency of an iris-based identification system, an indexing mechanism has been proposed. In this chapter, the indexing technique with low dimensional feature vector for an iris-based biometric identification system is described. The method is based on the iris texture features. Gabor energy features of iris texture are computed to create a twelve dimensional index key vector. A new storing structure and retrieving technique have been proposed to store and retrieve the iris data from the database. The block diagram of the iris biometric data indexing method is given in Fig. 4.1. The different tasks involved in this approach are discussed in this chapter. The rest of the chapter is organized as follows. An overview of Gabor filter will be discussed in Section 4.1. In Section 4.2, the preprocessing technique for iris image is described. Section 4.3 will describe the feature extraction technique for iris biometric. A new index key generation method for iris biometric data will be introduced in Section 4.4. The storing and retrieving techniques of iris data shall be 111

4. Iris Biometric Data Indexing Enrollment process Iris image

Feature extraction

Preprocessing

Index key generation

Storing

Database

Query iris image

Preprocessing

Feature extraction

Index key generation

Retrieving

Top matches

Query retrieving process

Figure 4.1: An overview of the proposed iris biometric data indexing approach.

given in Section 4.5 and Section 4.6, respectively. The performance evaluation of the indexing method will be presented in Section 4.7. The comparison with existing work will be reported in Section 4.8. Finally, Section 4.9 will give the summary of this chapter.

4.1

Preliminaries of Gabor Filter

Gabor filter is used in this work. For better understanding of work, the concept of the Gabor filter is breifly discussed in this section. Gabor transform theory was first proposed by D. Gabor in 1946 [16]. Daugman [8] proposed two-dimensional (2-D) Gabor transform theory in 1985. It has been observed that the 2-D Gabor filter is an effective method for time-frequency analysis [8, 10]. In the spatial domain, a 2-D Gabor filter consists of a sinusoidal wave modulated by a Gaussian envelope. It performs a localized and oriented frequency analysis of a 2-D signal. A 2-D Gabor function g(x, y) and its Fourier transform G(u, v) can be expressed as in Eq. (4.1) and Eq. (4.2) [19, 23], respectively. ( g(x, y) =

1 2πσx σy

)

[

1 exp − 2

(

112

x2 y2 + σx2 σy2

)

] + 2πjW x

(4.1)

4.1. Preliminaries of Gabor Filter { [ ]} 1 (u − W )2 v2 G(u, v) = exp − + 2 2 σu2 σv

(4.2)

where, σu = 1/2πσx and σv = 1/2πσy . In Eq. (4.1), σx and σy are the standard deviations of the Gaussian envelope along x and y directions, respectively and determine the filter bandwidth. Here, (x, y) and (u, v) are the spatial domain and frequency domain coordinates, respectively. W is the center frequency of the filter in frequency domain and j is √ called the imaginary unit, where j = −1. A multi-resolution Gabor filter (also called multi-channel Gabor wavelet) is a set of filter banks with different scales (frequencies) and orientations. The Gabor filter forms a complete but nonorthogonal basis set. Expanding a signal using this basis provides a localized frequency description, thus capturing local features or energy of the signal. Texture features can then be extracted from this group of energy distributions. The Gabor wavelet can be represented with a mother wavelet and derived the appropriate Gabor functions by different rotations and scales. Let g(x, y) be the mother wavelet of the Gabor filter family. The set of Gabor filter bank gm,n (x, y) can be generated from g(x, y) as expressed in Eq. (4.3). gm,n (x, y) = a−2m g(x′ , y ′ )

(4.3)

where a is a constant such that a > 1, m, n are two integer variables, and x′ and y ′ are represented in Eq. (4.4). x′ = a−m (x cos θn + y sin θn ) ′

y =a

−m

(4.4)

(−x sin θn + y cos θn )

In Eq. (4.4), θn = nπ/K, m = 0, 1, . . . , S −1, and n = 0, 1, . . . , K − 1. The parameters S and K are the total number of scales and orientations in the multi-resolution decomposition, respectively. The values of S and K are to be chosen to ensure that the half-peak magnitude 113

4. Iris Biometric Data Indexing support of the filter responses in the frequency spectrum touch each other. The filter parameters a, σu and σv can be computed as follows: ( a= σu =

Uh Ul

)

1 S−1

[

π (a − 1)Uh √ and σv = tan( − σu2 ) 2 2K 2 ln (a + 1) 2 ln 2 Uh2

]

(4.5)

where Ul and Uh denote the lower and upper center frequencies of interest, respectively. The Gabor feature space consists of responses calculated with a multi-resolution Gabor filter at several different scales (frequencies) and orientations. The response of a Gabor filter to an image is obtained by a 2-D convolution. Convolution of an image with the kernel gives a response that is proportional to how well the local features in the image match the kernel. Let I(x, y) denotes an image and GIm,n (x, y) denotes the response of a Gabor filter at the mth scale and nth orientation to an image point (x, y). The Gabor filter response to an image is defined in Eq. (4.6). GIm,n (x, y) =

∑∑ x1

I(x1 , y1 )gm,n (x − x1 , y − y1 )

(4.6)

y1

The Gabor filtered image has both real and imaginary components, as the response of the Gabor filter is complex. The magnitude of the Gabor filtered image is calculated by using Eq. (4.7). |GIm,n (x, y)| =

√

ReGIm,n (x, y)2 + ImGIm,n (x, y)2

(4.7)

where ReGIm,n (x, y) and ImGIm,n (x, y) are the real and imaginary parts of the Gabor filtered image, respectively. In this work, multi-resolution Gabor filter is used to extract features from iris images. 114

4.2. Preprocessing

4.2

Preprocessing

Iris localization and normalization are used as preprocessing tasks for a captured eye image. First, in localization, the iris part is isolated from an eye image. To localize the iris part, the eye image is preprocessed by applying downscaling and color level transform. The downscaling of the eye image is done to reduce the search area for pupil and iris boundaries. The original eye image is with the resolution 320×280 and the eye image is downscaled to the resolution 160×140. Figure 4.2(a) shows a sample eye image. The color level transform is applied to minimize the influence of irrelevant edges. In other words, the color level transform increases the intensity differences between the pupil and iris parts, and iris and sclera parts which in turn helps to detect the pupil and iris boundaries efficiently and accurately [12, 13]. The next step is to find the pupil boundary. To detect the pupil boundary, the color level transformed image is converted to a binary image and all connected components in the binary image are found. Then, the small irrelevant components which may occur due to eyelashes, eyelids and noise are removed. Pupil component is selected by counting the number of pixels within the calculated average radius. The pupil boundary is found from the pupil component by edge detection and edge connection. The pupil centroid is determined by calculating the centroid of all pixels within the pupil boundary. Figure 4.2(b) shows an eye image with detected pupil. Iris boundary detection is the next step in the iris localization. To detect the iris boundary, the eye image is divided into left and right images at the pupil centroid. Color level transform is applied on both left and right images to enhance the contrast between iris and sclera boundaries. The image preprocessing technique, namely dilation [17], is followed to color level transformed image to reduce the noise effect in edge detection. The dilated image is then thresholded to create a binary image and to detect the vertical edges which mainly occur 115

4. Iris Biometric Data Indexing

(a) Original eye image

(d) Localized iris image

(b) Pupil boundary detected eye image

(e) Normalized iris image

(c) Iris boundary detected eye image

(f ) Enhanced iris image

Figure 4.2: Preprocessing result of a sample iris image [12, 14].

due to iris boundary. Irrelevant edges which occur due to noise and eyelashes are removed after the edge detection. Iris boundary and eyelid boundary are detected by checking the pixel connectivity and drawing the small lines in particular directions. Figure 4.2(c) shows detected iris boundary in the eye image. Finally, resizing of pupil and iris boundary information is done. The detected iris part is shown in Fig. 4.2(d). Detailed of the pupil and iris boundary detection have been reported in [12, 13]. After completion of iris localization, the iris part is wrapped into a fixed size rectangular block to make the iris sample scale and translation invariant. This process is called iris normalization. A detailed about iris image normalization technique can be found in [14]. The wrapping process is done by transforming the iris texture from Cartesian to polar coordinate. The Daugman’s homogeneous rubber sheet 116

4.3. Feature Extraction model [11] is applied to normalize the iris texture. The normalized iris image is shown in Fig. 4.2(e). Then the normalized iris is enhanced to make the iris texture illumination invariant by applying Ma et al. [21] technique. This enhanced image is used in the feature extraction. An example of an enhanced normalized iris image is shown in Fig. 4.2(f).

4.3

Feature Extraction

Gabor energy features are extracted in different scales and orientations from a normalized iris image as stated in this section. It is known that the response of a Gabor filter to an image is obtained by a 2-D convolution operation. Let I(x, y) be the normalized iris image and GIm,n (x, y) denotes the response of Gabor filter in the mth scale and nth orientation to an image at point (x, y) on the image plane. The Gabor filtered image can then be obtained using Eq. (4.6) as stated in Section 4.1. This approach uses the multi-resolution Gabor filter to extract the iris texture features. For this purpose, the Gabor filter is applied in each scale and orientation, and the responses are computed at each pixel of the image. These responses are called Gabor coefficient values which are complex. The magnitude values are calculated at each pixel of the image. The Gabor energy is estimated by summing up the square values of the magnitude of Gabor responses at each pixel as expressed in Eq. (4.8). GEm,n =

∑

[|GIm,n (x, y)|]2

(4.8)

x,y

It is observed that the range of values of Gabor energy features is very high. It is also examined that intra-feature range distribution is higher for lower values of m and n. A non-linear function such as log is used to make the intra-feature range distribution similar. It effectively 117

4. Iris Biometric Data Indexing maps the high range of Gabor energy features’ values to a lower range of values. Thus, for a given iris image the Gabor energy features can be represented in the form of a matrix as shown in Eq. (4.9).    GF =   

log(GE0,0 )

log(GE0,1 )

...

log(GE0,K−1 )

log(GE1,0 ) .. .

log(GE1,1 ) .. .

...

log(GE1,K−1 ) .. .

...

log(GES−1,0 ) log(GES−1,1 ) . . . log(GES−1,K−1 )

      S×K

(4.9) The rows and columns denote the number of scales (S) and number of orientations (K) of the multi-resolution Gabor filter. It may be noted that the feature vector in our approach consists of S ×K number of features.

4.4

Index Key Generation

The extracted Gabor energy features in different scales and orientations are used to generate a key for iris database indexing. In this approach, a key represents a feature vector which consists of S × K number of features’ values. The ith value of an index key is denoted as f (i) in Eq. (4.10). f (i) = GFm,n

(4.10)

where GFm,n represents the logarithm value of the Gabor energy feature in the mth scale and nth orientation. In Eq. (4.10), m = 0, 1, . . . , S − 1 and n = 0, 1, . . . , K − 1, and i = n + (m × K) + 1. In other words, the ith feature of an index key represents the logarithmic value of a Gabor energy feature in a particular scale and orientation. Note that the length of the index key is S × K which is denoted as F L. Suppose, there are P number of subjects and for each subject there are Q number of samples. The index key for the q th sample of the pth subject is denoted by Eq. (4.11) where fp,q (i) represents the 118

4.5. Storing ith feature of the q th sample of the pth subject, q = 1, 2, . . . , Q and p = 1, 2, . . . , P . indxp,q =< fp,q (1) fp,q (2) · · · fp,q (i) · · · fp,q (F L) >

4.5

(4.11)

Storing

To store the iris data, first, an index space is created in the database, and then the iris data are stored into that index space of the database. The descriptions of creating index space and storing mechanism are discussed in the following sub sections.

4.5.1

Index Space Creation

For a given iris image, index keys are created as mentioned in Section 4.4. An index organization is proposed to store any index key for any sample. The index organization is shown in Fig. 4.3. The organization consists of F L number of tables. Each table is corresponding to each feature of the index key. The length of a table for a feature depends on the minimum and maximum values for all samples of all subjects. Thus, the length of the table for the ith feature is calculated using Eq. (4.12). T Li = Fmaxi − Fmini + 1

(4.12)

where Fmaxi and Fmini are the maximum and minimum values for all samples of all subjects of the ith feature, and i varies from 1 to F L. The Fmaxi and Fmini are then calculated using Eq. (4.13). The index value of the table corresponding to the ith feature is started with 1 and end with T Li . Fmaxi = max(fp,q (i))

∀ q ∈ Q and ∀ p ∈ P

Fmini = min(fp,q (i))

∀ q ∈ Q and ∀ p ∈ P 119

(4.13)

4. Iris Biometric Data Indexing Table1

Tablei

MDL 1

TableFL

MDL

MDL

1

1

IDL MDL 2

IDL MDL

IDL MDL

2

2

IDL

!

IDL

!

!

MDL TL1

IDL

IDL

!

!

MDL TLi

IDL

! MDL

TLFL

IDL

Figure 4.3: Index space organization.

Each entry of a table contains two lists (see Fig. 4.3). The list M DL contains a set of values which are calculated as the median values of all samples of a subject for a feature representing the table. The list IDL contains a set of identities which uniquely identify an iris sample.

4.5.2

Storing Iris Data

To store the index keys in the database, the values of a feature corresponding to all given samples of a subject are sorted. Then the center value of the feature values for that subject is calculated. Note that median or mean value can give the center value. The median value gives the central position which minimizes the average of the absolute deviations. On the other hand, the mean value gives the center position biasing toward the extreme value. For example, if any feature value is very low or very high for a particular sample, then the mean value is close to that feature value of the sample. Hence, the median value is used in this approach. For any ith feature, the median value of a given Q number of samples for a subject say p is calculated using Eq. (4.14). p

M edi = M edian(fp,q (i)) 120

∀q ∈ Q

(4.14)

4.5. Storing Note that if there are P number of subjects and each subject has Q number of samples, then N = P × Q number of median values and the identities are stored in each table. Let the unique identifier for the q th sample of the pth subject is denoted as Idpq and the ith feature value is denoted as fp,q (i). The median value and Idpq corresponding to the fp,q (i) are stored at the ti th position in the ith table. Given the value of fp,q (i), the value of index say ti , in the ith table is calculated using Eq. (4.15). ti = fp,q (i) − Fmini + 1

(4.15)

To enroll an additional samples of a subject which is already enrolled in the database, the median value of all enrolled samples with the additional samples for that subject is recalculated. Then, the all samples of the subject are enrolled into the database. If a new subject comes to enroll, first, the median value of each feature for all samples of the new subject is calculated. Then, the minimum and maximum values of each feature are checked to confirm whether the values are within the range of the table or not. If feature values lie within the range of the table then the identity and the median value of the new subject are enrolled. If the values do not fall within the range, then the size of table is increased with the new minimum and maximum values and the enrolled samples are reorganized. Finally, all samples of the new subject are enrolled. Illustration: The approaches to store and index iris biometric data are illustrated with an example. Suppose, there are data of 10 subjects with 5 sample each. Thus, the total number of samples to be stored and indexed is 50. Further, 3 scales and 4 orientations are considered for Gabor feature extraction. Hence, the length of the index key is 12. That is, 12 tables are needed in the database to store the 12 features for each index key. The index keys for 50 samples are created as discussed in Section 4.4. Figure 4.4 shows the index keys of 5 121

4. Iris Biometric Data Indexing f (1)

f (2)

f (3)

f (4)

f (5)

f (6)

f (7)

f (8)

f (9)

f (10)

f (11)

f (12)

id

indx16 indx 62

316 318

296 297

285 283

273 272

258 255

247 245

231 229

213 214

205 199

187 188

166 169

157 155

Id16 Id 62

indx 36

317

296

285

275

257

246

232

211

202

185

170

155

Id 36

6 4 6 5

318

297

281

273

254

245

231

211

202

184

169

156

Id 64

319

298

280

274

255

246

233

210

203

185

169

154

Id 56

indx indx

Figure 4.4: Index keys of the 6th subject with 5 samples.

samples and all data pertaining to say, sixth subject for an instance. Let assume that the minimum and maximum values of the first feature analyzing all fifty samples are found to be 100 and 350, respectively. Thus, the length of the first table in the database is 251 (Fmax1 − Fmin1 + 1). Similarly, the length of the other tables in the database can be calculated from the maximum and minimum values of features. Now, the first sample of the sixth subject needs to be enrolled into the database. Among all the index keys of the sixth subject, the first sample is highlighted as shown in Fig. 4.4. Now, the median value for each feature of the sixth subject is to be found. The values of the first feature of the sixth subject are shown as a rectangular block in Fig 4.4. From Fig. 4.4, it can be seen that the median value of the first feature of the sixth subject is 318. The first feature value of the first sample of the sixth subject is 316. The median value (318) of the sixth subject is stored in the M DL and the identity of the first sample of the sixth subject (Id61 ) in the IDL at the 217th (316 - 100 + 1 ) location of the first table of the database, respectively. Other features of the first sample of the sixth subject can be stored likewise. The rest of the samples are stored in the same way. Algorithm 4.1 formally states the techniques of creating tables in the database. The different steps of enrollment of all samples for all subjects are summarized in Algorithm 4.2. The following notations are used in the algorithms.

122

4.5. Storing Notations used in our algorithms P = Number of subjects Q = Number of samples of each subject F L = Number of features in an index key fp,q (i) is the ith feature of the q th sample of the pth subject Idpq is the identity of q th sample of the pth subject M DL is the list of median values IDL is the list of iris sample identities Fmini is the minimum value of the ith Fmaxi is the maximum value of the ith T ablei is the table for the ith feature Si is the set for retrieved iris template corresponding to the ith feature CSET is the candidate set

Algorithm 4.1 Creating tables for indexing Input: All index keys for all person’s Output: Tables in index space 1: for i = 1 to F L do 2: for all p = 1, 2, . . . , P do 3: for all q = 1, 2, . . . , Q do 4: Fmaxi = max(fp,q (i)) // Finding the maximum of all samples of all subjects 5:

Fmini = min(fp,q (i))

6:

end for end for T Li = Fmaxi − Fmini + 1

// Finding the minimum of all samples of all subjects

7: 8:

// Calculate the length of the table for

Create T ablei of length T Li 10: end for 9:

each feature

// Create the table for each feature

123

4. Iris Biometric Data Indexing Algorithm 4.2 Enrollment all samples for all subjects into the database Input: All index keys for all subjects with identities Output: Tables in index space with enrolled samples 1: for p = 1 to P do 2: for i = 1 to F L do pF 3: ∀q∈Q medi = M edian(fp,q (i)) 4: end for // Store feature vector into database 5: for all q = 1, 2, . . . , Q do 6: for i = 1 to F L do 7: ti = fp,q (i) − Fmini + 1 // Calculate the index for feature 8: 9: 10: 11: 12:

Add p Fmedi to T ablei [ti ].M DL Add Idpq to T ablei [ri ].IDL end for end for end for

4.6

value fp,q (i)

Retrieving

Once all subjects are enrolled into the database, it can be to retrieve data for a match. As a task of retrieving, the iris templates are to be found from the database which are the most similar to the query iris template. To do this, first, the index key of length F L from the query iris is generate as discussed in Section 4.4. The index of the query iris is represented in Eq. (4.16). indxt =< ft (1) ft (2) · · · ft (3) · · · ft (F L) >

(4.16)

Note that, only a set of feature values is presented exists for the given query iris image. There is no median value. At the time of retrieval, a set of median values and sample identities corresponding to each feature value are retrieved. For the ith feature (with feature value say ft (i)) of the query index key, a particular position of the 124

4.6. Retrieving ith table in the database is accessed based on the value of ft (i). The position of the ith table is decided by Eq. (4.17). T abIndexi = ft (i) − Fmini + 1

(4.17)

A set of identities (IDL) is retrieved from T abIndexi location of the ith table. Let this set be Si . The minimum and maximum median values (M edmax and M edmin ) are also found from the list M DL at T abIndexi location of the ith table. Next, the IDL sets corresponding to the values M edmin − δ to M edmax + δ are retrieved from T ablei and add these identities to Si . Here, δ is a threshold value which would be decided empirically (see Section 4.7.4.2). In this way, the candidate identities are retrieved for the ith feature and stored in a temporary list Si . Similarly, the identities for other features are retrieved and stored in the respective temporary lists. In next step, the most common identities among all sets Si ’s are found and rank of each identity is computed. First, all Si ’s are merged ∑ to S, that is, S = Si . To count the number of occurrences of each subject appeared in S, a candidate set (CSET ) is maintained which contains to fields, namely id and vote. The id filed stores the unique subjects’ identities in S and vote filed stores the number of occurrences of that identities in S. The vote corresponding to an identity is increased when it occurs in the set S. Then, the CSET is sorted in descending order based on the values of vote field. The rank one is assigned to the subject correspond to the highest value of the vote, rank two to the next highest value of the vote and so on. It means that the maximum occurrence is assigned as rank one and other ranks are given in descending order of number of occurrences. The highest rank indicates that the retrieved identities is most similar to the query image. Illustration: Let consider that the index keys of 10 subjects and 5 samples for each subject are stored in the database. Thus, there are 50 125

4. Iris Biometric Data Indexing index keys in the said database. Let assume the minimum and maximum values of the 1st feature (with respect to these 50 index keys) are 100 and 350, respectively. The length of the 1st table in the database is 251 (Fmax1 −Fmin1 +1). Figure 4.5(a) shows the 1st table for the 1st feature. Now consider the index key indxt which is generated from the query iris. In this example, 3 scales and 4 orientations are considered for Gabor feature extraction as in the storing time. Hence, the length of the index key is 12. The index key generated from the query iris template of length 12 is shown in Fig. 4.5(b). Here, the 1st feature value (ft (1)) of the query iris template is 317 (see Fig. 4.5(b)). Knowing this, all median values are retrieved from the M DL at location 218 (317 − 100 + 1) of the 1st table. It can be shown that the median values in the M DL at the 218th location of T able1 ’s table are 315 and 318 (see Fig. 4.5(a)). The minimum and maximum median values in the M DL are 315 and 318, respectively. All identities in the IDL at location 218 (317 − 100 + 1) are also added to S1 . Next, the identities in the IDL are added from a range of locations based on the minimum and maximum median values and preassigned threshold value δ. Let the value of δ is chosen as 1. Therefore, the range of locations is 215 (315 − 100 − 1 + 1) to 220 (318 − 100 + 1 + 1). All identities in IDL at locations 215 to 220 are added into S1 . The set S1 is shown in Fig. 4.5(c). Similarly, sets for the rest of the features can be created. The identities’ position in the other tables are not shown due to clarity of the figure and the limitation of space in the paper. For continuity of the illustration, assume the other eleven sets of retrieved identities, as shown in Fig. 4.5(d). All retrieved sets are merged to S. Finally, the number of occurrences of particular Idp in S is calculated and the rank is assigned. To do this, each unique subjects of S is inserted into the CSET . The vote corresponding to a subject is increased when it occurs in the set S. Figure 4.5(e) shows the CSET which consists the 126

4.7. Performance Evaluation Table1

f1 f 2 f 3 f 4 f 5 f 6 f 7 f8 f 9 f10 f11 f12 indx ' 317 295 284 274 257 245 232 212 202 184 169 154

MDL

1

IDL

(b) Index key vector for query iris template

215 216 217 218 219 220 221

MDL

{311}

S1 ={Id12 , Id13 , Id 24 , Id 52 , Id 33 , Id16 , Id 62 , Id 64 , Id56 }

IDL

{Id 52 }

(c) Set S1 for the 1st feature

MDL

{311}

IDL

{Id 24 }

S2 ={Id13 , Id15 , Id 34 , Id 54 , Id16 , Id 36 , Id 37 }

MDL

{315, 318}

IDL

{Id13 ,Id16 }

S3 ={Id14 , Id 34 , Id 44 , Id16 , Id 62 , Id 36 , Id 72 , Id 74 }

MDL

{315, 318}

IDL

{Id 33 , Id 36 }

MDL

{318, 319}

S4 ={Id12 , Id 33 , Id 34 , Id 53 , Id16 , Id 36 , Id 64 , Id 56 , Id 74 } S5 ={Id 52 , Id 34 , Id 44 , Id 62 , Id 36 , Id 64 , Id 56 , Id84 }

IDL

{Id12 , Id 62 , Id 64 }

S6 ={Id11 , Id13 , Id14 , Id 52 , Id 34 , Id16 , Id 62 , Id 36 , Id 64 , Id 56 , Id18 , Id 85 }

MDL

{318}

IDL

{Id 56 }

S7 ={Id12 , Id13 , Id15 , Id 22 , Id 42 , Id16 , Id 36 , Id 64 , Id 74 }

MDL

{319, 323}

IDL

1 3

5 1

{Id ,Id }

S8 ={Id13 , Id 52 , Id 33 , Id 36 , Id 64 , Id82 } S9 ={Id 32 , Id 52 , Id 36 , Id 64 , Id 56 } S10 ={Id 32 , Id 33 , Id 53 , Id 36 , Id 64 , Id 56 , Id18 , Id83} S11 ={Id15 , Id 62 , Id 36 , Id 64 , Id 56 , Id83 , Id85 }

MDL

251

IDL

S12 ={Id 33 , Id 53 , Id 62 , Id 36 , Id 64 , Id 56 , Id 74 , Id82 , Id84 } st

(a) Index table of the database for the 1 feature

(d) Sets for other eleven features

id

vote

id

rank

Id1 Id2 Id3 Id4 Id6 Id7 Id8

14 9 12 5 41 6 10

Id6 Id1 Id3 Id8 Id2 Id7 Id4

1 2 3 4 5 6 7

(e) Vote for each subject present in CSET

(f) Subjects with their ranks

Figure 4.5: Retrieving a match for a query iris image from a database.

vote values for each subject identity. Figure 4.5(f) shows the subjects with corresponding rank and the sixth subject being the rank 1 will be retrieved as the best match. The steps for retrieving iris template from a database are summarized in Algorithm 4.3.

4.7

Performance Evaluation

To study the efficacy of the proposed iris biometric data indexing approach, a number of experiments has been conducted. This section presents the experiments carried out and the experimental results observed. 127

4. Iris Biometric Data Indexing Algorithm 4.3 Retrieving iris template from database Input: Index key of query iris (indxt ) Output: Retrieved identity set with their ranks 1: for i = 1 to F L do 2: T abIndexi = ft (i) − Fmini + 1 // Calculate the table location 3: Si = T ablei [T abIndexi ].IDL 4: M edmaxi = max(T ablei [T abIndexi ].M DL) − Fmini + 1 5: M edmini = min(T ablei [T abIndexi ].M DL) − Fmini + 1 6: for k = M edmini − δ to M edmaxi + δ do // δ = offset ∪ value 7: Si = Si T ablei [T abIndexi ].IDL // Add iris identities to Si 8: end for 9: S = S + Si // Merge all set Si to S 10: end for 11: Add unique subject identity in S into the id field of CSET 12: Initialize all vote of the CSET to zero // Calculate the number of occurrence of each subject p 13: for Each Idq in S do 14: CSET [Idp ] → vote = CSET [Idp ] → vote + 1 15: end for 16: Sort all CSET in descending order based on the values of vote 17: Assign rank based on the value of vote

4.7.1

Performance Metrics

Accuracy and efficiency are the two main criteria usually considered to measure the performance of biometric indexing techniques. The accuracy of biometric indexing approach is commonly evaluated by the Hit Rate (HR) and Penetration Rate (P R) [27, 18, 6, 7]. The HR is the percentage of probes for which the correct identity is retrieved for a gallery by the indexing mechanism [18]. The P R is the percentage of the database retrieved for a query to get a correct match [18]. Let Ng be the number of entries in the database and Np be the number of query in the probe set. For a query sample if Ni is the number of entries retrieved for ith probe then the P R is defined as in (4.18). 128

4.7. Performance Evaluation If Nc (Nc < Np ) is the number of query sample for which successful matches are found within the top r retrieved candidates then the HR at rank r is defined as in Eq. (4.19). Np 1 ∑ Ni PR = Np Ng

(4.18)

i=1

HR =

Nc Np

(4.19)

Another metric called Cumulative Match Score (CM S) is also used to measure the performance of a biometric indexing technique. The CM S gives the probability of at least one correct identity present within a top rank which also represents the cumulative HR at different ranks. In other words, the HR at different ranks is represented with Cumulative Match Characteristics (CMC) curve. False Positive Identification Rate (F P IR) and False Negative Identification Rate (F N IR) is also used to check the accuracy of an identification system. To calculate F N IR and F P IR of an identification system [22] without indexing and with indexing Eq. (4.20) and Eq. (4.21) are followed, respectively. F N IR = F N M R F P IR = 1 − (1 − F M R)N F N IR = ER + (1 − ER) × F N M R F P IR = 1 − (1 − F M R)N ×P R

(4.20)

(4.21)

Here, F M R and F N M R are the False Match Rate and False Non Match Rate of a system, respectively. The F M R and F N M R are calculated from the genuine and imposter score distribution. In Eq. (4.20) and (4.21), N is the number of samples in the database, P R is the penetration rate of the indexing system and ER is the indexing error. The ER is calculated using Eq. (4.22) where HR represents the hit 129

4. Iris Biometric Data Indexing rate of the indexing technique. ER =

4.7.2

100 − HR 100

(4.22)

Databases

In this experiment, five different iris image databases are used: Bath University Iris Database (BATH) [1], CASIA-IrisV3-Interval Iris Database (CASIAV3I) [2], CASIA-IrisV4-Thousand Iris Database (CASIAV4T) [3], Multimedia University Iris Database Version-2 (MMU2) [4] and West Virginia University Iris Database Release 1 (WVU) [5]. A detailed description of each database is given in the following. Bath University Iris Database (BATH): The BATH database [1] contains 1000 eye images of 25 persons. Each person has 20 images of left eyes and 20 images of right eyes. These images are high resolution (1280×960 pixels), and have been compressed with JPEG-2000 to 0.5 bits/pixel. The database is created in conjunction with the University of Bath and Smart Sensors Limited. It may be noted that the iris data of left and right eyes of a person are different [9], therefore, 50 unique subjects are considered in BATH database. CASIA-IrisV3-Interval Iris Database (CASIAV3I): CASIAV3I database [2] is collected by the Institute of Automation at the Chinese Academy of Sciences (CASIA). This database contains 2639 eye images of 249 persons. The eye images in the database are captured from 395 eyes. The iris images are captured with the close-up iris camera developed by Center for Biometrics and Security Research (CBSR). The most of the images are taken in two sessions, with at least one month interval in indoor environment. The resolution of image is 640×480 pixels. From the CASIAV3I database 395 unique subjects are considered as the two eyes are different for a person. 130

4.7. Performance Evaluation CASIA-IrisV4-Thousand Iris Database (CASIAV4T):

This

database is also developed Institute of Automation at the Chinese Academy of Sciences [3]. The CASIAV4T database contains 200000 eye images of 1000 persons and each person has 10 left and 10 right eye images. All iris images are 640×480 pixels resolution and collected under near infrared illumination using IKEMB-100 camera. A total of 2000 unique subjects are considered for the CASIAV4T database. Multimedia University Iris Database Version-2 (MMU2): The MMU2 database is collected at Multimedia University [4]. It contains 995 eye images from 100 persons. Five images are captured from each eye of a person. There are 5 left eye images which are excluded from the database due to cataract disease. The images are captured using Panasonic BM-ET100US camera and with 320×238 resolution. The total number of unique subjects considered for this database is 199. West Virginia University Iris Database (WVU): This database is created by Center for Identification Technology Research, West Virginia University [5]. The database contains 3099 eye images from 244 persons. Left and right eye images are captured from 241 and 236 persons, respectively. In the WVU database, one to twenty samples are captured for each eye. The images are captured using OKI IRISPASSh handheld device with 480×640 pixels resolution. Total 477 unique subjects are considered from WVU database in our experiment. A summary of all the databases is given in Table 4.1.

4.7.3

Evaluation Setup

To evaluate the performance of the proposed approach, all samples of all virtual users are divided into two sets: Gallery and Probe. The samples in the Gallery are enrolled into the index database and samples in the Probe are used as query to search the index database. The 131

4. Iris Biometric Data Indexing Table 4.1: Characteristics of databases used in our experiments Size

# Persons

# Samples

# Unique subjects

BATH

1000

25

20

50

CASIAV3I

2639

249

2 to 10

395

CASIAV4T

20000

1000

10

2000

MMU2

995

100

5

199

WVU

3099

244

1 to 20

477

Database

80% samples of each subject are used to create the Gallery and other 20% to create the Probe. The samples of the Gallery Set are selected randomly from each subject. The experiments are done with an Intel Core2Duo processor (2.00 GHz) and 2.0-GB memory. GNU Compiler Collection (GCC) 4.3 compiler is used to develop compile the program.

4.7.4

Validation of the Parameter Values

The experiments are involved with different parameters and resources. Eventually, the reported experimental results are subjected to the validity of the above resources and assumptions on values of parameters. In the following, the values of different parameters are validated. The validity [20] is assessed on the basis of experimental evaluation of the variables. In the experiment, three parameters namely number of scales (S), number of orientations (K) and δ are considered. First, the assumed values of number of scales and number of orientations of Gabor filter are validated. Next, the value of δ is validated. 4.7.4.1

Values of S and K

In the experiment, the number of scales (S) and orientations (K) for feature extraction are decided so that it should give better results. It may be noted that if the number of scales and number of orien132

4.7. Performance Evaluation tations increase, then the number of features in index key increases. On the other hand, the memory requirement to store the index keys also increases when the number of features in index key increases. Further, HR increases as the number of features increases. Hence, there is a trade-off between the number of scales and orientations (and hence memory overhead) with the HR. The experimentation is done with different scales and orientations, and the result on the CASIAV3I database is shown in Fig. 4.6. In this experiment, 10 different scales and 10 different orientations are considered. Figure 4.6(a) and (b) show the HR and P R for different combination of scales and orientations, respectively. It is observed that a higher number of scales and

100 90 80 HR

70 60 50 40 30 10 8

10 6

8 k

6

4

4

2

2 0

s

0

(a) HR with different scales and orientations

70 60 50

PR

40 30 20 10 0 10 8

10 6

8 6

4 k

4

2

2 0

s

0

(b) P R with different scales and orientations

Figure 4.6: HR and P R with different scales and orientations.

133

4. Iris Biometric Data Indexing orientations gives the good HR but this also gives high P R which is not desirable. On the other hand, small values of number of scales and orientations give good P R but it give poor HR which is also not desirable. From Fig. 4.6, it is observed that the HR does not change significantly when the number of scales and orientations are increased beyond 3 and 4, respectively. Again, from Eq. (4.23), it is observed that the memory requirement to store the indexed features is directly proportional to the number of features used in indexing. In other words, with large values of number of scales and orientations memory requirement increases greatly without much improvement in HR. With this observation the value of scale is chosen as 3 and orientation is chosen as 4. In fact, this combination gives a better HR and P R as shown in Fig. 4.6. 4.7.4.2

Value of δ

To decide the value of δ, HR and P R are calculated for the different values of δ. Figure 4.7 shows the result for different values of δ. From Fig. 4.7(a), it is observed that the HR does not increase significantly beyond the value of δ = 5. On the other hand, from Fig. 4.7(b), it is observed that P R is changing rapidly after δ = 5. Considering this trend it is appropriate to set the value of δ as 5.

4.7.5

Evaluation

The efficiency of the iris indexing technique is judged with respect to accuracy, searching time and memory requirement. The experimentations are done with the gallery and probe sets created in Section 4.7.3. 4.7.5.1

Accuracy

In this experiment, the best match for each sample in the Probe set at the first rank is analyzed. The rank one HR, P R and retrieving time are calculated to measure the accuracy. The results are summarized 134

4.7. Performance Evaluation 100

HR

90

80

70

60 0

2

4

6

8

10

8

10

(a) HR for different values of δ 40

PR

30 20 10 0 0

2

4

6

(b) P R for different values of δ

Figure 4.7: HR and P R for different values of δ.

in Table 4.2. From Table 4.2, it is observed that the HR for WVU is considerably less because the quality of the iris images in WVU database are poor than the other databases. It may be noted that the average P R of BATH, CASIAV3I|indexCASIAV3I, CASIAV4T, MMU2 and WVU iris databases are 11.6%, 15.3%, 16.9%, 14.3% and 10.8%, respectively. From Table 4.2, it is observed that the retrieving time in CASIAV4T is slightly higher than the other databases though the size of database is significantly larger than others. The result is also substantiated in terms of CM S which gives the probability of at least one correct identity is present within a top rank. How CM S varies with rank is shown in Fig. 4.8 as CMC curve. It is obvious that CM S increases with the increase of rank. From Fig. 4.8, it is observed that 99.4%, 98.8%, 98,7, 98.9% and 98.8% CM Ss are pos135

4. Iris Biometric Data Indexing Table 4.2: HR and P R for different iris image databases HR

PR

Retrieving time (ms)

BATH

82.38

11.6

0.013

CASIAV3I

82.38



15.3

0.016

CASIAV4T

84.64



16.9

0.018

MMU2

83.92

14.3

0.011

WVU

80.64

10.8

0.016

Database



 

100 95 BATH

90 CMS



CASIAV3I 85

CASIAV4T MMU2

80

WVU

75 1

2

3

4

5

10 15 Rank

20

30

40

50

60 

Figure 4.8: CM C curves with reference to different databases.

sible within top 60 ranks for BATH, CASIAV3I, CASIAV4T, MMU2 and WVU databases, respectively. In the existing literature on iris indexing, the best reported result [24] so far is 98.56% CM S within top 100 rank for CASIAV3I iris database. In other words, if top 100 rank is considered in this approach then the CM S will eventually increase. Further, the performance of this method is analyzed with respect to F P IR and F N IR on CASIAV3I database. To do this, first F M R and F N M R are calculated as follows. Each query template of the P robe set is matched with each template in the Gallery set using 136



4.7. Performance Evaluation



16 14 12

Without Indexing With Indexing

FNIR

10 8 6 4 2 0 0

20

40

60

80

100

FPIR

Figure 4.9: F P IR vs. F N IR CASIA-V3-Interval database.

 curves



with and without indexing for

Daugman’s [11] iris recognition method. In this experiment, 3,000 genuine pairs and 1,242,608 imposter pairs are considered from the Gallery and P robe of CASIAV3I database. Then, the genuine score and imposter score are calculated using Daugman’s method [11] for each genuine and imposter pairs, respectively. Finally, F N IR and F P IR are calculated for an identification system [22] without indexing and with indexing using Eq. (4.20) and (4.21), respectively. The tradeoff between F P IR and F N IR for the identification system without indexing and with indexing is shown in Fig. 4.9 for CASIAV3I iris database. From the experimental results, it may be interpreted that 0.72% F P IR and 3.95% F N IR can be achieved with indexing and 3.93% F P IR and 3.83% F N IR without indexing. From Fig. 4.9, it is observed that using this indexing approach low F P IR for an F N IR can be achieved. 137

4. Iris Biometric Data Indexing 4.7.5.2

Searching Time

The run-time complexity is analyzed with big-O notation for gallery match corresponding to a query sample. Let N be the total number of samples enrolled in the database (from P number of subjects), and each subject has Q number of samples. With reference to the Algorithm 4.3, given a query template, the identities of subjects (IDL) and median values (M DL) stored at T abIndex position are retrieved from the ith (i = 1, 2, . . . , 12) table (T ablei ). T abIndex is calculated using Eq. (4.17). The time complexity of this process is O(1). Next, the IDL sets are retrieved corresponding to the values min(M DL) − δ to max(M DL) + δ from T ablei and added these names to Si for rank calculation. This process can be accomplished in O(1) time. Let IL be the average size of the IDL list. The rank calculation process can be done in O(IL) time. In the worst case, when all samples are stored in one position then IL is equal to N , which is very unlikely to occur. The search efficiency is analyzed by measuring the average time taken to retrieve iris templates from the database for a given query iris. Let tp be the average time to perform addition, subtraction and assignment operations. This indexing approach requires 6 comparisons to retrieve candidates corresponding to one feature and a candidate set of size IL is retrieved using 12 features (see Algorithm 4.3). Therefore, the time taken (Tr ) to retrieve a candidate set of size IL is (tp ×6)×12. Let td be the time to compare query iris template with one stored iris template for matching. Hence, the search time using this indexing approach (Ts ) is Tr +td ×IL. On the other hand, linear search requires td × N time. Thus, this indexing approach takes less time than the linear search approach because IL M (5.1) otherwise

In Eq. (5.1), M and V are the estimated mean and variance of I(x, y), respectively, and M0 and V0 are the desired mean and variance values, respectively.

5.1.2

Segmentation

In this step, the foreground fingerprint region which contains the ridges and valleys is separated from the image. The foreground region has a very high gray scale variance value [28]. Hence, a variance based thresholding method [16] is used for segmentation. First, the normalized fingerprint image is divided into a number of non-overlapping blocks and the gray scale variance is calculated for each block. Next, the foreground blocks and background blocks are decided based on a global threshold value. The threshold value is fixed for all blocks. Then, the gray-scale variance value of each block is compared with the global threshold value. If the variance is less than the global threshold value, then the block is assigned as background region, otherwise, it is assigned as foreground region. Finally, the background region is removed from the image and the fingerprint region is identified which 151

5. Fingerprint Biometric Data Indexing is used in subsequent stages of preprocessing and feature extraction.

5.1.3

Local Orientation Estimation

The orientation field of a fingerprint image represents the local orientation of the ridges and valleys in a local neighborhood. To compute the orientation image, Hong et al. [16] least mean square estimation method is used. In this method, block-wise orientations of the segmented image are calculated. First, the whole image is divided into a number of blocks each of size 16 × 16 and the gradient ∂x and ∂y along x- and y- axis are calculated, respectively. The gradients are computed using Sobel operator [14]. Next, the local orientation θ of each block centered at pixel (x, y) is calculated using Eq. (5.2), (5.3) and (5.4) [16]. θ(x, y) =

1 tan−1 2

x+ w 2

Vx (x, y) =

∑

(

Vy (x, y) Vx (x, y)

) (5.2)

y+ w 2

∑

2∂x (u, v)∂y (u, v)

(5.3)

∂x2 (u, v)∂y2 (u, v)

(5.4)

u=x− w v=y− w 2 2 x+ w 2

Vy (x, y) =

∑

y+ w 2

u=x− w 2

v=y− w 2

∑

Finally, smoothing of the orientation field in a local neighborhood is performed to remove the noise present in the orientation image. For this purpose, the orientation image is converted into a continuous vector field and then Gaussian filter [16, 28] is applied on the orientation image for smoothing.

5.1.4

Local Frequency Image Representation

The frequency image represents the local ridge frequency which is defined as the frequency of the ridge and valley structures in a lo152

5.1. Preprocessing cal neighborhood along a direction normal to the local ridge orientation [16]. Local ridge frequency can be computed as the reciprocal of the number of pixels between two consecutive ridges in a region. Hong et al. [16] method is used to calculate the ridge frequency. In this method, the normalized fingerprint image is divided into a number of non-overlapping blocks. An oriented window is defined in the ridge coordinate system along a direction orthogonal to the local ridge orientation at the center pixel for each block. The average number of pixels between two consecutive ridges are calculated. Let T (x, y) be the average number of pixels of block centered at (x, y). Then, the frequency F (x, y) of the block (x, y) is calculated using Eq. (5.5) [16]. F (x, y) =

1 T (x, y)

(5.5)

If no consecutive peaks are there in the block, then the frequency of the block is assigned a value of −1 to differentiate it from the valid frequency values.

5.1.5

Ridge Filtering

A 2-D Gabor filter is used to remove efficiently the undesired noise and enhance the ridge and valley structures of the fingerprint image. The Gabor filter is employed because it has both frequency and orientation selective properties [28] which adjust the filter to give maximal response to ridges at a specific orientation and frequency in the fingerprint image. Therefore, the Gabor filter preserves the ridge and valley structures and also reduces noise. An even-symmetric Gabor filter tuned with local ridge frequency and orientation is used to filter the ridges in a fingerprint image. The orientation and frequency parameters of the even-symmetric Gabor filter are determined by the ridge orientation and frequency information of the fingerprint. The detailed method can be found in [16]. 153

5. Fingerprint Biometric Data Indexing

5.1.6

Binarization and Thinning

The binarization is used to improve the contrast between the ridges and valleys in a fingerprint image by converting a gray level image into a binary image [28]. The binary image is useful for the extraction of minutiae points. The gray level value of each pixel is checked in the enhanced image. If the value is greater than a predefined threshold value, then the pixel value is set to one, otherwise, it is set to zero. The binary image contains two levels of information, the foreground ridges and the background valleys. Finally, thinning morphological operation is performed prior to the minutiae extraction. Thinning is performed by successively eroding the foreground pixels until they are one pixel wide. A standard thinning algorithm [15] is used to perform thinning operation. This algorithm performs thinning using two sub iterations. In each subiteration, the neighborhood of each pixel is examined in the binary image based on a particular set of pixel-detection criteria, whether the pixel can be deleted or not. These processes continue until there are no more pixels for deletion. It may be noted that the application of the thinning algorithm to a fingerprint image preserves the connectivity of the ridge structures while forming a thinning image of the binary image. This thinned image is then used in the subsequent minutiae extraction process.

5.1.7

Minutiae Point Extraction

Two main minutiae features namely ridge ending and ridge bifurcation are considered in this work. The Crossing Number (CN) [25, 28] method is followed to extract these minutiae features. This method extracts the ridge endings and bifurcations from the thinned image by scanning the local neighborhood of each ridge pixel using a 3 × 3 window [25]. A 3 × 3 window centered at point P and its eight neighbor points Pi (i = 1, 2, . . . , 8) are shown in Fig. 5.3(a). The CN at 154

5.1. Preprocessing

P4

P3

P2

P5

P

P1

P6

P7

P8

(a) Pixel position in a 3 × 3 window

(b) CN=1 (ridge ending)

(c) CN=3 (bifurcation)

Figure 5.3: Pixel position and crossing number for two different types of minutiae.

point P is defined as half the sum of the differences between pairs of adjacent pixels in the eight-neighborhood (see Eq. (5.6)) [25, 28]. Note that value of each Pi (i = 1, 2, . . . , 8) is either 0 or 1, as the image is binarized (see Section 5.1.6). 1∑ |Pi − Pi+1 | with P9 = P1 2 8

CN =

(5.6)

i=1

Using the properties of the CN, the ridge ending and bifurcation are detected for CN = 1 and CN = 3, respectively. Figure 5.3(b) shows a ridge ending with CN = 1 and Fig. 5.3(c) shows a bifurcation minutiae point with CN = 3. ANSI/NIST-ITL format [24] is used to store the minutiae points. The origin of the fingerprint image is located at the bottom-left corner and y-coordinate is measured from the bottom to upward. The position and the angle of a minutiae feature are stored in (x, y, θ, T ) format where (x, y) is the location coordinate of a minutiae in the fingerprint image, θ is the minutiae angle and T is the type of minutiae. The value of T is R for ridge end type minutiae and B for bifurcation type minutiae. Minutiae angle (θ) is measured anti-clockwise with the horizontal x-axis. The angle of a ridge ending is calculated by measuring the angle between the horizontal x-axis and the line starting at 155

5. Fingerprint Biometric Data Indexing the ridge end point and running through the middle of the ridge as shown in Fig. 5.4(a). The angle of a bifurcation is calculated by measuring the angle between the horizontal x-axis and the line starting at the minutia point and running through the middle of the intervening valley between the bifurcating ridges as shown in Fig. 5.4(b). Extracted minutiae using the above mentioned method may have many false minutiae due to breaks in the ridge or valley structures. These artifacts are typically introduced by the noise at the time of taking fingerprint impression. A break in a ridge causes two false ridge endings to be detected, while a break in a valley causes two false bifurcations. In order to remove all these false minutiae, the shorter distance minutiae removal method of Chatterjee et al. [13] is followed. In this method, the distance between two ridge ending points or two bifurcation points, and orientations of that points are checked. If the distance is less than some minimum distance say δ, and the orientations of two points are opposite, then both minutiae points are removed. Here, the distance between two points is considered as Euclidean distance. Euclidean distance (ED) between two minutiae points A and B with positions (xA , yA ) and (xB , yB ) is defined as: ED(A, B) =

√

(xA − xB )2 + (yA − yB )2

y

y

(x, y)

(x, y)

x

x

(a) Ridge ending

(b) Bifurcation

Figure 5.4: Minutiae position and minutiae orientation.

156

(5.7)

5.2. Feature Extraction Let two minutiae points A and B are represented as (xA , yA , θA , TA ) and (xB , yB , θB , TB ), respectively. The Euclidean distance between two minutiae points is represented as ED(A, B). The following heuristic is applied to remove the false minutiae points. IF (TA = TB ) THEN IF (ED(A, B) < δ) AND (210 ≤ |θA − θB | ≤ 150) THEN REMOVE A and B from the list of minutiae ENDIF ENDIF In the above heuristic, the value of δ is chosen as the average distance between two ridges in the fingerprint image whose value is typically taken as 10. The difference of the orientations (|θA − θB |) is checked between 150 to 210 degree for the oppositeness of the minutiae points. Note that the exact value of |θA − θB | is 180 degree when the minutiae points are opposite. The tolerance value of ±30 degree is considered due to the curvature nature of the ridges.

5.2

Feature Extraction

The feature extraction step consists of two tasks: two closest point triangulation and triplet generation. Detailed descriptions of these tasks are given in the following sub-sections.

5.2.1

Two Closest Points Triangulation

To capture the structural information from the distribution of minutiae points, the concept of two closest point triangulation is proposed. A two closest point triangle is defined with three minutiae points such that for any point, the distance from other two points are minimum. In other words, let mi be a minutiae point. Other two points say mj 157

5. Fingerprint Biometric Data Indexing and mk form a two closest point triangle if the distances between mi and mj , and mi and mk are minimum compared to all other minutiae points in the fingerprint. Figure 5.5(a) illustrates the definition of the two closest point triangle with four minutiae points. It may be noted that the triangulation covers less geometric area of a fingerprint and implies a meaningful feature. A set of two closest point triangles can be obtained from a given fingerprint image. Figure 5.5(b) shows the two closest point triangles obtained from minutiae points of a sample fingerprint.

5.2.2

Triplet Generation

A two closest point triangle with three minutiae points represents a triplet. First, all such triplets are identified. Let M = {m1 , m2 , . . . , m|M | } be a set of |M | minutiae points in a fingerprint image. A set of minutiae triplets T = {t1 , t2 , . . . , t|T | } from M need to be identified. The number of triplets in the set T is |T |, where |T | = |M |. First, the distance matrix D is calculated as shown in Eq. (5.8). Each entry in D denotes the distance between two minutiae points. For example, di,j represents the distance between the ith and j th minutiae points. Here, di,j is calculated as the Euclidean distance using Eq. (5.7). 

d1,1

d1,2

...

d1,j

  d2,1 d2,2 . . . d2,j   .. .. .. ..  . . . . D=  d di,2 . . . di,j  i,1  .. .. .. ..  . . . .  d|M |,1 d|M |,2 . . . d|M |,j

...

d1,|M |

... .. .

d2,|M | .. .

... .. .

di,|M | .. .

           

(5.8)

. . . d|M |,|M |

The triplet corresponding to the ith minutiae mi in the triplet set T is represented with ti , where ti =< mi , mj , mk > and 1 ≤ i, j, k ≤ |M |. The minutiae points mj and mk of triplet ti are determined by the 158

5.2. Feature Extraction 250

ml 200

di,l 150

mi di,j

100

mj

di,k 50

mk

0 50

(a) Two closest point triangulation

100

150

200

250

(b) Two closest point triangle with sample minutiae points of a fingerprint image

m4 (100, 135)

t1 = (m1, m2, m3) t2 = (m2, m1, m5)

m1 (80, 120) m2 (90, 110)

t3 = (m3, m2, m1) t4 = (m4, m1, m2) t5 = (m5, m2, m3)

m3 (71, 100) m5 (95, 90)

(c) Duplicate two closest point triangles

(d) Duplicate triplets

Figure 5.5: Two closest point triangulation with sample fingerprint.

criteria defined in Eq. (5.9). mj = minIndex(D) ∃di,j ∈ D : 1 ≤ j ≤ |M | & i ̸= j mk = minIndex(D) ∃di,k ∈ D : 1 ≤ k ≤ |M | & i ̸= j & i ̸= k (5.9) In Eq. (5.9), minIndex function returns the minutiae point having the minimum distance from the ith minutiae. Each ti generated using the above mentioned criteria is added to the triplet set T. Note that there may be duplicate triplets in T because the minutiae points with the minimum distance may be same for two neighborhood points. Figure 5.5(c) and (d) show the closest point triangles and corresponding triplets generated from the five minutiae points of Fig. 5.5(c). Initially, 159

5. Fingerprint Biometric Data Indexing the number of triplets in T is 5. It can be shown that the triplets t1 and t3 generated from minutiae points m1 and m3 are same. Hence, one of these duplicate triplets (t1 or t3 ) is removed from T. After removing the triplet t3 , the set contains t1 , t2 , t4 and t5 . In this way, the triplet set T is generated, where |T | is the number of triplets generated from |M | minutiae points. It may be noted that |T | is at most equal to |M | for a fingerprint image.

5.3

Index Key Generation

A fingerprint image is represented with a triplet set T. A number of index keys from each triplet in T is generated for a fingerprint image. An index key is characterized with a 8-dimensional feature vector which consists of geometric information of the triplet, relative orientation information of each minutia point associated with the triplet and texture information of the local ridge structure around each minutiae point of the triplet. Let ti be the ith triplet in T where 1 ≤ i ≤ |T | and indxi be the index key generated from ti , which contains of eight feature values (f i (1), f i (2), f i (3), f i (4), f i (5), f i (6), f i (7) and f i (8)). The three minutiae points m1 , m2 and m3 of ti are the vertices of a triangle in a 2-D space (see Fig. 5.6). Let l1 , l2 and l3 be the lengths of the sides of the triangle such that l1 ≤ l2 ≤ l3 , and θ1 , θ2 and θ3 be the orientations of the minutiae points m1 , m2 and m3 , respectively for the triplet ti . The angle of the longest side of the triangle with respect to the horizontal axis is denoted by θL . Figure 5.6 shows the different components of the triangle corresponding to ti . The feature values (f i (1), f i (2), · · · , f i (8)) of the ith index key are computed as follows. f i (1): The feature f i (1) is defined as the ratio of the two smallest sides of triangle and calculated using Eq. (5.10). f i (1) = 160

l2 l1

(5.10)

5.3. Index Key Generation 1

m1 l3 L 2

m2 l2 l1

3

m3

Figure 5.6: Components of a minutiae triplet.

f i (2): The feature f i (2) of index key is defined as interior angle between the two smallest sides of triangle. Here, f i (2) is the angle between l1 and l2 as in Eq. (5.11). f i (2) = ∠m2 m3 m1

(5.11)

f i (3), f i (4) and f i (5): These features are the orientation of minutiae m1 , m2 , m3 with respect to the angle of the longest side of triangle θL . Here, f i (3), f i (4) and f i (5) are calculated using Eq. (5.12). f i (3) = |θL − θ1 | f i (4) = |θL − θ2 |

(5.12)

f i (5) = |θL − θ3 | f i (6), f i (7) and f i (8): These three features represent the Gabor energy features at the location of three minutiae (m1 , m2 and m3 ) of the triplet, respectively. A 2-D Gabor filter is used on 20 × 20 block size centered at each minutiae point of enhanced fingerprint image. The block size 20 is considered because the inter ridge distance of fingerprint image captured at 500 dpi is 10. The Gabor energy feature gives numerical summary of local ridge configuration around the minutiae. 161

5. Fingerprint Biometric Data Indexing Let I(x, y) be an enhanced fingerprint image and G(ω, θ) be the Gabor filter [16] with frequency ω and orientation θ. The Gabor response GRω,θ is obtained by 2-D convolution of the image I(x, y) and Gabor filter G(ω, θ). Here, ω is the average local ridge frequency of the fingerprint and can be computed using the method given in [16], w is the size of the filter and θ is the angle of the minutia (θ = θ1 , θ2 , θ3 ). The response of Gabor filter is computed using Eq. (5.13).

GRω,θ (x, y) =

2 ∑

w

2 ∑

u= −w 2

v= −w 2

w

G(ω, θ)I(x − u, y − v)

(5.13)

These responses are also called Gabor coefficient values which are complex. The magnitude value of Gabor coefficient is calculated at each pixel in the block of a minutiae point using Eq. (5.14). Finally, the Gabor energy (GEω,θ ) at each minutiae is calculated as the sum of squares of the magnitude values of GRω,θ at every pixel using Eq. (5.15). |GRω,θ (x, y)| =

√ Re(GRω,θ (x, y))2 + Im(GRω,θ (x, y))2 GEω,θ =

∑

[|GRω,θ (x, y)|]2

(5.14) (5.15)

x,y

The values of f i (6), f i (7) and f i (8) are calculated using Eq. (5.16). f i (6) = GEω,θ1 f i (7) = GEω,θ2

(5.16)

f i (8) = GEω,θ3 All index keys are generated corresponding to all triplets of the fingerprint images of each subject. The values of each feature are in different range for all index keys of fingerprint images. To bring the values of each feature into a common range, min-max normal162

5.4. Storing ization [17] is applied on each feature of fingerprint images. Suppose, there are P number of subjects and each subject has Q number of samples to be enrolled into the database. The index keys of the q th sample of the pth subject is represented in Eq. (5.17), where p = 1, 2, . . . , P and q = 1, 2, . . . , Q. In Eq. (5.17), the ith index key of the q th sample of the pth subject is denoted as indxip,q . Let |Tp,q | be the number of index keys generated from the fingerprint image of the q th sample of the pth subject. Hence, the total number of index keys obtained from ∑ all persons is N = P,Q p=1,q=1 |Tp,q |. Each index key can be viewed as a single point in an 8-dimensional hyperspace. Thus, each fingerprint image is a collection of points residing in the 8-dimensional space. These index keys and the corresponding subject’s identities are stored into the database so that it can be easily retrieved the identities of subjects at the time of searching. The identity of the q th sample of the pth subject is retrieved with Idpq . indx1p,q

= [

1 (1) fp,q

1 (2) fp,q

···

1 (f l) fp,q

···

1 (8) fp,q

]

indx2p,q

= [

2 (1) fp,q

2 (2) fp,q

···

2 (f l) fp,q

···

2 (8) fp,q

]

···

···

indxip,q

= [

i (1) fp,q

= [

T fp,qp,q (1)

···

··· i (2) fp,q

···

i (f l) fp,q

···

···

Tp,q indxp,q

T fp,qp,q (2)

··· ···

i (8) fp,q

···

···

T fp,qp,q (f l)

···

T fp,qp,q (8)

]

]

(5.17)

5.4

Storing

In this work, three different storing techniques are explored to index the fingerprint data. First, the linear storing technique is investigated where all index keys and corresponding subjects’ identities are stored linearly. Then, clustering technique is examined. In this technique, all index keys are divided into a number of groups based on the feature 163

5. Fingerprint Biometric Data Indexing values of index keys. The index keys and the corresponding subject’s identities in each group are stored linearly for each cluster. Finally, the clustered kd-tree technique is analyzed to store all index keys. In this technique, all index keys are divided into a number of groups. The index keys and the corresponding subject’s identities in each group are stored in a kd-tree [7].

5.4.1

Linear Index Space

In this approach, all index keys and subject identities are stored linearly. In other words, a 2-D index space is created to store index keys and the corresponding identities. The size of the index space would be N × (F L + 1) where N is the total number of index keys generated from all persons and F L (F L = 8) is the number of feature values in an index key. The first eight columns of each row in the 2-D space contain the eight feature values of an index key and the last column contains the identity of the subject corresponding to the index key. The method of creating linear index space is summarized in Algorithm 5.1. In Step 2 to 12 of Algorithm 5.1, all index keys and identity of a person are stored in the linear index space. Among these steps, Step 5 to 7 store the eight key values into the first eight columns of the linear index space and Step 8 stores the identity of the person into the 9th column of the linear index space. Figure 5.7 shows the structure of the linear index space in the database. In Fig. 5.7, LN IN DX is the two dimensional index space ∑ of size N × (F L + 1). In this structure, N = P,Q p=1,q=1 |Tp,q | number of index keys can be stored from the P subjects with Q samples each. Here, first all index keys of the 1st subject are stored, then all index keys of the 2nd subject and so on. In Fig. 5.7, the entry i (1) f i (2) · · · f i (d) · · · f i (8) > and Idp in LN IN DX rep< fp,q q p,q p,q p,q

resent the ith index key and the identity of the q th sample of the pth subject, respectively. 164

5.4. Storing Algorithm 5.1 Creating linear index space Input: All index keys of all samples of all subjects (indx11,1 , indx21,1 , T

T

1,1 1,2 . . ., indx1,1 ; indx11,2 , indx21,2 , . . ., indx1,2 ; . . . ; indx1P,Q , indx2P,Q ,

T

P,Q . . ., indxP,Q ) Output: Linear Index Space (LN IN DX[ ][ ]) 1: inc = 1 // P is the number of subjects 2: for p = 1 to P do // Q is the number of samples 3: for q = 1 to Q do // Tp,q is the number of index keys for the q th sample of the pth

subject 4: 5: 6:

7: 8:

for i = 1 to Tp,q do for f l = 1 to 8 do LN IN DX[inc][f l] = indxip,q [f l] // Copy eight feature valend for LN IN DX[inc][9] = Idpq

ues into the first eight columns

// Copy identity of the subject in the ninth column

inc = inc + 1 10: end for 11: end for 12: end for 9:

5.4.2

Clustered Index Space

In this approach, all index keys are divided into a number of groups. The grouping is done by performing k-means clustering on all index √ keys using the 8-dimensional feature values. N/2 number of clusters are chosen which is the rule of thumb of clusters for any unsupervised clustering algorithm [23] to balance between the number of clusters and the number of items within a single cluster. An eight dimensional index key is treated as a eight dimensional point in the cluster space. Before doing k-means clustering, the fingerprint data is preprocessed so that clustering algorithm converges to a local optima. In prepro165

5. Fingerprint Biometric Data Indexing 1 1 1 1 1 1 1 1 1 f1,1 (1) f1,1 (2) f1,1 (3) f1,1 (4) f1,1 (5) f1,1 (6) f1,1 (7) f1,1 (8)

T1,1 1,1

T1,1 1,1

T1,1 1,1

T1,1 1,1

T1,1 1,1

T1,1 1,1

T1,1 1,1

T1,1 1,1

f

i1,q

i i i i i i i i (2) f1,q (3) f1,q (4) f1,q (5) f1,q (6) f1,q f1,q (1) f1,q (7) f1,q (8)

(1) f

(2) f

(3) f

(4) f

(5) f

(6) f

(7) f

(8)

index keys of the 1st sample

1 1

Id

Id1q

ith index key of the qth sample

Q-1

1 1 1 1 1 1 1 1 (7) f1,Q (8) T1,q +1 f1,Q (1) f1,Q (2) f1,Q (3) f1,Q (4) f1,Q (5) f1,Q (6) f1,Q

Id1Q

q=1 Q

T1,q

T1,Q 1,Q

f

T1,Q 1,Q

(1) f

T1,Q 1,Q

(2) f

T1,Q 1,Q

(3) f

T1,Q 1,Q

(4) f

T1,Q 1,Q

(5) f

T1,Q 1,Q

(6) f

T1,Q 1,Q

(7) f

(8)

1st Subject

T1,1

Id11

index keys of the Qth sample

1 Q

Id

q=1

i p,q

i i i i i i i i (5) f p,q (7) f p,q (8) (2) f p,q (3) f p,q (4) f p,q (6) f p,q f p,q (1) f p,q

Id qp

Tp,q +1

1 1 1 1 1 1 1 1 f P,1 (1) f P,1 (2) f P,1 (3) f P,1 (4) f P,1 (5) f P,1 (6) f P,1 (7) f P,1 (8)

Id1P

P-1,Q

p=1,q=1

P-1,Q

Tp,q +TP,1 f

TP,1 P,1

(1) f

TP,1 P,1

(2) f

TP,1 P,1

(3) f

TP,1 P,1

(4) f

TP,1 P,1

(5) f

TP,1 P,1

(6) f

TP,1 P,1

(7) f

TP,1 P,1

(8)

ith index key of the qth sample of the pth subject

index keys of the 1st sample

P 1

Id

i i i i i i i i f P,q (1) f P,q (2) f P,q (3) f P,q (4) f P,q (5) f P,q (6) f P,q (7) f P,q (8)

Id qP

1 1 1 1 1 1 1 1 Tp,q +1 f P,Q (1) f P,Q (2) f P,Q (3) f P,Q (4) f P,Q (5) f P,Q (6) f P,Q (7) f P,Q (8)

Id QP

i P,q P,Q-1

p=1,q=1 P,Q

Tp,q +1 f

TP,Q P,Q

(1) f

TP,Q P,Q

(2) f

TP,Q P,Q

(3) f

TP,Q P,Q

(4) f

TP,Q P,Q

(5) f

TP,Q P,Q

(6) f

TP,Q P,Q

T

ith index key of the qth sample

Pth Subject

p=1,q=1

index keys of the Qth sample

(7) f P,QP,Q (8) Id QP

p=1,q=1

LNINDX

Figure 5.7: Linear index space.

cessing, the distances of all points from the origin are compared and the density of the points from the distance histogram is found. From √ this histogram, the top N/2 picks are selected and the cluster centers are randomly initialized with these points. Then, the k-means clustering is applied on all eight dimensional points in the cluster space (i.e. all index keys) and the centroid is found for each cluster. The cluster centroid represents the point which is equal to the average values of all the eight dimensional points in the cluster. Then, key points are assigned to the clusters such that the distance of a key point from the centroid of assigned cluster is the minimum than the distances from the centroids of other clusters. If any key point is found with almost same distances from the two or more cluster centers and or some isolated points with large distance from cluster center, those points are removed from the resulting partition to reduce the effect of outliers. 166

5.4. Storing The distance is measured as Euclidean distance between key point and cluster centroid. Finally, when all points are assigned, the centroids of the cluster is recalculated. The centroid of each cluster is stored in a two dimensional space (see Fig. 5.8). The size of the space is √ K × (F L + 1), where K is the number of clusters (here, K = N/2) and F L (F L = 8) is the number of features in an index key. Let Ni be the number of index keys present in the ith cluster and f j (l) be the lth (l = 1, 2, . . . , 8) feature value of the j th index key in the ith cluster. Then, the lth centroid value dcni (l) of the ith cluster is calculated using Eq. (5.18). dcni (l) =

Ni 1 ∑ f j (l) for l = 1 to 8 and i = 1 to K Ni

(5.18)

j=1

Thus, eight centroid index values (dcni (1) dcni (2) · · · dcni (8)) f(1) f(2) f(3) f(4) f(5) f(6) f(7) f(8) id 1 2

N1 Cluster[1]->LNINDX dcn(1) dcn(2) dcn(3) dcn(4) dcn(5) dcn(6) dcn(7) dcn(8)

clid

1 dcn1(1) dcn1(2) dcn1(3) dcn1(4) dcn1(5) dcn1(6) dcn1(7) dcn1(8)

cl1 f(1) f(2) f(3) f(4) f(5) f(6) f(7) f(8) id 1 2

i dcni(1) dcni(2) dcni(3) dcni(4) dcni(5) dcni(6) dcni(7) dcni(8)

cli

Ni Cluster[i]->LNINDX K dcnK(1) dcnK(2) dcnK(3) dcnK(4) dcnK(5) dcnK(6) dcnK(7) dcnK(8)

clK

CLTCNTRD

f(1) f(2) f(3) f(4) f(5) f(6) f(7) f(8) id 1 2

NK Cluster[K]->LNINDX

Figure 5.8: Clustered index space.

167

5. Fingerprint Biometric Data Indexing can be calculated of the ith cluster using Eq. (5.18) and the centroid values are stored in a 2-D space (CLT CN T RD in Fig. 5.8). In Fig. 5.8, CLT CN T RD is the array for storing the centroid values of all clusters. The first eight columns of each row store the 8-dimensional centroid values (dcn(l), l = 1, 2, . . . , 8) for a cluster and the last column stores the cluster id (it is represented as clid in Fig. 5.8) which uniquely identifies each cluster. For each cluster, a linear index space (Cluster → LN IN DX) is created as shown in Fig. 5.8. In Fig. 5.8, Cluster[i] → LN IN DX is the linear index space for the ith cluster. The feature values of an index key in Cluster[i] → LN IN DX are denoted as < f (1), f (2), · · · , f (8) > and the identity of the subject corresponding to the index key is represented as id. Note that the index keys within a cluster are stored in a linear fashion. The method for creating clustered index space is summarized in Algorithm 5.2. In Step 2 to 12 of Algorithm 5.2, all index keys of all samples of all subjects are stored into a linear index space. In Step 13, k-meansClustering function takes the linear index space as input and divides all index key into K groups. Each group is stored in Cluster[ ] → LN IN DX[ ][ ] 2-D space of the clustered index space. In Step 15, Centroid function calculates the centroid of a cluster. The centroid values of a cluster are stored in CLT CN T RD of cluster index space. The cluster id of each cluster (here cluster number) is stored in the 9th column of CLT CN T RD in Step 16.

5.4.3

Clustered kd-tree Index Space

First, clusters of all index keys are created as discussed above. Then each index key of a cluster is stored into a kd-tree. A kd-tree is a data structure for storing a finite set of points from a k-dimensional space [7]. The kd-tree is a binary tree in which every node stores a k-dimensional point. In other words, the node of a kd-tree stores an eight dimensional point. The node structure of kd-tree is shown 168

5.4. Storing Algorithm 5.2 Creating clustered index space Input: All index keys of all samples of all subjects (indx11,1 , indx21,1 , T

T

1,1 1,2 . . ., indx1,1 ; indx11,2 , indx21,2 , . . ., indx1,2 ; . . . ; indx1P,Q , indx2P,Q ,

T

P,Q . . ., indxP,Q ) Output: Cluster centroids (CLT CN T RD[][]) and linear index spaces (Cluster[] → LN IN DX[][]) corresponding to each cluster

1: 2: 3: 4: 5: 6:

inc = 1 for p = 1 to P do for q = 1 to Q do for i = 1 to Tp,q do for f l = 1 to 8 do LN IN DX[inc][f l] = indxip,q [f l] // Copy feature values to the first eight columns

7: 8:

end for LN IN DX[inc][9] = Idpq

// Copy identity of the subject in the ninth column

9: 10: 11: 12: 13:

14: 15: 16:

inc = inc + 1 end for end for end for // Grouping of all index keys into K different clusters Cluster[ ] → LN IN DX[ ][ ] = K-meanClustering(LN IN DX[ ][ ], K) // K is the number of clusters for k = 1 to K do // Calculate the centroid of each cluster CLT CN T RD[k][ ] = Centroid(Cluster[k] → LN IN DX) CLT CN T RD[k][9] = k // Copy the cluster id of linear index space corresponding to a cluster in the ninth column of clustered index space

17:

end for

in Fig. 5.9(a). In a node of a kd-tree, keyV ector field stores the kdimensional index key and Split field stores the splitting dimension or a discriminator value. The lef tT ree and rightT ree store a kd-tree representing the pointers to the left and right of the splitting plane, 169

5. Fingerprint Biometric Data Indexing

keyVector: k-dimensional point Split: splitting dimension leftTree: A kd-tree representing the points to the left of the splitting plane rightTree: A kd-tree representing the points to the right of the splitting plane

P1 =

P7 =

P2 =

P8 =

P3 =

P9 =

P4 =

P10 =

P5 =

P11 =

P6 =

P12 =

(a) Components of a kd-tree node

(b) 3D points

Split=1 P1=[20, 16, 18]

Split=2 P3=[17, 26, 14]

Split=3 P5=[19, 15, 28]

Split=1 P10=[12, 20, 35]

Split=2 P2=[24, 12, 17]

Split=3 P6=[18, 32, 27]

Split=1 P12=[15, 32, 18]

Split=3 P7=[27, 11, 24]

Split=1 P8=[18, 34, 33]

Split=1 P11=[30, 11, 29]

Split=3 P4=[27, 22, 37]

Split=1 P9=[25, 17, 28]

(c) Kd-tree example

Figure 5.9: Structure of a kd-tree and an example of kd-tree with three dimensional points.

respectively. For an example, let there are twelve three dimensional points (P 1, P 2, . . . , P 12) as shown in Fig. 5.9(b) and a kd-tree with these points is shown in Fig. 5.9(c). The first point P 1 is inserted at the root of the kd-tree. At the time of insertion, one of the dimension is chosen as a basis (Split) of dividing the rest of the points. In this example, the value of Split in the root node is 1. In other words, if the value of the first dimension of the current point to be inserted is less than the same of the root, then the point is stored in lef tT ree otherwise in rightT ree. This means all items to the left of root will have the first dimension value less than that of the root and all items to the right of the root will have greater than (or equal to) that of the root. The point P 2 is inserted in the right kd-tree and P 3 is in170

5.4. Storing serted in the left kd-tree of the root. When the point P 4 is inserted, the first dimension value of P 4 is compared with the root and then the second dimension value of P 4 is compared with the second dimension value of P 2 at next level. The point P 4 is inserted in the right kd-tree of the point P 2. Similarly, all other points are inserted into the kd-tree. First dimension will be chosen again as the basis (Split=1 ) for discrimination at level 3. All eight dimensional points (index keys) are stored within a cluster in a kd-tree data structure. The FLANN library [1, 5, 26] is used to implement the kd-tree in this work. The maximum height of the optimized kd-tree with N number of k-dimensional point is ⌊log2 (N )⌋ [7]. Figure 5.10 shows the schematic of a clustered kd-tree index space. In Fig. 5.10, CLT CN T RD stores the centroid of the K clusters. The centroid value of the ith cluster is represented as < dcni (l) dcni (2) · · · dcni (8) >. The index keys within a cluster are stored into a kd-tree data structure and a unique id is assigned to the each kd-tree. The id of a kd-tree is stored at the last column of the CLT CN T RD. In Fig. 5.10, rooti represents the id of the ith kd-tree. The kd-treei stores the index keys of the ith cluster as shown in Fig. 5.10. The methods for creating clustered kd-tree index spaces are sumdcn(1) dcn(2) dcn(3) dcn(4) dcn(5) dcn(6) dcn(7) dcn(8) root 1 dcn1(1) dcn1(2) dcn1(3) dcn1(4) dcn1(5) dcn1(6) dcn1(7) dcn1(8) root1

kd-tree1

i dcni(1) dcni(2) dcni(3) dcni(4) dcni(5) dcni(6) dcni(7) dcni(8) rooti

kd-treei

K dcnK(1) dcnK(2) dcnK(3) dcnK(4) dcnK(5) dcnK(6) dcnK(7) dcnK(8) rootK

kd-treeK

CLTCNTRD

Figure 5.10: Clustered kd-tree index space.

171

5. Fingerprint Biometric Data Indexing marized in Algorithm 5.3. In Step 2 to 12 of Algorithm 5.3, all index keys of all samples of all subjects are stored in a linear index space.

Algorithm 5.3 Creating clustered kd-tree index space Input: All index keys of all samples of all subjects (indx11,1 , indx21,1 , T

T

1,1 1,2 . . ., indx1,1 ; indx11,2 , indx21,2 , . . ., indx1,2 ; . . . ; indx1P,Q , indx2P,Q ,

T

P,Q . . ., indxP,Q ) Output: Cluster centroids (CLT CN T RD[ ][ ]) and kd-trees (kd − treek ) corresponding to each cluster 1: inc = 1 2: for p = 1 to P do 3: for q = 1 to Q do 4: for i = 1 to Tp,q do 5: for f l = 1 to 8 do 6: LN IN DX[inc][f l] = indxip,q [f l] 7: end for 8: LN IN DX[inc][9] = Idpq 9: inc = inc + 1 10: end for 11: end for 12: end for 13: Cluster[ ] → LN IN DX[ ][ ] = K-meanClustering(LN IN DX[ ][ ], K) // K is the number of clusters 14: for k = 1 to K do 15: CLT CN T RD[k][ ] = Centroid(Cluster[k] → LN IN DX) // Each key of a cluster inserted into a kd-tree index space // Nk is the number of index keys in kth cluster 16: for j = 1 to Nk do 17: Insert Cluster[k] → LN IN DX[j][ ] into Kd-treek 18: end for 19: CLT CN T RD[k][9] = Kd-treek // Copy id of kd-tree corre-

sponding to a cluster in the ninth column of clustered index space 20:

end for 172

5.5. Retrieving Step 13 takes the linear index space as input and divides all index keys into k groups using k-meansClustering function. In Steps 14 to 20 of Algorithm 5.3, the clustered kd-tree index space is created. In Step 15, Centroid function calculates the centroid of a cluster. The centroid values of a cluster are stored in CLT CN T RD of clustered index space. In Step 16 to 18, each index key of the k th cluster is inserted into the k th kd-tree and in Step 19, the id of the k th kd-tree is stored in the 9th column of k th row of clustered index space (CLT CN T RD).

5.5

Retrieving

Retrieving is the process of generating a candidate set (CSET ) from the enrolled fingerprints for a given query fingerprint. Note that the fingerprints in the CSET are the most likely to match with the query fingerprint. To obtain the CSET for a given query fingerprint image, first the index keys are generated corresponding to each triplet of the query fingerprint using index key generation technique discussed in Section 5.3 and each dimension is normalized using min-max normalization [17] to bring each dimension into common range. Let Tt be the total number of index keys of the query fingerprint (see Eq. (5.19)). In Eq. (5.19), indxit denotes the ith index key of the query fingerprint. Then, each index key of the query fingerprint is compared with the stored index keys in the database and the CSET is generated. The CSET contains two fields: id and vote. The id and vote fields of the CSET store the identity of the subject and the number of vote received for that subject. When an index key of the query fingerprint is matched with a stored index key then a vote is counted corresponding to that identity. Finally, the rank is calculated based on the vote received by each identity. The first rank is given corresponding to the maximum vote received, second rank is given corresponding to the 173

5. Fingerprint Biometric Data Indexing next maximum vote received and so on. indx1t

= [

ft1 (1)

ft1 (2)

···

ft1 (d)

···

ft1 (8)

]

indx2t

= [

ft2 (1)

ft2 (2)

···

ft2 (d)

···

ft2 (8)

]

···

···

indxit

fti (1)

= [

···

··· fti (2)

···

···

fti (d) ···

··· ···

fti (8)

]

(5.19)

···

Tt (2) · · · f Tt (d) · · · f Tt (8) ] = [ ftTt (1) fp,q t t

indxTt t

In this work, three different searching techniques are investigated to retrieve the fingerprint from the database for three different storage structures discussed in the previous section. These three searching techniques are linear search (LS), clustered search (CS) and clustered kd-tree search (CKS) for linear, clustered and clustered kd-tree index spaces, respectively. These techniques are described in the following.

5.5.1

Linear Search (LS)

In this approach, the linear index space is searched for a query fingerprint. For an index key of the query fingerprint, whole linear index space is searched and the distances are calculated with all index key stored in linear index space. The distance is calculated as Euclidean distance. The Euclidean distance between the ith index key (indxit ) of the query fingerprint and the j th index key (indxjp,q ) of the q th sample of the pth subject in the database is calculated as in Eq. (5.20). √ EDi,j =

8 ∑

j (fti (f l) − fp,q (f l))2 where i = 1, 2, . . . , Tt

f l=1

j = 1, 2, . . . , Tp,q ; ∀p and ∀q (5.20) The identity is stored and a vote is casted on that identity in the CSET corresponding to the minimum distance. If the j th index key in 174

5.5. Retrieving the database produces minimum Euclidean distance for the ith index key of query fingerprint, then a vote is added for the subject identity (Idj ) of the j th index key. In the similar way, the votes are casted for other index keys of the query fingerprint. The identity which receives the highest number of votes will be ranked first in the CSET . The method for generating CSET using linear search is described in Algorithm 5.4. Step 3 to 7 of Algorithm 5.4 calculate the distances between a query index key and all stored index keys. In Step 8, the minimum distance among all calculated distances is determined. Match index corresponding to the minimum distance is selected in Step 9 and the id is retrieved from linear index space at that index in Step 10. Step 11 to 18 generate the CSET and CSET is sorted according to received votes in Step 20.

5.5.2

Clustered Search (CS)

In this technique, the clustered index space is searched to retrieve the candidate set. For an index key of query fingerprint, first a cluster is found which contains index keys similar with the query index key. Then, the all identities are retrieved from the identified cluster. To do this, first the Euclidean distance is calculated between an index key of the query fingerprint and all stored centroid values of clusters using Eq. (5.21). In Eq. (5.21), indxit is the ith index key of the query fingerprint and dcnj is the centroid values of the j th cluster. √ EDi,j =

8 ∑

(fti (f l) − dcnj (f l))2 where j = 1, 2, . . . , K

f l=1

(5.21)

& i = 1, 2, . . . , Tt The cluster is selected which has the minimum distance between a query index key and the centroid values of that cluster. If centroid values of the j th cluster in the database produce minimum distance, then all identities are retrieved from the linear index space 175

5. Fingerprint Biometric Data Indexing Algorithm 5.4 Candidate set generation in LS from linear index space Input: All index keys from query fingerprint image (indx1t , indx2t , . . ., indxTt t ) and linear index space (LN IN DX[ ][ ]) Output: Candidate Set (CSET [ ] → (id, vote)) 1: inc = 1 // Tt is the total number of query index key 2: for i = 1 to Tt do 3: for j = 1 to N do // N is the √ total number of index keys in the database 8 ∑ 4: EDi,j = (fti (f l) − LN IN DX[j][f l])2 f l=1

17:

end for EDmin = min(EDi ) // Select minimum distance Select j corresponding to EDmin and assign to idmin id = LN IN DX[idmin ][9] subid = SubjectId(id) // Copy subject id from id if subid ∈ / CSET [ ] → (id) then CSET [inc] → id = subid CSET [inc] → vote = CSET [inc] → vote + 1 inc = inc + 1 else CSET [id] → vote = CSET [id] → vote + 1 end if end for

18:

Sort CSET [ ] → (id, vote) in descending order based on vote

5: 6: 7: 8: 9: 10: 11: 12: 13: 14: 15: 16:

(Cluster[i] → LN IN DX[ ][ ]) of the j th cluster in the database. Similarly, the clusters are found for all other index keys of the query fingerprint. The voting is done with the retrieved identities from the clusters to generate the CSET . The method for generating CSET from the clustered index space is summarized in Algorithm 5.5. Step 2 to 7 of Algorithm 5.5 find the matched cluster for an index key of query fingerprint. 176

5.5. Retrieving Algorithm 5.5 Candidate set generation in CS from clustered index space Input: All index keys from query fingerprint image (indx1t , indx2t , . . ., indxTt t ), Cluster centroid index space(CLT CN T RD[ ][ ]) and linear index space for each cluster (Cluster[ ] → LN IN DX[ ][ ]) Output: Candidate Set (CSET [ ] → (id, vote)) inc = 1 // Tt is the total number of query index key 2: for i = 1 to Tt do // Find matched cluster, K is the number of clusters 3: for k = 1 √ to K do 8 ∑ 4: EDi,k = (fti (f l) − dcnk (f l))2 1:

f l=1

5: 6: 7: 8: 9: 10: 11: 12: 13: 14: 15: 16: 17: 18: 19: 20:

end for EDmin = min(EDi ) Select cluster id corresponding to EDmin and assign to CL // Let NCL be the total number of index key in the CLth cluster for i = 1 to NCL do id = Cluster[CL] → LN IN DX[i][9] subid = SubjectId(id) // Select the subject id from id if subid ∈ / CS[ ] → (id) then // Generate candidate set CSET [inc] → id = subid CSET [inc] → vote = CSET [inc] → vote + 1 inc = inc + 1 else CSET [id] → vote = CSET [id] → vote + 1 end if end for end for Sort CSET [ ] → (id, vote) in descending order based on vote

5.5.3

Clustered kd-tree Search (CKS)

In this approach, first the cluster is found to which a query index key belongs. Then, the kd-tree is searched corresponding to that clus177

5. Fingerprint Biometric Data Indexing ter. The cluster is found which have the minimum Euclidean distance between a query index key and the stored centroid index key. The method is discussed in the previous section. If centroid values of the ith cluster produces minimum Euclidean distance, the ith kd-tree is searched in the database. The method for creating CSET from the clustered kd-tree index space is summarized in Algorithm 5.6. Step 2 to 7 of Algorithm 5.6 find the matched cluster for an index key of the query fingerprint. Step 8 finds the nearest neighbors of query index key from the kd-tree corresponding to matched cluster. FLANN library [1, 5, 26] is used to search the kd-tree. The subject identity of the nearest neighbor is found in Step 9 and 10. The sorted CSET is generated in Step 11 to 19.

5.6

Performance Evaluation

To study the efficacy of the proposed fingerprint data indexing approach, a number of experiments have been conducted on different fingerprint databases with different experimental setups. The efficacy of the proposed method is measured with different aspects like accuracy, efficiency, searching time and memory requirement. In this section, first the descriptions of different fingerprint databases and the experimental setups used in this approach are presented. Then, the experimental results with this approach are discussed.

5.6.1

Databases

In this experiments, six widely used fingerprint databases are considered. The first two databases are obtained from National Institute of Science and Technology (NIST) Special DB4 database [4, 29]. The remaining databases are obtained from Fingerprint Verification Competition (FVC) 2004 databases [2, 22]. Detailed description of each database is given in the following. 178

5.6. Performance Evaluation Algorithm 5.6 Candidate set generation in CKS from clustered kdtree index space Input: All index keys from query fingerprint image (indx1t , indx2t , . . ., indxTt t ), Cluster centroid index space(CLT CN T RD[ ][ ]) and linear index space for each cluster (Kd − tree[ ] ) Output: Candidate Set (CSET [ ] → (id, vote)) 1: inc = 1 // Tt is the total number of query index key 2: for i = 1 to Tt do // Find matched cluster, K is the number of clusters 3: for k = 1 √ to K do 8 ∑ 4: EDi,k = (fti (f l) − dcnk (f l))2 f l=1

5: 6: 7:

end for EDmin = min(EDi ) Select cluster id corresponding to EDmin and assign to CL // Find nearest neighbor of a query index key from Kd-tree (Kdtree[CL] )

8: 9: 10: 11: 12: 13: 14: 15: 16: 17: 18: 19:

N N = f indN earestN eighbour(Kd-tree[CL] , indxQ q ) id = retrieveIdF romN N (N N ) subid = SubjectId(id) if subid ∈ / CSET [ ] → (id) then CSET [inc] → id = subid CSET [inc] → vote = CSET [inc] → vote + 1 inc = inc + 1 else CSET [id] → vote = CSET [id] → vote + 1 end if end for Sort CSET [ ] → (id, vote) in descending order based on vote

NIST DB4: The NIST Special Database 4 [4, 29] contains 4000 8-bit gray scale fingerprint images from 2000 fingers. Each finger has two different image instances (F and S). The fingerprint images are captured at 500 dpi resolution from rolled fingerprint impressions scanned from cards. The size of each fingerprint image is 512 × 512 pixels with 179

5. Fingerprint Biometric Data Indexing 32 rows of white space at the bottom of the fingerprint image. The fingerprints are uniformly distributed in the five fingerprint classes: arch (A), left loop (L), right loop (R), tented arch (T) and whorl (W). NIST DB4 (Natural): The natural distribution of fingerprint images [4, 18] is A = 3.7%, L = 33.8%, R = 31.7%, T = 2.9% and W = 27.9%. A subset of NIST DB4 is obtained by reducing the cardinality of the less frequent classes to resemble a natural distribution. The data set contains 1,204 fingerprint pairs (the first fingerprints from each class have been chosen according to the correct proportion). FVC2004 DB1: The first FVC 2004 database [2, 22] contains 880 fingerprint images of 110 different fingers. Each finger has 8 impressions in the database. The images are captured at 500 dpi resolutions using optical sensor (“V300" by CrossMatch) [22]. The size of each image is 640 × 480 pixels. FVC2004 DB2: The second FVC 2004 database [2, 22] contains 880 fingerprint images captured from 110 different fingers and 8 impressions per finger. The images are captured at 500 dpi resolutions using optical sensor (“U.are.U 4000" by Digital Persona) [22]. The size of each image is 328×364 pixels. FVC2004 DB3: The third FVC 2004 database [2, 22] contains 880 fingerprint images of 110 different fingers captured at 512 dpi resolutions with thermal sweeping sensor (“Finger Chip FCD4B14CB" by Atmel) [22]. Each finger has 8 impressions in the database. The size of each image is 300×480 pixels. FVC2004 DB4: The fourth FVC 2004 database [2, 22] is generated synthetically using SFinGe V3 tool [3, 11]. This database consists of 880 fingerprint images of 110 fingers and 8 impression per finger. The images are generated with about 500 dpi resolutions and the size of images is 288×384 pixels. 180

5.6. Performance Evaluation

5.6.2

Evaluation Setup

To evaluate the proposed indexing methods, each fingerprint database is partitioned into two sets: Gallery and P robe. The Gallery set contains the fingerprint images enrolled into the index database and the P robe set contains the fingerprint images used for query to search the index database. In this experiment, one type of Gallery set (G1) and one type of P robe set (P) are created for each NIST database and three types of Gallery set (G1, G3 and G5) and one type of P robe set (P) are created for each FVC 2004 database to conduct the different experiments. • G1: The gallery set is created with the first fingerprint impression of each finger in the NIST databases and one fingerprint impression from the first six impressions of each finger in FVC 2004 databases. • G3: In this gallery set, three fingerprint impressions from the first six impressions of each finger in FVC 2004 databases are used. • G5: This gallery set contains five fingerprint impressions from the first six impressions of each finger in FVC 2004 databases. • P: The probe set is created with the second fingerprint impression of each finger in NIST databases and the seventh and eighth impressions of each finger in FVC 2004 databases. All methods described in this approach are implemented using C language and compiled with GCC 4.3 compiler on the Linux operating system. Methods are evaluated with Intel Core2Duo processor of speed 2.00GHz and 2GB RAM. 181

5. Fingerprint Biometric Data Indexing

5.6.3

Evaluation

The approach is evaluated with different experiments to measure the performance of the proposed indexing method. The descriptions of these experiments and the results with these experiments are given in the following subsections.

5.6.3.1

Accuracy

The accuracy of the proposed fingerprint data indexing approach is measured with respect to the parameters: HR, P R and CM S. The definition of these parameters are given in Section 4.7.1. Here, the accuracy of the proposed method is studied with single sample enrollment as well as multiple sample enrollments into the database. Performance of three search techniques: To investigate the performance of the LS , CS and CKS with single sample enrollment into the database, G1 and P are used as Gallery and P robe sets of all NIST (NIST DB4 and NIST DB4 Natural) and FVC 2004 (FVC 2004 DB1, FVC 2004 DB2, FVC 2004 DB3 and FVC 2004 DB4) databases. G1 is enrolled into the database and the samples in P are used to probe. Results on rank one HR, P R and searching time (ST ) for LS, CS and CKS are given in Table 5.1. From Table 5.1, it is observed that LS gives the maximum rank one HR whereas CKS gives the minimum rank one HR. However, the P R is 100% for LS whereas the P R for CKS is 13.55%. Also the ST in CKS is less than the other two searching techniques. The performances of LS, CS and CKS are also substantiated with CM S at different ranks. The CM S is represented by CMC curve. Figure 5.11 shows the CMC curves of LS, CS and CKS for two NIST databases and four FVC 2004 databases. In Fig. 5.11, it can be shown that the CM S is increased at higher rank. For NIST DB4 database, 95.90%, 95.05% and 94.20% CM S can be attained (see 182

5.6. Performance Evaluation Table 5.1: HR, P R and searching time (ST ) for the different fingerprint databases with single enrollment (gallery G1) into the database LS DB

CS

CKS

HR

PR

ST (ms)

FVC2004 DB1

90.91

100

33.102

89.09

28.38

10.342

86.36

11.93

0.426

FVC2004 DB2

90.45

100

32.693

88.18

32.03

7.852

85.45

13.17

0.485

FVC2004 DB3

83.64

100

117.725

80

33.44

25.887

78.18

14.75

1.57

FVC2004 DB4

89.09

100

26.925

85.91

25.56

9.395

84.09

10.89

0.349

NIST DB4

85.3

100

325.928 80.4

32.56

25.455

78.7

15.34

1.193

NIST DB4 Natural

87.7

100

28.67

18.411

81.1

15.22

0.842

213

HR

83.45

PR

ST (ms)

HR

PR

ST (ms)

Fig. 5.11(a)) and 96.20%, 95.55% and 95.10% CM S for NIST DB4 Natural database (see Fig. 5.11(b)) at the rank five in LS, CS and CKS, respectively. From Fig. 5.11(a) and (b), it is observed that if the rank increases from 5 to 15, then the CM S increases to 99.65%, 98.50% and 97.55% in LS, CS and CKS for NIST DB4 database, respectively. With reference to NIST DB4 (Natural) database, for the same increment in rank, the CM S are 99.70%, 99.10% and 98.80% in LS, CS and CKS, respectively. For FVC2004 DB1 94.09%, 92.73% and 90.45% CM S can be achieved (Fig. 5.11(c)), 94.09%, 91.82% and 90.0% CM S for FVC2004 DB2 (Fig. 5.11(d)), 88.18%, 87.27% and 85.91% CMS for FVC2004 DB3 (Fig. 5.11(e)) and 95.91%, 90.45% and 88.18% CM S for FVC2004 DB4 (Fig. 5.11(f)) at rank five in LS, CS and CKS, respectively. Figure 5.11(c), (d), (e) and (f) show that the CM S increases to 100%, 98.64% and 94.55% for FVC2004 DB1, 100%, 97.27% and 95.91% for FVC2004 DB2, 97.73%, 95.0% and 94.09% for FVC2004 DB3 and 100%, 98.64% and 95.0% for FVC2004 DB4 when rank increases to 15% in LS, CS and CKS, respectively. 183

  

Biometric Data Indexing 5. Fingerprint   

100

CMS

95

NISTDB4 LS G1

90

CS G1 CKS G1 85

80 0

5

10

15 Rank

20

25

30 



(a) CMC curves with NIST DB4 database 

100

95 CMS

NIST D4 Natural LS G1

90

CS G1 CKS G1 85

80 0

5

10

15 Rank

20

25

30 

(b) CMC curves with NIST DB4 Natural  database 

100

95 CMS

FVC2004 DB1 LS G1

90

CS G1 CKS G1 85

80 0

5

10

15 Rank

20

25

30 

(c) CMC curves with FVC 2004 DB1 database  

Figure 5.11: CMC curves with one enrolled sample per fingerprint in LS, CS and CKS for the different databases.

184

5.6. Performance Evaluation



100

95 CMS

FVC2004 DB2 LS G1

90

CS G1 CKS G1 85

80 0

5

10

15 Rank

20

25

30 

2004 DB2 database (d) CMC curves with FVC   

100

CMS

95

FVC2004 DB3 LS G1

90

CS G1 CKS G1 85

80 0

5

10

15 Rank

20

25

30 

(e) CMC curves with FVC 2004 DB3 database   

100

CMS

95

FVC2004 DB4 LS G1

90

CS G1 CKS G1 85

80 0

5

10

15 Rank

20

25

30 

(f ) CMC curves with FVC 2004 DB4 database  

Figure 5.11: CMC curves with one enrolled sample per fingerprint in  LS, CS and CKS for the different databases (contd.). 

185



5. Fingerprint Biometric Data Indexing Effect of multiple samples enrollment: The effects of enrollments of multiple samples per fingerprint into the index database with LS, CS and CKS performances are also studied. The performances are measured on G1, G3 and G5 Gallery sets and P P robe set of each FVC 2004 database (FVC 2004 DB1, FVC 2004 DB2, FVC 2004 DB3 and FVC 2004 DB4). The evaluation results of three searching techniques are presented in the following. a) LS: The results of LS with different number of samples enrollment into the database ae given in Table 5.2. It is observed that rank one HR increases for more number of enrollments. The performance is also analyzed with CM S of LS when one (G1), three (G3) and five (G5) samples per fingerprint are enrolled into the database. Figure 5.12 (a), (b), (c) and (d) show the CMC curves of LS for FVC2004 DB1, DB2, DB3 and DB4 databases, respectively. It is observed that LS gives better result when five samples per fingerprint are enrolled in the database. b) CS: The performance of CS with one (G1), three (G3) and five (G5) samples enrollment per fingerprint into the database is reported in Table 5.3. From Table 5.3, it is observed that CS is also perform better with respect to rank one HR when multiple number of samples are enrolled into the database. For multiple enrollment, P R is Table 5.2: HR, PR and searching time (ST) in LS with the different number of enrollments (Gallery G1, G3 and G5) into the database G1

DB FVC2004 DB1 FVC2004 DB2 FVC2004 DB3 FVC2004 DB4

G3

G5

HR

PR

ST (ms)

HR

PR

ST (ms)

HR

PR

ST (ms)

90.91

100

33.102

92.27

100

109.13

92.73

100

159.045

90.45

100

32.693

91.36

100

77.155

91.82

100

155.873

83.64

100

117.725 84.55

100

356.29

85.91

100

671.011

89.09

100

26.925

100

103.115 90.45

100

170.812

90

186

5.6. Performance Evaluation

 

100

95 CMS

FVC2004 DB1 LS G1

90

LS G3 LS G5 85

80 0

5

10

15 Rank

20

25

30 

(a) CMC  curves of LS with gallery G1, G3 and G5 for FVC 2004 DB1 database





100

CMS

95

FVC2004 DB2 LS G1

90

LS G3 LS G5 85

80 0

5

10

15 Rank

20

25

30 

(b) CMC curves of LS with gallery G1, G3 and G5 for FVC 2004 DB2 database 



100

95 CMS

FVC2004 DB3 LS G1

90

LS G3 LS G5 85

80 0 

5

10

15 Rank

20

25

30 

(c) CMC curves of LS with gallery G1, G3 and G5 for FVC 2004 DB3 database

Figure 5.12: CMC curves of LS with the different number of enrolled samples for the different databases.

187

5. Fingerprint Biometric Data Indexing 

100

CMS

95

FVC2004 DB4 LS G1

90

LS G3 LS G5 85

80 0

5

10

15 Rank

20

25

30 

(d) CMC curves of LS with gallery G1, G3 and G5 for FVC 2004 DB4 database

Figure 5.12: CMC curves of LS with the different number of enrolled samples for different databases (contd.). Table 5.3: HR, P R and searching time (ST ) in CS with the different number of enrollments (gallery G1, G3 and G5) into the database G1

DB FVC2004 DB1 FVC2004 DB2 FVC2004 DB3 FVC2004 DB4

G3

G5

PR

ST (ms)

HR

PR

ST (ms)

HR

PR

ST (ms)

89.09

28.38

10.342

89.55

26.89

20.035

90

25.91

25.594

88.18

32.03

7.852

89.55

28.88

14.396

90.45

27.96

18.69

80

33.44

25.887

80.91

30.74

43.221

82.73

29.52

65.78

85.91

25.56

9.395

87.27

25.52

13.745

88.18

24.14

21.356

HR

increased a little but required searching time is more. The performance of CS is analyzed with CM S for multiple samples enrollment. The CMC curves are obtained in CS with FVC2004 DB1, DB2, DB3 and DB4 databases with multiple samples enrollment are shown in Fig. 5.13 (a), (b), (c) and (d), respectively. It is observed that CS is produced better performance when more number of samples per fingerprint are enrolled in the database. c) CKS: The results of CKS with the different number of samples enrolled into the database are presented in Table 5.4. It can be shown that rank one HR increases for more number of enrollments without 188

5.6. Performance Evaluation 

100

95 CMS

FVC2004 DB1 CS G1

90

CS G3 CS G5 85

80 0

5

10

15 Rank

20

25

30 

(a) CMC curves of CS with gallery G1, G3 and G5 for FVC 2004 DB1 database





100

95 CMS

FVC2004 DB2 CS G1

90

CS G3 CS G5 85

80 0

5

10

15 Rank

20

25

30 

(b) CMC  curves of CS with gallery G1, G3 and G5 for FVC 2004 DB2 database





100

95 CMS

FVC2004 DB3 CS G1

90

CS G3 CS G5 85

80 0 

5

10

15 Rank

20

25

30 

(c) CMC curves of CS with gallery G1, G3 and G5 for FVC 2004 DB3 database

Figure 5.13: CMC curves of CS with the different number of enrolled samples for the different databases.

189

5. Fingerprint Biometric Data Indexing 

100

95 CMS

FVC2004 DB4 CS G1

90

CS G3 CS G5 85

80 0

5

10

15 Rank

20

25

30 

(d) CMC curves of CS with gallery G1, G3  and G5 for FVC 2004 DB4 database

Figure 5.13: CMC curves of CS with the different number of enrolled samples for the different databases (contd.). Table 5.4: HR, P R and searching time (ST ) in CKS with the different number of enrollments (gallery G1, G3 and G5) into the database G1

DB FVC2004 DB1 FVC2004 DB2 FVC2004 DB3 FVC2004 DB4

G3

G5

PR

ST (ms)

HR

PR

ST (ms)

HR

PR

ST (ms)

86.36

11.93

0.426

87.27

12.02

0.706

88.64

12.32

1.041

85.45

13.17

0.485

86.82

13.87

0.55

87.27

14.54

0.768

78.18

14.75

1.57

79.55

15.18

2.121

81.82

15.61

2.885

84.09

10.89

0.349

85

10.88

0.766

85.45

11.29

0.684

HR

affecting the P R. It can be also observed that CKS requires very less time for large number of samples enrollment. The performance of CKS is also substantiated with CM S. The CM S at the different ranks of cluster kd-tree search with multiple samples enrollment per fingerprint into the database is shown in Fig. 5.14. The results are obtained with the G1, G3 and G5 Gallery sets and P P robe set of all FVC 2004 databases. The CMC curves with this approach for FVC2004 DB1, DB2, DB3 and DB4 are shown in Fig. 5.14 (a), (b), (c) and (d), respectively. It is shown that CKS is also performed better when more number of samples per fingerprint are enrolled in the database. 190

5.6. Performance Evaluation

100

95

CMS

FVC2004 DB1 CKS G1

90

CKS G3 CKS G5 85

80 0

5

10

15 Rank

20

25

30 

(a) CMC curves of CKS with G1, G3 and G5 for FVC 2004 DB1





100

95 CMS

FVC2004 DB2 CKS G1

90

CKS G3 CKS G5 85

80 0

5

10

15 Rank

20

25

30 

(b) CMC curves of CKS with G1, G3 and G5 for FVC 2004 DB2





100

CMS

95

FVC2004 DB3 CKS G1

90

CKS G3 CKS G5 85

80 0 

5

10

15 Rank

20

25

30 

(c) CMC curves of CKS with G1, G3 and G5 for FVC 2004 DB3

Figure 5.14: CMC curve of CKS with the different number of enrolled samples for the different databases.

191

5. Fingerprint Biometric Data Indexing 

100

95 CMS

FVC2004 DB4 CKS G1

90

CKS G3 CKS G5 85

80 0

5

10

15 Rank

20

25

30 

(d) CMC curves of CKS with G1, G3 and G5 for FVC 2004 DB4

Figure 5.14: CMC curves of CKS with the different number of enrolled samples for the different databases (contd.).

5.6.4

Searching Time

In this section, the searching time complexity of the proposed three searching techniques are calculated. The execution time and the average number of comparisons required in LS, CS and CKS with different number of enrolled samples for the different databases are also reported. a) LS: Let Tn be the number of index keys from N fingerprints enrolled in the database and each fingerprint has Tp number of index keys on the average. Let Tq be the average number of index keys in the query fingerprint. In LS, Tn × Tq number of comparisons are required to retrieve a set of similar index keys with their identities and Tq log Tq comparisons are required to sort the retrieved identities based on their rank. Thus, the average time complexity of LS (denoted as TLS ) is calculated as in Eq. (5.22). TLS

= Tn × Tq + Tq log Tq = N × Tp × Tq + Tq log Tq = O(N )

(5.22)

√ b) CS: Let Tn index keys are stored in K (K= Tn /2) clusters (rule 192

5.6. Performance Evaluation of thumb of clusters for any unsupervised clustering algorithm [23] for balance between number of clusters and the number of points in √ a single cluster) and each cluster contains Tk (Tk = 2Tn ) number of index keys on the average. Thus, Tq × K number of comparisons are required to find the matched clusters. A set of Tq × Tk similar index keys with their identities are retrieved for a query fingerprint in Algorithm 5.5. The average number of comparisons required to sort the retrieved identities according to their rank is (Tq ×Tk ) log(Tq ×Tk ). Hence, the complexity of the CS can be calculated as in Eq. (5.23). TCS = Tq × K + (Tq × Tk ) log(Tq × Tk ) √ √ √ = Tq × T2n + (Tq × 2Tn ) log(Tq × 2Tn ) √ Tq × N ×Tp √ = [1 + 2 log Tq + log 2 + log(N × Tp )] √ 2 = O( N × log N )

(5.23)

c) CKS: In CKS, Tq ×K number of comparisons are needed to find the matched clusters and Tq log Tk number of comparisons are required to retrieve a set of similar index keys with their identities from the kdtree for a query fingerprint. The CKS retrieves Tq number of index keys with their identities (see Algorithm 5.6). On the average Tq log Tq number of comparisons are needed to sort the retrieved data according to their rank. Therefore, the complexity of the CKS is calculated in Eq. (5.24). TCKS

= Tq × K + Tq log Tk + Tq log Tq √ √ = Tq × T2n + Tq log 2Tn + Tq log Tq [√ ] √ T = √q2 N × Tp + √12 (log 2 + log(N × Tp )) + 2 log Tq √ = O( N ) (5.24)

The execution time (in Intel Core-2 Duo 2.00 GHz processor and 2GB RAM implementation environment) of LS, CS and CKS with different fingerprint databases are shown in Table 5.5. It is observed 193

5. Fingerprint Biometric Data Indexing Table 5.5: Execution time and average number of comparisons required in LS, CS and CKS with the different number of enrolled samples (ES)

DB

ES

Execution time (in ms)

Average number of comparisons

CS

CKS

LS

CS

CKS

110

33.102

10.342

0.426

122600

36937

1938

330

109.130

20.035

0.706

404187

71555

3210

550

159.045

25.594

1.041

722931

98437

4165

110

32.693

7.852

0.485

112734

32717

1732

330

77.155

14.396

0.550

335457

59984

2749

550

155.873

18.690

0.768

577309

81263

3489

110

117.725

25.887

1.570

511847

117669

5232

330

356.290

43.221

2.121

1549088

216107

8483

550

671.011

65.780

2.885

2580813

286002

10687

110

26.925

9.395

0.349

117064

32397

1746

330

103.115

13.745

0.766

343716

59760

2734

550

170.812

21.356

0.684

569373

79097

3420

NIST DB4

2000

325.928

25.455

1.193

1303712

110676

4418

NIST DB4 Natural

1204

213.304

18.411

0.842

790016

83685

3509

FVC2004 DB4

FVC2004 DB3

FVC2004 DB2

FVC2004 DB1

LS

that the execution time for CKS is less than the other two searching methods. It is also observed that there is no significant changes in the execution time of the CKS when the number of enrolled sample increases. On the other hand, the execution time rapidly increases for LS and CS for enrollment of multiple samples. The average number of comparisons required for LS, CS and CKS are shown in Table 5.5. Hence, it may be concluded that CKS requires the least number of comparisons to retrieve and rank the similar identities for a given query. 194

5.7. Comparison with Existing Work

5.6.5

Memory Requirements

In this approach, each value of an index key requires 4 bytes memory. There are 8 key values in the each index key and one identity field is required for each index key. 216 identities can be stored with 2 bytes identity field. Hence, 34 bytes is required to store an index along with identity in linear index space. In clustered index space, the memory is needed to store the cluster centroid additional to the linear index space. Each cluster centroid requires 32 bytes memory and 2 bytes memory for cluster id. On the other hand, in clustered kd-tree index space, it is needed to store the cluster centroid and the kd-trees for all clusters. Similar to the clustered index space, each cluster centroid requires 32 bytes memory and 2 bytes memory for cluster id. A single node of kd-tree requires 46 bytes memory. Table 5.6 shows the memory requirements to store different number of fingerprints for different databases for linear, cluster and clustered kd-tree index spaces.

5.7

Comparison with Existing Work

There are several work [8, 6, 20, 30, 27, 9, 12, 18] for fingerprint identification methods. The most of the existing approaches are tested on the NIST DB4 and NIST DB4 Natural database and reported the results with error rate (ERR) at different number of top matches. To compare the proposed work with existing work, ERR is computed as 100 − HR at different ranks. Here, HR denotes hit rate and defines in Eq. (4.19). Figure 5.15 shows the comparison of ERR at different ranks of three proposed techniques with existing work. Figure 5.15(a) and (b) report the comparison on NIST DB4 database and Fig. 5.16(a) and (b) report the comparison on NIST DB4 Natural database. From Fig. 5.15(a), it is evident that all three proposed techniques outperform other existing approaches reported (on NIST DB4 database at all 195

5. Fingerprint Biometric Data Indexing Table 5.6: Memory requirements (in KB) of linear, cluster and clustered kd-tree index spaces

Clustered kd-tree

Cluster

Linear

Index space

#Fp in DB

FVC2004 FVC2004 FVC2004 FVC2004 DB1 DB2 DB3 DB4

NIST DB4

NIST DB4 Nl

110

108.18

113.12

233.95

120.96

92.7

93.27

330

357.03

336.95

708.46

355.47

278.08

279.84

550

638.73

579.99

1180.44

588.96

463.48

466.4

1204

1014.62

1021.16

2000

1685.26

110

109.50

114.48

235.91

122.39

93.93

94.50

330

359.46

339.30

711.88

357.9

280.23

281.99

550

641.98

583.08

1184.85

592.08

466.27

469.19

1204

1018.74

1025.28

2000

1690.54

110

134.96

141.10

290.96

150.85

115.74

116.44

330

443.46

550

792.27

418.58

878.57

441.54

345.66

347.84

719.55

1462.60

730.66

575.33

578.94

1204

1257.47

1265.55

2000

2087.07

penetration rates). Figure 5.15(b) shows that Cappelli et al. (2011) [12] indexing performs better than the proposed indexing techniques at low rank (with respect to the result with NIST DB4 database). However, the proposed indexing methods perform better at a little higher rank (top 11 rank). From Fig. 5.16(a), it is clear that the proposed approaches outperform other reported work; although Fig. 5.16(b) shows that Cappelli (2011) [9] and Cappelli et al. (2011) [12] methods perform better than the proposed methods at lower rank for NIST DB4 Natural database. However, the proposed approaches are comparable with Cappelli (2011) [9] and Cappelli et al. (2011) [12] methods at a little higher rank (top 13 rank). Ross and Mukherjee (2007) [27], Liang et al. (2007) [19] and Zhang et al. (2008) [30] publish the results on FVC 2004 DB1 database. Liang et al. (2007) [19] also publish the result using Bhanu and Tan 196

    with Existing Work 5.7. Comparison 

25 Bhanu and Tan (2003) Gyaourova and Ross (2008) Proposed LS Proposed CKS

20

Germain et al. (1997) Liu et al. (2007) Proposed CS

ERR

15

10

5



0 0



10

20

30

Rank

   



(a) NIST DB4 

25

ERR

20

Cappelli (2011)

Cappelli et al. (2011)

Jiang et al. (2006)

Proposed LS

Proposed CS

Proposed CKS

15

10

5

0 0

10

20 Rank

30 

(b) NIST DB4

Figure 5.15: Comparison with existing work for NIST DB4 database.

(2003) [8] and Bebis et al. (1999) [6] method for FVC 2004 DB1 database. Zhang et al. (2008) [30] also report the result of Liu et al. (2007) [21] method on FVC 2004 DB1 database. The proposed work is compared with these existing works. Figure 5.17 shows the comparison of ERR at different ranks of three proposed techniques with existing work. From Fig. 5.17 it is clearly evident that the proposed approaches outperform the approaches in [8, 6, 20, 30, 27] method. However, Liang et al. (2007) [19] method is performed better at rank but the proposed methods perform better from at a little higher rank (top 13 rank). 197

 

5. Fingerprint Biometric Data Indexing

 

25

ERR

20

Cappelli et al. (1999)

Jiang et al. (2006)

Lee et al. (2005)

Li et al. (2006)

Lumini et al. (1997)

Proposed LS

Proposed CS

Proposed CKS

15

10

5 

0



0

10



20

30

Rank







(a) NIST DB4 Natural



25 Cappelli (2011) Liu et al. (2006) Proposed LS Proposed CKS

ERR

20

Cappelli et al. (2011) Liu et al. (2007) Proposed CS

15

10

5

0 0

10

20

30

Rank



(b) NIST DB4 Natural   Figure 5.16: Comparison with existing work for NIST DB4 Natural database. 

25

20

Ross and Mukherjee (2007)

Linag et al. (2007)

Bhanu and Tan (2003)*

Bebis et al. (1999)*

Zhang et al. (2008)

Liu et al. (2007)**

Proposed LS Proposed CKS

ERR

15

Proposed CS *result published in Liang et al. (2007) **result published in Zhang et al. (2008)

10

5

0 0

10

20

30 Rank



Figure 5.17: database.

40

50 

Comparison with existing work for FVC2004 DB1

198

5.8. Summary Table 5.7: Comparison of search complexity and execution time of the existing work Search complexity

Search time (ms)

Implementation environment

FP in DB

Bhanu and Tan [8]

O(N)

1000

Sun Ultra 2

2000

Cappelli [9]

O(N + NT logNT )

1.6

Intel Core 2 Quad 2.66 GHz

2000

Cappelli et al. [10]

O(N)

900

Pentium 200MHz

1204

Cappelli et al. [12]

O(|V | × L × bs )

16

Intel Quad Core, 2.66 GHz

2700

Jiang et al. [18]

O(N)

67

Intel Pentium 4, 2.26 GHz

2000

Proposed LS

O(N)

326

Intel Core 2 Duo, 2.00 GHz

2000

Proposed CS

√ O( N × logN )

25

Intel Core 2 Duo, 2.00 GHz

2000

√ O( N )

1

Intel Core 2 Duo, 2.00 GHz

2000

Methods

Proposed CKS

The proposed approach is also compared with the existing approaches with respect to searching complexity and the execution time. The result is reported in Table 5.7. From the reported result it is observed that proposed clustered kd-tree search executes faster than the all other approaches.

5.8

Summary

A new set of eight-dimensional feature vectors for fingerprint indexing has been proposed in this work. The eight-dimensional rotation and scale invariant feature vectors are calculated from the local topology of the minutiae triplets which are generated by two closest point triangulation of minutiae points. The invariance properties of the feature vector make the fingerprint indexing technique accurate for the different fingerprints captured from the different devices and sensors. 199

5. Fingerprint Biometric Data Indexing Three different searching techniques, LS, CS and CKS are explored with three different index spaces. It has been observed that LS technique produces the highest HR but this technique is computationally expensive than the other two approaches. On the other hand, CKS perform faster retrieval than the other two searching techniques though the HR is little lower than others. The CS gives moderate hit rate and also takes moderate time for retrieval. The results show that CS approach have good balance between response time and accuracy. Further, it has been observed that enrollment of multiple samples of a subject improve the performance of indexing system. Finally, the comparison with the existing indexing techniques shows that the performance of the proposed approach is better with respect to accuracy and the searching time.

200

BIBLIOGRAPHY

Bibliography [1] FLANN - Fast Library for Approximate Nearest Neighbors.

URL http://www.cs.ubc.ca/~mariusm/index.php/

FLANN/FLANN. (Accessed on May, 2012). [2] FVC2004.

FVC2004:

Verification Competition.

the Third International Fingerprint URL http://bias.csr.unibo.it/

fvc2004/databases.asp. (Accessed on December, 2011). [3] SFinGe. Synthetic Fingerprint Generator. URL http://bias. csr.unibo.it/research/biolab/sfinge.html.

(Accessed on

January, 2012). [4] NIST Special Database 4. NIST 8-Bit Gray Scale Images of Fingerprint Image Groups(FIGS), 1992. URL http://www.nist. gov/srd/nistsd4.cfm. (Accessed on January, 2012). [5] S. Arya and D. M. Mount. Algorithms for Fast Vector Quantization. In Proceedings of DCC ’93: Data Compression Conference, pages 381–390, Snowbird, UT, 1993. [6] G. Bebis, T. Deaconu, and M. Georgiopoulos. Fingerprint Identification using Delaunay Triangulation. In Proceedings of the IEEE International Conference on Intelligence, Information, and Systems, pages 452–459, Bethesda, USA, October-November 1999. doi: http://dx.doi.org/10.1109/ICIIS.1999.810315. [7] J. L. Bentley. Multidimensional Binary Search Trees Used for Associative Searching. Communications of the ACM, 18(9):509–517, 1975. ISSN 0001-0782. doi: http://dx.doi.org/10.1145/361002. 361007. [8] B. Bhanu and X. Tan. Fingerprint Indexing Bsed on Novel Features of Minutiae Triplets. IEEE Transactions on Pattern Anal201

5. Fingerprint Biometric Data Indexing ysis and Machine Intelligence, 25(5):616–622, 2003. ISSN 01628828. [9] R. Cappelli. Fast and Accurate Fingerprint Indexing Based on Ridge Orientation and Frequency. IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics, 41(6):1511– 1521, 2011. [10] R. Cappelli, A. Lumini, D. Maio, and D. Maltoni. Fingerprint Classification by Directional Image Partitioning. IEEE Transactions on Pattern Analysis and Machine Intelligence, 21(5):402– 421, 1999. [11] R. Cappelli, D. Maio, and D. Maltoni. Synthetic FingerprintDatabase Generation. In Proceedings of the 16th International Conference on Pattern Recognition, volume 3, pages 744–747, QC, Canada, August 2002. [12] R. Cappelli, M. Ferrara, and D. Maltoni. Fingerprint Indexing Based on Minutia Cylinder-Code. IEEE Transactions on Pattern Analysis and Machine Intelligence, 33(5):1051–1057, 2011. [13] A. Chatterjee, S. Mandal, G. M. A. Rahaman, and A. S. M. Arif. Fingerprint Identification and Verification System by Minutiae Extraction using Artificial Neural Network. International Journal of Computer and Information Technology (IJCIT), 1(1):12–16, 2010. [14] R. C. Gonzalez and R. E. Woods. Digital Image Processing. Prentice Hall, 2nd edition, 2002. [15] Z. Guo and R. W. Hall. Parallel Thinning with Two-subiteration Algorithms. Communications of the ACM, 32(3):359–373, 1989. [16] L. Hong, Y. Wan, and A. Jain. Fingerprint Image Enhancement: Algorithm and Performance Evaluation. IEEE Transactions on 202

BIBLIOGRAPHY Pattern Analysis and Machine Intelligence, 20(8):777–789, 1998. ISSN 0162-8828. [17] A. K. Jain, A. Ross, and S. Prabhakar. An Introduction to Biometric Recognition. IEEE Transactions on Circuits and Systems for Video Technology, 14(1):4–20, 2004. [18] X. Jiang, M. Liu, and A. C. Kot. Fingerprint Retrieval for Identification. IEEE Transactions on Information Forensics and Security, 1(4):532–542, 2006. [19] X. Liang, A. Bishnu, and T. Asano. A Robust Fingerprint Indexing Scheme using Minutia Neighborhood Structure and LowOrder Delaunay Triangles. IEEE Transactions on Information Forensics and Security, 2(4):721–733, 2007. [20] M. Liu, X. Jiang, and A. C. Kot. Fingerprint Retrieval by Complex Filter Responses. In Proceedings of the 18th International Conference on Pattern Recognition, volume 1, pages 1042–1045, Hong Kong, China, August 2006. [21] M. Liu, X. Jiang, and A.C. Kot. Efficient Fingerprint Search Based on Database Clustering. Pattern Recognition, 40(6):1793– 1803, June 2007. [22] D. Maio, D. Maltoni, R. Cappelli, J. L. Wayman, and A. K. Jain. FVC2004: Third Fingerprint Verification Competition. In Proceedings of the International Conference on Biometric Authentication, pages 1–7, Hong Kong, China, July 2004. [23] K. V. Mardia, J. T. Kent, and J. M. Bibby. Multivariate Analysis. Academic Press, 1980. [24] R. M. McCabe and E. M. Newton.

Information Technology:

American National Standard for Information Systems - Data For203

5. Fingerprint Biometric Data Indexing mat for the Interchange of Fingerprint, Facial, & Other Biometric Information - Part 1. ANSI/NIST-ITL 1-2007 Revision of ANSI/NIST-ITL 1-2000, 2007. [25] B. M. Mehtre. Fingerprint Image Analysis for Automatic Identification. Machine Vision and Applications, 6(2-3):124–139, 1993. doi: 10.1007/BF01211936. [26] M. Muja and D. G. Lowe. Fast Approximate Nearest Neighbors with Automatic Algorithm Configuration. In Proceedings of the International Conference on Computer Vision Theory and Applications (VISAPP’09), pages 331–340, 2009. [27] A. Ross and R. Mukherjee. Augmenting Ridge Curves with Minutiae Triplets for Fingerprint Indexing. In Proceedings of the SPIE Conference on Biometric Technology for Human Identification IV, volume 6539, pages 65390C–65390C–12, Orlando, USA, April 2007. [28] R. Thai.

Fingerprint Image Enhancement and Minutiae Ex-

traction.

Honours Programme Report, School of Computer

Science and Software Engineering, The University of Western Australia, 2003. URL http://www.csse.uwa.edu.au/~pk/ studentprojects/raymondthai/RaymondThai.pdf.

(Accessed

on June, 2011). [29] C. J. Watson and C. L. Wilson. NIST Special Database 4, Fingerprint Database. Technical report, National Institute of Standards and Technology, 1992. [30] Y. Zhang, J. Tian, K. Cao, P. Li, and X. Yang. Improving Efficiency of Fingerprint Matching by Minutiae Indexing. In Proceedings of the 19th International Conference on Pattern Recognition, pages 1–4, Tampa, FL, December 2008. 204

Chapter 6

Face Biometric Data Indexing Face-based biometric identification system with a large pool of data requires huge computation time to search an individual’s identity from database due to the high dimensionality of the face biometric features. In this work, a new indexing technique is proposed for face-based biometric identification system to narrow down the search space. The technique is based on the Speed Up Robust Feature (SURF) key points and descriptors [10]. Index keys are generated for a face image from the key points and descriptors of the face image. A two level index space is created based on the extracted key points. A storing and retrieving techniques have been proposed to store and retrieve the face data to or from the proposed index space. Figure 6.1 gives an overview of the proposed approach. The rest of the chapter is organized as follows. Different steps of preprocessing technique for face image will be outlined in Section 6.1. The feature extraction methodology for face will be discussed in Section 6.2. Index key generation method for face biometric will be presented in Section 6.3. The storing and retrieving techniques for face 205

6. Face Biometric Data Indexing Enrollment process Face image

Preprocessing

Feature extraction

Index key generation

Storing

Database

Query face image

Preprocessing

Feature extraction

Index key generation

Retrieving

Top matches

Query retrieving process

Figure 6.1: An overview of the proposed face biometric data indexing approach.

data shall be given in Section 6.4 and Section 6.5, respectively. Performance of the proposed method with different face databases will be analyzed in Section 6.6. Section 6.7 will present the comparison with existing work. The chapter will be concluded in Section 6.8.

6.1

Preprocessing

Input face image of a biometric system contains background and is usually of not good quality. To extract the features from a face image, the background is needed to remove and the image quality should be enhanced. This makes feature extraction task easy and also ensures the quality of the extracted features. Some essential preprocessing tasks is required to follow for the above mentioned reason. The steps followed in the preprocessing are described in the following.

6.1.1

Geometric Normalization

All input face images may not be of same size and aligned in the same direction (e.g. due to the movement of head at the time of capturing). Geometric normalization process proposed by Blome et al. [13] and Beveridge et al. [12] is followed to align and scale the images so that the face images are in the same position, same size and 206

6.1. Preprocessing at the same orientation. To get geometric normalized image, first, the eye coordinates are searched from the face image using Blome et al. [14, 15] method. Let (XL , YL ) and (XR , YR ) be the detected left and right eye coordinates, respectively.The angle (θ) between horizontal axis and the line passing through the two eye coordinates is calculated as shown in Fig. 6.2(a). The angle θ is calculated using Eq. (6.1). θ = tan−1 (

YR − YL ) XR − XL

(6.1)

To lines up the eye coordinates, the image is rotated by −θ angle. Let I(x′ , y ′ ) be transformed image of I(x, y) after rotation by −θ angle. The rotated image is shown in Fig. 6.2(b). Transformation relation between original image pixel and resultant image pixel is given in Eq. (6.2). [

x′ y′

]

[ =

cos(−θ) − sin(−θ) sin(−θ)

cos(−θ)

]

[ ×

x y

] (6.2)

After rotating the face image, the face part is detected from the rotated image. To do this, Viola and Jones’s [30] face detection algorithm is used. A snapshot of the detected face is shown in Fig. 6.2(c). To make the face image scale invariant, the detected face part (DW × DH) is mapped into a fixed size image (F W ×F H) by applying scaling transformation. The scaling factor s is calculated using Eq. (6.3). s=

FW DW

(6.3)

In this approach, F H = 150 and F W = 130 are considered, because, in general, aspect ratio of a face image would be 1.154 [13]. The detected image is wrapped into fixed size image by applying translation and followed by scaling. Let I(x, y) and I(x′′ , y ′′ ) be the detected and fixed size face image. The transformation is done using Eq. (6.4). In Eq. (6.4), (tx , ty ) is the top left coordinate of the detected face im207

6. Face Biometric Data Indexing age (see Fig. 6.2(c)). Figure 6.2(d) shows the fixed image of width F W and height F H. [

x′′

]

y ′′

6.1.2

[ =

s 0 0 s

]

([ ×

x

]

y

[ −

tx

]) (6.4)

ty

Face Masking

Masking is the process of separating the foreground region from the background region in the image. The foreground region correspond to the clear face area containing the significant feature value, which is called the area of interest. The masking is done to ensure that the face recognition system does not respond to features corresponding to

(tx, ty) (XL, YL)

DH

(XR, YR)

DW

(a) Face image before align

(b) Face image after align

(c) Detected face boundary

FH

FW

(d) Fixed face image

size

(e) Masked face image

(f ) Enhanced face image

Figure 6.2: Face images after different preprocessing tasks.

208

6.1. Preprocessing background, hair, clothing etc. An elliptical mask [13, 12] is used such that only the face from forehead to chin and cheek to cheek is visible. Let (xc , yc ) is the center of the elliptical mask which is just below the eye level. The following mathematical model [13] is applied to each face image to mask the background region. Let I(x, y) be the detected face image obtained after geometric normalization. The masked image I ′ (x, y) is calculated as per the following rule.

′

I (x, y) =

   I(x, y) if  

(x−xc )2 a2

+

(y−yc )2 b2

−1≤0

otherwise

0

where, a and b are minor axis and major axis of elliptical shape, respectively. Figure 6.2(e) shows the face image after applying mask on geometric normalized image where only face part is present in the image and the background has been masked out.

6.1.3

Intensity Enhancement

Intensity enhancement is required to reduce the variations due to lighting and sensor differences. The intensity enhancement is done in two steps [13]. First, the histogram of the unmasked face part is equalized and then the intensity of the image is normalized to a mean of zero and standard deviation of one. To equalize the histogram of an image, the cumulative distribution (cdf ) of the histogram is calculated. The cdf at each gray level v represents the sum of its occurrence and the occurrence of the previous gray level in image. Then the histogram equalization is done by Eq. (6.5). ( h(v) = round

cdf (v) − cdfmin (F W × F H) − cdfmin 209

) × (L − 1)

(6.5)

6. Face Biometric Data Indexing where cdfmin is the minimum value of the cumulative distribution function, F W and F H are the width and height of image, and L is the number of gray levels used. Next, the intensity values of the equalized face image is normalized. Intensity normalization is used to standardize the intensity values in an image by adjusting the range of gray level values so that it lies within a desired range of values. Let I(i, j) represents the gray level value at pixel (i, j) of histogram equalized image and N (i, j) represents the gray level value at pixel (i, j) in normalized image. The normalization is done using Eq. (6.6).  √ )2   M0 + V0 ×(I(i,j)−M V N (i, j) = √   M − V0 ×(I(i,j)−M )2 0 V

if I(i, j) > M (6.6) otherwise

where M and V are the estimated mean and variance of I(i, j), respectively, and M0 and V0 are the desired mean and variance values, respectively. The pixel values is selected to have a mean of zero and a standard deviation of one in the experiment. Figure 6.2(f) shows the intensity enhancement image of the geometric normalization image.

6.2

Feature Extraction

Speed Up Robust Feature (SURF) [9, 10] method is known as a good image interest points (also called key points) and feature descriptors detector. SURF method is applied in this approach. This method is used because it has several advantages over other feature extraction methods. The most important property of an interest point detector using SURF method is its repeatability. Repeatability means that the same interest points will be detected in different scales, orientations and illuminations of a given image. Another advantage is that the 210

6.2. Feature Extraction SURF method is computationally very fast. In addition to these, the SURF feature provides scale, rotation and illumination invariant feature descriptors. The feature extraction method consists of three different tasks: key point detection, orientation assignment and key point descriptor extraction. In this section, first the different steps of the key point detection method is presented. The orientation assignment of each key point is discussed next. Then the method of feature descriptor extraction at each key point is described.

6.2.1

Key Point Detection

Bay et al. [9, 10] method is followed to detect key points from a face image. The method is based on Hessian matrix approximation [10] which uses integral images to reduces the computation time of key points detection. The method consists of three steps. First, the scale space of the preprocessed image is created, which helps to detect key points at different scales of the image. Next, Hessian matrix [23, 10] is calculated at each pixel position in the different scale space images and the value of the determinant of the Hessian matrix is computed. Note that all detected key points are not necessarily discriminant because the determinant of the Hessian matrix does not produce local maximum or minimum response at all detected points. Hence, the most discriminant key points are localized from all detected key points. In the following, the detailed descriptions of the above mentioned steps are presented. 6.2.1.1

Scale Space Creation

Scale space of an image is represented by an image pyramid as shown in Fig. 6.3(a). To create an image pyramid, a fixed size Gaussian filter is applied to an image and then sub sampled filtered image [19]. This process should be done repeatedly to get the higher layered images of a 211

6. Face Biometric Data Indexing scale space. Since, the repeated sub sampling of image demands more computation time, this is done in a different way. A Gaussian filter is applied into the image keeping the image size fixed but increasing the filter size (see Fig. 6.3(b)). This gives the same result of decreasing the size of image and finds Gaussian convolution using same size of filter and then scale up the image due to the cascading properties of Gaussian filter [19, 10]. Scale space is made of number of octaves; where an octave represents a series of filter responses obtained by convolving the same input image with doubling the filter size. Again, an octave is divided into a constant number of scale levels. To create scale space Bay et al. [10] method is followed. Eight distinct Gaussian filters are constructed with different sizes and different standard deviations. The Gaussian filter of size m × m and standard deviation σ is represented as Gm×m (σm ). The eight Gaussian filters are G9×9 (σ9 ), G15×15 (σ15 ), G21×21 (σ21 ), G27×27 (σ27 ), G39×39 (σ39 ), G51×51 (σ51 ), G75×75 (σ75 ) and G99×99 (σ99 ) (for details please see the reference [10]). Base level of scale space is created with filter size 9 × 9 and standard deviation (σ) 1.2 (also referred as scale s=1.2). Next, scale spaces are obtained by changing the size of filter. As the filter size is changed to obtain subsequent scale space the associate Gaussian scale is needed to change as

Low

Image size

Filter size

High

Fixed size filter

High

Fixed size image

Low (b) Scale space creation by increasing filter size

(a) Scale space creation by reducing image size

Figure 6.3: Approaches to scale space creation.

212

6.2. Feature Extraction well so that ratios of layout remain constant. This standard deviation (scale) of current filter can be calculated using Eq. (6.7). σc = σb ×

Sizec Sizeb

(6.7)

In Eq. (6.7), σc and σb be the standard deviation of current filter and base filter, respectively, and Sizec and Sizeb denote the current filter size and base filter size, respectively. 6.2.1.2

Hessian Matrix Creation

SURF detector [10] is based on determinant of Hessian matrix [23, 10]. Blob-like structures which differ in properties like brightness or color compared to the surrounding are detected at locations where the determinant of Hessian matrix is maximum. Let f (x, y) be a function of two variables, then Hessian matrix H is defined as in Eq. (6.8). [ H(f (x, y)) =

∂2f ∂x ∂2f ∂x∂y

∂2f ∂x∂y ∂2f ∂y

] (6.8)

Hence, determinant of above matrix will be as follow: det(H(f(x, y))) =

∂2f ∂2f − ∂x ∂y

(

∂2f ∂x∂y

)2 (6.9)

The maxima and minima can be computed from the above determinant formula. This will help to classify the points into maxima and minima based on the sign of result. Let x (x, y) be a point in an image I(x, y). Then Hessian matrix is calculated as follows. [ H(x, σm ) =

Lxx (x, σm ) Lxy (x, σm ) Lxy (x, σm ) Lyy (x, σm )

∂ 2 Gm×m (σm ) , ∂x2 ∂ 2 Gm×m (σm ) . Further, ∂x∂y

Here, Lxx (x, σm ) = Lxy (x, σm ) =

213

Lyy (x, σm ) =

] (6.10) ∂ 2 Gm×m (σm ) ∂y 2

and

Lxx (x, σm ) represents the con-

6. Face Biometric Data Indexing volution of the Gaussian second order derivative of Gaussian filter at scale σm with the image I(x, y) at point x; similarly the Lxy (x, σm ) and Lyy (x, σm ). These entire derivatives are known as Laplacian of Gaussian [19]. The determinant of each pixel is calculated to find the interest points. To reduce the computation time of calculating Laplacian of Gaussian, Bay et al. approximation method [10] is followed. The approximation of the Laplacian of Gaussian is done using box filter representations of the respective Gaussian filters. Figure 6.4 shows the weighted box filter approximations of discretized and cropped second order Gaussian derivatives in the x, y and xy-directions. Suppose, Dxx , Dyy and Dxy represent the weighted box filter response [10] of the second order Gaussian derivatives Lxx , Lyy and Lxy , respectively. Then Hessian determinant with box filter approximation is calculated using Eq. (6.11) [10]. det(Happrox ) = Dxx Dyy − (0.9Dxy )2

(6.11)

The above determinant value is referred to as the blob response at location x = (x, y) with scale σm . This method of Hessian matrix creation is applied for each point of each scale space for a face image. 6.2.1.3

Key Point Localization

Key points are detected based on the determinant value of the Hessian matrix. As the high determinant value at a point represents more discriminant [9] point, those points are considered where the determinant values are high. For this reason, first, a threshold value is chosen and the points whose values are less than the threshold are removed. Bay et al. [9] show that the threshold value 600 is good for detecting the discriminant key point from an image with average contrast and sharpness. In this experiment, the threshold value is chosen as 600. Next, non-maximal suppression [19] is performed to find the candidate key 214

6.2. Feature Extraction

(a) Derivative in x- direction (Lxx )

(b) Derivative in y- direction (Lyy )

(c) Derivative in xy- direction (Lxy )

(d) Approximated derivative in x- direction (Dxx )

(e) Approximated derivative in y- direction (Dyy )

(f ) Approximated derivative in xy- direction (Dxy )

Figure 6.4: Gaussian partial 2nd order derivative and their approximation.

points. To do this, each point of scale space is compared with its 26 neighbors (see Fig. 6.5) and the local maxima is found. Eight points are considered from the scale space at which non-maximal suppression is done and 9 points from both above and below the scale space of that. Finally, to localize key points the maxima of the determinant of the Hessian matrix is interpolated in scale and image space. To do this, the method proposed by Brown and Lowe [16] is used.

6.2.2

Orientation Assignment

An orientation is assigned to each key point to extract rotation invariant features from the input face image. The orientation is important 215

Scale

6. Face Biometric Data Indexing

Figure 6.5: Non-maximal suppression by checking the nearest neighbor in subsequent scale spaces.

(a) Haar filter in x-direction

(b) Haar filter in y-direction

(c) Orientation assignment to a key point

Figure 6.6: Haar filters in x- and y- directions and orientation of a key point.

because the feature descriptors are extracted relative to this orientation in later stage. To find the orientation of a key point, first, a circular area centered with the key point of radius 6σ is created where σ is the standard deviation of the Gaussian filter [10] at the current scale space. Then, the Haar wavelet response [20] is calculated at each point within the circular area in x and y directions. Haar filter of size 4σ is used. The Haar filter responses in x and y directions are calculated using two filters as shown in Fig. 6.6(a) and (b). Then, weighted response of Haar wavelet response is calculated with Gaussian filter. The Gaussian filter is centered with interest point and the value σ of Gaussian is taken as 2. The weighted responses are represented by points in vector space as shown in Fig. 6.6(c). To find dominating orientation, the resultant vector in window of size 60 degree is computed 216

6.2. Feature Extraction dx |dx| dy |dy|

5×5 sample points

dy dx

Figure 6.7: SURF descriptor extraction at a key point.

as shown in Fig. 6.6(c). The longest vector is taken as the orientation of that key point. A set of key points is detected from an image and orientation of each key point is estimated. A key point can be represented with position, orientation, scale space in which the key point is detected, Laplacian value and the determinant of Hessian matrix. Let k 1 , k 2 , . . ., k L be the L detected key points of an input image. The key points of an image is represented as shown in Eq. (6.12). k1

= x1

y1

θ1

σ ls1

hs1

k2

= x2

y2

θ2

σ ls2

hs2

...

...

kL

= xL y L

(6.12)

θL σ lsL hsL

In Eq. (6.12), (x, y) and θ represent the position and orientation of a key point, respectively; σ (σ ∈ σ9 , σ15 , σ21 , σ27 , σ39 , σ51 , σ75 , σ99 ) denotes scale space at which key point is detected (see Section 6.2.1.1); ls and hs represent the Laplacian value and determinant of Hessian matrix, respectively.

6.2.3

Key Point Descriptor Extraction

In this step, the feature descriptors are extracted at each key point from the scale space images as follows. Scale space images are cre217

6. Face Biometric Data Indexing ated by applying Gaussian filter on the images (as discussed in Section 6.2.1.1). SURF method [10] is followed to extract the feature descriptors from the face image. First, a square window of size 20σ is created where σ is the scale or standard deviation of the Gaussian filter at which key point is detected. The window is centered at key point position and the direction of window is the same with the orientation of the key point (see Fig. 6.7). Now, the window is divided into 4×4 square sub regions and within each sub-region 25 (5×5) regularly distributed sample points are placed. Haar wavelet responses [20] is calculated at each sample point of a sub-region in x and y directions. The Haar wavelet responses are weighted with a Gaussian filter with standard deviation 3.3σ centered at key point to reduce the effect of geometric deformations and localization errors. Let dx and dy be the Haar wavelet responses at each sample point within each sub-region. ∑ ∑ ∑ ∑ dx, |dx|, dy and |dy| are considered as four features at each sub-region. Hence, sixty four (4 × 4 × 4) descriptors are created corresponding to each key point. The extracted feature descriptors from a face image is shown in Eq. (6.13). In Eq. (6.13), d1 , d2 , . . . , di , . . . , dT represent descriptors of T key points and f dij represents the j th descriptor of the ith key point. d1

= f d11

f d12

f d13

. . . f d1j

. . . f d164

d2

= f d21

f d22

f d23

. . . f d2j

. . . f d264

f di3

. . . f dij

. . . f di64

f dT3

. . . f dTj

. . . f dT64

... di

... =

f di1

=

f dT1

... dT

6.3

f di2

(6.13)

... f dT2

Index Key Generation

All key points and feature descriptors are extracted from all face images. A face image can be represented with a set of index keys. The 218

6.3. Index Key Generation set of index keys are generated from the extracted key points of a face image such that for each key point there is an index key. More precisely, the key point information, feature descriptors and identity of a person are used as the constituents of an index key. An index key is represented as a row vector of sixty nine elements. The first four values of an index key contain the sign of Laplacian, position and orientation of a key point. Next sixty four values of the index key hold the feature descriptors corresponding to the key point and the last value keeps the identity of a person. The first four values are used to index the database and the feature descriptors are used to search the identity of a person. Suppose, there are P number of subjects and each has Q number of samples. Index keys for all samples of all subjects are T

1 , . . . , k p,q ) and generated. Let Tp,q be the number of key points (kp,q p,q T

p,q feature descriptors (d1p,q , . . . , dp,q ) extracted from the face image of

the q th sample of the pth subject. Note that Tp,q may vary from one face image to another. Thus, Tp,q number of index keys are generated for the face image the q th sample of the pth subject. The index keys of the q th sample of the pth subject is presented in Eq. (6.14). The ith index key (indxip,q ) of the q th sample of the pth subject is generi ) and corresponding feature descriptors ated by the ith key point (kp,q

(dip,q ), and the identity (Idpq ) of the q th sample of the pth subject. In i , θ i and di represent the sign of Laplacian, Eq. (6.14), lsip,q , xip,q , yp,q p,q p,q

x and y positions, orientation and feature descriptors of the ith key i ), and Idp represents the identity of the face image of the point (kp,q q

q th sample of the pth subject. indx1p,q

= ls1p,q

x1p,q

1 yp,q

1 θp,q

d1p,q

Idpq

indx2p,q

= ls2p,q

x2p,q

2 yp,q

2 θp,q

d2p,q

Idpq

i yp,q

i θp,q

dip,q

Idpq

T

θp,qp,q

T

Idpq

... indxip,q

... =

lsip,q

=

Tp,q lsp,q

... Tp,q indxp,q

xip,q ... T

p,q xp,q

yp,qp,q 219

T

p,q dp,q

(6.14)

6. Face Biometric Data Indexing

6.4

Storing

Once all index keys are generated, the face data are to be stored into the database. To do this, first, a two-level index space is created in the database. Then the face data are stored into the index space in two ways: linear structure and kd-tree structure. The techniques of index space creation and storing are described in the following.

6.4.1

Index Space Creation

An index space needs to be created to store all index keys into the database. The created index space helps to find a match corresponding to a query in fast and accurate manner. To create index space, first four components of an index key are used. These are the sign of Laplacian (ls), positions (x and y) and orientation (θ) of a key point. All index keys can be classified into two groups based on the sign of the Laplacian value (ls) because this value distinguishes the brightness (dark and light) at a key point position. Note that all face images are aligned in the same direction and scaled to the same size in the preprocessing step. Hence, a key point will occur at the same or near to the same position in the image and the orientation of the key point will remain almost same although, the face images are captured at different time. So, the index keys in each group can be divided into sub-groups based on the positions (x and y) and orientation (θ) of a key point. Due to the above characteristics of key points a two-level index space is proposed to store the index keys. In the first level, the index space is divided based on the value of ls of index keys. The value of ls can be either ‘−1’ for low intensity value or ‘+1’ for high intensity value for a key point. Hence, the first level index space (LS) is divided into two sub-index spaces (LS1 and LS2 ) as shown in Fig. 6.8. In the second level, each sub-index space is divided into a number of cells 220

6.4. Storing 1 LS1

3 2 ! 1

1

1 2 3 y

x

2

2 LS2 LS

First level index space

3 2 ! 1

3

1

1 2 3

INDX

y

x

3 2

INDX

Second level index space

Figure 6.8: Proposed index space to store all index keys of all face images.

based on the positions (x and y) and orientation (θ) of key points. The second level index space (IN DX) is represented as a three dimensional index space. The three dimensions of index space are x, y and θ. Each dimension is in different scales. To bring each dimension in the same scale, each dimension is normalized. To do this, each dimension of the second level index space is quantized into the same number of units. Each dimension is quantized into δ number of units. The value of δ is decided experimentally (discussed in Section 6.6.3). Each three dimensional index space in the second level is refered as an index cube. Each index cube contains δ 3 number of cells. Figure 6.8 shows two three-dimensional index cubes for storing the index keys. Now, all index keys are stored into the index space based on the first four values of the index keys. Note that a number of index keys may map into a single cell of an index cube because index values of a set of index keys may fall within the same range. The cell positions are found for all index keys. To do this a set of hash functions are defined based on the sign value of Laplacian, positions and orientation 221

6. Face Biometric Data Indexing of a key point. Let ls, (x, y) and θ be the sign value of Laplacian, positions and orientation of a key point, respectively. Then the hash functions are defined in Eq. (6.15). In Eq. (6.15), ls′ , x′ , y ′ , and θ′ are the normalized cell index of the two-level index space and F H and F W represent the height and width of the normalized face image, respectively. ls′ = Hls (ls), where Hls (ls) = x′

= Hx (x),

where Hx (x)

=

y′

= Hy (y),

where Hy (y)

=

θ′

= Hθ (θ),

where Hθ (θ)

=

ls+3 2 ⌈ (x+1)×δ FW ⌉ (y+1)×δ ⌈ FH ⌉ ⌈ (θ+1)×δ 360 ⌉

(6.15)

The storing of an index key into the proposed two-level index space is illustrated with an example. Let indx =< ls = −1, x = 55, y = 89, θ = 45, f d1 , f d2 , . . . , f d64 , Id > be an index key of a face image of size 130 × 150 (F W × F H) , f d1 to f d64 are the 64 feature descriptors of that key point and Id be the identity of the subject. To store the feature descriptors (f d1 , f d2 , . . . , f d64 ) and the identity (Id) into the index space, the hash functions (defined in Eq. (6.15)) are applied on ls, x, y and θ of the key point. The feature descriptors will be stored into the first index cube (LS1 ) of the first level of index space because the value of ls′ is 1 after applying the hash function on ls. The cell position of the index cube in second level index space (IN DX) is decided by applying the hash function on x, y and θ. Let assume that each dimension of second level index space is divided into 15 units (i.e. δ = 15). After applying the hash function on x, y and θ the value of x′ , y ′ and θ′ are 7, 9 and 2, respectively. Hence, the feature descriptors (f d1 , f d2 , . . . , f d64 ) and identity (Id) of the index key (indx) is stored at [7,9,2] location in the first index cube which is represented as LS1 → IN DX[7][9][2]. It may be noted that a cell of an index cube can contain a set of feature descriptors and identities of index keys. To store the feature 222

6.4. Storing descriptors and identities of index keys, two storing structures: linear storing structure and kd-tree storing structure have been proposed. These storing structures are discussed in the following.

6.4.2

Linear Storing Structure

In this technique, a 2-D linear index space (LN IN DX) is created for each cell of the index cube. Each linear index space is assigned a unique identity (lid) and this identity is stored into the corresponding cell of the index cube. Note that there are δ 3 number of cells in each index cube. Hence, 2δ 3 number of linear index spaces are created to store all index keys using linear storage structure. The linear index space (LN IN DX) stores the 64-dimensional feature descriptors (f d1 , f d2 , . . . , f d64 ) and identities (Id) of index keys. Figure 6.9 shows the linear index space (LN IN DXi ) for the ith cell of the first index cube (LS[1] → IN DX). The cell stores the identity (lidi ) of the linear index space (LN IN DXi ). The ith cell of the index cube LS[1] → IN DX is represented as CELL[i]. All index keys of the ith cell are stored into the linear index space (LN IN DXi ). Hence, the size of the linear index space (LN IN DXi ) is Ni × (64 + 1) where Ni is the number of index keys in the ith cell and an index key contains 64dimensional feature descriptor. The method for creating linear index space is summarized in Algorithm 6.1. In Step 7 of Algorithm 6.1, the index cube are found from the first level index space. The cell position of the index cube is found in Step 8 to Step 10. Step 11 calculates the identity of linear index space and Step 12 assigns that identity to a cell of an index cube. The descriptor values and the identities of index keys are found in Step 13 and 14.

6.4.3

Kd-tree Storing Structure

A kd-tree is created for each cell of an index cube and assign a unique identity (kid) to each kd-tree. The identity of the kd-tree is stored 223

6. Face Biometric Data Indexing

fd1 fd2 fd3 ... fd64 Id lidi LS[1]

INDX[x][y][!] (CELL[i])

1 2 3 j Ni-1 Ni LNINDXi

Figure 6.9: Linear index space to store all index keys of the ith cell.

into the corresponding cell of the index cube. There are 2δ 3 number of cells in the index space. Hence, the total number of kd-trees required is 2δ 3 . All index keys of a cell are stored into a kd-tree. A kd-tree is a data structure for storing a finite set of points from a k-dimensional space [8, 7, 25] and the structure is defined in Section 5.4.3. All sixty four dimensional points (descriptors of an index key) are stored within a cell into a kd-tree data structure. Kd-tree insertion method is applied to store the index keys. This method is proposed by Arya and Mount [7, 8] which follows Bentley [11] kd-tree insertion method. The maximum height of the optimized kd-tree with N number of k-dimensional point is ⌊log2 (N )⌋ [11]. The kd-tree structure for the ith cell is shown in Fig. 6.10. In Fig. 6.10, KDIN DXi is the kd-tree for the ith cell of the first index cube (LS[1] → IN DX). The cell stores the identity (kidi ) of the ith kd-tree (KDIN DXi ). The ith cell of the first index cube LS[1] → IN DX is represented as CELL[i]. The method for creating kd-tree index space is summerized in Algorithm 6.2. Step 7 of Algorithm 6.2 finds the position of first level index space and Step 8 to Step 10 calculate the cell position in the index cube. Step 11 calculates the unique identity of the kd-tree. The identity of the kd-tree is assigned into a cell of an index cube in Step 12. Finally, the descriptor values and the identity of a subject are inserted into the kd-tree in Step 15. 224

6.5. Retrieving Algorithm 6.1 Creating index space with linear storing structure Input: All index keys of all person’s face image (indx11,1 , indx21,1 , . . ., T

T

1,1 1,2 indx1,1 ; indx11,2 , indx21,2 , . . ., indx1,2 ; . . . ; indx1P,Q , indx2P,Q , . . .,

T

P,Q indxP,Q ). Two level index space (LS[ ] → IN DX[ ][ ][ ]). Output: Index space (LS[ ] → IN DX[ ][ ][ ]) with linearly stored index keys for each cell (CELL[ ] → LN IN DX[ ][ ])

1: 2: 3: 4: 5: 6: 7: 8: 9: 10: 11: 12:

for c = 1 to 2 × δ 3 do inc[c] = 1 // Initialize linear index counter end for for p = 1 to P do for q = 1 to Q do for i = 1 to Tp,q do ls = Hls (lsip,q ) // Decide first level index space // Decide cell location of second level index space x = Hx (xip,q ) i ) y = Hy (yp,q i ) θ = Hθ (θp,q lid = (ls − 1) × δ 3 + (x − 1) × δ 2 + (y − 1) × δ + θ LS[ls] → IN DX[x][y][θ] = lid

//

Copy id of linear index space into a cell // Copy descriptor values of index key into linear index space 13: 14: 15: 16: 17: 18:

CELL[lid] → LN IN DX[inc[lid]] = dip,q // Copy identity of person into linear index space CELL[lid] → LN IN DX[inc[lid]][65] = Idpq inc[lid] = inc[lid] + 1 end for end for end for

6.5

Retrieving

Querying is the process of retrieving a set of candidates from the enrolled face templates corresponds to a query template. The retrieved templates are most likely to match with the query face template. Two different searching techniques have been investigated to retrieve the 225

6. Face Biometric Data Indexing

KDINDX kidi LS[1]

INDX[x][y][!] (CELL[i])

KDINDXi

Figure 6.10: Kd-tree to store the index keys of the ith cell of the index space.

face templates from the database of two different storage structures discussed previously. In each searching technique, first the index keys are generated corresponding to the query face using index key generation technique (discussed in Section 6.3). Let the index keys generated from a query face are represented in Eq. (6.16). indx1t

= ls1t

x1t

yt1

θt1

d1t

indx2t

= ls2t

x2t

yt2

θt2

d2t

yti

θti

dit

ytLt

θtLt

t dL t

... indxit

... =

lsit

=

t lsL t

... t indxL t

xit

(6.16)

... t xL t

The ith query index key is represented as indxit = lsit , xit , yti , θti , dit , where lsit , xit , yti and θti , dit represent the sign of Laplacian, x and y position, orientation, feature descriptor of the ith key point (kti ) of the query face image, respectively. Then, indexing is applied on the first level index space using the value of ls of the query index key. The indexing is done using hash functions defined in Eq. (6.15). The first level of indexing selects the index cube for a query index key. Let assume that the value of ls of the ith index key of query is −1. Then index cube (LS[1] → IN DX) in the first level index space (LS) is selected for the ith query index key. Next, hash functions (defined in 226

6.5. Retrieving Algorithm 6.2 Creating index space with kd-tree storing structure Input: All index keys of all person’s face image (indx11,1 , indx21,1 , . . ., T

T

1,1 1,2 indx1,1 ; indx11,2 , indx21,2 , . . ., indx1,2 ; . . . ; indx1P,Q , indx2P,Q , . . .,

T

P,Q indxP,Q ). Two level index space (LS[ ] → IN DX[ ][ ][ ]). Output: Index space (LS[ ] → IN DX[ ][ ][ ]) with kd tree stored index keys for each cell (CELL[ ] → KDIN DX).

1: 2: 3: 4: 5: 6: 7: 8: 9: 10: 11: 12: 13: 14: 15: 16: 17: 18: 19:

for c = 1 to 2 × δ 3 do inc[c] = 1 // Initialize linear index counter end for for p = 1 to P do for q = 1 to Q do for i = 1 to Tp,q do ls = Hls (lsip,q ) // Decide first level index space // Decide cell location of second level index space x = Hx (xip,q ) i ) y = Hy (yp,q i ) θ = Hθ (θp,q kid = (ls − 1) × δ 3 + (x − 1) × δ 2 + (y − 1) × δ + θ // Copy id of the kd-tree into a cell LS[ls] → IN DX[x][y][θ] = kid T emp[ ] = dip,q T emp[65] = Idpq Insert T emp into Kd-treekid inc[kid] = inc[kid] + 1 end for end for end for

Eq. (6.15)) are applied on the value of x, y and θ of the query index key to find the cell position of the index cube in the second level index space. Then, the candidate set (CSET ) is generated by counting the vote received for each identity of the retrieved index keys from the database. The CSET contains two fields: id and vote. The id and vote fields of the CSET store the identities of subjects and the number 227

6. Face Biometric Data Indexing of votes received for an identity. To generate the CSET , the linear or kd-tree structures are searched corresponding the identity stored in the selected cell of an index cube and find the closest match in the linear or kd-tree index space. If x, y and θ of the ith index key of a query select the LS[1] → IN DX[x][y][θ] cell of the index cube LS[1] → IN DX and retrieve the j th linear or kd-tree identity then the closest match is found in the LN IN DXj linear index space for linear search and KDT REEj for kd-tree search. Finally, ranks are given based on the votes received by each identity. The search techniques are described in the following.

6.5.1

Linear Search

In linear search, first the cell position is found in an index cube for a query index key. Then, the linear index space of that cell is searched. Euclidean distance between feature descriptor of a query index key and all the feature descriptors stored in the linear index space is computed to find a match. Let the j th cell in the index cube is selected for the ith index key of a query. Then, the linear index space (CELL[j] → LN IN DX) corresponding to the j th cell is selected to find a match. The Euclidean distances between the feature descriptors of the ith index key of the query and all the descriptors stored in the linear index space (CELL[j] → LN IN DX) is computed using Eq. (6.17). The identity is retrieved corresponding to the minimum distance. The retrieved identity is then placed in the CSET and cast a vote for this identity. The same procedure is followed for all other index keys of the query face. The linear searching method is summarized in Algorithm 6.3. In Step 2 of Algorithm 6.3, the length of each linear index space is initialized. Step 6 and Step 7 to 9 find the index of the first and second level index spaces, respectively. The cell id is calculated in Step 10. In Step 11 to 19, the minimum distance for an index key of query face is found and the identity is retrieved 228

6.5. Retrieving corresponding to the minimum distance. Step 23 to 31 generate the CSET for a query index key. Finally, the CSET is sorted in Step 35. EDi,j = EuclidDist(dj , dit ), ∑ j i 2 where, EuclidDist(dj , dit ) = 64 f =1 (f df −t f df )

(6.17)

Algorithm 6.3 Candidate set generation in linear search from index space Input: All index keys from query face image (indx1t , indx2t , . . ., t indxL t ), index space (LS[ ] → IN DX[ ][ ][ ]) with linearly stored index keys for each cell (CELL[ ] → LN IN DX[ ][ ]). Output: Candidate Set (CSET [ ] → (id, vote)) 1: 2: 3: 4: 5: 6: 7: 8: 9: 10: 11: 12: 13: 14: 15: 16: 17: 18:

for cellid = 1 to 2 × δ 3 do KEY S[cellid] = # of keys in CELL[cellid] → LN IN DX[ ][ ] end for idc = 1 for i = 1 to Tt do // Tt is the total number of query index key ′ i ls = Hls (lst ) // Find cell location of second level index space x′ = Hx (xit ) y ′ = Hy (yti ) θ′ = Hθ (θti ) // Calculate cell id of an index cube cellid = LS[ls] → IN DX[x′ ][y ′ ][θ′ ] // Retrieve the matched id from CELL[cellid] → LN IN DX[ ][ ] M inDist = ∞ // KEY S[cellid] is the total number of index key in the cellidth cell for j = 1 to KEY S[cellid] do EDi,j = EucleadDist(CELL[cellid] → LN IN DX[j], dit ) if EDi,j ≤ M inDist then m=1 M atchId[m] = CELL[cellid] → LN IN DX[j][65] else if EDi,j = M inDist then m=m+1 229

6. Face Biometric Data Indexing 19: 20: 21: 22: 23: 24: 25: 26: 27: 28: 29: 30: 31: 32: 33: 34: 35:

M atchId[m] = CELL[cellid] → LN IN DX[j][65] end if end for for j = 1 to m do id = M atchId[m] subid = SubjectId(id) // Generate candidate set if subid ∈ / CSET [ ] → (id) then CSET [idc] → id = subid CSET [idc] → vote = 1 idc = idc + 1 else idx = IdIndex(CSET [], subid) CSET [idx] → vote = CSET [idx] → vote + 1 end if end for end for Sort CSET [ ] → (id, vote) in descending order based on vote

6.5.2

Kd-tree Search

In kd-tree search, first the cell position is found in an index cube for an index key of a query. Then the identity of a kd-tree (kid) is retrieved from the cell and search the kd-tree corresponding to the retrieved kdtree identity. The hash functions are applied to find the cell position in the index cube. Let the j th cell in the index cube is selected for the ith query index key. Then kd-tree index space (CELL[j] → KDIN DX) corresponding to the j th cell is searched to find a match. Approximate nearest neighbor search [7, 8, 26, 3] is applied to reduce the searching time. Arya and Mount’s [7, 3] approximate k nearest neighbor search method is used to search the kd-tree. In this technique, only the k closest bins of the kd-tree are examined and a priority queue is used to identify the closest bins based their distances from query. The expected searching complexity of the nearest neighbor search can be reduced to O(kdlogn) and space complexity is O(dn). For this pur230

6.5. Retrieving pose, a public domain library (FLANN) [26, 3] for faster approximate nearest neighbors search is available. In the proposed approach, this library is utilized for implementing kd-tree algorithms. The identities corresponding to the closest match are retrieved from the kd-tree. The retrieved identities are then placed in the CSET and cast a vote for this identity. The same procedure is followed for all other index keys of the query face. The searching method for kd-tree index space is summarized in Algorithm 6.4. Step 3 and Step 4 to 6 of Algorithm 6.4 find the index of the first and second level index spaces, respectively. In Step 7, the cell identity of an index cube is calculated for a query index key. Step 8 finds the approximate nearest neighbors for a query index key and Step 9 retrieves the identities corresponding to the nearest neighbors. In Step 12 to 17, the CSET is generated for a query index key and sort the CSET in Step 20.

Algorithm 6.4 Candidate set generation in kd-tree search from index space Input: All index keys from query face image (indx1t , indx2t , . . ., t indxL t ) with linearly stored index keys for each cell (CELL[ ] → KDIN DX[ ]). Output: Candidate Set (CSET [ ] → (id, vote)) 1: 2: 3: 4: 5: 6: 7: 8: 9: 10:

idc = 1 for i = 1 to Lq do // Lq is the total number of query index key ls′ = Hls (lsit ) // Find first level index space // Find cell location of second level index space x′ = Hx (xit ) y ′ = Hy (yti ) θ′ = Hθ (θti ) cellid = LS[ls] → IN DX[x][y][θ] // Retrieve the matched id from Kd-tree (Kd-tree[cellid] ) N N = f indAN N (Kd-tree[cellid] , dit ) id = retrieveIdF romN N (N N ) // Select id of nearest neighbor subid = SubjectId(id) // Copy subject id from id 231

6. Face Biometric Data Indexing

11: 12: 13: 14: 15: 16:

// Generate candidate set if subid ∈ / CSET [ ] → (id) then CSET [idc] → id = subid CSET [idc] → vote = 1 idc = idc + 1 else idx = IdIndex(CSET [], subid) // Return index of subject id in CSET

CSET [idx] → vote = CSET [idx] → vote + 1 18: end if 19: end for 20: Sort CSET [ ] → (id, vote) in descending order based on vote 17:

6.6

Performance Evaluation

The proposed method has been evaluated with different face databases in different experimental setups to measure the performance of face indexing approach. The performance metrics are defined in Section 4.7.1. In this section, first the face databases used in this experiment and then different experimental setups which are used are described. Then, experimental results observed in experiments with respect to the different performance metrics defined in Section 4.7.1 are presented. A comparison of the proposed approach with existing face indexing approaches is given in this section.

6.6.1

Database

The experiments are performed on two widely used large face databases, namely Color FERET [28, 2, 5] and FRGC V2.0 [27, 29, 4]. The experiments are also carried out on CalTech 256 [18, 1] face database. Detailed description of each database is given in the following. Color FERET Face Database: The FERET database is developed for the Facial Recognition Technology (FERET) program [2, 5]. The database is designed by the Defense Advanced Research Products 232

6.6. Performance Evaluation Agency (DARPA) [5] during 1993 to 1997 to give common standard for face recognition experiments. The database contains 11338 images from 994 different subjects. These images are collected in different sessions. The resolution of the captured images is 256 × 384 pixel. The database contains 2722 frontal images with different facial expressions (Neutral and Alternate). There are 1364 images with neutral expression and 1358 images with alternate expression. Figure 6.11(a) and (b) shows the four images with different facial expressions of two different subjects. FRGC 2.0 Face Database: FRGC Still version 2.0 data set [27, 29, 4] is collected at University of Notre Dame as a part of Face Recognition Grand Challenge (FRGC) program. The primary goal of the FRGC program is to promote and advance the face recognition technology. This database contains color face images, which are taken in different lightning conditions and environments. The database consists of 24038 frontal face images of 466 subjects. These images are captured in Fall 2003 and Spring 2004 semesters of 20032004 academic year. A total of 16024 images from all subjects are captured in indoor environment with two different protocols (FERET and Mugshot) and two different facial expressions (Neutral and Smiley) [27]. The resolution of each image is either 1704 × 2272 pixel or 1200 × 1600 pixel. The images are collected in 4007 subject sessions. Four images (FERET-Neutral, FERET-Smiley, Mugshot-Neutral and Mugshot-Smiley) are captured in each subject session. The database contains 4007 FERET-Neutral, 4007 FERET-Smiley, 4007 MugshotNeutral and 4007 Mugshot-Smiley face images. Figure 6.11(c) and (d) shows four images with two facial expressions of two different subjects. FRGC Still version 2.0 data set [27, 29, 4] contains 8014 face images which are captured in outdoor environment with different backgrounds and illuminations. Figure 6.11(e) shows two face images of two different subjects in different backgrounds. 233

6. Face Biometric Data Indexing

(a) Frontal face image samples without expression from FERET database

(b) Frontal face image samples with expression from FERET database

(c) Frontal face image samples without expression from FRGC database

(d) Frontal face image samples with expression from FRGC database

(e) Sample face images in outdoor environment from FRGC database

(f ) Sample face images from CalTech database

Figure 6.11: Sample images from FERET, FRGC and CalTech256 databases.

CalTech 256 Face Database: Caltech-256 object category data set [1, 18] contains a total of 30607 images from 256 different categories. In this experiment, face category images of the Caltech-256 data set are used. The face category set consists of 432 face images from 28 subjects. Each face image is captured in complex background with different facial expressions. Figure 6.11(f) shows two face images of two different subjects from CalTech 256 face database.

6.6.2

Evaluation Setup

To evaluate the proposed indexing method, each face database is partitioned into two sets: Gallery and P robe. The Gallery set contains the face images which are enrolled into the index database and P robe set contains the face images which are used as queries to search the in234

6.6. Performance Evaluation dex database. In this experiment, different Gallery and P robe sets are created for each database. The descriptions of different Gallery and P robe sets for FERET, FRGC and CalTech256 databases are given in Table 6.1. All methods described in this approach are implemented using C programming language and OpenCV [6] image processing library in the Linux operating system. All methods are evaluated with Intel Core-2 Duo processor of speed 2.00GHz and 2GB RAM. Table 6.1: Description of Gallery and P robe sets of FERET, FRGC and CalTech256 face databases.

FERET

DB

Name

#images

#subjects

Gallery11

994

994

Gallery12

994

994

P robe11

992

992

P robe12

370

250

P robe13

366

247

P robe14

736

250

P robe15

228

75

Gallery21

466

466

Gallery22

932

466

Description First face image with neutral facial expression of first session for all subjects. First face image with neutral facial expression of first session for all subjects. First face image with alternate facial expression of first session for all subjects. Face images with neutral facial expressions of other sessions for all subjects. Face images with alternate facial expressions of other sessions for all subjects. Face images with neutral and alternate facial expressions of other sessions for all subjects. Face images with neutral and alternate facial expressions of other sessions for all subjects. But images are captured with minimum twelve months difference. First face image with neutral facial expression of first session for all subjects. Images are captured with FERET protocol. First face image with neutral and smiley facial expressions of first session for all subjects. Images are captured with FERET protocol. Continued to next page

235

6. Face Biometric Data Indexing

DB

Name

P robe21

P robe22

P robe23

P robe24

FRGC

P robe25

P robe26

P robe27

P robe28

P robe29

P robe30

P robe31

P robe32

Table 6.1 – continued from previous page #ima#subDescription ges jects First face image with smiley facial expression 466 466 of first session for all subjects. Images are captured with FERET protocol. Face images with neutral facial expressions of other sessions for all subjects. Images are cap3541 411 tured with FERET protocol. Face images with smiley facial expressions of other sessions for all subjects. Images are cap3541 411 tured with FERET protocol. Face images with neutral and smiley facial ex7082 411 pressions of other sessions for all subjects. Images are captured with FERET protocol. Face images with neutral and smiley facial expressions of other sessions for all subjects. Images are captured with FERET protocol. The 1134 193 time difference from first captured image is minimum six months. First face image with neutral facial expression 466 466 of first session for all subjects. Images are captured with Mugshot protocol. First face image with smiley facial expression of first session for all subjects. Images are cap466 466 tured with Mugshot protocol. Face images with neutral facial expressions of 3541 411 other sessions for all subjects. Images are captured with Mugshot protocol. Face images with smiley facial expressions of 3541 411 other sessions for all subjects. Images are captured with Mugshot protocol. Face images with neutral and smiley facial ex7082 411 pressions of other sessions for all subjects. Images are captured with Mugshot protocol. Face images with neutral and smiley facial expressions of other sessions for all subjects. Im1134 193 ages are captured with Mugshot protocol. The time difference from first captured image is minimum six months. 8014 466 Face images with at outdoor environment. Continued to next page

236

6.6. Performance Evaluation

DB

Name Gallery41

CalTech

Gallery42

P robe41

P robe42 P robe43

6.6.3

Table 6.1 – continued from previous page #ima#subDescription ges jects One face image from each subject. Face image 28 28 is selected randomly. Eighty percent face images of each subject. 350 26 Face images are selected randomly. One face image from each subject. Face im26 26 age is selected randomly from the rest of the Gallery41. All face images of each subject except the 404 26 Gallery41 face images. All face images of each subject except the 82 26 Gallery42 face images.

Validation of the Parameter Value

The experimental results reported in this chapter are subjected to the validity of the value of the parameter used. The value of dimension quantization of the second level index space (δ) is considered as an important parameter in this experiment. The validity is assessed on the basis of experimental evaluation of the variable. The value of δ is validated in the following. To determine the value of the number of quantization (δ) of each dimension of the second level index space, the following experiment has been done. Kd-tree based search is performed for a set of query images with different values of δ. The value of δ is varied from 2 to 50 with increment of 2. This experiment is conducted with FERET, FRGC and CalTech databases. Gallery11 and P robe11 for FERET database, Gallery21 and P robeP 21 for FRGC database, and Gallery41 and P robe41 for CalTech database are used for this experiment. The rank one HR and P R for different values of δ as observed are shown in Fig. 6.12. From Fig. 6.12(a), it is observed that for FERET database rank one HR decreases nearly 2% when the value of δ is changed 2 to 14 whereas rank one HR decreases more than 4% when the value of δ is changed 16 to 20. On the other hand, the P R decreases more than 237



6. Face Biometric Data Indexing

 



100

100

FERET (Gallery11-Probe11)

80

60

60 %

%

80

PR HR

40

0 0

10

20

į

30

40

50

0

10

20



(a) HR and P R with Gallery11  and P robe11 for different values of  δ  



100

į

30

40

50 

(b) HR and P R with Gallery21 and P robe21 for different values of δ



CalTech (Gallery41-Probe41)



80

PR HR



60



%



HR

20

0



PR

40

20



FRGC (Gallery21-Probe21)



40



20



0



0





10

20

į

30

40

50 

 (c) HR and P R with Gallery41 and P robe41 for different values of δ

Figure 6.12: HR and P R with FERET, FRGC and CalTech256 databases for different values of δ.

85% when the value of δ is changed 2 to 14 but P R decreases only 5% for the change of δ value 14 to 20 for FERET database. The similar trend is observed for FRGC database also (see Fig. 6.12(b)). On the other hand, the value of δ equal to 12 gives better performance than the other values of δ for Caltech database (see Fig. 6.12(c)). Hence, in other experiments, the value of δ equal to 15 is chosen for FERET and FRGC databases, and 12 for Caltech database, respectively. However, user may choose the other values of δ according to their requirements. 238

6.6. Performance Evaluation

6.6.4

Evaluation

A number of experiments have been conducted to evaluate the accuracy, searching time and memory requirement of the proposed face indexing method. The descriptions of these experiments and the result of each experiment are given in the following.

6.6.4.1

Accuracy

The accuracy is measured with respect to HR, P R and CM S metrics. In the first experiment, the performance of the proposed method with and without applying indexing is shown. Next, the performances of the proposed indexing method with different P robe sets is presented. The effect of multiple number of samples enrollment into the database is shown in the next experiment. Finally, the F P IR and F N IR are measured for the face-based identification system with the proposed indexing technique. Performances with and without indexing: In this experiment, the performance of the system with and without applying the proposed indexing technique is compared. Gallery11 and P robe11 for FERET, Gallery21 and P robe21 for FRGC databases, and Gallery41 and P robe41 for CalTech databases are used in this experiment. The CMC curves with and without indexing for different databases are shown in Fig. 6.13. Figure 6.13(a) shows that the approach without indexing gives better cumulative match score for FERET database. Whereas, from Fig. 6.13(b) and (c), it can be seen that the approach with and without indexing gives almost the same CM S after the 15th rank for FRGC database and after the 7th rank for CalTech database. The rank one HR, P R and average searching time (ST ) are reported for linear and kd-tree search using indexing and without indexing in Table 6.2. From Table 6.2, it can be seen that kd-tree search with indexing achieves 95.57%, 97% and 92.31% HR with 9.70%, 12.55% and 239

6. Face Biometric Data Indexing

 

100



100

FERET (Gallery11-Probe11)

FRGC (Gallery21-Probe21)

99

CMS

CMS

98 97 96

Without indexing With indexing

Without indexing With indexing

95

96 0

10

20

30

40

50

Rank

0

20

30 Rank

40

50 

(b) CMC curve with and without indexing for FRGC Gallery21 and P robe21 sets



100

10



and without (a) CMC curve with   indexing for FERET Gallery11   and P robe11 sets

CalTech256 (Gallery41-Probe41)   

CMS



98



95



Without indexing With indexing

 

90 0

2

4

6

8

10

Rank

(c) CMC curve  with and without indexing for CalTech256  Gallery41 and P robe41 sets

  



 Figure 6.13: CMC curve with and without indexing for FERET, FRGC and CalTech256 databases.  



7.14% P R for FERET, FRGC and CalTech databases, respectively. It is also observed that kd-tree search requires less average searching time for all three databases. Performance with different probe sets: In this experiment, the performances of linear and kd-tree search have been checked with the proposed indexing method for different P robe sets. The P robe sets are created with different conditions as discussed in Table 6.1. The all images of Gallery11 (with neutral expression), Gallery21 (with neutral expression) and Gallery41 are enrolled into the database for 240

6.6. Performance Evaluation Table 6.2: Performance of the proposed approach with and without indexing using linear and kd-tree search Database

FERET

FRGC

CalTech

Performance

Linear

Kd-tree

Without indexing

With indexing

Without indexing

With indexing

HR

97.28

95.57

97.28

95.57

PR

100

10.54

49.43

ST (ms)

4.46 × 10

114.00

2.21 × 10

HR

98.93

97.00

98.93

PR

100.00

16.36

51.47

ST (ms)

2.78 × 10

92.55

1.43 × 10

HR

96.15

92.31

96.15

PR

100

26.91

61.78

ST (ms)

1.24 × 10

8.10

7.64 × 10

5

5

4

7.90 5

85.40 97.00 12.55

5

71.02 92.31 23.72

3

7.14

FERET, FRGC and CalTech databases, respectively and all probe sets are used to test the indexing performances of linear and kd-tree search. Figure 6.14(a), (b) and (c) show the CMC curves of all probe sets of FERET, FRGC, CalTech256 databases, respectively. From Fig. 6.14(a) and (b), it can be noted that the CM Ss are reduced for the P robe sets which contain the face images captured in more than six month difference. On the other hand, face images captured with different expressions but in the same session give the better results than the others. It is observed that face images with complex background (P robe32 of FRGC database) give less CM S than the others. However, face images with complex background for CalTech database give above 90% CM S. The rank one HR, P R and ST for linear and kd-tree search for different P robe sets are given in Table 6.3. It is observed that the P R and ST for kd-tree based search are less for all P robe sets. Performance of multiple enrollments of a subject into the index space: The experiment has been done to check the effect of multiple enrollments on the performance. In this experiment, all sam241

6. Face Biometric Data Indexing 

100



FERET (Gallery11)

95 90

90

CMS

CMS

95

85

Probe11 Probe13 Probe15

80 0

10

20

Probe12 Probe14

85 Probe21 Probe24 Probe27 Probe30

80 

30

40



50

Rank

75 0

10

 

Probe22 Probe25 Probe28 Probe31

20 30 Rank

 

  (a) CMC curves for different P robe sets with FERET Gallery11

Probe23 Probe26 Probe29 Probe32

40

50 

 (b) CMC curves for different P robe sets with FRGC Gallery21



CalTech256 (Gallery41) 100

CMS



FRGC (Gallery21)

100

Probe41 Probe42 Probe43

95

90 0

5

10

15

20

25

Rank 



 (c) CMC curves for different P robe sets with CalTech256 Gallery41

Figure 6.14: CMC curves for different P robe sets with single enrolment of a subject with FERET, FRGC and CalTech256 databases.

ples of Gallery12 for FERET, Gallery22 for FRGC and Gallery42 for CalTech databases are enrolled, and all P robe sets of FERET, FRGC and CalTech databases are used for testing. The CMC curves of all P robe sets are shown in Fig. 6.15. From Fig. 6.15, it can be seen that 100% CM S is achieved for P robe11 and P robe21 because the images in the P robe11 and P robe21 sets are also in the Gallery12 and Gallery22, respectively. It is observed that in multiple enrollments of a subject, CM Ss for other probe sets are increased than that of the single enrollments. The P R, rank one HR and searching time for lin242

6.6. Performance Evaluation Table 6.3: Performance of different P robe sets with single enrolment of a subject in linear and kd-tree search Linear

CalTech

FRGC

FERET

DB

Kd-tree

Probe set HR

PR

ST (ms)

HR

PR

ST (ms)

P robe11

95.57

10.54

114.00

95.57

7.90

85.40

P robe12

87.44

11.13

121.98

87.44

8.34

91.38

P robe13

86.11

11.28

122.32

86.11

8.45

91.64

P robe14

86.75

11.20

121.29

86.75

8.39

90.87

P robe15

82.44

10.62

114.29

82.44

7.96

85.62

P robe21

97.00

16.36

92.55

97.00

12.55

71.02

P robe22

90.40

14.66

82.48

90.40

11.25

63.29

P robe23

80.41

16.13

92.01

80.41

12.38

70.60

P robe24

85.40

15.40

87.66

85.40

11.81

67.27

P robe25

87.31

15.69

88.48

87.31

12.04

67.89

P robe26

98.07

14.62

81.73

98.07

11.22

62.71

P robe27

96.36

16.20

91.73

96.36

12.43

70.39

P robe28

89.70

14.41

81.72

89.70

11.06

62.71

P robe29

79.39

16.07

91.24

79.39

12.33

70.01

P robe30

84.54

15.24

84.86

84.54

11.69

65.12

P robe31

85.46

15.42

86.88

85.46

11.83

66.67

P robe32

81.40

15.47

86.38

81.81

11.87

66.28

P robe41

92.31

26.91

8.19

92.31

23.72

7.22

P robe42

94.55

27.52

8.29

94.55

24.27

7.31

P robe43

93.90

27.72

8.48

93.90

24.44

7.47

ear and kd-tree based search is computed with multiple enrollments. The results are summarized in Table 6.4. From this experiment, it is observed that better rank one HR is achieved without affecting the P R. Nevertheless, a higher searching time is required to search the database with multiple enrollments. 243











6. Face Biometric Data Indexing









100

FRGC (Gallery22)

100

FERET (Gallery12) 95

CMS

CMS

95 90

90

85

Probe11 Probe13 Probe15

Probe12 Probe14

85

80

80 0

10

20

30

40

50

Rank

0

10

20

 

30

Probe22 Probe24 Probe26 Probe28 Probe30 Probe32

40

50

Rank

 

 (a) CMC curves for different P robe sets with FERET  Gallery12



 for different (b) CMC curves P robe  sets with FRGC Gallery22



CalTech256 (Gallery42) 100

98 CMS



Probe21 Probe23 Probe25 Probe27 Probe29 Probe31

Probe41 Probe42

96

Probe43

94 0

5

10

15

20

25

Rank 



 (c) CMC curves for different P robe sets with CalTech256 Gallery42

Figure 6.15: CMC curves for different P robe sets with multiple enrolment of a subject with FERET, FRGC and CalTech256 databases.

FPIR and FNIR: The performance of the proposed system is analyzed with respect to F P IR and F N IR.

Gallery11, Gallery21

and Gallery42 sets are used as Gallery sets, and P robe11, P robe21 and P robe43 are used as P robe sets for FERET, FRGC and CalTech databases, respectively. There are 992, 466 and 1297 genuine scores and 985056, 216690 and 27403 imposter scores are computed for FERET, FRGC and CalTech databases, respectively. The trade-off between FPIR and FNIR for the identification system without indexing and with indexing is shown in Fig. 6.16. The equal error rates 244

6.6. Performance Evaluation Table 6.4: Performance of different P robe sets for multiple enrolments of a subject in linear and kd-tree search

CalTech

FRGC

FERET

DB

Linear

Probe set

Kd-tree

HR

PR

ST (ms)

HR

PR

ST (ms)

P robe11

100

9.33

207.66

100

6.74

149.98

P robe12

88.94

9.85

215.93

88.94

7.11

155.96

P robe13

86.56

9.99

216.59

86.56

7.21

156.43

P robe14

87.73

9.92

214.91

87.73

7.16

155.22

P robe15

82.31

9.40

205.35

82.31

6.79

148.31

P robe21

100

13.69

164.07

100

10.14

121.54

P robe22

91.11

12.27

146.87

91.11

9.09

108.79

P robe23

93.93

13.50

163.63

93.93

10.00

121.21

P robe24

92.52

12.88

153.78

92.52

9.54

113.91

P robe25

95.59

13.13

157.14

95.59

9.73

116.40

P robe26

98.93

12.23

148.34

98.93

9.06

109.88

P robe27

99.57

13.56

160.79

99.57

10.04

119.11

P robe28

90.23

12.06

144.72

90.23

8.93

107.20

P robe29

93.31

13.45

160.29

93.31

9.96

118.73

P robe30

91.77

12.75

151.78

91.77

9.45

112.43

P robe31

93.30

12.90

151.07

93.30

9.56

111.90

P robe32

87.00

12.94

153.26

87.44

9.59

113.53

P robe41

96.15

14.21

55.70

96.15

11.12

43.58

P robe42

97.52

14.54

57.35

97.52

11.38

44.87

P robe43

98.78

14.64

57.85

98.78

11.46

45.27

(EER) of the system without indexing are 6.38%, 5.12% and 15.78% for FERET, FRGC and CalTech databases, respectively. Whereas, the EERs with indexing are 6.06%, 4.87% and 14.36% for FERET, FRGC and CalTech databases, respectively. 6.6.4.2

Searching Time

The time complexities of linear and kd-tree based search techniques are analyzed in the proposed index space. Let N be the total number of face images enrolled in the database and Tp be the average number of index keys in each enrolled face image. Thus, the total number of index keys in the index space is Tn = Tp ×N . If there are K number of 245



6. Face Biometric Data Indexing









20

20

15

15 FNIR

FNIR



Without indexing

10

With indexing

5

5

0

0 0

10

20

30 FPIR

40

50



0

60 

(a) FPIR vs. FNIR curve for FERET database

 

10

20

30 FPIR

40

50

60 

(b) FPIR vs.  FNIR curve for FRGC database  

30



25



Without indexing With indexing

20 FNIR



Without indexing With indexing

10

 

15



10



5

 

0 0

10

20

30 FPIR

40

50

60





(c) FPIR vs.  FNIR curve for FRGC database

Figure 6.16: FPIR vs. FNIR curve with and without indexing for FERET, FRGC and CalTech256 databases.

cells in all index cubes, then the average number of index keys in each cell is Tk = Tn /K. Let Tq be the average number of index keys in each query face image. To perform linear search or kd-tree based search in the index space, first, the index cell position is found for an index key of a query using hash functions defined in Eq. (6.15). This operation requires O(1) computation time for both type of searching. The time complexity analysis of linear and kd-tree based searching within the located cell are given in the following. Linear search: In linear search, Tk × Tq number of comparisons are required to retrieve a set of similar index keys and their identities, and Tq log Tq comparisons are required to sort the retrieved identities 246

6.6. Performance Evaluation based on their ranks. Thus, the average time complexity of linear search (denoted as TLS ) can be calculated as follows. TLS = O(1) × Tq + Tk × Tq + Tq log Tq = O(1) × Tq +

N ×Tp K

× Tq + Tq log Tq

(6.18)

= O(N ) Kd-tree search: The number of comparisons required in kd-tree based search to find a set of nearest index keys and their identities are log Tk ×Tq , and to sort the retrieved identities based on their ranks are Tq log Tq . Thus, the average time complexity of kd-tree based search (denoted as TKS ) can be calculated as follows.

TKS

= O(1) × Tq + log Tk × Tq + Tq log Tq = O(1) × Tq + log

N ×Tp K

× Tq + Tq log Tq

(6.19)

= O(log N ) The searching time and the average number of comparisons required for linear and kd-tree based search techniques is calculated with the proposed index space. To compute these, different number of samples are enrolled into the index space for FERET, FRGC and CalTech databases. The execution time (in Intel Core-2 Duo 2.00 GHz processor and 2GB RAM implementation environment) of linear and kd-tree search with FERET, FRGC and CalTech databases are shown in Fig. 6.17(a), (b) and (c), respectively. It is observed that the execution time for kd-tree search is less than the linear search method. It is also observed that the rate of increment in execution time for kd-tree based search is less when the number of enrolled sample increases. The average number of comparisons for linear and kd-tree based search is given in Fig. 6.18. From Fig. 6.18, it can be seen that the rate of increment in number comparisons is also less for kd-tree based search. Hence, it may be concluded that to retrieve the similar identities for a 247

6. Face Biometric Data Indexing 



1 Linear Kd-tree

0

FRGC Database

1

0.5 Linear Kd-tree

0 0

500

1000 1500 # Samples

2000

0

200



searching time for (a) Average  FERET database

 

400 600 # Samples

800

1000 

time for (b) Average searching  FRGC database  

60 Searching time (ms)



×102

1.5

FERET Database

2

Searching time (ms)

Searching time (ms)

×102

CalTech256 Database

 

40

 

20



Linear Kd-tree



0



0

100 200 # Samples

300

(c) Average searching time for  CalTech256 database 



   

 Figure 6.17: Average searching time with different sizes of databases for FERET, FRGC and CalTech256 databases. 

given query, kd-tree based search within index cell is better than the  linear search.

  

6.6.4.3

Memory Requirement 

Here, the memory requirement is analyzed for two proposed storing 

structures. Let b1 and b2 bytes memory are required to store the refer ence of the index cubes into the first level index space and the reference of linear or kd-tree index spaces into the index cube, respectively. Let m bytes are required to store a feature value of an index key and 2 bytes are required to store an identity of a subject. If there are P sub248

6.6. Performance Evaluation

 



15

×103

FERET Database

×103

FRGC Database

# Comparisons

# Comparisons

10 10

5 Linear Kd-tree

Linear Kd-tree

0

0 0

500

1000 1500 # Samples

0

2000

×103

200



  number of compar(a) Average isons for FERET database

400 600 # Samples

800

1000 

 number of compar(b) Average isons for FRGC database

CalTech256 Database

4 # Comparisons



5

2 Linear Kd-tree

0 0 

100

200 # Samples

300 

 number of compar(c) Average isons for CalTech256 database

Figure 6.18: Average number of comparisons with different sizes of databases for FERET, FRGC and CalTech256 databases.

jects and each subject has Q samples then the memory requirement for linear and kd-tree index spaces can be computed using Eq. (6.20) and (6.21), respectively. In Eq. (6.20) and (6.21), Tp,q represents the number of index keys of the q th sample of the pth subject and δ denotes the number of quantization levels of the second level index space. MLS = 2 × (b1 + δ 3 × b2 ) +

Q P ∑ ∑

(64 × m + 2) × Tp,q

(6.20)

(64 × m + 14) × Tp,q

(6.21)

p=1 q=1

MKD = 2 × (b1 + δ × b2 ) + 3

Q P ∑ ∑ p=1 q=1

249

6. Face Biometric Data Indexing In this approach, 2 bytes are required to store the reference of index cube into a cell of first level index space and 4 bytes are required to store the reference of linear or kd-tree index space into a cell of index cube. There are 64 feature values in an index key and 4 bytes are required to store a feature value. The identity of a subject is also stored with each index key. The identity field requires 2 bytes extra memory for each index key. 216 identities can be stored with 2 bytes identity field. Hence, a total of 258 bytes are required to store an index key along with the identity in linear index space. In kd-tree based index space, a single node of kd-tree requires 270 bytes memory. Figure 6.19(a), (b) and (c) show the memory requirements for linear and 





40

FERET Database

30

Memory (MB)

Memory (MB)

40



20 10

Linear index space Kd-tree index space

0

FERET Database

30 20 10

Linear index space Kd-tree index space

0 0

500

1000 1500 # Samples

2000

0

500



 for (a) Memory requirements FERET database 

1000 1500 # Samples

 





40



FERET Database

     



Memory (MB)







 for (b) Memory requirements FRGC database





2000

30

 

20



10

Linear index space Kd-tree index space

 

0 0

500

1000 1500 # Samples

2000

 (c) Memory requirements for CalTech256 database  

 

 

Figure 6.19: Memory requirements with different sizes of databases  for FERET, FRGC and CalTech256 databases. 

250    

6.7. Comparison with Existing Work kd-tree index spaces to store different number of samples for FERET, FRGC and CalTech databases, respectively. From Fig. 6.19, it is observed that the memory requirements are almost same for the linear and kd-tree based index spaces.

6.7

Comparison with Existing Work

The proposed approach is compared with three existing face indexing approaches [22, 24, 21]. To compare the proposed work, Gallery11, Gallery21 and Gallery41 are used as Gallery sets, and P robe11, P robe21 and P robe41 are used as P robe sets for FERET, FRGC and CalTech databases, respectively. The comparison result is reported in Table 6.5. The comparison is done with respect to rank one HR, P R and average searching time (ST ). From Table 6.5, it can be seen that the approach gives better performance than the existing approaches. Table 6.5: Comparison of the proposed approach with existing approaches Approach Lin et al. [22]

Mohanty et al. [24]

Kaushik et al. [21]

Proposed

Performance

FERET

FRGC

CalTech

HR

83.87

85.19

73.08

PR

41.07

45.49

83.99

ST (ms)

2697.40

1738.35

895.27

HR

94.15

95.06

80.77

PR

25.58

24.95

60.5

ST (ms)

745.61

623.56

46.77

HR

95.87

97.64

88.46

PR

16.96

18.61

27.49

ST (ms)

456.24

398.78

14.59

HR

95.57

97.00

92.31

PR

7.90

12.55

23.72

ST (ms)

85.40

71.02

7.22

251

6. Face Biometric Data Indexing

6.8

Summary

In this chapter, an approach of creating a two-level indexing mechanism is discussed for a face biometric-based identification system. The work has been published in [17]. The proposed indexing mechanism reduces search space for face-based identification system. In this technique, a set of sixty nine dimensional index keys is created using SURF feature extraction method from a face image. Among sixty nine dimensions, only four dimensions are considered to create the two-level index space. In the first level of indexing, the index keys are grouped based on the sign of Laplacian value and in the second level, the index keys are grouped based on the position and the orientation. A set of similar identities for a query are found from the two-level index space using a hashing technique. The hashing technique requires O(1) time complexity to retrieve the identities. Linear and kd-tree based searching mechanism have been explored to search the identities within the two-level index space. The approach is tested with FERET, FRGC and CalTech face databases. The experimental result shows that kd-tree based search performs better than the linear search. On an average 95.57%, 97% and 92.31% rank one HR with 7.90%, 12.55% and 23.72% P R can be achieved for FERET, FRGC and CalTech databases, respectively. Further, this approach gives better HR when multiple samples of a subject are enrolled into the database. On the average 8.21%, 11.87% and 24.17% search space reduction are possible for different probe sets of FERET, FRGC and CalTech, respectively. With the proposed indexing approach, the computation time advantage can be achieved without compromising the accuracy compared to traditional face-based person identification systems. The limitations of the proposed approach is that it does not give good results under different poses (e.g. left or right profile) of face images.

252

BIBLIOGRAPHY

Bibliography [1] CalTech 256. CalTech 256 Face Database. URL http://www. vision.caltech.edu/Image_Datasets/Caltech256/. (Accessed on October, 2012). [2] FERET.

The Facial Recognition Technology (FERET)

Database.

URL http://www.itl.nist.gov/iad/humanid/

feret/feret_master.html. (Accessed on December, 2011). [3] FLANN - Fast Library for Approximate Nearest Neighbors.

URL http://www.cs.ubc.ca/~mariusm/index.php/

FLANN/FLANN. (Accessed on May, 2012). [4] NIST FRGC.

Face Recognition Grand Challenge (FRGC).

URL http://www.nist.gov/itl/iad/ig/frgc.cfm. (Accessed on May, 2012). [5] NIST FERET. The Color FERET Database: Version 2. URL http://www.nist.gov/itl/iad/ig/colorferet.cfm. (Accessed on February, 2012). [6] OpenCV.

The Open Source Computer Vision (OpenCV) Li-

brary. URL http://opencv.willowgarage.com/wiki/Welcome. (Accessed on December, 2011). [7] S. Arya and D. M. Mount. Algorithms for Fast Vector Quantization. In Proceedings of DCC ’93: Data Compression Conference, pages 381–390, Snowbird, UT, 1993. [8] S. Arya, D. M. Mount, N. S. Netanyahu, R. Silverman, and A. Y. Wu. An Optimal Algorithm for Approximate Nearest Neighbor Searching in Fixed Dimensions. Journal of the ACM, 45(6):891– 923, 1998. 253

6. Face Biometric Data Indexing [9] H. Bay, T. Tuytelaars, and L. V. Gool. SURF: Speeded Up Robust Features. In Proceedings of the 9th European Conference on Computer Vision, volume LNCS-3951, pages 404–417, Graz, Austria, May 2006. doi: http://dx.doi.org/10.1007/11744023_32. [10] H. Bay, A. Ess, T. Tuytelaars, and L. V. Gool. SURF: Speeded Up Robust Features. Computer Vision and Image Understanding, 110(3):346–359, 2008. doi: http://dx.doi.org/10.1016/j.cviu. 2007.09.014. [11] J. L. Bentley. Multidimensional Binary Search Trees Used for Associative Searching. Communications of the ACM, 18(9):509–517, 1975. ISSN 0001-0782. doi: http://dx.doi.org/10.1145/361002. 361007. [12] J. R. Beveridge, D. Bolme, B. A. Draper, and M. Teixeira. The CSU Face Identification Evaluation System.

Machine Vision

and Applications, 16(2):128–138, 2005. doi: http://dx.doi.org/ 10.1007/s00138-004-0144-7. [13] D. S. Bolme, J. R. Beveridge, M. L. Teixeira, and B.A. Draper. The CSU Face Identification Evaluation System: Its Purpose, Features and Structure. In Proceedings of the 3rd International Conference on Computer Vision Systems, pages 304–313, Graz, Austria, April 2003. [14] D. S. Bolme, J. R. Beveridge, and B. A. Draper. FaceL: Facile Face Labeling. In Proceedings of the 7th International Conference on Computer Vision Systems, volume LNCS-5815, pages 21–32, Li`ege, Belgium, October 2009. doi: http://dx.doi.org/10.1007/ 978-3-642-04667-4_3. [15] D. S. Bolme, B. A. Draper, and J. R. Beveridge. Average of Synthetic Exact Filters. In Proceedings of the Computer Vision 254

BIBLIOGRAPHY and Pattern Recoginition, pages 2105–2112, Miami, Florida, June 2009. [16] M. Brown and D. G. Lowe. Invariant Features from Interest Point Groups. In Proceedings of the British Machine Vision Conference, pages 656–665, Cardiff, Wales, September 2002. [17] J. Dewangan, S. Dey, and D. Samanta. Face images database indexing for person identification problem. International Journal of Biometrics and Bioinformatics (IJBB), 7(2):93–122, 2013. [18] G. Griffin, A. Holub, and P. Perona. Caltech-256 Object Category Dataset. Technical report, California Institute of Technology, Pasadena, CA, USA, 2007. [19] R. Jain, R. Kasturi, and B.G. Schunck. Machine Vision. McGrawHill Inc, New York, 1st edition, 1995. [20] A. Jensen and A. L. Cour-Harbo. Ripples in Mathematics: The Discrete Wavelet Transform. Berlin, Heidelberg. Springer-Verlag, 1st, 2001. [21] V. D. Kaushik, A. K. Gupta, U. Jayaraman, and P. Gupta. Modified Geometric Hashing for Face Database Indexing. In Proceedings of the 7th international conference on Advanced Intelligent Computing Theories and Applications: with Aspects of Artificial Intelligence, pages 608–613, Zhengzhou, China, August 2011. [22] K. H. Lin, K. M. Lam, X. Xie, and W. C. Siu. An Efficient Human Face Indexing Scheme using Eigenfaces. In Proceedings of the IEEE International Conference on Neural Networks and Signal Processing, pages 920–923, Nanjing, China, December 2003. [23] T. Lindeberg. Feature Detection with Automatic Scale Selection. International Journal of Computer Vision, 30(2):79–116, 1998. doi: http://dx.doi.org/10.1023/A:1008045108935. 255

6. Face Biometric Data Indexing [24] P. Mohanty, S. Sarkar, R. Kasturi, and P. J. Phillips. Subspace Approximation of Face Recognition Algorithms: An Empirical Study. IEEE Transactions on Information Forensics and Security, 3(4):734–748, 2008. [25] Andrew Moore. A Tutorial on Kd-trees. Technical report, University of Cambridge, 1991. URL http://www.cs.cmu.edu/~awm/ papers.html. [26] M. Muja and D. G. Lowe. Fast Approximate Nearest Neighbors with Automatic Algorithm Configuration. In Proceedings of the International Conference on Computer Vision Theory and Applications (VISAPP’09), pages 331–340, 2009. [27] J. Phillips and P. J. Flynn. Overview of the Face Recognition Grand Challenge. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 947–954, San Diego, USA, June 2005. [28] P. J. Phillips, H. Moon, S. A. Rizvi, and P. J. Rauss. The FERET Evaluation Methodology for Face-Recognition Algorithms. IEEE Transactions on Pattern Analysis and Machine Intelligence, 22 (10):1090–1104, 2000. doi: http://dx.doi.org/10.1109/34.879790. [29] P. J. Phillips, P. J. Flynn, T. Scruggs, K. W. Bowyer, and W. Worek. Preliminary Face Recognition Grand Challenge Results. In Proceedings of the 7th International Conference on Automatic Face and Gesture Recognition, pages 15–24, Southampton, UK, April 2006. [30] P. Viola and M. J. Jones. Robust Real-time Face Detection. International Journal of Computer Vision, 57(2):137–154, 2004.

256

Chapter 7

Multimodal Biometric Data Indexing Multi-biometric based person identification is gaining its importance and present trend is to process a large amount of biometric data which is in the order of millions. The problem in such situations is to deal with high dimensional features from two or more biometric traits. This demands high computation time to identify a query template. In this work, an approach to index a large pool of multi-biometric data has been proposed so that the matching process can be accomplished in a real time without compromise in accuracy of person identification. The proposed indexing technique is based on the relative scores. First, a small set of reference subjects is selected. Then, the subjects are enrolled (retrieved) into (from) the database using proposed indexing approach. At the time of enrollment (retrieving), the relative scores are calculated against the set of reference subjects corresponding to each trait. The scores are combined using SVM based score level fusion technique. These scores are used to generate index key for a subject. Based on the index code values, the subject identities are stored into the database. Index spaces are created in the database 257

7. Multimodal Biometric Data Indexing and store subjects’ identities into index space based on the relative index key values. At the time of querying, a candidate set is created for a query index key corresponding to each biometric trait. A new rank level fusion technique is introduced on the retrieved candidate sets using SVM rank. The different steps in the proposed approach are shown in Fig. 7.1. Figure 7.1(a) shows an overview of the reference subject selection and Fig. 7.1(b) shows the different steps of enrolling a subject into the database and retrieving a set of subjects from the database. The rest of the chapter is organized as follows. Feature extraction and score calculation methodologies from multi-biometric traits will be discussed in Section 7.1 and 7.2, respectively. In Section 7.3, the techniques of reference subject selection will be described. Section 7.4 Ref. subject N Iris image Fingerprint image

Feature extraction

Ref. subject 2 Ref. subject 1

Reference subject selection

Score calculation

Sample selection

...

Iris

Subject selection

Fingerprint

Face image

Face

(a) Reference subject selection Enrollment process Enrolling subject

Iris image Fingerprint image

Feature extraction

Face image

Reference score calculation

Score level fusion Score normalization

Score fusion

Index Key generation

Storing

Ref. subjects Iris Fingerprint

Database

Face

Query subject

Iris image Fingerprint image Face image

Feature extraction

Reference score calculation

Score level fusion Score Score normalization fusion

Index Key generation

Retrieving

Rank level fusion

Query retrieving process

(b) Identification system with indexing

Figure 7.1: Overview of the proposed approach.

258

Retrieved subjects Iris Fingerprint Face

7.1. Feature Extraction will brief about the score calculation against the selected reference subjects.

Score fusion technique will be described in Section 7.5.

Section 7.6 will explain the proposed index key generation method for multimodal biometric system. The storing and retrieving techniques shall be discussed in Section 7.7 and Section 7.8, respectively. Section 7.9 will introduce the proposed rank level fusion technique. Performances of the proposed indexing method shall be presented in Section 7.10. Section 7.12 will summarize the chapter.

7.1

Feature Extraction

In this work, a subject having a multiple number of samples is considered. A sample is characterized with the images of multiple biometric traits. A sample of a subject is identified with a unique identity and all samples of a subject are grouped together. Let assume that there are P number of subjects and each having Q number of samples, and three biometric traits are considered for each sample. Figure 7.2 shows the representation of samples for a subject with multiple biometric traits. In Fig. 7.2, B1, B2, and B3 represent three biometric traits and B1pq denotes the q th sample of the pth subject for the biometric trait B1. The extracted feature set of a subject for all samples and all biometric traits is denoted by Eq. (7.1). F (B1pq ) represents the feature set of the q th sample of the pth subject for the biometric trait B1. The number of feature vectors for a biometric trait depends on the feature extraction method. F (B1pq ) may contains a single feature vector or a set of feature vectors. Eq. (7.2) shows the feature vectors of the F (Bqp ) where B denotes a biometric trait. In Eq. (7.2), KB and LB represent the number of feature vectors extracted from a sample and the number of features in a single feature vector for the biometric B, respectively. 259

Subject 1

Sample 1 Trait 1 B111 Trait 2 B211 Trait 3 B311

Subject p

Sample 1 Trait 1 B11p Trait 2 B21p Trait 3 B31p

Subject P

7. Multimodal Biometric Data Indexing

Sample 1 Trait 1 B11P Trait 2 B21P Trait 3 B31P

...

Sample q Trait 1 B11q Trait 2 B21q Trait 3 B31q

...

Sample q Trait 1 B1pq Trait 2 B2pq Trait 3 B3pq

...

Sample q Trait 1 B1qP Trait 2 B2qP Trait 3 B3qP

...

Sample Q Trait 1 B11Q Trait 2 B21Q Trait 3 B31Q

...

Sample Q Trait 1 B1pQ Trait 2 B2pQ Trait 3 B3pQ

...

Sample Q Trait 1 B1QP Trait 2 B2QP Trait 3 B3QP

Figure 7.2: Representation of multimodal biometric samples.

F (B1p1 ) F (B1p2 ) ( ) F B1pq

F (B2p1 ) F (B2p2 ) ··· ( ) F B2pq (· · · )

( ) F B1pQ F B2pQ

   ( p)   F Bq =    

f d12 (Bqp ) · · · f d11 (Bqp ) p 2 f d22 (Bqp ) · · · f d1 (Bq ) ··· ··· ··· f dk2 (Bqp ) · · · f dk1 (Bqp ) ··· ··· ··· KB KB p p f d1 (Bq ) f d2 (Bq ) · · ·

F (B3p1 ) F (B3p2 ) ( ) F B3pq

(7.1)

( ) F B3pQ f d1d (Bqp ) f d2d (Bqp ) ··· f dkd (Bqp ) ··· KB f dd (Bqp )

··· ··· ··· ··· ··· ···

f d1LB (Bqp ) f d2LB (Bqp ) ··· f dkLB (Bqp ) ··· KB f dLB (Bqp )

        

(7.2) In this approach, three biometric traits namely iris, fingerprint and face are considered. These biometric traits (in this case, B1, B2 and B3 are iris, fingerprint and face biometric traits, respectively) are commonly used in different applications [17, 18] due to their uniqueness, stability, reliability and accessibility. The state of art techniques are used to extract features from these biometric traits. Daugman’s 260

7.2. Score Calculation IrisCode-based method [6, 7], Jain’s filter bank-based method [16] and Du’s SURF-based method [8] are applied to extract features from iris, fingerprint and face biometric traits, respectively. Summary of the extracted features from iris, fingerprint and face biometric traits are given in Table 7.1.

7.2

Score Calculation

The similarity between two samples of a biometric trait are measured and represented as a score value. Let F (Bqp ) and F (Bnm ) be the feature vectors of the q th sample of the pth subject and the nth sample of the mth subject, respectively for a biometric trait B. The score m,n for a biometric trait B is represented as Sp,q (B) and calculate using

Eq. (7.3). ( ) m,n Sp,q (B) = SCOREB (F Bqp , F (Bnm ))

(7.3)

Thus, using Eq. (7.3), the similarity score value between two samples belong to a particular biometric trait can be measured. It may Table 7.1: Feature summary of different biometric traits Biometric trait (B)

Feature extraction method

KB

LB

FB

Iris (B1)

Daugman’s IrisCode [7]

1

2048

2048

Fingerprint (B2)

Jain’s FilterBank [16]

8

64

512

Face (B3)

Du’s SURF feature [8]

70 to 130

Score calculation method (SCOREB ()) Hamming

261

Jain’s Euclidean distance [16]

4200 Du’s Euclidean to distance [8] 8320 → Length of feature 64

KB →# of feature vectors /sample, LB vector, FB → # of total features /sample

distance [7]

7. Multimodal Biometric Data Indexing be noted that SCOREB () function is different for different biometric traits. Existing methods are used to calculate the scores for different biometric traits and described in Table 7.1. Daugman’s Hamming distance method [7], Jain’s Euclidean distance method [16] and Du’s Euclidean distance method [8] are used to calculate the scores for iris, fingerprint and face, respectively.

7.3

Reference Subject Selection

The proposed multimodal biometric indexing approach is based on the relative scores. To calculate relative scores, the reference subjects need to be decided. All individuals’ data will be inserted into the database based on the relative scores with respect to the reference subjects. The reference subjects with a single sample are chosen in such way that the scores generated by individuals with respect to the reference subjects give a high degree of distinctiveness. Let < p B1p , B2p , . . . , BQ > be all Q samples of the pth subject (p = 1, 2, . . . , P )

where P is the total number of subjects with the biometric trait B. M subjects (Br11 , Br22 , . . . , BrMM ) with single sample are to be selected as reference subjects from all samples of all P subjects for the biometric trait B and let BrMM denotes the selected sample of the M th subject. The selection of reference subjects with single sample for a biometric trait is done in two steps. The details of these steps are described in the following.

7.3.1

Sample Selection

In this step, a distinct sample has been chosen for each subject for a biometric trait which gives high variance in scores compared to other samples for that biometric trait of that subject. Let Brp , the rth sample of the pth subject for a biometric trait B, gives the maximum variance among all samples of the pth subject for the biometric B. To select 262

7.3. Reference Subject Selection Brp , first, scores of the ith sample (Bip ) of the pth subject with all other samples (Bjp ; i ̸= j and j = 1, 2, . . . , Q) of the pth subject are calculated for the biometric trait B and the variance vp,i (B) of the scores is computed for the ith sample. The scores for the ith sample with all other samples for the pth subject is represented in Eq. (7.4) and the score variance for the ith sample of the pth subject is calculated in Eq. (7.5).

p,j Sp,i (B) =< Sp,i (B) > ∀j; and j ̸= i; j = 1, 2, . . . Q;

vp,i (B) =

∑ 1 p,j (Sp,i (B) − µp,i (B))2 Q−1

(7.4)

(7.5)

1≤j≤Q j̸=i

In Eq. (7.4), Sp,i (B) denotes the score vector of length Q − 1 and p,j Sp,i (B)

denotes the score between the ith and j th samples of the pth

subject for B. In Eq. (7.5), µp,i (B) represents the mean of all scores for the ith sample of the pth subject for biometric trait B and calculated using Eq. (7.6). µp,i (B) =

∑ p,j 1 Sp,i (B) Q−1

(7.6)

1≤j≤Q j̸=i

The variance for the all samples (Bip ; i = 1, 2, . . . , Q) of the pth subject is calculated using Eq. (7.4), (7.5) and (7.6) and a variance vector vp (B) is created for the pth subject (see Eq. (7.7)).

vp (B) =< vp,1 (B), vp,2 (B), . . . , vp,i (B), . . . , vp,Q (B) >

(7.7)

The maximum variance from vp (B) is determined and the corresponding sample are selected as a distinct sample of the pth subject 263

7. Multimodal Biometric Data Indexing for the biometric trait B. Let vp,r (B) be the maximum variance in vp (B). The rth sample (Brp ) is selected as a distinct sample for the pth subject for B and represented as Brpp . In this way, the distinct samples (Br11 , Br22 , . . . , BrPP ) are selected for all P subjects of biometric trait B.

7.3.2

Subject Selection

Now, there are all subjects with one distinct sample (Br11 , Br22 , . . . , BrPP ) for a biometric trait. In this step, M subjects will be selected as reference subjects from these subjects which yield the high diversity in scores with respect to other subjects. In other words, Br11 , Br22 , . . . , BrMM are to be selected as reference subjects which give top M variances among all P subjects. It may be noted that a reference subject contains single biometric template from each trait. To choose the reference subjects, the same maximum variation finding strategy is followed as in sample selection. Here, the main difference is that the variation among all subjects are computed rather than all samples of a subject. First, a score vector Sp,r (B) is calculated for the pth subject (see Eq. (7.8)) with all other subjects for a biometric trait B and the score variance is computed for the pth subject in Eq. (7.9).

q

q,r Sp (B) =< Sp,r q = 1, 2, . . . P ; p (B) > ∀q; and q ̸= p;

vp (B) =

∑ 1 q,rq 2 (Sp,r p (B) − µp (B)) P −1

(7.8)

(7.9)

1≤q≤P q̸=p

In Eq. (7.8), rp and rq are the selected samples for the pth and q th q

q,r subject, respectively and Sp,r p (B) is the score between the selected

samples of the pth and q th subjects for the biometric trait B. The length of the score vector Sp,r (B) is P − 1. In Eq. (7.9), µp (B) is the mean of all scores between the pth subject and all other subjects for the 264

7.3. Reference Subject Selection biometric trait B. The value of µp (B) is calculated using Eq. (7.10). µp (B) =

∑ q,rq 1 Sp,rp (B) P −1

(7.10)

1≤q≤P q̸=p

A variance vector of all subjects is calculated for the biometric trait B using Eq. (7.9) and the vector is represented with v(B) (see Eq. (7.11)).

v(B) =< v1 (B), v2 (B), . . . , vp (B), . . . , vP (B) >

(7.11)

The top M variances can be found from v(B) and M subjects are selected corresponding to the top variances as a reference subjects for each biometric trait. Let Brmm (m = 1, 2, . . . , M ) be the subject with top mth variances. Br11 , Br22 , . . . , BrMM are selected as reference subjects for the biometric trait B and represent as R1 (B), R2 (B), . . . , RM (B). The reference subjects contain single template for each biometric trait. In this way, reference subjects can be selected for all other biometric traits. The extracted features of three biometric traits (B1, B2 and B3) of reference subjects are defined in Eq. (7.12). The feature vector of the mth (m = 1, 2, . . . , M ) reference subject for biometric trait B is represented in Eq. (7.13). In Eq. (7.13), KB and LB denote the number of feature vectors extracted from the biometric trait B and the length of a feature vector, respectively. ( ) 1 F B1R ( ) 2 F B1R F

( (

F

m B1R

M B1R

) )

( ) 1 F B2R ( ) 2 F B2R (

···

F B2R

m

)

( ) 1 F B3R ( ) 2 F B3R ( m) F B3R

( ··· M) ( ) M F B2R F B3R 265

(7.12)

7. Multimodal Biometric Data Indexing

        m R F (B )=       

m

···

f d1d (B R )

f d21 (B R )

m

···

f d2d (B R )

···

f d11 (B R )

m m

m

· · · f d1LB (B R ) m

· · · f d2LB (B R )

···

···

···

···

m f dk1 (B R )

···

m f dkd (B R )

···

m f dkLB (B R )

···

···

···

···

···

···

Rm ) B f dK d (B

···

Rm ) B f dK LB (B

Rm ) B f dK 1 (B

               

(7.13)

7.4

Reference Score Calculation

Scores of all samples against all reference subjects are calculated using Eq. (7.3). The match score of the q th sample (q = 1, 2, . . . , Q) of the pth subject (p = 1, 2, . . . , P ) against the mth reference subject m (see Eq. (7.14)). (m = 1, 2, . . . , M ) for B is denoted as Sp,q

( m) ( ) m Sp,q (B) = SCORE(F Bqp , F B R )

(7.14)

The scores for the q th samples of the pth subject with respect to all reference subjects are given in Eq. (7.15). In Eq. (7.15), Sm p,q denotes a tuple which contains scores of the q th samples of the pth against the mth reference subject for all biometric traits. Scores for all p (p = 1, 2, . . . , P ) and all q (q = 1, 2, . . . , Q) are calculated, where P and Q are the number of subjects and number of samples for each subject, respectively. 1 (B1) S 1 (B2) S 1 (B3) > S1p,q = < Sp,q p,q p,q 2 (B1) S 2 (B2) S 2 (B3) > S2p,q = < Sp,q p,q p,q

··· Sm p,q

··· =
Sp,q

(7.15)

7.5. Score Level Fusion

7.5

Score Level Fusion

After the calculation of scores for all samples of each biometric trait„ next task is to combine the scores of all biometric traits for a sample. Note that the scores of a biometric trait is in different scales. Hence, the scores of each biometric trait are normalized before combining it. The scores of three biometric traits are combined using SVM classifier [19, 22]. In the following, the score normalization technique followed by score fusion (combining) technique are discussed.

7.5.1

Score Normalization

Several score normalization techniques (min-max, z-score, tanh, sigmoid, reduction of high-scores effect (RHE), etc.) exist in literature [15, 33, 12, 31]. Table 7.2 shows the effectiveness of the different score normalization methods with respect to robustness, scalability, efficiency and ease of use. From Table 7.2, it can be seen that RHE method [12] is more robust and efficient among all other normalization techniques. Hence, RHE normalization technique has been applied in this work. All scores of each biometric trait are normalized using Table 7.2: Characteristics of the different score normalization methods Score normalization method

Robustness

Scalability

Efficiency

Ease of use

Min-max [15, 31]

No

No

High

Easy

Decimal scaling [15, 31]

No

No

Low

Easy

tanh [15, 31]

Yes

No

Moderate

Hard

Sigmoid [15, 31]

Yes

No

Moderate

Hard

z-score [15, 12, 31]

No

Yes

High

Hard

Reduction of High Score (RHE) [12]

Yes

Yes

High

Moderate

267

7. Multimodal Biometric Data Indexing Eq. (7.16). sm p,q (B) =

m (B) − min(S(B)) Sp,q {mean(S∗ (B)) + std(S∗ (B))} + min(S(B))

(7.16)

m In Eq. (7.16), sm p,q (B) is the normalized score of Sp,,q (B) for the

biometric B. S(B) denotes the distribution of all scores and S∗ (B) denotes the genuine score distribution [12] for the biometric B.

7.5.2

Score Fusion

The normalized scores are used to calculate the multimodal score for a sample. There exist several score fusion methods (maximum rule, minimum rule, sum rule, product rule, weighted sum rule, likelihood ratio, SVM and etc.) [15, 12, 24, 30, 31, 29]. The characteristics of the different fusion methods are shown in Table 7.3. From Table 7.3, it can be seen that the SVM based score fusion is the most efficient with respect to the different parameters like accuracy, robustness and scalability. Hence, SVM classifier is preferable to classify the genuine and imposter subjects using the normalized scores [25, 13, 9, 26, 12, 32]. In this work, SVM classifier [19, 22] is used to calculate the multimodal score because SVM-based classification performs better for biometric data classification [9, 12, 32]. m m m Let sm p,q =< sp,q (B1) sp,q (B2) sp,q (B3) > be the normalized score

vector of the q th sample of the pth subject with respect to the mth reference subject. Then, the SVM classification function for sm p,q is represented using kernel trick [19, 34] (see Eq (7.17)). F (sm p,q ) =

Ns ∑

αi yi K(Si , sm p,q )

(7.17)

i=1

In Eq. (7.17), Ns is the number of support vectors, Si denotes the ith

support vector, yi is the class associated with Si , K(·, ·) represents

a non-linear kernel function and αi represents Lagrange multiplier as268

7.5. Score Level Fusion Table 7.3: Characteristics of the different score fusion methods Fusion method

Robustness

Maximum rule [24, 31]

No

No

Moderate

No training required

Minimum rule [24, 31]

No

No

Low

No training required

Sum rule [24, 31]

Yes

No

Moderate

No training required

Product rule [24, 31]

No

No

Low

No training required

Scalability Accuracy

Remarks

Weighted sum rule [24, 31]

Yes

Yes

High

Accuracy depends on the weight of classifiers. Hence, deciding the weights of the classifiers is an important task.

Likelihood ratio [24, 31]

Yes

Yes

High

Score statistics is required in advance and training required

SVM [25, 31]

Yes

Yes

High

Training required

m sociated with Si . The sign of F (sm p,q ) represents the class of the sp,q m and the absolute value of F (sm p,q ) indicates the confidence of sp,q being

in that class. In this approach, the confidence value F (sm p,q ) is considered as a combined score. The combination of all biometric traits is considered as multimodal biometric trait and denoted it as B4. The multimodal score of the q th sample of the pth subject with respect to m , that is, S m = F (sm ). the mth reference subject is represented by Sp,q p,q p,q

To decide the support vectors (Si ), classes (yi ) of each support vector and Lagrange multipliers (αi ) associated with each support vector, the SVM classifier is trained with a set of training samples. 280134 training data are used to train the SVM classifier. These data are created from 100 users of the Gallery set. Detail description of the training procedure is given in Section 7.10.3. 269

7. Multimodal Biometric Data Indexing m (B4)) for all samples (q = Note that the multimodal scores (Sp,q

1, 2, . . . , Q) of all subjects (p = 1, 2, . . . , P ) against all reference subjects (m = 1, 2, . . . , M ) calculated as mentioned above (also see Eq. (7.17)) are in different scales than the unimodal biometric traits. Hence, the multimodal scores are normalized using RHE normalization technique (see Eq. (7.16)) and the normalized multimodal score of the q th sample of the pth subject against the mth reference subject is represented with sm p,q (B4). Now, there are four normalized score values for a sample with respect to a reference subject. Four normalized score values of the q th sample of the pth subject against the mth reference subject is denoted m m m with a vector < sm p,q (B1) sp,q (B2) sp,q (B3) sp,q (B4) >. The normal-

ized score vector for all samples of all subjects against all reference subjects are generated and represented in Eq. (7.18) where p = 1, 2, . . . , P and q = 1, 2, . . . , Q. These score vectors are used to generate index key for a sample. Index key generation technique is discussed in the next section. < s1p,q (B1) s1p,q (B2) s1p,q (B3) s1p,q (B4) > < s2p,q (B1) s2p,q (B2) s2p,q (B3) s2p,q (B4) > ···


Index Key Generation

To generate an index key for a sample of a subject the normalized score values obtained as discussed above are used. Note that there are four normalized score values with respect to a reference subject and there are M reference subjects. Hence, M index keys are created and each index key is of four dimensions corresponding to four normalized 270

7.6. Index Key Generation score values. Each dimension is called a key feature of an index key. To compute an index key feature the distribution of the scores of a biometric trait with respect to a reference subject for all samples of all subjects are used. Let sm (B) denotes a vector which contains normalized scores of all samples of all subjects against the mth reference subject for the biometric trait B. sm (B) is represented in Eq. (7.19). m m m sm (B) =< sm 1,1 (B) s1,2 (B) . . . sp,q (B) . . . sP,Q (B) >

(7.19)

The distribution of the score vector sm (B) may not be uniform. But the target is to generate uniformly distributed index keys which helps to store the identities of subjects into the database in a well distributed manner. So, uniformly distributed score vector is constructed prior to index key generation. To do this, histogram equalization [10] technique is applied on the score values of the vector sm (B). The m (B) for a score sm (B) of the q th sample of the pth equalized score fp,q i

subject against the mth reference subject for the biometric B is calculated in Eq. (7.20). In Eq. (7.20), T (·) is a transfer function on the input normalized score to get the equalized score, pSm (B) denotes probability density function (PDF) of scores against the mth reference subject for the biometric trait B, minsm (B) represents the minimum value in the vector sm (B) and r is the dummy variable for integration. ∫ m fp,q (B)

=

T (sm p,q (B))

sm p,q (B)

= s=minsm (B)

psm (B) (r)dr

(7.20)

The equalized score in Eq. (7.20) is used as a key feature of an index key. The index key of the q th sample of the pth subject for the mth reference subject is represented with indxm p,q . M index keys are generated for a sample of a subject. All M index keys of the q th sample of the pth subject are shown in Eq. (7.21). 271

7. Multimodal Biometric Data Indexing

1 (B1) f 1 (B2) f 1 (B3) f 1 (B4) > indx1p,q = < fp,q p,q p,q p,q 2 (B1) f 2 (B2) f 2 (B3) f 2 (B4) > indx2p,q = < fp,q p,q p,q p,q

...

...

m m m m indxm p,q = < fp,q (B1) fp,q (B2) fp,q (B3) fp,q (B4) >

...

(7.21)

...

M M M M indxM p,q = < fp,q (B1) fp,q (B2) fp,q (B3) fp,q (B4) >

7.7

Storing

The index keys for all samples of all subjects are generated according to the proposed index key generation method. To store the identities of subjects based on the index keys, first index spaces are created into the database and then the identities are stored into the index spaces. The methods for index space creation and storing data are discussed in the following.

7.7.1

Index Space Creation

An index space into the database corresponding to each reference subject is created. If there are M reference subjects then M index spaces are created. Figure 7.3 shows an overview of the mth index space which is corresponding to the mth reference subject. Each index space contains one table for each biometric trait. There are four biometric traits B1, B2, B3 and B4 where B4 represents multimodal trait in this approach. Four tables in each index space are created. In Fig. 7.3, th table for a biometric trait B is denoted with T ablem B (in the m index

space) and the length of the table is denoted with LB . The length of each table depends on the enrollments of subjects into the database 272

7.7. Storing

Tablem B1

mth index space Tablem Tablem B2 B3

Tablem B4

1 IDL 2 IDL 3 IDL

1 IDL 2 IDL 3 IDL

1 IDL 2 IDL 3 IDL

1 IDL 2 IDL 3 IDL

LB1 IDL

LB2 IDL

LB3 IDL

LB4 IDL

Figure 7.3: Index space for the mth reference subject.

and is decided experimentally (see Section 7.10.5.2). The index values of the cells for the table T ablem B are represented with 1, 2, . . . , LB . Each cell of the table contains a list called IDL which stores a set of identities of subjects (see Fig. 7.3).

7.7.2

Storing Multimodal Biometric Data

The identity of each sample is stored into the database based on the generated index keys which is stated as follows. For the key value m (B) of the index key indxm , the mth index space and table T ablem fp,q p,q B

are selected. The unique identity of the q th sample of the pth subject is represented as Idpq where p and q represent the subject number and sample number of the subject, respectively. The identity Idpq are stored into the identity list IDL corresponding to a cell of the table T ablem B . The cell position in the table is calculated based on the index key value. The cell position (tB ) in the table T ablem B for the key m (B) is found using Eq. (7.22) and Eq. (7.23). In Eq. (7.22) value fp,q

and Eq. (7.23), LB denotes the length of the table, and minf m (B) and maxf m (B) represent the minimum and maximum values of all key values of all samples of all subjects against the mth reference subject 273

7. Multimodal Biometric Data Indexing for the biometric B, respectively and calculated using Eq. (7.24). ⌈ tB = where,

∆=

m (B)−min m fp,q f (B) ∆

+

∆ 2

⌉

maxf m (B) − minf m (B) LB −1

m (B)) ∀p ∈ P and ∀q ∈ Q maxf m (B) = max(fp,q m (B)) minf m (B) = min(fp,q

∀p ∈ P and ∀q ∈ Q

(7.22) (7.23)

(7.24)

Note that if there are P number of subjects with Q number of samples each, M number of reference subjects and each subject has B biometric traits then there are M index spaces and each index space has B number of tables. N = P × Q number of identities are stored in each table. Illustration: The storing technique is illustrated with an example. Let there be five subjects (P = 5) and each subject has four samples (Q = 4) to be enrolled into the database. Also assume that there are three reference subjects (R1 , R2 and R3 ). The four dimensional index keys are generated for the third sample (q = 3) of the fifth subject (p = 5) with respect to all three reference subjects which is shown in Fig. 7.4(a). Three index spaces are created corresponding to each reference subject in the database and each index space contains four tables. Figure 7.4(b) shows all tables of the first index space. Let assume that the length of each table is 5, and the range of key values for B1, B2, B3 and B4 with respect to the first reference subject are 0 to 0.8, 0 to 0.7, 0 to 0.5, and 0 to 1, respectively. Now, it is shown how to store the identity of a subject for the first index key (highlighted row in Fig. 7.4(a)). The first index space is selected to store identity Id53 corresponding to the first index key indx15,3 . The identity will be stored into all tables (T able1B1 , T able1B2 , T able1B3 and T able1B4 ) of the first index space. The cell positions tB1 , tB2 , tB3 and tB4 of the T able1B1 , T able1B2 , T able1B3 and T able1B4 for the indx15,3 are calculated using Eq. (7.22), respectively. The calculated cell positions 274

7.8. Retrieving index indx

1 5,3

f(B1) f(B2) f(B3) f(B4) id : 0.713 0.498 0.237 0.744 Id 53

2 indx 5,3 : 0.785 0.505 0.254 0.756 Id 35

indx 35,3 : 0.763 0.566 0.332 0.782 Id 53 (a) Index key of the 3rd sample of the 5th subject

Table1B1

Table1B4

Table1B3

Table1B2

1

{Id 42 ,...}

1

{Id13 ,...}

1

{Id 34 ,...}

1

{Id 22 ,...}

2 3

{Id13 ,...} {Id12 ,...}

2 3

{Id 34 ,...} { Id 53 ,...}

2 3

{ Id 53 ,...} {Id 42 ,...}

2 3

{Id14 ,...} {Id12 ,...}

4

{ Id 53 ,...}

4

{Id15 ,...}

4

{Id 33 ,...}

4

{ Id35 ,...}

5

{Id 33 ,...}

5

{Id 54 ,...}

5

{Id14 ,...}

5

{Id 54 ,...}

(b) Database of the 1st index space

Figure 7.4: Example of storing the 3rd sample of the 5th subject into the 1st index space of the database.

for four tables are 4, 3, 2 and 4, respectively. The sample identity (Id53 ) is stored into the identity list (IDL) at the 4th , 3rd , 2nd and 4th cell positions of T able1B1 , T able1B2 , T able1B3 and T able1B4 respectively. The highlighted cells in Fig. 7.4(b) show the positions for storing the identity (Id53 ).

7.8

Retrieving

Once all samples of individuals are enrolled into the database, it can be used to retrieve data for a match. In retrieving, the samples are found from the database which are the most similar to a query sample. The index keys are generated for the query sample as discussed in Section 7.6 and the query index keys are represented in Eq. (7.25) where indxm denotes the query index key correspond to the mth reference subject, f m (B) denotes key feature value corresponding to the 275

7. Multimodal Biometric Data Indexing biometric trait B and M represents the number of reference subjects. indx1 indx2 ... indxm ... indxM

= =

< f 1 (B1) < f 2 (B1)

=

< f m (B1)

= < f M (B1)

f 1 (B2) f 2 (B2) ... m f (B2) ... f M (B2)

f 1 (B3) f 2 (B3)

f 1 (B4) > f 2 (B4) >

f m (B3)

f m (B4) >

f M (B3)

f M (B4) >

(7.25)

Now, a set of identities of subjects are retrieved for each query index key. For this purpose, a candidate set is maintained for each biometric trait. Note that there are four biometric traits and each trait is related to a dimension of an index key. Let the candidate sets are CSET (B1), CSET (B2), CSET (B3) and CSET (B4) for the biometric traits B1, B2, B3 and B4, respectively. Each candidate set consists of two fields: id and vote. The retrieved subject identities are stored into the id field and number of occurrences of the subject identities are stored into the vote field. To retrieve the subject identities for the mth query index key indxm , the mth index space is searched and the list of subject identities IDL is retrieved from a particular cell of the table in the mth index space for a key feature value of the indxm . For the key feature value f m (B) of the indxm , IDL from the tB th cell of the T ablem B is retrieved. The cell position tB in the table is calculated by Eq. (7.22) and Eq. (7.23). The retrieved identities for the f m (B) are stored into id field of the candidate set CSET (B). The number of occurrence of each subject identities is counted and stored in the vote field of the candidate set CSET (B) corresponding to the subject identity. Note that a subject have more than one sample in the database, and the superscript and subscript values refer to the subject and sample identifiers, respectively. The count is done based on the subject identifier. For example, if Id52 , Id51 , Id32 identities are retrieved for biometric B then Id5 and Id3 are stored into the id field, and 2 and 1 in the vote field of candidate set CSET (B), respectively. 276

7.8. Retrieving The key values calculated corresponding to a reference subject for a query may differ from the key values of the matched subjects. This may leads to decrease the accuracy of the proposed system. To address this limitation, the ±δ neighbor cell positions of the cell tB is considered in a table. The identity lists (IDL) are retrieved from tB − δ to tB + δ cell positions of the table. The value of δ is decided experimentally (see Section 7.10.5.3). In this way, all candidate sets CSET (B1), CSET (B2), CSET (B3) and CSET (B4) are generated for all biometric traits B1, B2, B3 and B4. These candidate sets are used for rank level fusion to rank the retrieved identities. Illustration: The retrieving of candidate set from the database for a given query subject is illustrated with an example. In this example, 3 reference subjects are considered to generate the index key. Let assume that the database stores the identities for 5 subjects and each subject has four samples. The database consists of four index spaces. Figure 7.5(a) shows the stored identities in the tables T able1B1 , T able2B1 and T able3B1 of all index spaces ( for 1st , 2nd and 3rd index spaces, respectively) for the biometric trait B1. In each table, the identities of all subjects are stored. The index keys of a query subject are shown in Fig. 7.5(b). The candidate set CSET (B1) is generated for the B1. To generate CSET (B1), the f (B1) key values of all index keys (see Fig. 7.5(b)) are considered and the respective positions are found in the three tables using Eq. (7.22). The positions in the T able1B1 , T able2B1 and T able3B1 for the key values f (B1) = 0.417, 0.465, 0.378 are 3, 3 and 2. The identities are retrieved from the 3rd , 3rd and 2nd positions of the T able1B1 , T able2B1 and T able3B1 , respectively, and count the number of occurrences of each subject. From Fig. 7.5(a), it can be seen that Id1 , Id4 and Id5 occur 10, 4 and 7 times, respectively. Therefore, Id1 , Id4 and Id5 in CSET (B1) receive 10, 4 and 7 votes, respectively (see Fig. 7.5(c)). Similarly, CSET (B2), CSET (B3) and 277

7. Multimodal Biometric Data Indexing Table1B1

Table3B1

Table 2B1

1

{Id11 , Id 32 ,Id 42 }

1

{Id 22 ,Id 32 , Id 24 }

1

{Id13 ,Id 32 ,Id 34 }

2

{Id12 ,Id 22 , Id 42 , Id13 }

2

{Id 32 ,Id 33 ,Id 34 }

2

3

{Id12 , Id13 , Id14 , Id 43 , Id15 , Id 52 }

3

{ Id11 , Id12 , Id13 , Id14 , Id 43 , Id 52 , Id 54 }

{ Id11 , Id12 , Id14 , Id14 , Id 44 , Id15 , Id 52 , Id 54 }

3

{Id13 ,Id 22 ,Id 42 }

4

{Id 34 ,Id 53 ,Id 54 }

4

{Id12 , Id13 ,Id15 }

4

{Id 33 ,Id 42 ,Id 53 }

5

{Id 32 ,Id 33 ,Id14 ,Id 44 }

5

{Id 24 ,Id14 ,Id 44 ,Id 53 }

5

{Id12 , Id 32 ,Id 34 }

(a) Tables of the biometric trait (B1) from all index spaces of the database

index f(B1) f(B2) f(B3) f(B4) indx1 : 0.417 0.156 0.497 0.691 indx 2 : 0.465 0.241 0.451 0.645 3

indx : 0.378 0.162 0.438 0.752

CSET(B1)

CSET(B2)

CSET(B3)

CSET(B4)

id

vote

id

vote

id

vote

id

vote

Id1

10

Id3

8

Id2

5

Id3

6

Id4

4

Id1

6

Id3

9

Id4

9

Id5

7

Id2

3

Id1

3

Id1

5

(b) Index keys for query subject

(c) Candidate sets for all biometric traits

Figure 7.5: Example of retrieving candidate sets for a given query.

CSET (B4) can be generated for the biometric traits B2, B3 and B4, respectively.

7.9

Rank Level Fusion

For a query subject a candidate set is retrieved corresponding to each trait. A candidate set for a biometric trait contains a set of retrieved subjects and a vote for each retrieved subject. Therefore, each retrieved subject has different order of similarities based on different biometric traits. To decide the order of similarity for the retrieved subjects with the query, the rank level fusion is done among all candidate sets. Rank level fusion is done in two steps. These steps are described in the following. 278

7.9. Rank Level Fusion

7.9.1

Creating Feature Vector for Ranking

In Section 7.8, four candidate sets (CSET (B1), CSET (B2), CSET (B3) and CSET (B4)) have been retrieved. From these candidate sets, a set of feature vectors are created for ranking. Let consider a total of N subject identities are retrieved in all four candidate sets for a query. Each subject identity has own vote in each candidate set. The votes of the retrieved subjects is represented in each candidate set as shown in Table 7.4. In Table 7.4, Idi denotes the identity of the ith retrieved subject and vi (B) denotes the vote for that subject identity in the candidate set CSET (B). For each retrieved subject, a feature vector is created from the number of votes of that subject in each candidate set. The feature vector for the subject identity Idi is represented in Eq (7.26). vi = < v i (B1) v i (B2) v i (B3) v i (B4) >

7.9.2

(7.26)

SVM Ranking

In this step, each feature vector is ranked using SVM ranking method [20, 34]. Unlike SVM classification function, which outputs a distinct class for a feature vector, the ranking function gives an ordering of feature Table 7.4: Retrieved subjects and their votes in each candidate set

Vote

Subject Id

CSET (B1)

CSET (B2)

CSET (B3)

CSET (B4)

Id1

v 1 (B1)

v 1 (B2)

v 1 (B3)

v 1 (B4)

Id2

v 2 (B1)

v 2 (B2)

v 2 (B3)

v 2 (B4)

...

...

...

...

...

Idi

v i (B1)

v i (B2)

v i (B3)

v i (B4)

...

...

...

...

...

IdN

v N (B1)

v N (B2)

v N (B3)

v N (B4)

279

7. Multimodal Biometric Data Indexing vectors. The ranking function outputs a score for each feature vector, from which a global ordering of feature vectors is constructed [20, 34]. Let vi feature vector is preferred to vj feature vector and specified as vi ≻ vj . The objective is to find a global function F (·) which outputs a score such that F (vi ) > F (vj ) for any vi ≻ vj . The global ranking function F (·) on a feature vector vi can be computed using SVM [20, 34] and represented in Eq. (7.27). In Eq. (7.27), K(·, ·) is kernel function, Ns is the number of support vectors, (Vk − Vl ) is the pair wise difference support vector and αk,l is the Lagrange multiplier associated with the (Vk − Vl ). i

F (v ) =

Ns ∑

αk,l K(Vk − Vl , vi )

(7.27)

k,l

The pair wise difference support vector (Vk −Vl ) and the Lagrange multiplier αk,l for the global ranking function F (·) are computed from a set of labeled training data. The detail method to train the SVM for ranking can be found in [20, 22]. To generate the set of training data, T number of queries are randomly selected from the gallery set and a set of feature vectors is created for each query using the method discussed in Section 7.9.1. Let Rt be the training data generated from the tth query (t = 1, 2, . . . , T ) and is represented in Eq. (7.28). NT T Rt = {(v1t , rt1 ), (v2t , rt2 ), · · · , (vit , rti ) · · · , (vN t , rt )}

(7.28)

In Eq. (7.28), vit denotes the ith feature vector for the tth query and rti is the rank of vit . Note that the training data Rt follows strict ordering [20], that is, rti < rtj if vit ≻ vjt . The ranks of the feature vectors in the training data are given by manually or any automatic ranking method [20, 22]. In this approach, a rank is assigned to each feature vector of a training data by an automatic method which is discussed in the following. 280

7.9. Rank Level Fusion The ranking method for training data is based on a weighted feature vector. Let wvti be the weighted value of the ith feature vector vit for the tth training data. The weighted value wvti is computed by Eq. (7.29). wvti = [vti (B1) + vti (B2) + vti (B3) + vti (B4)] × wti

(7.29)

In Eq. (7.29), wti represents the weight of the ith feature vector vit for the tth training data. The weight is assigned to each feature vector of a training data in such a way that if the identity related to the feature vector is present in all candidate sets then it gets higher preference than the other feature vector. The weight wti is computed using Eq. (7.30). wti =

B4 ∑

{ eB

where eB =

B=B1

0 if vti (B) = 0 1 otherwise.

(7.30)

The weighted values are calculated for all feature vectors of a training data using Eq. (7.29) and rank the feature vector based on the weighted values. The rank one is given to the feature vector with the highest weighted value, rank two to the feature vector with the second highest weighted value and so on. In this way, the rank is assigned to each feature vector of all training sets. Illustration: The SVM-based ranking of each retrieved subject is illustrated with an example. Figure 7.6(a) shows the retrieved candidate sets for a query. There are 5 unique subjects among all candidate sets. A feature vector is generated for each subject identity. For the subject identity Id1 , there are 10, 6, 3 and 5 votes in CSET (B1), CSET (B2), CSET (B3) and CSET (B4). Hence, the feature vector v1 corresponding to Id1 is < 10 6 3 5 >. The feature vectors for all retrieved subjects are shown in Figure 7.6(b). The value of SVM ranking function is computed for each feature vector which is shown in 281

7. Multimodal Biometric Data Indexing CSET(B1) id vote 1

Id Id4 Id5

10 4 7

CSET(B3) id vote

CSET(B2) id vote 3

Id Id1 Id2

8 6 3

CSET(B4) id vote

Id1: v1 = 2

2

3

3

4

4

5

5

Id : v = < 0 3 5 0> Id : v = < 0 8 9 6>

6 9 5

Id : v = < 4 0 0 9>

(a) Candidate sets with retrieved data

(b) Feature vectors for all subjects in candidate sets

2

Id Id3 Id1

5 9 3

3

Id Id4 Id1

Id : v = < 7 0 0 0>

id

vi F(vi) Rank

Id1

v1

1.6

1

Id2

v2

0.6

4

3

Id

3

v

0.9

2

Id4

v4

0.7

3

5

5

0.5

5

Id

v

(c) Subject with rank after SVM

Figure 7.6: An example of SVM-based rank to each retrieved subject for a query.

Figure 7.6(c). In Figure 7.6(c), it can be seen that the feature vector v1 related to Id1 gives the maximum value for SVM-ranking function. The rank 1 is assigned to the subject identity Id1 .

7.10

Performance Evaluation

To study the efficacy of the proposed multimodal biometric data indexing approach, a number of experiments have been conducted on virtual multimodal database and measured the performances with the metric defined in Section 4.7.1. This section starts with the description of database. Then, the experiments carried out and the experimental results are presented.

7.10.1

Database

There are few publicly available multimodal databases with iris, fingerprint and face biometric traits. Out of which, WVU [5, 4] and BioSecure [1] are the two multimodal database which contain three biometric traits, namely iris, fingerprint and face. But the authority of the WVU database does not disclose the privacy of the users. Hence, 282

7.10. Performance Evaluation WVU face database is also not available to the research community. On the other hand, BioSecure [1] database contains less number of users and this database is not freely available to the research community. As a way out, a set of virtual users are created from three publicly available large unimodal databases and the experiments are performed on these virtual user dataset. CASIAV3I [2], WVU [5, 4] and FRGC Still version 2.0 [27, 28, 3] unimodal biometric databases are used for iris, fingerprint and face biometric traits, respectively. The description of these databases are given in the following. Iris Database: The CASIAV3I database [2] contains 2639 eye images from 395 eyes of 249 persons. One to twenty six images are captured from each eye. It may be noted that the iris data of left and right eyes of a person are different. Hence, each eye is treated as a unique subject. The summary of CASIAV3I iris database is given in Table 7.5. Figure 7.7(a) shows two sample eye images of CASIA database. At least two samples from each subject are considered so that use can be at least one sample for enrollment into the database and one sample for probing. There are 372 subjects which have at least two samples for each subject. These subjects are used to create virtual users in this experiments. Fingerprint Database: The WVU fingerprint database [5, 4] contains 7136 images of 270 persons. The images are captured from 4 different fingers (left index, left thumb, right index, right thumb). Three to twenty images are captured from each finger. As each finger of a person is different, the each finger is considered as unique subject. Hence, there are 1080 unique subjects in the WVU fingerprint database. The features can not be extracted from the fingerprint images of 320 subject. So, 750 subjects are used to create virtual users in this experiment. The summary of WVU fingerprint database is given in Table 7.5. Two Sample fingerprint images selected randomly from WVU database are shown in Fig. 7.7(b). 283

7. Multimodal Biometric Data Indexing Table 7.5: Summary of biometric databases and virtual users DB

Modality

#sub

#sam

Size

Remark

CASIAV3I

Iris

395

1 to 26

2639

372 subjects have at least 2 samples and use to create virtual users

WVU

Fingerprint

1080

3 to 20

7136

Minutiae points are extracted successfully from 750 subjects which are used to create virtual users.

FRGC

Face

466

2 to 44

4007

All 466 subjects are used to create virtual users

Virtual users

iris + fingerprint + face

372

2 to 20

2625

Subjects are selected randomly from each trait

#sub → # unique subjects, #sam → # sample/ subject

Face Database: FRGC Still version 2.0 database [27, 28, 3] contains 16028 frontal face images of 466 subjects. The images from each subject are captured in indoor environment with two different protocols (FERET and Mugshot) and two different facial expressions (Neutral and Smiley) [27]. In this experiment, a subset of 4007 face images of 466 persons has been taken with neutral expression captured with FERET protocol (FERET-Neutral). These images are captured in Fall 2003 and Spring 2004 semester of 2003-2004 academic year. The images from each subject are captured in indoor environment. A brief summary of the FRGC database is given in Table 7.5. All these images are used for virtual user creation. Figure 7.7(c) shows two face images of two different subjects. Virtual Users: A sample (instance) of a virtual user consists of three images from three biometric traits, namely iris, fingerprint and face. The virtual users are created from the above mentioned databases. Those subjects from the databases are considered which have at least two samples because at least one sample is needed for enrollment and one sample for probe. Note that there are 372, 1080 and 466 subjects have at least two samples for iris, fingerprint and face biometric databases, respectively. From these three databases 372 virtual users 284

7.10. Performance Evaluation

(a) Sample iris images

(b) Sample fingerprint images

(c) Sample face images

Figure 7.7: Sample images of CASIAV3I iris, WVU fingerprint and FRGC face databases.

(subjects) can be created with three biometric traits. To create virtual user (subject), first, all samples of a subject are selected from the database of each biometric trait. The subject is selected randomly. Then, a sample image from each biometric trait of the subject is taken and used as a sample image of virtual user. It may be noted that each subject has some uniqueness with respect to each biometric trait. Hence, the subjects or samples are not used to create a virtual user which are already selected for a virtual user. This helps to create virtual users with unique multi biometric traits. In this procedure, 372 virtual users are created and each virtual user has 2 to 20 samples, resulting in a total of 2625 samples. The summary of virtual users is given in Table 7.5.

7.10.2

Evaluation Setup

To evaluate the performance of the proposed approach, all samples of all virtual users are divided into two sets: Gallery and Probe. The samples in the Gallery set are enrolled into the index database and samples in the Probe set are used as query to search the index database. 80% samples of each subject are used to create the Gallery 285

7. Multimodal Biometric Data Indexing set and other 20% to create the Probe set. The samples of the Gallery set are selected randomly from each subject. The experiments have been done with an Intel Core2Duo processor (2.00 GHz) and 2.0-GB memory. The developed programs are executed with GCC 4.3 compiler.

7.10.3

Training of SVM-based Score Fusion Module

SVM-based score fusion technique is used to combine the scores of different traits and the SVM needs to train before using it in score fusion. The model [22, 21, 14] is trained with known training samples. The training samples contain the scores of all traits and their classes. The class of the training sample is genuine if the score is calculated from the samples of the same subject and imposter if the samples are from different subjects. To create training samples, first, 100 subjects with 2 to 10 samples per subject are selected form the Gallery set. From these selected samples, 2,498 genuine and 2,77,636 imposter pairs are created. Then, the scores between each pair of sample for each biometric trait are calculated using Eq. (7.3) given in Section 7.2 and the scores are normalized using the RHE score normalization method described in Section 7.5.1. Thus, a total of 2,80,134 training data have been created. Each training data has been assigned either genuine or imposter class based on pairs (genuine or imposter). The SVM is trained with these training data and perform 5-fold Cross validation [22, 14] to measure the performance with training data. 96.75% cross validation accuracy has been observed, that is, the SVM training is of good classification accuracy. SVMligtht tool [22, 21] is used for training and testing the SVM model.

7.10.4

Training of SVM-based Ranking Module

To train the SVM in rank level fusion, an enrolled database, a set of query samples and the retrieved candidate sets for each query with 286

7.10. Performance Evaluation their ranking are required. For this purpose, 1 sample from 100 subjects is randomly selected as query from the gallery set and the rest of the samples of Gallery set are enrolled into the database. For each query sample, four candidate sets are retrieved corresponding to four biometric traits using the proposed retrieving technique (see Section 7.8). To create training data for SVM ranking, it has to be assigned a rank to each candidates retrieved in the candidate set. To do this, a feature vector is created corresponding to each candidate retrieved candidate set using the proposed method described in Section 7.9.2. An initial rank is given to each training data using Eq. (7.29) as mentioned in Section 7.8. In this way, 100 training sets are created from the 100 query samples. 5-fold cross validation [14, 23] is performed with these training data and observe 92.46% cross validation accuracy. The SVMRank tool [23, 20] is used to train and test the SVM in ranking module.

7.10.5

Validation of the Parameter Values

Three parameters are used in this proposed indexing approach. The first parameter is the number of reference subjects (M ) which is used in index key generation. The second parameter is the size of the table (LB ) in index space which is used in storing the biometric data into the index space. The third parameter is the number of neighbor cells (δ) of a table. The values of these three parameters are validated experimentally. The experimental validations are given in the following. 7.10.5.1

Number of Reference Subjects (M )

In this approach, the index keys are generated with respect to a number of reference subjects. The number of index keys depends on the number of reference subjects. If more number of index keys are considered from more number of reference subjects then better HR can be achieved, however, the P R is also increased. So, the number of 287

7. Multimodal Biometric Data Indexing reference subjects should be chosen in such a way that it gives good HR with low P R. To do this, experiment is performed with different number of reference subjects (M = 5, 10, 15, 20 and 25). To evaluate the result without biasing toward reference subject the samples of all reference subjects are removed from the gallery and probe sets and the HR and P R are measured. The result is shown in Fig. 7.8. It is observed that the HR remains almost same though the number of reference subjects is more than 10 and the P R increases for more than 10 reference subjects. Hence, 10 reference subjects are considered in this approach.

7.10.5.2

Size of Table (LB )



The HR and P R of the proposed indexing technique depend on the 

number of entries into a cell of the index table. If number of entries 

into the cell are more then the chances of finding a query subject  within the cell increase; as a result HR increases and vice-versa. At  the same time, the P R also increases as more number of samples  the cell which need to be retrieved at the time of querying. The in 

100 90 80 70 60 50 40 30 20 10 0

%

HR PR

5



10

15 # Reference subject

20

25 

Figure 7.8: HR and P R with different number of reference subjects.

288

7.10. Performance Evaluation number of entries depends on the number of cells of a table which is referred as table size. The number of entries into a cell decreases when table size increases and vice versa. Hence, the HR and P R of different number of enrolled samples are measured with different table sizes. In this experiment, sizes of the table is calculated by taking different percentages of the total number of enrolled samples. The table size is considered 10% to 100% of enrolled samples with step of 10% increment. The experiments have been done with 500, 1000, 1500 and 2000 enrolled samples and results are presented in Fig. 7.9. From Fig. 7.9(a), it is observed that HR decreases rapidly beyond the table size as 30% of the total enrolled samples for different number of enrollments. Whereas in Fig. 7.9(b), it can be seen that P R decreases when the table size is less than 20% of the total enrolled samples for different number of enrollments. Hence, the size of the table is chosen as 20% of the total number of enrolled samples.

7.10.5.3

Number of Neighbor Cells (δ)

For an index key, a set of stored identities are retrieved from a cell and its neighborhood cells of a table. If more number of cells are considered then the HR will be increased but the P R will be also increased. Hence, there should be a choice for selecting the number of neighborhood cells (δ). For this purpose, an experiment is conducted with different values of δ. The values of δ are chosen from 1 to 10 and the HR and P R are measured at each δ value. The result is presented in Fig. 7.10. From Fig. 7.10, it can be seen that by incrementing the value of δ from 1 to 5 the HR is increased from 95.81% to 99.55% with the increase of P R 2.57%. However, incrementing the value of δ from 5 to 10, the HR increases only 0.30% with the increase of P R 4.93%. The value of δ = 5 gives 99.55% HR at 13.86% P R. The value of δ = 5 is selected in the experiment. 289

7. Multimodal Biometric Data Indexing

100

HR

90

80 2000 70

100% 90% 80% 70% 60% 50% 40% 30% 20% 10% Table size

1000 500

1500 es pl am #S

(a) HR with different number of enrolled samples and different table sizes

20 18

PR

16 14 12 2000

10 8 60%

1500 50%

40%

es

1000 30%

20%

Table size

10%

500

pl

am #S

(b) P R with different number of enrolled samples and different table sizes

Figure 7.9: HR and P R with different number of enrolled samples and different table sizes.

7.10.6

Evaluation

The efficiency of the proposed indexing technique is judged with respect to accuracy, searching time and memory requirement. After selecting the 10 reference subjects, the Gallery set contains 1957 samples and P robe set contains 668 samples from 372 subjects with iris, fingerprint and face biometric traits. Experiments are evaluated with 290

7.10. Performance Evaluation these Gallery and P robe sets. In experiment, the indexing is performed using iris, fingerprint and face trait separately, and different combinations of these three traits (multimodal) also.

7.10.6.1

Accuracy

To analyze the accuracy of the proposed approach, HR and P R are measured. The HR and P R achieved by indexing with iris, fingerprint, face and different combinations of these three traits are reported in Table 7.6. From Table 7.6, it can be seen that the indexing with three traits gives 99.55% HR which is higher than the indexing with uninodal or combination of any two traits. However, the P R is little higher than the unimodal or combination of any two traits. The result is also substantiated in terms of CM S which gives the probability of at least one correct identity present within a top rank. How CM S varies with rank is shown in Fig. 7.11 as CMC curve for 

indexing with each unimodal traits as well as for the multimodal in

100 80 60 %

PR HR

40 20 0 0

2

4

į

6

8

10 

 Figure 7.10: HR and P R for different values of δ with the proposed indexing technique.  

291  

7. Multimodal Biometric Data Indexing dexing. From Fig. 7.11, it is observed that 91.62%, 92.96% and 86.98% CM Ss are possible at the top 30 ranks for indexing with iris, fingerprint and face, respectively. On the other hand, indexing with the combination of three traits gives 99.25% CM S at the 30th rank. Further, the performance of the proposed method is analyzed with respect to F P IR and F N IR. To do this, first F M R and F N M R are calculated as follows. Each query template of the P robe set is Table 7.6: HR and P R of the proposed indexing technique with unimodal and multimodal traits Biometric trait

HR

PR

Iris

93.56

14.63

Fingerprint

95.96

12.98

90.27

15.86

97.75

17.62

97.21

16.44

93.86

17.18

99.55

17.77

Face Iris+Fingerprint Fingerprint+Face

  

Iris+Face



Iris+Fingerprint+Face



100 95

CMS

90 85 Iris+Fingerprint+Face Iris Fingerprint Face Iris+Fingerprint Fingerprint+Face Iris+Face

80 75 70 0

5

10

15 Rank 

20

25

30 

Figure 7.11: CMC curves of the proposed indexing technique with different combination of biometric traits.  



292

7.10. Performance Evaluation



10 9 8 7

FNIR

6

Without indexing Indexing with iris

5

Indexing with fingerprint 4

Indexing with face

3

Multimodal indexing

2 1 0 0

10

20

30 FPIR

40

50

60 



Figure 7.12: F P IR versus F N IR curves of the proposed identifica indexing and with iris, fingerprint, face and tion  system without any multimodal based indexing.

matched with each template in the Gallery set using SVM classification. 3738 genuine pairs and 13,03,538 imposter pairs are chosen from the Gallery and P robe, and the genuine score and imposter scores are calculated for each genuine and imposter pairs, respectively. Finally, F N IR and F P IR are calculated for the identification system without indexing and with indexing using Eq. (4.20) and (4.21), respectively. The trade-off between F P IR and F N IR for the identification system without indexing is shown in Fig. 7.12. Figure 7.12 also shows the trade-off between F P IR and F N IR for indexing with iris, fingerprint, face and combining of three traits. From the experimental results, it may be interpreted that 5.22% F N IR can be achieved at 1% F P IR without indexing where as 2.72%, 2.43%, 3.04% and 2.42% F N IR can be achieved at 1% F P IR for iris, fingerprint, face and multimodal (combination of iris, fingerprint and face) indexing. From Fig. 7.12, also it can be observed that using the proposed indexing approach a lower F P IR can be achieved for an F N M R. 293

7. Multimodal Biometric Data Indexing 7.10.6.2

Searching Time

First, the run-time complexity is analyzed with big-O notation for gallery match corresponding to a query sample. Let N be the total number of samples enrolled in the database from P number of individuals and an individual has L number of samples. In this approach, for given a query template, constant time is required to find a position in the table and retrieve the list of identities (IDL) stored at the position for a key value of a query index key. Adding these list into a set SB can be done in constant time. There are 10 index keys and each has 4 key values. Hence, the time complexity of retrieving candidates from the table for a given query is O(1). Each retrieved candidate is processed to create feature vector for rank level fusion. Let IL be the average number of candidates retrieved from the database. Note that IL