Wavelet Analysis and Its Applications: Second International Conference, WAA 2001 Hong Kong, China, December 18–20, 2001 Proceedings [1 ed.] 3540430342, 9783540430346

Citation preview

Lecture Notes in Computer Science Edited by G. Goos, J. Hartmanis, and J. van Leeuwen

2251

3

Berlin Heidelberg New York Barcelona Hong Kong London Milan Paris Tokyo

Yuan Y. Tang Victor Wickerhauser Pong C. Yuen Chun-hung Li (Eds.)

Wavelet Analysis and Its Applications Second International Conference, WAA 2001 Hong Kong, China, December 18-20, 2001 Proceedings

13

Series Editors Gerhard Goos, Karlsruhe University, Germany Juris Hartmanis, Cornell University, NY, USA Jan van Leeuwen, Utrecht University, The Netherlands Volume Editors Yuan Y. Tang Pong C. Yuen Chun-hung Li Hong Kong Baptist University Department of Computer Science Kowloon Tong, Hong Kong E-mail:{yytang/pcyuen/chli}@comp.khbu.edu.hk Victor Wickerhauser Washington University, Department of Mathematics Campus Box 1146, Cupples I St. Louis, Missouri 63130, USA E-mail: [email protected]

Cataloging-in-Publication Data applied for Die Deutsche Bibliothek - CIP-Einheitsaufnahme Wavelet analysis and its applications : second international conference ; proceedings / WAA 2001, Hong Kong, China, December 18 - 20, 2001. Yuan Y. Tang ... (ed.). - Berlin ; Heidelberg ; New York ; Barcelona ; Hong Kong ; London ; Milan ; Paris ; Tokyo : Springer, 2001 (Lecture notes in computer science ; Vol. 2251) ISBN 3-540-43034-2

CR Subject Classification (1998): E.4, H.5, I.4, C.3, I.5 ISSN 0302-9743 ISBN 3-540-43034-2 Springer-Verlag Berlin Heidelberg New York This work is subject to copyright. All rights are reserved, whether the whole or part of the material is concerned, specifically the rights of translation, reprinting, re-use of illustrations, recitation, broadcasting, reproduction on microfilms or in any other way, and storage in data banks. Duplication of this publication or parts thereof is permitted only under the provisions of the German Copyright Law of September 9, 1965, in its current version, and permission for use must always be obtained from Springer-Verlag. Violations are liable for prosecution under the German Copyright Law. Springer-Verlag Berlin Heidelberg New York a member of BertelsmannSpringer Science+Business Media GmbH http://www.springer.de © Springer-Verlag Berlin Heidelberg 2001 Printed in Germany Typesetting: Camera-ready by author, data conversion by DA-TeX Gerd Blumenstein Printed on acid-free paper SPIN 10845973 06/3142 543210

Preface

The first international conference on wavelet analysis and its applications was held in China in 1999. Following the success of the first conference, the second international conference (ICWAA 2001) was held in Hong Kong in December 2001. The objective of this conference is to provide a forum for researchers working on both wavelet theory and its applications. By idea-sharing and discussions on the state of the art in wavelet theory and applications, ICWAA 2001 is aimed to stimulate the future development, explore novel applications, and exchange ideas for developing robust solutions. By August 2001, we had received 67 full papers submitted from all over the world. To ensure the quality of the conference and proceedings, each paper was reviewed by three reviewers. After a thorough review process, the program committee selected 24 regular papers for oral presentation and 27 short papers for poster presentation. In addition to these 24 oral presentations, there were 3 invited talks delivered by distinguished researchers, namely Prof. John Daugman from Cambridge University, UK, Prof. Bruno Torresani from Inria, France, and Prof. Victor Wickerhauser, from Washington University, USA. We must add that the program committee and the reviewers did an excellent job within a tight schedule. We wish to thank all the authors for submitting their work to ICWAA 2001 and all the participants, whether you came as a presenter or an attendee. We hope that there was ample time for discussion and opportunity to make new acquaintances. Finally, we hope that you experienced an interesting and exciting conference and enjoyed your stay in Hong Kong.

October 2001

Yuan Y. Tang, Victor Wickerhauser Pong C. Yuen, C. H. Li

Organization

The Second International Conference on Wavelet Analysis and Applications is organized by the Department of Computer Science, Hong Kong Baptist Univeristy and IEEE Hong Kong Section Computer Chapter.

Organizing Committee

Congress Chair:

Ernest C. M. Lam

General Chairs:

John Daugman Ernest C. M. Lam

Program Chairs:

Yuan Y. Tang Victor Wickerhauser P. C. Yuen

Organizing Chair:

Kelvin C. K. Wong

Local Arrangement Chair:

William K. W. Cheung

Registration & Finance Chair: K. C. Tsui Publications Chairs:

C. H. Li M. W. Mak

Workshop Chair:

Samuel P. M. Choi

Publicity Chair:

C. S. Huang

Sponsors

Hong Kong Baptist University Croucher Foundation IEEE Hong Kong Section Computer Chapter

Organization

VII

Program Committee Metin Akay Akram Aldroubi Claudia Angelini Algirdas Bastys T. D. Bui Elvir Causevic Mariantonia Cotronei Hans L. Cycon Dao-Qing Dai Wolfgang Dahmen Donggao Deng T. N. T. Goodman D. Hardin Daren Huang Wen-Liang Hwang Rong-Qing Jia P. Jorgensen K. S. Lau Seng-Luan Lee Jian-Ping Li Wei Lin Guixing Luan Hong Ma Peter Oswald Lizhong Peng Valrie Perrier S. D. Riemenschneider Zuowei Shen Guoxiang Song Georges Stamon Chew-Lim Tan Michael Unser Jianzhong Wang Yueshen Xu Lihua Yang Rongmao Zhang Xingwei Zhou

Dartmouth College Vanderbilt University Istituto per Applicazioni della Matematica Vilnius University Concordia University Everest Biomedical Instrument Company Universita’ di Messina Fachhochschule fur Technik und Wirtschaft Berlin Zhongshan University Technische Hochschule Aachen Zhongshan University University of Dundee Vanderbilt University Zhongshan University Institute of Information Science University of Alberta University of Iowa Chinese University of Hong Kong National University of Singapore Logistical Engineering University Zhongshan University Shenyang Inst. of Computing Technology Sichuan University Bell Laboratories, Lucent Technologies Peking University Domaine Universitaire West Virgina University National University of Singapore XiDian University University Rene Descartes National University of Singapore Batiment de Microtechnique Sam Houston State University University of North Dakota Zhongshan University Shenyang Inst. of Computing Technology Nankai University

Table of Contents

Keynote Presentations Personal Identification in Real-Time by Wavelet Analysis of Iris Patterns . . . . 1 J. Daugman, OBE Hybrid Representations of Audiophonic Signals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 B. Torresani Singularity Detection from Autocovariance via Wavelet Packets . . . . . . . . . . . . . 3 M. V. Wickerhauser

Image Compression and Coding Empirical Evaluation of Boundary Policies for Wavelet-Based Image Coding . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 C. Schremmer Image-Feature Based Second Generation Watermarking in Wavelet Domain . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16 S. Guoxiang and W. Weiwei A Study on Preconditioning Multiwavelet Systems for Image Compression . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22 W. Kim and C.-C. Li Reduction of Blocking Artifacts in Both Spatial Domain and Transformed Domain . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37 W.-K. Ling and P. K.-S. Tam Simple and Fast Subband De-blocking Technique by Discarding the High Band Signals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44 W.-K. Ling and P. K-S. Tam A Method with Scattered Data Spline and Wavelets for Image Compression . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .49 L. Guan and L. Feng

Video Coding and Processing A Wavelet-Based Preprocessing for Moving Object Segmentation in Video Sequences . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54 L.-C. Liu, J.-C. Chien, H. Y. Chuang, and C.-C. Li

X

Table of Contents

Embedded Zerotree Wavelet Coding of Image Sequence . . . . . . . . . . . . . . . . . . . . 65 M. J´erˆ ome and N. Ellouze Wavelet-Based Video Compression Using Long-Term Memory Motion-Compensated Prediction and Context-Based Adaptive Arithmetic Coding . . . . . . . . . . . . . . . . . . . . . . . . . . . 76 D. Marpe, T. Wiegand, and H. L. Cycon Wavelets and Fractal Image Compression Based on Their Self-Similarity of the Space-Frequency Plane of Images . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87 Y. Ueno

Theory Integration of Multivariate Haar Wavelet Series . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99 S. Heinrich, F. J. Hickernell, and R.-X. Yue An Application of Continuous Wavelet Transform in Differential Equations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107 H.-Z. Qu, C. Xu, and Z. Ruizhen Stability of Biorthogonal Wavelet Bases in L2 (R) . . . . . . . . . . . . . . . . . . . . . . . . . 117 P. F. Curran and G. McDarby Characterization of Dirac Edge with New Wavelet Transform . . . . . . . . . . . . . 129 L. Yang, X. You, R. M. Haralick, I. T. Phillips, and Y. Y. Tang Wavelet Algorithm for the Numerical Solution of Plane Elasticity Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 139 Y. Shen and W. Lin Three Novel Models of Threshold Estimator for Wavelet Coefficients . . . . . . 145 S. Guoxiang and Z. Ruizhen The PSD of the Wavelet-Packet Modulation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 151 M. Li, Q. Peng, and S. Zhong Orthogonal Multiwavelets with Dilation Factor a . . . . . . . . . . . . . . . . . . . . . . . . . 157 S. Yang, Z. Cheng, and H. Wang

Image Processing A Wavelet-Based Image Indexing, Clustering, and Retrieval Technique Based on Edge Feature . . . . . . . . . . . . . . . . . . . . . . . . . . 164 M. Kubo, Z. Aghbari, K. S. Oh, and A. Makinouchi Wavelet Applications in Segmentation of Handwriting in Archival Documents . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 176 C. L. Tan, R. Cao, and P. Shen

Table of Contents

XI

Wavelet Packets for Lighting-Effects Determination . . . . . . . . . . . . . . . . . . . . . . . 188 A. Z. Kouzani, and S. H. Ong Translation-Invariant Face Feature Estimation Using Discrete Wavelet Transform . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 200 K. Ma and X. Tang Text Extraction Based on Nonlinear Frame . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 211 Y. Guan and L. Zhang A Wavelet Multiresolution Edge Analysis Method for Recovery of Depth from Defocused Images . . . . . . . . . . . . . . . . . . . . . . . . . . . . 217 Q. Wang, W. Hu, J. Hu, and K. Hu Construction of Finite Non-separable Orthogonal Filter Banks with Linear Phase and Its Application in Image Segmentation . . . . . . . . . . . . 223 H. Chen and S. Peng Mixture-State Document Segmentation Using Wavelet-Domain Hidden Markov Tree Models . . . . . . . . . . . . . . . . . . . . . . 230 Y. Y. Tang, Y. Hou, J. Song, and X. Yang Some Experiment Results on Feature Analyses of Stroke Sequence Free Matching Algorithms for On-Line Chinese Character Recognition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 237 M. L. Tak Automatic Detection Algorithm of Connected Segments for On-line Chinese Character Recognition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 242 M. L. Tak

Signal Processing

Speech Signal Deconvolution Using Wavelet Filter Banks . . . . . . . . . . . . . . . . . 248 W. Hu and R. Linggard A Proposal of Jitter Analysis Based on a Wavelet Transform . . . . . . . . . . . . . . 257 J. Borgosz and B. Cyganek Skewness of Gabor Wavelets and Source Signal Separation . . . . . . . . . . . . . . . . 269 W. Yu, G. Sommer, and K. Daniilidis The Application of the Wavelet Transform to Polysomnographic Signals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .284 M. MacCallum and A. E. A. Almaini Wavelet Transform Method of Waveform Estimation for Hilbert Transform of Fractional Stochastic Signals with Noise . . . . . . . . . 296 W. Su, H. Ma, Y. Y. Tang, and M. Umeda

XII

Table of Contents

Multiscale Kalman Filtering of Fractal Signals Using Wavelet Transform . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 305 J. Zhao, H. Ma, Z.-S. You, and M. Umeda General Analytic Construction for Wavelet Low-Passed Filters . . . . . . . . . . . . 314 J. P. Li and Y. Y. Tang A Design of Automatic Speech Playing System Based on Wavelet Transform . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 321 Y. Liu, J. Cen, Q. Sun, and L. Yang General Design of Wavelet High-Pass Filters from Reconstructional Symbol . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 326 L. Yang, Q. Chen, and Y. Y. Tang Realization of Perfect Reconstruction Non-uniform Filter Banks via a Tree Structure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 331 W.-K. Ling and P. K.-S. Tam Set of Decimators for Tree Structure Filter Banks . . . . . . . . . . . . . . . . . . . . . . . . . 336 W.-K. Ling and P. K.-S. Tam Set of Perfect Reconstruction Non-uniform Filter Banks via a Tree Structure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 341 W.-K. Ling and P. K.-S. Tam Systems and Applications Joint Time-Frequency Distributions for Business Cycle Analysis . . . . . . . . . . .347 S. Md. Raihan, Y. Wen, and B. Zeng The Design of Discrete Wavelet Transformation Chip . . . . . . . . . . . . . . . . . . . . . 359 Z. Razak and M. Yaacob On the Performance of Informative Wavelets for Classification and Diagnosis of Machine Faults . . . . . . . . . . . . . . . . . . . . . . . . 369 H. Ahmadi, R. Tafreshi, F. Sassani, and G. Dumont A Wavelet-Based Ammunition Doppler Radar System . . . . . . . . . . . . . . . . . . . . 382 S. H. Ong and A. Z. Kouzani The Application of Wavelet Analysis Method to Civil Infrastructure Health Monitoring . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 393 J. P. Li, S. A. Yan, and Y. Y. Tang Piecewise Periodized Wavelet Transform and Its Realization, Properties and Applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 398 W.-K. Ling and P. K.-S. Tam Wavelet Transform and Its Application to Decomposition of Gravity Anomalies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 404 H. Zunze

Table of Contents

XIII

Computations of Inverse Problem by Using Wavelet in Multi-layer Soil . . . . 411 B. Wu, S. Liu, and Z. Deng Wavelets Approach in Choosing Adaptive Regularization Parameter . . . . . . 418 F. Lu, Z. Yang, and Y. Li DNA Sequences Classification Based on Wavelet Packet Analysis . . . . . . . . . .424 J. Zhao, X. W. Yang, J. P. Li, and Y. Y. Tang The Application of the Wavelet Transform to the Prediction of Gas Zones . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 430 X. W. Yang, J. Zhao, J. P. Li, J. Liu, and S. P. Zeng Parameterizations of M-Band Biorthogonal Wavelets . . . . . . . . . . . . . . . . . . . . . . 435 Z. Zhang and D. Huang Author Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .449

Author Index

Aghbari, Z. . . . . . . . . . . . . . . . . . . . . 164 Ahmadi, H. . . . . . . . . . . . . . . . . . . . . 369 Almaini, A. E. A. . . . . . . . . . . . . . . 284 Borgosz, J. . . . . . . . . . . . . . . . . . . . . 257 Cao, R. . . . . . . . . . . . . . . . . . . . . . . . . 176 Cen, J. . . . . . . . . . . . . . . . . . . . . . . . . 321 Chen, H. . . . . . . . . . . . . . . . . . . . . . . 223 Chen, Q. . . . . . . . . . . . . . . . . . . . . . . 326 Cheng, Z. . . . . . . . . . . . . . . . . . . . . . . 157 Chien, J.-C. . . . . . . . . . . . . . . . . . . . . 54 Chuang, H. Y. . . . . . . . . . . . . . . . . . . 54 Curran, P. F. . . . . . . . . . . . . . . . . . . 117 Cycon, H. L. . . . . . . . . . . . . . . . . . . . . 76 Cyganek, B. . . . . . . . . . . . . . . . . . . . 257 Daniilidis, K. . . . . . . . . . . . . . . . . . . 269 Daugman, J. . . . . . . . . . . . . . . . . . . . . . 1 Deng, Z. . . . . . . . . . . . . . . . . . . . . . . . 411 Dumont, G. . . . . . . . . . . . . . . . . . . . 369 Ellouze, N. . . . . . . . . . . . . . . . . . . . . . .65 Feng, L. . . . . . . . . . . . . . . . . . . . . . . . . 49 Guan, L. . . . . . . . . . . . . . . . . . . . . . . . .49 Guan, Y. . . . . . . . . . . . . . . . . . . . . . . 211 Guoxiang, S. . . . . . . . . . . . . . . . 16, 145 Haralick, R. M. . . . . . . . . . . . . . . . . 129 Heinrich, S. . . . . . . . . . . . . . . . . . . . . . 99 Hickernell, F. J. . . . . . . . . . . . . . . . . .99 Hou, Y. . . . . . . . . . . . . . . . . . . . . . . . .230 Hu, J. . . . . . . . . . . . . . . . . . . . . . . . . . 217 Hu, K. . . . . . . . . . . . . . . . . . . . . . . . . . 217 Hu, W. . . . . . . . . . . . . . . . . . . . 217, 248 Huang, D. . . . . . . . . . . . . . . . . . . . . . 435

J´erˆome, M. . . . . . . . . . . . . . . . . . . . . . 65 Kim, W. . . . . . . . . . . . . . . . . . . . . . . . . 22 Kouzani, A. Z. . . . . . . . . . . . . 188, 382 Kubo, M. . . . . . . . . . . . . . . . . . . . . . . 164 Li, C.-C. . . . . . . . . . . . . . . . . . . . . 22, 54 Li, J. P. . . . . . . . . . 314, 393, 424, 430 Li, M. . . . . . . . . . . . . . . . . . . . . . . . . . 151 Li, Y. . . . . . . . . . . . . . . . . . . . . . . . . . .418 Lin, W. . . . . . . . . . . . . . . . . . . . . . . . . 139 Ling, W.-K. . . 37, 44, 331, 336, 341, 398 Linggard, R. . . . . . . . . . . . . . . . . . . . 248 Liu, J. . . . . . . . . . . . . . . . . . . . . . . . . . 430 Liu, L.-C. . . . . . . . . . . . . . . . . . . . . . . . 54 Liu S. . . . . . . . . . . . . . . . . . . . . . . . . . 411 Liu, Y. . . . . . . . . . . . . . . . . . . . . . . . . 321 Lu, F. . . . . . . . . . . . . . . . . . . . . . . . . . 418 Ma, H. . . . . . . . . . . . . . . . . . . . . 296, 305 Ma, K. . . . . . . . . . . . . . . . . . . . . . . . . 200 MacCallum, M. . . . . . . . . . . . . . . . . 284 Makinouchi, A. . . . . . . . . . . . . . . . . 164 Marpe, D. . . . . . . . . . . . . . . . . . . . . . . 76 McDarby, G. . . . . . . . . . . . . . . . . . . .117 Oh, K. S. . . . . . . . . . . . . . . . . . . . . . . 164 Ong, S. H. . . . . . . . . . . . . . . . . 188, 382 Peng, Q. . . . . . . . . . . . . . . . . . . . . . . . 151 Peng, S. . . . . . . . . . . . . . . . . . . . . . . . 223 Phillips, I. T. . . . . . . . . . . . . . . . . . . 129 Qu, H.-Z. . . . . . . . . . . . . . . . . . . . . . . 107 Raihan, S. Md. . . . . . . . . . . . . . . . . 347 Razak, Z. . . . . . . . . . . . . . . . . . . . . . . 359

450

Author Index

Ruizhen, Z. . . . . . . . . . . . . . . . 107, 145 Sassani, F. . . . . . . . . . . . . . . . . . . . . . 369 Schremmer, C. . . . . . . . . . . . . . . . . . . . 4 Shen, P. . . . . . . . . . . . . . . . . . . . . . . . 176 Shen, Y. . . . . . . . . . . . . . . . . . . . . . . . 139 Sommer, G. . . . . . . . . . . . . . . . . . . . 269 Song, J. . . . . . . . . . . . . . . . . . . . . . . . 230 Su, W. . . . . . . . . . . . . . . . . . . . . . . . . 296 Sun, Q. . . . . . . . . . . . . . . . . . . . . . . . . 321 Tafreshi, R. . . . . . . . . . . . . . . . . . . . . 369 Tak, M. L. . . . . . . . . . . . . . . . . 237, 242 Tam, P. K.-S. 37, 44, 331, 336, 341, 398 Tan, C. L. . . . . . . . . . . . . . . . . . . . . . 176 Tang, X. . . . . . . . . . . . . . . . . . . . . . . . 200 Tang, Y. Y. 129, 230, 296, 314, 326, 393, 424 Torresani, B. . . . . . . . . . . . . . . . . . . . . . 2 Umeda, M. . . . . . . . . . . . . . . . 296, 305 Ueno, Y. . . . . . . . . . . . . . . . . . . . . . . . . 87 Wang, H. . . . . . . . . . . . . . . . . . . . . . . 157 Wang, Q. . . . . . . . . . . . . . . . . . . . . . . 217 Weiwei, W. . . . . . . . . . . . . . . . . . . . . . 16 Wen, Y. . . . . . . . . . . . . . . . . . . . . . . . 347

Wickerhauser, M. V. . . . . . . . . . . . . . 3 Wiegand, T. . . . . . . . . . . . . . . . . . . . . 76 Wu, B. . . . . . . . . . . . . . . . . . . . . . . . . 411 Xu, C. . . . . . . . . . . . . . . . . . . . . . . . . . 107 Yaacob, M. . . . . . . . . . . . . . . . . . . . . 359 Yan, S. A. . . . . . . . . . . . . . . . . . . . . . 393 Yang, L. . . . . . . . . . . . . . 129, 321, 326 Yang, S. . . . . . . . . . . . . . . . . . . . . . . . 157 Yang, X. . . . . . . . . . . . . . . . . . . . . . . .230 Yang, X. W. . . . . . . . . . . . . . . 424, 430 Yang, Z. . . . . . . . . . . . . . . . . . . . . . . . 418 You, X. . . . . . . . . . . . . . . . . . . . . . . . . 129 You, Z.-S. . . . . . . . . . . . . . . . . . . . . . 305 Yu, W. . . . . . . . . . . . . . . . . . . . . . . . . 269 Yue, R.-X. . . . . . . . . . . . . . . . . . . . . . . 99 Zeng, B. . . . . . . . . . . . . . . . . . . . . . . . 347 Zeng, S. P. . . . . . . . . . . . . . . . . . . . . .430 Zhang, L. . . . . . . . . . . . . . . . . . . . . . . 211 Zhang, Z. . . . . . . . . . . . . . . . . . . . . . . 435 Zhao, J. . . . . . . . . . . . . . . . . . . 424, 430 Zhao, J. . . . . . . . . . . . . . . . . . . . . . . . 305 Zhong, S. . . . . . . . . . . . . . . . . . . . . . . 151 Zunze, H. . . . . . . . . . . . . . . . . . . . . . . 404

Personal Identification in Real-Time by Wavelet Analysis of Iris Patterns John Daugman, OBE The Computer Laboratory, University of Cambridge, UK

Abstract. The central issue in pattern recognition is the relation between within-class variability and between-class variability. These are determined by the various degrees-of-freedom spanned by the patterns themselves, and by the selectivity of the chosen feature encoders. An interesting application of 2D wavelets in computer vision is the automatic recognition of personal identity by encoding and matching the complex patterns visible at a distance in each eye’s iris. Because the iris is a protected, internal, organ whose random texture is highly unique and stable over life, it can serve as a kind of living password or passport that one need not remember but is always in one’s possession. I will describe wavelet demodulation methods that I have developed for this problem over the past 10 years, and which are now installed in all existing commercial systems for iris recognition. The principle that underlies iris recognition is the failure of a test of statistical independence performed on the phase angle sequences of iris patterns. Quadrature 2D Gabor wavelets spanning 3 octaves in scale enable the complex-valued assignment of local phasor coordinates to iris patterns. The combinatorial complexity of these phase sequences spans about 244 independent degrees-of-freedom, and generates binomial distributions for the Hamming Distances (a similarity metric) between different irises. In six public independent field trials conducted so far using these algorithms, involving several millions of iris comparisons, there has never been a single false match recorded. The time required to locate and to encode an iris into quantized wavelet phase sequences is 1 second. Then database searches are performed at a rate of 100,000 irises/second. Data will be presented in this talk from 2.3 million IrisCode comparisons. This wavelet application could be used in a wide range of settings in which persons’ identities must be established or confirmed by large scale database search, without relying upon cards, keys, documents, secrets, passwords or PINs.

Y. Y. Tang et al. (Eds.): WAA 2001, LNCS 2251, p. 1, 2001. c Springer-Verlag Berlin Heidelberg 2001


Hybrid Representations of Audiophonic Signals Bruno Torresani LATP, CMI, Universit´e de Provence, France

Abstract. A new approach for modeling audio signal will be presented, in view of efficient encoding. The method is based upon hybrid models featuring transient, tonal and stochastic components in the signal. The three components are estimated and encoded independently using a strategy very much in the spirit of transform coding. The signal models involve nonlinear expansions on local trigonometric bases, and binary trees of wavelet coefficients. Unlike several existing approaches, the method does not rely on any prior segmentation of the signal. The talk is based on joint works with L. Daudet and S. Molla.

Y. Y. Tang et al. (Eds.): WAA 2001, LNCS 2251, p. 2, 2001. c Springer-Verlag Berlin Heidelberg 2001


Singularity Detection from Autocovariance via Wavelet Packets M. Victor Wickerhauser Department of Mathematics, Washington University, USA

Abstract. We use the eigenvalues of a version of the autocovariance matrix to recognize directions at which the Fourier transform of a function is slowly decreasing, which provides us with a technique to detect singularities in images. In very high dimensions, we show how the wavelet packet best-basis algorithm can be used to compute these eigenvalues approximately, at relatively low computational complexity.

Y. Y. Tang et al. (Eds.): WAA 2001, LNCS 2251, p. 3, 2001. c Springer-Verlag Berlin Heidelberg 2001


Empirical Evaluation of Boundary Policies for Wavelet-Based Image Coding Claudia Schremmer Praktische Informatik IV Universit¨ at Mannheim, 68131 Mannheim, Germany [email protected]

Abstract. The wavelet transform has become the most interesting new algorithm for still image compression. Yet there are many parameters within a wavelet analysis and synthesis which govern the quality of a decoded image. In this paper, we discuss different image boundary policies and their implications for the decoded image. A pool of gray–scale images has been wavelet–transformed at different settings of the wavelet filter bank and quantization threshold and with three possible boundary policies. Our empirical evaluation is based on three benchmarks: a first judgment regards the perceived quality of the decoded image. The compression rate is a second crucial factor. Finally, the best parameter settings with regard to these two factors is weighted with the cost of implementation. Contrary to the JPEG2000 standard, where mirror padding is implemented, our investigation proposes circular convolution as the boundary treatment. Keywords: Wavelet Analysis, Boundary Policies, Empirical Evaluation

1

Introduction

Due to its outstanding performance in compression, the wavelet transform is the focus of new image coding techniques such as the JPEG2000 standard [8,4]. JPEG2000 proposes a reversible (Daub 5/3–tap) and an irreversible (Daub 9/7– tap) wavelet filter bank. However, since we were interested in how filter length affects the quality of image coding, we investigated the orthogonal and separable wavelet filters developed by Daubechies [2]. These belong to the group of wavelets used most often in image coding applications. They specify a number n0 of vanishing moments: if a wavelet has n0 vanishing moments, then the approximation order of the wavelet transform is also n0 . Implementations of the wavelet transform on still images entail other aspects as well: speed, decomposition depth, and boundary treatment policies. Long filters require more computing time than short ones. Furthermore, the (dyadic) wavelet transform incorporates the aspect of iteration: the low–pass filter defines an approximation of the original signal that contains only half as many coefficients. This approximation successively builds the input for the next approximation. For compression purposes, coefficients in the time–scale domain Y. Y. Tang et al. (Eds.): WAA 2001, LNCS 2251, pp. 4–15, 2001. c Springer-Verlag Berlin Heidelberg 2001


Empirical Evaluation of Boundary Policies

5

are discarded and the synthesis quality improves with the number of iterations on the approximation. Finally, the wavelet transform is mathematically defined only within a signal; image applications thus need to solve the boundary problem. Depending on the boundary policy selected, the number of iterations in a wavelet transform might vary with the filter length. Moreover, the longer the filter length, the more important the boundary policy becomes. In this work, we investigate the effects of three different boundary policies in combination with different wavelet filter banks on a number of gray–scale images. A first determining factor is the visual perception of a decoded image. As we will see, although the quality varies strongly with the selected image, for a given image it remains relatively unconcerned about the parameter settings. A second crucial factor is therefore the expected compression rate. Finally, the cost of implementation weights these two benchmarks. Our empirical evaluation leads us to recommend circular convolution as the boundary treatment, contrary to JPEG2000 which proposes padding. The article is organized as follows. In Section 2, we cite related work on wavelet filter evaluation. Section 3 reviews the wavelet transform and details the aspects that are important for our survey. In Section 4, we present the technical evaluation of the wavelet transform and detail our results. The article ends in Section 5 with an outlook on future work.

2

Related Work

Villasenor’s group researches wavelet filters for image compression. In [10], the focus is on biorthogonal filters, and the evaluation is based on the information preserved in the reference signal, while [3] focuses on a mathematically optimal quantizer step size. In [1], the evaluation is based on lossless as well as on subjective lossy compression performance, complexity and memory usage. An interpretation of why the observations are made is nevertheless lacking. Strutz has thoroughly researched the dyadic wavelet transform in [9]: the design and construction of different wavelet filters is investigated, as are good Huffman and arithmetic encoding strategies. An investigation of boundary policies, however, is lacking.

3

The Wavelet Transform

A wavelet is an (ideally) compact function, i.e., outside a certain interval it vanishes. Implementations are based on the fast wavelet transform, where a given wavelet (i.e., mother wavelet) is shifted and dilated so as to provide a base in the function space. That is, a one–dimensional function is transformed into a two– dimensional space, where it is approximated by coefficients that depend on time (determined by the translation parameter) and on scale, i.e., frequency (determined by the dilation parameter). The localization of a wavelet in time spread (σt ) and frequency spread (σω ) has the property σt σω = const. However, the resolution in time and frequency depends on the frequency. This is the so–called

6

Claudia Schremmer

zoom phenomenon of the wavelet transform: it offers high temporal localization for high frequencies while offering good frequency resolution for low frequencies. 3.1

Wavelet Transform and Filter Banks

By introducing multiresolution, Mallat [7] made an important contribution to the application of wavelet theory to multimedia: the transition from mathematical theory to filters. Multiresolution analysis is implemented via high–pass, respectively, band–pass filters (i.e., wavelets) and low–pass filters (i.e., scaling functions): The detail coefficients (resulting from the high–pass, respectively, band–pass filtering) of every iteration step are kept apart, and the iteration starts again with the remaining approximation coefficients (from application of the low–pass filter). This multiresolution theory is ‘per se’ defined only for one–dimensional wavelets on one–dimensional signals. As still images are two– dimensional discrete signals and two–dimensional wavelet filter design remains an active field of research [5][6], current implementations are restricted to separable filters. The successive convolution of filter and signal in both dimensions opens two potential iterations: – standard: all approximations, even in mixed terms, are iterated, and – non–standard: only the purely low–pass filtered parts of every approximation enter the iteration. In this work, we concentrate on the non–standard decomposition. 3.2

Image Boundary

A digital filter is applied to a signal by convolution. Convolution, however, is defined only within a signal. In order to result in a reversible wavelet transform, each signal coefficient must enter into filter length/2 calculations of convolution (here, the subsampling process by factor 2 is already incorporated). Consequently, every filter longer than two entries, i.e., every filter except Haar, requires a solution for the boundary. Furthermore, images are signals of a relatively short length (in rows and columns), thus the boundary treatment is even more important than e.g. in audio coding. Two common boundary policies are padding and circular convolution. Padding Policies. With padding, the coefficients of the signal on either border are padded with filter length-2 coefficients. Consequently, each signal coefficient enters into filter length/2 calculations of convolution, and the transform is reversible. Many padding policies exist; they all have in common that each iteration step physically increases the storage space in the wavelet domain. In [11], a theoretical solution for the required storage space (depending on the signal, the filter bank and the iteration level) is presented. Nevertheless, its implementation remains sophisticated.

Empirical Evaluation of Boundary Policies

7

Circular Convolution. The idea of circular convolution is to ‘wrap’ the end of a signal to its beginning or vice versa. In so doing, circular convolution is the only boundary treatment to maintain the number of coefficients for a wavelet transform, thus simplifying storage management1 . A minor drawback is that the time information contained in the time–scale domain of the wavelet–transformed coefficients ‘blurs’: the coefficients in the time–scale domain that are next to the right border (respectively, left border) also affect signal coefficients that are located on the left (respectively, right). The selected boundary policy has an important impact on the iteration behavior of the wavelet transform. It does not affect the iteration behavior of padding policies. However, with circular convolution, the decomposition depth varies with the filter length: the longer the filter, the fewer the number of decomposition iterations possible. For example, for an image of 256 × 256 pixels, the Daub–2 filter bank with 4 coefficients allows a decomposition depth of 7, while the Daub–20 filter bank with 40 coefficients has reached signal length after only 3 decomposition levels. Thus, the evaluation presented in Tables 1 to 4 is based on a decomposition depth of level 8 for the two padding policies, while the decomposition depth for circular convolution varies from 7 to 3, according to the selected filter length.

4

Empirical Evaluation

4.1

Set-Up

Our empirical evaluation sought the best parameter settings for the choice of the wavelet filter bank and for the image boundary policy to be implemented. The performance was evaluated according to the criteria: 1. visual quality, 2. compression rate, and 3. complexity of implementation. The quality was rated based on the peak signal–to–noise ratio (PSNR)2 . The compression rate was simulated by a simple quantization threshold: the higher the threshold, the more coefficients in the time–scale domain are discarded, the higher is the compression rate. More precisely, the threshold was carried out only on the parts of the image that have been high–pass filtered (respectively, band–pass filtered) at least once. That is, the approximation of the image was excluded from the thresholding due to its importance for the image synthesis. 1

2

Storage space, however, expands indirectly: an image can be stored with integers, while the coefficients in the time–scale domain require floats. When org(x, y) depicts the pixel value of the original image at position (x, y), and dec(x, y) denotes the
pixel value of the decodedimage at position (x, y), then PSNR [dB] = 10 · log

xy



xy

2552

(org(x,y)−dec(x,y))2

.

8

Claudia Schremmer

Our evaluation was set up on the six gray–scale images of size 256 × 256 pixels demonstrated in Figure 1. These test images have been chosen in order to comply with different features: – contain many small details: Mandrill, Goldhill, – contain large uniform areas: Brain, Lena, Camera, House, – be relatively symmetric at the left–right and top–bottom boundaries: Mandrill, Brain, – be very asymmetric with regard to these boundaries: Lena, Goldhill, House, – have sharp transitions between regions: Brain, Lena, Camera, House, and – contain large areas of texture: Mandrill, Lena, Goldhill, House. 4.2

Results

Image-Dependent Analysis. The detailed evaluation results for the six test images are presented in Tables 1 and 2. Some interesting observations made from these two tables and their explanations are as follows: – For a given image and a given quantization threshold, the PSNR remains astonishingly constant for different filter banks and different boundary policies. – At high thresholds, Mandrill and Goldhill yield the worst quality. This is due to the large amount of details in both images. – House produces the overall best quality at a given threshold. This is due to its large uniform areas. – Due to their symmetry, Mandrill and Brain show good quality results with padding policies. – The percentage of discarded information at a given threshold is far higher for Brain than for Mandrill. This is due to the uniform black background of Brain, which produces small coefficients in the time–scale domain, compared to the many small details in Mandrill which produce large coefficients and thus do not fall below the threshold. – With regard to the heuristic for compression, and for a given image and boundary policy, Table 2 reveals that • the compression ratio for zero padding increases with increasing filter length, • the compression ratio for mirror padding decreases with increasing filter length, and • the compression ratio for circular convolution varies, but most often stays almost constant. The explanation is as follows. Padding an image with zeros, i.e., black pixel values, most often produces a sharp contrast to the original image, thus the sharp transition between the signal and the padding coefficients results in large coefficients in the fine scales, while the coarse scales remain unaffected. This observation, however, is put into a different perspective for longer filters: With longer filters, the constant run of zeros at the boundary does not show

Empirical Evaluation of Boundary Policies

9

strong variations, and the detail coefficients in the time–scale domain thus remain small. Hence, a given threshold cuts off fewer coefficients when the filter is longer. With mirror padding, the padded coefficients for shorter filters represent a good heuristic for the signal adjacent to the boundary. Increasing filter length and accordingly, longer padded areas, however, introduces too much ‘false’ detail information into the signal, resulting in many large detail coefficients that ‘survive’ the threshold. Image-Independent Analysis. The above examples reveal that most phenomena are signal–dependent. As a signal–dependent determination of best– suited parameters remains academic, our further reflections are made on the average image quality and the average amount of discarded information as presented in Tables 3 and 4 and the corresponding Figures 2 and 3. Figure 2 visualizes the coding quality of the images, averaged over the six test images. The four plots represent the quantization thresholds λ = 10, 20, 45 and 85. In each graphic, the visual quality (quantified via PSNR) is plotted against the filter length of the Daubechies wavelet filters. The three boundary policies: zero padding, mirror padding and circular convolution are regarded separately. The plots obviously reveal that the quality decreases with an increasing threshold. More important are the following statements: – Within a given threshold, and for a given boundary policy, the PSNR remains almost constant. This means that the quality of the coding process depends hardly or not at all on the selected wavelet filter bank. – Within a given threshold, mirror padding produces the best results, followed by circular convolution. Zero padding performs worst. – The gap between the performance of the boundary policies increases with an increasing threshold. Nevertheless, the differences observed above with 0.28 dB maximum gap (at the threshold λ = 85 and the filter length of 40 coefficients) are so marginal that they do not actually influence visual perception. As the visual perception is neither influenced by the choice of filter nor by the boundary policy, the coding performance has been studied as a second benchmark. The following observations are made in Figure 3. With a short filter length (4 to 10 coefficients), the compression ratio is almost identical for the different boundary policies. This is not astonishing as short filters involve only little boundary treatment, and the relative importance of the boundary coefficients with regard to the signal coefficients is negligible. More important for our investigation is that: – The compression heuristic for each of the three boundary policies is inversely proportional to their quality performance. In other words, mirror padding discards the least number of coefficients at a given quantization threshold, while zero padding discards the most.

10

Claudia Schremmer

– With an increasing threshold, the gap between the compression ratios of the three policies narrows. In the overall evaluation, we have seen that mirror padding performs best with regard to quality, while it performs worst with regard to compression. Inversely, zero padding performs best with regard to compression and worst with regard to quality. Circular convolution holds the midway in both aspects. On the other hand, the gap in compression is by far superior to the differences in quality. Calling to mind the coding complexity of the padding approaches, compared to the easy implementation of circular convolution (see Section 3.2), we strongly recommend to implement circular convolution as the boundary policy in image coding.

5

Conclusion

We have discussed and evaluated the strengths and weaknesses of different boundary policies in relation to various orthogonal wavelet filter banks. Contrary to the JPEG2000 coding standard, where mirror padding is suggested for boundary treatment, we have proven that circular convolution is superior in the overall combination of quality performance, compression performance and ease of implementation. In future work, we will improve our heuristic on the compression rate and rely on the calculation of a signal’s entropy such as it is presented in [12] and [9].

References 1. Michael D. Adams and Faouzi Kossentini. Performance Evaluation of Reversible Integer–to–Integer Wavelet Transforms for Image Compression. In Proc. IEEE Data Compression Conference, page 514 ff., Snowbird, Utah, March 1999. 5 2. Ingrid Daubechies. Ten Lectures on Wavelets, volume 61. SIAM. Society for Industrial and Applied Mathematics, Philadelphia, PA, 1992. 4 3. Javier Garcia-Frias, Dan Benyamin, and John D. Villasenor. Rate Distortion Optimal Parameter Choice in a Wavelet Image Communication System. In Proc. IEEE International Conference on Image Processing, pages 25–28, Santa Barbara, CA, October 1997. 5 4. ITU. JPEG2000 Image Coding System. Final Committee Draft Version 1.0 – FCD15444-1. International Telecommunication Union, March 2000. 4 5. Jelena Kovaˇcevi´c and Wim Sweldens. Wavelet Families of Increasing Order in Arbitrary Dimensions. IEEE Trans. on Image Processing, 9(3):480–496, March 2000. 6 6. Jelena Kovaˇcevi´c and Martin Vetterli. Nonseparable Two– and Three–Dimensional Wavelets. IEEE Trans. on Signal Processing, 43(5):1269–1273, May 1995. 6 7. St´ephane Mallat. A Wavelet Tour of Signal Processing. Academic Press, San Diego, CA, 1998. 6 8. Athanassios N. Skodras, Charilaos A. Christopoulos, and Touradj Ebrahimi. JPEG2000: The Upcoming Still Image Compression Standard. In 11th Portuguese Conference on Pattern Recognition, pages 359–366, Porto, Portugal, May 2000. 4

Empirical Evaluation of Boundary Policies

11

9. Tilo Strutz. Untersuchungen zur skalierbaren Kompression von Bildsequenzen bei niedrigen Bitraten unter Verwendung der dyadischen Wavelet–Transformation. PhD thesis, Universit¨ at Rostock, Germany, May 1997. 5, 10 10. John D. Villasenor, Benjamin Belzer, and Judy Liao. Wavelet Filter Evaluation for Image Compression. IEEE Trans. on Image Processing, 2:1053–1060, August 1995. 5 11. Mladen Victor Wickerhauser. Adapted Wavelet Analysis from Theory to Software. A. K. Peters Ltd., Natick, MA, 1998. 6 12. Mathias Wien and Claudia Meyer. Adaptive Block Transform for Hybrid Video Coding. In Proc. SPIE Visual Communications and Image Processing, pages 153– 162, San Jose, CA, January 2001. 10

(a) Mandrill

(b) Brain

(c) Lena

(d) Camera

(e) Goldhill

(f) House

Fig. 1. Test images for the evaluation

12

Claudia Schremmer

Table 1. Detailed results of the quality evaluation with the PSNR on the six test images. The mean values over the images are given in Table 3 Quality of visual perception — PSNR [dB] Wavelet

zero mirror circular zero mirror circular zero mirror circular padding padding convol. padding padding convol. padding padding convol. Mandrill

Daub–2 Daub–3 Daub–4 Daub–5 Daub–10 Daub–15 Daub–20

18.012 18.157 18.169 18.173 17.977 17.938 17.721

17.996 18.187 18.208 18.167 17.959 17.934 17.831

Daub–2 Daub–3 Daub–4 Daub–5 Daub–10 Daub–15 Daub–20

14.298 14.414 14.231 14.257 14.268 14.246 14.046

14.350 14.469 14.239 14.216 14.274 14.258 14.065

Daub–2 Daub–3 Daub–4 Daub–5 Daub–10 Daub–15 Daub–20

10.905 10.988 10.845 10.918 10.907 10.845 10.784

10.885 10.970 10.839 10.969 10.929 10.819 10.872

Daub–2 Daub–3 Daub–4 Daub–5 Daub–10 Daub–15 Daub–20

9.095 9.206 9.160 9.171 9.207 9.083 9.071

9.121 9.184 9.152 9.208 9.193 9.161 9.142

Daub–2 Daub–3 Daub–4 Daub–5 Daub–10 Daub–15 Daub–20

17.334 17.532 17.529 17.489 17.539 17.747 17.474

17.346 17.560 17.591 17.448 17.541 17.530 17.527

Daub–2 Daub–3 Daub–4 Daub–5 Daub–10 Daub–15 Daub–20

14.387 14.473 14.438 14.460 14.468 14.408 14.384

14.365 14.452 14.438 14.505 14.400 14.406 14.370

Daub–2 Daub–3 Daub–4 Daub–5 Daub–10 Daub–15 Daub–20

12.213 12.032 12.150 12.077 12.061 12.074 11.798

12.242 12.122 12.178 12.133 12.197 12.059 11.975

Daub–2 Daub–3 Daub–4 Daub–5 Daub–10 Daub–15 Daub–20

11.035 11.092 10.943 11.018 10.815 10.779 10.688

11.161 11.176 11.152 11.148 11.064 11.005 11.031

Camera

Brain Threshold: 10 — Excellent overall quality 18.238 18.141 18.151 18.197 16.392 18.221 18.429 18.434 18.433 16.391 17.963 18.353 18.340 18.248 16.294 18.186 18.279 18.280 18.259 16.543 18.009 18.291 18.300 18.479 16.249 18.022 18.553 18.543 18.523 16.267 18.026 18.375 18.357 18.466 16.252 Threshold: 20 — Good overall quality 14.403 16.610 16.611 16.577 14.775 14.424 16.743 16.755 16.721 14.758 14.276 16.637 16.628 16.734 14.862 14.269 16.747 16.751 16.854 14.739 14.360 16.801 16.803 16.878 14.624 14.300 16.822 16.810 16.852 14.395 14.227 16.953 16.980 16.769 14.252 Threshold: 45 — Medium overall quality 10.910 14.815 14.816 14.747 13.010 10.948 15.187 15.150 15.052 12.766 10.885 15.014 15.029 15.056 12.820 10.949 15.036 15.031 14.999 12.913 10.913 14.989 15.013 15.212 12.447 10.815 15.093 15.133 15.064 12.577 10.843 14.975 14.934 14.882 12.299 Threshold: 85 — Poor overall quality 9.135 13.615 13.621 13.783 11.587 9.124 13.787 13.784 13.759 11.437 9.168 13.792 13.815 13.808 11.539 9.203 13.837 13.850 13.705 11.692 9.206 13.870 13.922 14.042 11.128 9.126 13.731 13.795 13.917 11.128 9.204 13.852 13.800 13.974 11.142 Goldhill Threshold: 10 — Excellent overall quality 17.371 16.324 16.266 16.412 19.575 17.625 16.322 16.296 16.358 19.640 17.577 16.241 16.212 16.342 19.560 17.389 16.214 16.193 16.154 19.613 17.383 16.307 16.223 16.317 19.482 17.523 16.012 16.067 16.033 19.653 17.484 16.322 16.245 16.319 19.550 Threshold: 20 — Good overall quality 14.396 13.937 13.940 13.898 17.446 14.426 13.872 13.892 13.858 17.525 14.430 13.828 13.836 13.753 17.468 14.427 13.743 13.743 13.711 17.454 14.409 13.762 13.785 13.798 17.592 14.414 13.687 13.730 13.697 17.260 14.362 13.700 13.782 13.731 17.476 Threshold: 45 — Medium overall quality 12.131 12.033 12.034 11.876 15.365 12.188 11.961 12.006 11.889 14.957 12.145 11.855 11.891 11.925 14.906 12.120 11.848 11.844 11.801 15.159 12.093 11.760 11.917 11.726 14.776 12.176 11.725 11.855 11.753 14.810 12.048 11.763 11.803 11.703 14.420 Threshold: 85 — Poor overall quality 11.041 10.791 10.805 10.844 13.530 11.080 10.943 10.916 10.754 13.488 11.046 10.861 10.904 10.740 13.524 11.129 10.826 10.935 10.738 13.114 10.987 10.824 10.972 10.771 13.158 10.982 10.737 10.838 10.607 13.073 11.090 10.709 10.819 10.766 13.173

Lena 16.288 16.402 16.355 16.561 16.278 16.304 16.470

16.380 16.350 16.260 16.527 16.214 16.288 16.238

14.765 14.817 14.918 14.946 14.840 14.631 14.597

14.730 14.687 14.735 14.815 14.699 14.477 14.353

13.052 13.138 13.132 13.301 13.066 12.954 12.877

12.832 12.903 12.818 12.983 12.795 12.686 12.640

11.902 11.793 11.806 11.790 11.430 11.610 11.694

11.577 11.516 11.636 11.872 11.555 11.475 11.597

House 19.563 19.630 19.558 19.555 19.388 19.671 19.495

19.608 19.621 19.584 19.566 19.732 19.726 19.524

17.480 17.594 17.647 17.458 17.635 17.276 17.449

17.471 17.612 17.351 17.465 17.689 17.266 17.240

15.437 15.476 15.080 15.382 15.246 15.090 15.033

15.155 15.118 15.180 15.244 14.872 14.969 14.609

13.804 13.726 13.613 13.903 13.695 13.357 13.257

13.703 13.627 13.510 13.111 13.434 13.123 13.678

Empirical Evaluation of Boundary Policies

13

Table 2. Heuristic for the compression rate of the coding parameters of Table 1: The higher the percentage of discarded information in the time–scale domain is, the higher is the compression ratio. The mean values over the images are given in Table 4 Discarded information in the time–scale domain — Percentage [%] Wavelet

zero mirror circular zero mirror circular zero mirror circular padding padding convol. padding padding convol. padding padding convol. Mandrill

Daub–2 Daub–3 Daub–4 Daub–5 Daub–10 Daub–15 Daub–20

42 43 44 45 53 59 65

41 42 42 41 38 35 32

Daub–2 Daub–3 Daub–4 Daub–5 Daub–10 Daub–15 Daub–20

63 64 65 66 70 74 78

63 63 63 62 58 56 51

Daub–2 Daub–3 Daub–4 Daub–5 Daub–10 Daub–15 Daub–20

86 86 87 87 88 90 92

86 86 86 85 82 79 74

Daub–2 Daub–3 Daub–4 Daub–5 Daub–10 Daub–15 Daub–20

96 96 96 96 97 97 97

96 96 96 95 93 91 86

Daub–2 Daub–3 Daub–4 Daub–5 Daub–10 Daub–15 Daub–20

78 77 77 77 77 80 81

80 79 79 78 74 71 66

Daub–2 Daub–3 Daub–4 Daub–5 Daub–10 Daub–15 Daub–20

86 86 86 86 86 88 88

88 88 88 87 85 82 78

Daub–2 Daub–3 Daub–4 Daub–5 Daub–10 Daub–15 Daub–20

93 93 94 94 93 94 95

95 95 95 94 93 91 88

Daub–2 Daub–3 Daub–4 Daub–5 Daub–10 Daub–15 Daub–20

97 97 97 97 97 97 98

98 98 98 97 96 95 93

Camera

Brain Lena Threshold: λ = 10 — Excellent overall quality 41 83 83 83 78 79 42 84 84 84 78 80 41 85 84 84 78 79 41 85 84 84 79 79 41 87 82 84 79 74 40 88 78 82 82 69 40 89 74 83 83 64 Threshold: λ = 20 — Good overall quality 63 91 91 91 87 89 64 92 91 91 87 89 63 92 91 91 87 88 63 92 91 91 87 90 63 93 89 91 88 83 62 93 86 91 89 79 63 94 82 91 90 74 Threshold: λ = 45 — Medium overall quality 87 96 96 96 94 95 87 96 96 96 94 95 87 96 96 96 94 95 87 96 96 96 95 94 87 97 94 96 94 91 87 97 91 96 95 88 87 97 89 96 96 83 Threshold: λ = 85 — Poor overall quality 97 98 98 98 97 98 97 98 98 98 97 98 97 98 98 98 97 97 97 98 98 98 98 97 97 98 97 98 97 94 97 98 95 98 98 92 98 98 93 99 98 88 Goldhill House Threshold: λ = 10 — Excellent overall quality 79 70 71 70 79 80 78 70 71 71 79 80 78 71 71 70 79 80 78 71 71 70 79 79 76 73 67 69 80 72 75 77 63 68 82 66 74 79 58 68 83 59 Threshold: λ = 20 — Good overall quality 88 85 87 86 87 88 88 85 87 86 87 88 88 86 86 86 87 88 88 86 86 86 87 87 87 86 83 86 87 81 86 89 79 86 89 75 86 89 73 86 89 69 Threshold: λ = 45 — Medium overall quality 95 94 96 95 93 95 95 95 96 95 94 95 95 95 95 95 94 94 95 95 95 96 94 94 95 95 92 96 94 89 95 95 89 96 95 84 95 96 85 96 95 78 Threshold: λ = 85 — Poor overall quality 98 97 98 98 97 98 98 98 98 98 97 97 98 98 98 98 97 97 98 98 98 99 97 97 98 98 96 99 97 93 98 98 93 99 97 89 98 98 90 99 98 84

79 80 79 80 78 77 77 88 89 89 89 88 88 88 95 95 96 96 96 96 96 98 98 98 98 98 98 99

80 80 79 79 78 77 76 88 88 87 88 87 87 87 94 95 95 95 95 94 95 98 97 98 98 98 98 99

14

Claudia Schremmer

Table 3. Average quality of the six test images. Figure 2 gives a more ‘readable’ plot of these digits

Wavelet Daub–2 Daub–3 Daub–4 Daub–5 Daub–10 Daub–15 Daub–20 Daub–2 Daub–3 Daub–4 Daub–5 Daub–10 Daub–15 Daub–20

Average image quality — PSNR [dB] zero mirror circular zero mirror circular padding padding convol. padding padding convol. Threshold λ = 10 Threshold λ = 20 17.630 17.602 17.701 15.242 15.252 15.246 17.745 17.752 17.768 15.298 15.330 15.288 17.691 17.711 17.662 15.244 15.284 15.213 17.719 17.701 17.680 15.233 15.270 15.257 17.641 17.615 17.689 15.253 15.290 15.306 17.695 17.675 17.686 15.136 15.185 15.168 17.616 17.654 17.676 15.135 15.207 15.114 Threshold λ = 45 Threshold λ = 85 13.057 13.078 12.942 11.609 11.736 11.681 12.982 13.144 13.016 11.659 11.763 11.643 12.932 13.025 13.002 11.637 11.740 11.651 12.992 13.110 13.016 11.610 11.806 11.626 12.823 13.061 12.935 11.500 11.713 11.666 12.854 12.985 12.911 11.422 11.628 11.538 12.673 12.916 12.788 11.439 11.624 11.718

Quality - Threshold 10 18

Quality - Threshold 20 zero-padding mirror-padding circular convolution

zero-padding mirror-padding circular convolution

15.4

17.8 15.2

PSNR

PSNR

17.6 15

17.4 14.8 17.2 14.6 17 4

6

8

10

20

30

40

4

6

8

10

20

Length of Wavelet Filter

Length of Wavelet Filter

Quality - Threshold 45

Quality - Threshold 85 12

zero-padding mirror-padding circular convolution

13.2

30

40

zero-padding mirror-padding circular convolution

11.8 13

PSNR

PSNR

11.6 12.8

11.4 12.6 11.2 12.4 11 4

6

8

10

20 Length of Wavelet Filter

30

40

4

6

8

10

20

30

40

Length of Wavelet Filter

Fig. 2. Visual quality of the test images at the quantization thresholds λ = 10, 20, 45 and 85. The values correspond to Table 3

Empirical Evaluation of Boundary Policies

15

Table 4. Average bitrate heuristic of the six test images. Figure 3 gives a more ‘readable’ plot of these digits

Wavelet Daub–2 Daub–3 Daub–4 Daub–5 Daub–10 Daub–15 Daub–20 Daub–2 Daub–3 Daub–4 Daub–5 Daub–10 Daub–15 Daub–20

Average discarded information — Percentage [%] zero mirror circular zero mirror circular padding padding convol. padding padding convol. Threshold λ = 10 Threshold λ = 20 72.0 72.3 72.0 83.2 84.3 84.0 71.8 72.7 72.5 83.5 84.3 84.3 72.3 72.5 71.8 83.8 84.0 84.0 72.7 72.0 72.0 84.0 83.8 84.2 74.8 67.8 71.0 85.0 79.8 83.7 78.0 63.7 69.8 87.0 76.2 83.3 80.0 58.8 69.7 88.0 71.2 83.5 Threshold λ = 45 Threshold λ = 85 92.7 93.8 93.7 97.0 97.7 97.8 93.0 93.8 93.8 97.2 97.5 97.7 93.3 93.5 94.0 97.2 97.3 97.8 93.5 93.0 94.2 97.3 97.0 98.0 93.5 90.2 94.2 97.3 94.8 98.0 94.3 87.0 94.0 97.5 92.5 98.0 95.2 82.8 94.2 97.8 89.0 98.7

Discarded Information - Threshold 10 100 95 90

90

85

85

80 75 70

80 75 70

65

65

60

60

55

55

50

50 4

6

8

10

20

30

40

4

6

8

10

20

Length of Wavelet Filter

Length of Wavelet Filter

Discarded Information - Threshold 45

Discarded Information - Threshold 85

100

100

zero-padding mirror-padding circular convolution

95 90

90

85

85

80 75 70

40

80 75 70

65

65

60

60

55

30

zero-padding mirror-padding circular convolution

95

Percentage (%)

Percentage (%)

zero-padding mirror-padding circular convolution

95

Percentage (%)

Percentage (%)

Discarded Information - Threshold 20 100

zero-padding mirror-padding circular convolution

55

50

50 4

6

8

10

20 Length of Wavelet Filter

30

40

4

6

8

10

20

30

40

Length of Wavelet Filter

Fig. 3. Average bitrate heuristic of the test images at the quantization thresholds λ = 10, 20, 45 and 85. The values correspond to Table 4

Image-Feature Based Second Generation Watermarking in Wavelet Domain Song Guoxiang and Wang Weiwei School of Science, Xidian University Xi’an, 710071, P.R.China

Abstract. An image-feature based second generation watermarking scheme is proposed in this paper. A host image is firstly transformed into wavelet coefficients and features are extracted from the lowest approximation. Then a watermark sequence is inserted in all high frequency coefficients corresponding to the extracted featured approximation coefficients. Original host image is not needed in watermarking detection, but the featured approximation coefficients position is necessary for robust detection. The correlation between the embedded watermark and all high frequency coefficients of a possibly corrupted watermarked image corresponding to the approximate coefficients at the same position as the original featured approximation coefficients is calculated and compared to a predefined threshold to see if the watermark is present. Experimental results show the watermark is very robust to common image processing, lossy compression in particular. Keywords: image feature, digital watermarking, wavelet transform

1

Introduction

Lately, multimedia and computer networking have known rapid development and expansion. This created an increasing need for systems that protect the copyright ownership for digital images. Digital watermarking is the embedding of a mark into digital content that can later be, unambiguously, detected to allow assertions about the ownership or provenience of the data. This makes watermarking an emerging technique to prevent digital piracy. To be effective, a watermark must be imperceptible within its host, discrete to prevent unauthorized removal, easily extracted by the owner, and robust to incidental and intentional distortions. Most of the recent work in watermarking can be grouped into two categories: spatial domain methods and frequency domain methods. Kutter et al. [1] refered both the spatial-domain and the transform domain techniques as first generation watermarking schemes and introduced the concept of second generation watermarking schemes which, unlike the first generation watermarking schemes, employ the notion of the data features. For images, features can be edges, corners, textured areas or parts in the image with specific characteristics. Features suitable for watermarking should have three basic properties: First, invariance Y. Y. Tang et al. (Eds.): WAA 2001, LNCS 2251, pp. 16–21, 2001. c Springer-Verlag Berlin Heidelberg 2001


Image-Feature Based Second Generation Watermarking in Wavelet Domain

17

to noise (lossy compression, additive, multiplicative noise, ect.) Second, covariance to geometrical transformations (rotation, translation, sub-sampling, change of aspect ratio, etc.) The last, localization (cropping the data should not alter remaining feature points). In this paper, we deal with the wavelet domain image watermarking method with the notion of second generation watermarking scheme. Previous wavelet domain watermarking schemes [2,3,4,5,6,7,8] added a watermark to a selected set of DWT coefficients in chosen subbands. The methods proposed in [2,3,6,8] requires the original image for detection, while the methods in [4,5,7] does not. However, the method [4] needs the embedded position and the corresponding subband label as well as two threshold value. For the method [5], if the watermarked image is tampered, the number of the coefficients that are greater than the larger threshold may not be equal to the size of the embeded watermark, thus there existed a problem for detection in calculating the correlation between the embedded watermark and the coefficients of a possibly modified watermarked image, whose absolute magnitude is above the larger threshold. The method [7] embedded watermarks into all HL and LH coefficients at levels 2 to 4, resulted in poor quality. Based on the concept of second generation watermarking scheme, we propose a wavelet domain watermarking method which embeds watermarks into all high frequency coefficients corresponding to the featured lowest approximation coefficients. First, the host image is transformed using DWT and features are extracted from the lowest approximation using the method in [9]. Then the watermark is embedded into all subband coefficients corresponding to the featured lowest approximate coefficients. Finally, the modified coefficients is inversely transformed to form the watermarked image. In the watermark detection, the original image is not needed, but for more robust detection, the featured lowest approximate coefficients position of the original image is required, which can be encrypted using private key encryption and stored in the image header. The correlation between the embedded watermark and all high frequency coefficients of a possibly corrupted watermarked image corresponding to the lowest approximate coefficients at the same position as the original featured approximation coefficients is calculated and compared to a predefined threshold to see whether the watermark is present or not. Experimental results show that the watermark is very robust to common image processing, lossy compression in particular. Even when the watermarked image is compressed by JPEG with a quality factor of one percent, the watermark is still present.

2

The Proposed Method

The original image is firstly decomposed using DWT with 8 taps Daubechies orthogonal filter [10] until the scale N to obtain multiresolution LHn , HLn , HHn (n = 1, 2, · · ·, N ) and the lowest resolution approximation LLN .There exists a tree structure between the coefficients [11] as shown in Fig.1(for N = 3). The

18

Song Guoxiang and Wang Weiwei

tree relation can be defined as follows: tree(LLN (x, y)) = tree(HLN (x, y)) ∪ tree(LHN (x, y)) ∪ tree(HHN (x, y)) (1) tree(HLn (x, y)) = tree(HLn−1 (2x − 1, 2y − 1)) ∪ tree(HLn−1 (2x, 2y − 1)) (2) ∪ tree(HLn−1 (2x − 1, 2y)) ∪ tree(HLn−1 (2x, 2y)) where n = N, N − 1, · · · , 2. For tree(LHn (x, y)), tree(HHn (x, y))(n = N, N − 1, · · · , 2), the definition is similar to (2). tree(HL1 (x, y)) = HL1 (x, y) tree(LH1 (x, y)) = LH1 (x, y) tree(HH1 (x, y)) = HH1 (x, y) For the experiments reported in this paper, N is taken as N = 4. 2.1

Feature Extraction

We use the method in [9] to extract features of the image. The difference is that we extract features from the lowest approximation components LLN of the DWT of the image, rather than from the original image. Since the size of LLN is 1/(4N ) times that of the original image, the time needed for extracting features is largely reduced.The feature extraction scheme is based on a decomposition of the image using Mexican-Hat wavelets. In two dimensions, the response of the Mexican-Hat mother avelet is defined as: ψ(x, y) = (2 − (x2 + y 2 ))e−(x

2

+y 2 )/2

(3)

The isotropic nature of the Mexican-Hat filter is well suited for detecting pointfeatures. Here we briefly describe the feature-detection procedure as follows: Firstly, define the feature-detection function, Pij (·, ·) as: Pij (k, l) = |Mi (k, l) − γMj (k, l)|

(4)

where Mi (k, l) and Mj (k, l) represent the responses of Mexican-Hat wavelets at the image location (k, l) for scales i and j respectively. For an image A, the wavelet response Mi (k, l) is given by: Mi (k, l) =< (2−i ψ(2−i (k, l))), A >

(5)

where < ·, · > denotes the convolution of its operands. We only consider wavelets on a dyadic scale. Thus, the normalizing constant is given by γ = 2−(i−j) . The operator | · | returns the absolute value of its parameter. Here we take i = 2 and j = 4 as in [9]. Secondly, determine points of local maxima of Pij (·, ·). These maxima correspond to the set of potential feature-points. A circular neighborhood with a radius of 5 points is used to determine the local maxima. Finally, accept a point of local maxima of Pij (·, ·) as a feature-point if the variance of the image-pixels in the neighborhood of the point is higher than a threshold. Here a 7 × 7 neighborhood around the point is used for computing the local variance. A candidate point is accepted as a feature-point if the corresponding local variance is larger than a threshold, which we take as 20.

Image-Feature Based Second Generation Watermarking in Wavelet Domain

2.2

19

Watermark Inserting

The original image I is firstly decomposed using DWT with 8 taps Daubechies orthogonal filter until the scale N = 4 to obtain multiresolution LHn , HLn , HHn (n = 1, 2, · · · , 4) and the lowest resolution approximation LL4 . Then featurepoints are extracted from LL4 using the method in 2.1. If LL4 (x, y) is a featurepoint, then some watermark bits x ∈ X are added to all the children notes of tree(LL4 (x, y)). X stands for a set of watermark x and the elements xl of x are given by the random noise sequence whose probability law has a normal distribution of zero mean and unit variance. Since for every tree(LL4 (x, y))), there are 255 children in all, except for the root, the size of the watermark x, denoted by M , is given by M = 255× the number of feature-points in LL4 ). The specific embedding method is as follows: For every feature-point LL4 (x, y), for every Wl ∈ tree(LL4 (x, y)) and Wl = LL4 (x, y) Wl′ ← Wl + α|Wl |xl

(6)

where wl and Wl′ denotes respectively the DWT coefficient of the original and watermarked image,α is a modulating factor, here we take α = 0.2. Finally, inversely transform the modified multiresolution subbands to obtain the watermarked image I ′ . 2.3

Watermark Detection

The original image is not required in the watermark detection, but for more robust detection, the feature-points position of the original image is indeed necessary. Firstly, A possibly corrupted watermarked image I˜ is decomposed as I in ˜ l ∈ tree(LL ˜ 4 (x, y)) ˜ 4 (x, y), all coefficients W 2.2. Then for every feature-point LL ˜ ˜ ˜ ˜ and Wl = LL4 (x, y) are taken out, where LL4 and Wl respectively represents ˜ the lowest resolution approximation and high frequency coefficients of I.We cal˜ culate the correlation z between Wl and all candidates y ∈ X of the embedded watermark x as: M
˜ l yl W z = 1/M (7) l=1

By comparing the correlation with a predefined threshold Sz , which is given in [7] to determine whether a given watermark is present or not. In theory, the threshold Sz is taken as M α
Sz = |Wl | (8) 2M l=1

In practice, the watermarked image would be attacked incidentally or intentionally, so for robust detection, the threshold is taken as Sx = r

M α
˜ |Wl |, 0 < r ≤ 1 2M l=1

(9)

20

3

Song Guoxiang and Wang Weiwei

Experimental Results

In order to confirm that the proposed watermarking scheme is effective, we performed some numerical experiments with some gray-scale standard images. Here we describe experimental results for the standard image ”lenna”(512 × 512 pixels, 8 bits/pixel) shown in Fig.2(a). Fig.2(b) shows the watermarked image with parameters α = 0.2, N = 4 and M = 4080. Next, we tested the robustness of the watermark against some common image processing operations on the watermarked image Fig.2(b). Fig.3 is the result of JPEG compression with quality factor of 1. The image after 11 × 11 mean filtering is shown in Fig.4. The image after adding white Gaussian noise of power 40db is shown in Fig.5. Fig.6 is the clipped image with only 25% center data left. Fig.7 shows the result of rotation counter clockwise by 10 degrees. The response of the watermark detector and the corresponding threshold for the untampered and attacked watermarked image are given in Tab.1. The threshold is calculated using the equation (10), where r = 2/3 . As shown in Tab.1, though image degradation is very heavy, the watermark is still easily recovered and the detector response is also well above the threshold. Numerical experiments with the other standard images have also demonstrated similar results.

4

Conclusions

An image-feature based wavelet domain second generation watermarking scheme is proposed in this paper. Experiments show that the watermark is very robust to common image processing, lossy compression and smoothing in particular. Even for the JPEG compressed version of the watermarked image with quality factor of 1%, the feature-points remain salient. Furthermore, we will investigate watermarking method that resistant to geometric attacks.

References 1. M. Kutter, S. K. Bhattacharjee, and T. Ebrahimi, ”Towards second generation watermarking scheme,” Proc. IEEE ICIP’99, Vol.1,1999 16 2. D. Kundur and D. Hatzinakos, ”A robust digital image watermarking method using wavelet-based fusion,” Proc. IEEE ICIP’97, vol.1, 1997, pp.544-547 17 3. X. G. Xia, C. G. Boncelet and G. R. Arce, ”A multiresolution watermark for digital images,” Proc. IEEE ICIP’97, Vol.1,1997, pp.548-551 17 4. H. Inoue, A. Miyazaki, A. Yamamoto, etal., ”A digital watermark bases on the wavelet transform and its robustness on image compression,” Proc. IEEE ICIP’98, Vol.2, 1998, pp.391-423 17 5. R. Dugad, K. Ratakonda and N. Ahuja, ”A new wavelet-based scheme for watermarking image,” Proc. IEEE ICIP’98, vol.2, 1998, pp.419-423 17 6. W. W. Zhu, Z. X. Xiong and Y. Q. Zhang, ”Multiresolution watermarking for images and video: a unified approach,” Proc. IEEE ICIP’98, vol.1, 1998, pp.465468 17

Image-Feature Based Second Generation Watermarking in Wavelet Domain

21

7. H. Inoue, A. Kiomiyazaki and T. Katsura, ”An image watermarking method based on the wavelet transform,” Proc. IEEE ICIP’99, vol.1, 1999, pp.296-300 17, 19 8. J. R. Kim and Y. S. Moon, ”A robust wavelet-based digital watermarking using Level-adaptive thresholding,” Proc. IEEE ICIP’99, vol.2, 1999, pp.226-230 17 9. S. K. Bhattacharjee and M. Kutter, ”Compression tolerant image authentication”, Proc. IEEE ICIP’98, Vol.1,1998 17, 18 10. I. Daubechies, ”Ten Lectures on Wavelets,” CBMS-NSF conference series in applied mathematics, SIAM Ed. 17 11. J. M. Shapiro, ”Embeded image coding using zerotrees of wavelet coefficients,” IEEE trans. On Signal Processing, Vol.41, No.12, 1993, pp.3445-3462 17

A Study on Preconditioning Multiwavelet Systems for Image Compression Wonkoo Kim and Ching-Chung Li University of Pittsburgh, Dept. of Electrical Engineering Pittsburgh, PA 15261, USA [email protected] [email protected]

Abstract. We present a study on applications of multiwavelet analysis to image compression, where filter coefficients form matrices. As a multiwavelet filter bank has multiple channels of inputs, we investigate the data initialization problem by considering prefilters and postfilters that may give more efficient representations of the decomposed data. The interpolation postfilter and prefilter are formulated, which are capable to provide a better approximate image at each coarser resolution level. A design process is given to obtain both filters having compact supports, if exist. Image compression performances of some multiwavelet systems are studied in comparison to those of single wavelet systems.

1

Nonorthogonal Multiwavelet Subspaces

Let us define a multiresolution analysis of L2 (R) generated by several scaling functions, with an increasing sequence of function subspaces {Vj }j∈Z in L2 (R): {0} ⊂ . . . ⊂ V−1 ⊂ V0 ⊂ V1 ⊂ . . . ⊂ L2 (R).

(1)

Subspaces Vj are generated by a set of scaling functions φ1 , φ2 , . . . , φr (namely, multiscaling functions) such that Vj := closL2 (R) < φm j,k : 1 ≤ m ≤ r, k ∈ Z >,

∀ j ∈ Z,

(2)

2 i.e., Vj is the closure of the linear span of {φm j,k }1≤m≤r, k∈Z in L (R), where j/2 m j φm φ (2 x − k), j,k (x) := 2

∀ x ∈ R.

(3)

Then we have a sequence of multiresolution subspaces {Vj } generated by a set of multiscaling functions, where the resolution gets finer and finer as j increases. ˙ Wj , ∀ j ∈ Z, Let us define inter-spaces Wj ⊂ L2 (R) such that Vj+1 := Vj + ˙ denotes a nonorthogonal direct sum. Wj where the plus sign with a dot (+) is the complement to Vj in Vj+1 , and thus Wj and Wl with j = l are disjoint but may not be orthogonal to each other. If Wj ⊥ Wl , ∀ j = l, we call them Y. Y. Tang et al. (Eds.): WAA 2001, LNCS 2251, pp. 22–36, 2001. c Springer-Verlag Berlin Heidelberg 2001 

A Study on Preconditioning Multiwavelet Systems for Image Compression

23

semi-orthogonal wavelet spaces [1]. By the nature of construction, subspaces Wj can be generated by r base functions, ψ 1 , ψ 2 , . . . , ψ r that are multiwavelets. The m subspace Wj is the closure of the linear span of {ψj,k }1≤m≤r, k∈Z : m Wj := closL2 (R) < ψj,k : 1 ≤ m ≤ r, k ∈ Z >,

∀ j ∈ Z,

(4)

where m ψj,k (x) := 2j/2 ψ m (2j x − k),

∀ x ∈ R.

(5)

We may express multiscaling functions and multiwavelets as vector functions:  1   1  ψ (x) φ (x)  ..   ..  (6) φ(x) :=  .  , ψ(x) :=  .  , ∀ x ∈ R. ψ r (x)

φr (x)

Also, in vector form, let us define φj,k (x) := 2j/2 φ(2j x − k) and ψ j,k (x) := 2j/2 ψ(2j x − k),

∀ x ∈ R.

(7)

Since the multiscaling functions φm ∈ V0 and the multiwavelets ψ m ∈ W0 1/2 m are all in V1 , and since V1 is generated by {φm φ (2x− k)}1≤m≤r, k∈Z , 1,k (x) = 2 2 there exist two ℓ matrix sequences {Hn }n∈Z and {Gn }n∈Z such that we have a two-scale relation for the multiscaling function φ(x):  φ(x) = 2 Hn φ(2x − n), x ∈ R, (8) n∈Z

which is also called as a two-scale matrix refinement equation (MRE), and for multiwavelet ψ(x):  Gn φ(2x − n), x ∈ R, (9) ψ(x) = 2 n∈Z

where Hn and Gn are r × r square matrices. We are interested in finite sequences of Hn and Gn , namely, FIR (Finite Impulse Response) filter pairs. Using the fractal interpolation, Geronimo, Hardin, and Massopust successfully constructed a very important multiwavelet system [2,3,4] which has two orthogonal multiscaling functions and two orthogonal multiwavelets. Their four matrix coefficients Hn satisfy the MRE for a multiscaling function φ(x): " √ #       H0 =

3 10 √

−

2 40

4 2 10 3 − 20

, H1 =

3 0 10 √ 9 2 1 40 2

, H2 =

0

√ 9 2 40

0 0√ 0 , H3 = , 3 − 20 − 402 0

(10)

and other four matrix coefficients Gn generate a multiwavelet ψ(x): " √ # " √ #  9√2 1   √2  2 9 2 3 3 G0 =

− 40 − 20 √ , G1 = 1 − 3202 − 20

40 9 20

− −2 − 40 0 , G2 = 409 3√202 , G3 = 1 0 0 − 20 20 20

(11)

24

Wonkoo Kim and Ching-Chung Li GHM multiscaling function 1

2 1.8 1.6 1.4 1.2 1 0.8 0.6 0.4 0.2 0

GHM multiscaling function 2 2

1 0.5 0 -0.5 0

0.5

1

1.5

2

GHM multiwavelet 1

2

-1

1

1

0.5

0.5

0

0

-0.5

-0.5

-1

-1

-1.5

0

0.5

(a) φ1

1

1.5

2

GHM multiwavelet 2

2 1.5

1.5

1.5

-2

-1.5 0

0.5

(b) φ2

1

1.5

2

-2

0

0.5

(c) ψ 1

1

1.5

2

(d) ψ 2

Fig. 1. Geronimo-Hardin-Massopust orthogonal multiscaling functions and multiwavelets Othogonal cardinal 2-balanced multiwavelet 1 Othogonal cardinal 2-balanced multiscaling function 1

Othogonal cardinal 2-balanced multiscaling function 2

2

2

1.5

1.5

1

1

0.5

0.5

0

0

-0.5

0

1

2

3

4

1

(a) φ

5

-0.5

2

2

1.5

1.5

1

1

0.5

0.5

0

0

-0.5

-0.5

-1

-1

-1.5 0

1

2

3

4

2

(b) φ

5

-2

Othogonal cardinal 2-balanced multiwavelet 2

-1.5 0

1

2

3

(c) ψ

1

4

5

-2

0

1

2

3

4

5

(d) ψ 2

Fig. 2. Cardinal 2-balanced orthogonal multiscaling functions and multiwavelets The GHM (Geronimo-Hardin-Massopust) orthogonal multiscaling functions are shown in Figure 1(a) and (b), and their corresponding orthogonal multiwavelets are shown in (c) and (d). The GHM multiwavelet system has very remarkable properties: its scaling functions and wavelets are orthogonal, very shortly supported, symmetric or antisymmetric, and it has second order approximation so that locally constant and locally linear functions are in Vj . Another example of orthogonal multiwavelet is shown in Figure 2[5,6,7], where multiscaling functions are shown in figures (a) and (b), and multiwavelet functions are shown in figures (c) and (d), respectively. Two scaling functions in each cardinal balanced multiwavelet system are the same functions up to a half integer shift in time, and also the wavelets are the same up to a half integer shift in time. The approximation orders of the cardinal balanced orthogonal multiwavelet systems are 2 for cardinal 2-balanced, 3 for cardinal 3-balanced, and 4 for cardinal 4-balanced systems. The cardinal 2-balanced orthogonal multiwavelet filters are given by     −1 −1 H(z) =

b(z) 0.5z , z −5 b(−1/z) 0.5z −2

G(z) =

−b(z) 0.5z , −z −5 b(−1/z) 0.5z −2

(12)

where b(z) = 0.015625+0.123015364784490z −1 +0.46875z −2 −0.121030729568979z −3+ 0.015625z −4 −0.001984635215512z −5 . For more details on cardinal balanced orthogonal multiwavelets, refer to the paper written by I. Selesnick [6]. We should note that a scalar system with one scaling function cannot combine symmetry, orthogonality, and the second order approximation together. Furthermore, the solution of a scalar refinement equation with four coefficients is supported on the interval [0,3], while multiscaling functions with four matrix coefficients can be supported on a shorter interval.

A Study on Preconditioning Multiwavelet Systems for Image Compression

25

˙ W0 , Since all elements of both φ(2x) and φ(2x − 1) are in V1 and V1 = V0 + ˜ n }n∈Z and {G ˜ n }n∈Z such that there exist two ℓ2 matrix sequences {H  T ˜ k−2n ˜ Tk−2n ψ(x − n) , ∀ k ∈ Z, H φ(x − n) + G (13) φ(2x − k) = n∈Z

which is called the decomposition relation of φ and ψ.1 ˜ n }, {G ˜ n }), which are We have two pairs of sequences ({Hn }, {Gn }) and ({H ˙ 0 . A carefully chosen pair of unique due to the direct sum relationship V1 = V0 +W sequences ({Hn }, {Gn }) can generate multiscaling functions and multiwavelets and thus multiwavelet subspaces; hence, they can completely characterize a multiwavelet analysis.

2

Multiwavelet Decomposition and Reconstruction

From the formulas (8), (9), and (13), the following signal decomposition and reconstruction algorithms can be derived. Let vj ∈ Vj and wj ∈ Wj so that   vj (x) := cTj,k φ(2j x − k); (14) cj,k · φ(2j x − k) = k∈Z

k∈Z

wj (x) :=



j

dj,k · ψ(2 x − k) =



dTj,k ψ(2j x − k),

(15)

k∈Z

k∈Z

where · denotes a dot product between two vectors and ·T denotes the transpose operator. The scale factor 2j/2 is not explicitly shown here for simplicity but ˙ Wj−1 , incorporated into the sequences cj,k and djk . By the relation Vj = Vj−1 + vj (x) := vj−1 (x) + wj−1 (x)   = cj−1,k · φ(2j−1 x − k) + dj−1,k · ψ(2j−1 x − k), k∈Z

(16) ∀ j ∈ Z.

k∈Z

Thus we have the following recursive decomposition (analysis) formulas:   ˜ n−2k cj,n = ˜ −n cj,2k−n , H H cj−1,k = ∀ j ∈ Z; n

dj−1,k =

 n

(17)

n

˜ n−2k cj,n = G



˜ −n cj,2k−n , G

∀ j ∈ Z.

(18)

n

An original data sequence c0 (={c0,k }k ) is decomposed into c1 and d1 data sequences, and the sequence c1 is further decomposed into c2 and d2 sequences, etc.. Keeping this process recursively, the original sequence c0 is decomposed into d1 , d2 , d3 , . . . . Note that this process continuously reduces the data size by half for each decomposed sequence but it conserves the total data size. 1

˜ and G ˜ and reversed indexing We here intentionally transposed the matrices of H instead of 2n − k, for some convenience in representing formulas of dual relationship.

26

Wonkoo Kim and Ching-Chung Li

j

j

✲ H ˜−

♠ 2 ✲ j−1 ❄

✲ 2♠ ✻

HT

✲ G ˜−

♠ 2 ✲ dj−1 ❄

✲ 2♠ ✻

GT

❄

♠✲ j +❤ ×2 ✻

(a) Filterbanks derived from multiwavelet analysis ✲ H ♠ ♠ ˜ 2 ✲ j−1 ✲ ✻ 2 H∗ ❄ ❄ ♠✲ +❤ ×2

j ∗ ♠ ♠ ✲ G ✲ ✲ ✻ ˜ 2 2 d G j−1 ❄ ✻ (b) Multiwavelet filterbanks by reverse indexing

Fig. 3. The multiwavelet transform filter banks. Filters are r × r matrices and data paths are r lines, where r = 2 in our examples. The multiwavelet systems (a) and (b) are equivalent, except that filter indices are all reversed between the two systems Let DK , K ≥ 1, be the subsampling (downsampling) operator defined by (DK x)[n] := x[Kn],

(19)

where K is a subsampling rate and x is a sequence of vector-valued samples. The decomposition formulas can be rewritten in the Z-transform domain as ˜ − (z)cj (z), cj−1 (z) = D2 H ˜ − (z)cj (z), dj−1 (z) = D2 G

(20) (21) T

where the superscript − denotes reverse indexing, i.e., H− := H∗ . From the two-scale relations (8), (9) and from (14), (15), we have the following recursive reconstruction (synthesis) formula: 

 T (22) Hk−2n cj−1,n + GTk−2n dj−1,n . cj,k = 2 n

Let UK , K ≥ 1, be the upsampling operator defined by n n x[ K ], if K is an integer; (UK x)[n] := 0, otherwise,

(23)

where K is an upsampling rate and x is a sequence of vector-valued samples. Then the reconstruction formula can be rewritten in the Z-transform domain as

 cj (z) = 2 HT (z)U2 cj−1 (z) + GT (z)U2 dj−1 (z)

(24)

˜

j−1 (z) = D2 H(z)c j (z), ˜ dj−1 (z) = D2 G(z)c j (z),

(25)

The decomposition and reconstruction systems implemented by multiwavelet filterbanks are shown in Figure 3, where the system (a) is the exact implementation of our equations derived. If we take reverse indexing for all filters, we have the system (b), and the multiwavelet decomposition formulas become

(26)

A Study on Preconditioning Multiwavelet Systems for Image Compression

27

and the reconstruction formula becomes

j (z) = 2 [H∗ (z)U2 j−1 (z) + G∗ (z)U2 dj−1 (z)] .

(27)

Note that the input data cj is a sequence of vector-valued data, every data path has r lines, and filters are r × r matrices. We restrict r = 2 in this study. Constructing a vector-valued sequence cj from a signal or an image is nontrivial. As an 1-D input signal is vectorized, the direction of filter indexing will affect the reconstructed signal in an undesirable way, if the vectorization scheme does not match with filter indexing. This effect does not happen in a scalar wavelet system, whose filters are not matrices. As we do not take reverse indexing for data sequences, we will take the system (a) of Figure 3 in our implementation. A prefilter for the chosen input scheme will be designed later in Section 5.

3

Biorthogonality and Perfect Reconstruction Condition

From the two-scale dilation equations (8), (9), and the decomposition relation (13), we have the following biorthogonality conditions: ˜ ∗ (z) H(z)H ˜ ∗ (z) H(z)G ˜ ∗ (z) G(z)H ˜ ∗ (z) G(z)G

˜ ∗ (−z) = Ir ; + H(−z)H ˜ ∗ (−z) = 0r ; + H(−z)G ˜ ∗ (−z) = 0r ; + G(−z)H ˜ ∗ (−z) = Ir , + G(−z)G

(28) (29) (30) (31)

which completely characterize the biorthogonality between the analysis filter ˜ G) ˜ and the synthesis filter pair (H, G). (Namely, H ⊥ G ˜ and H ˜ ⊥ G.) pair (H, 2 Let Hm (z) denote the modulation matrix of (H, G) as defined by   Hm (z) :=

H(z) H(−z) , G(z) G(−z)

(32)

˜ m (z) denote the modulation matrix of (H, ˜ G) ˜ similarly defined, then the and H above biorthogonality condition becomes   ∗    ∗ ˜ ∗m (z) = Hm (z)H

H(z) H(−z) G(z) G(−z)

˜ (z) G ˜ (z) Ir 0 H ˜ ∗ (−z) = 0 Ir = I2r . ˜ ∗ (−z) G H

(33)

From the decomposition and reconstruction formulas (20), (21) and (24), we have the following perfect reconstruction (PR) condition: ˜ ∗m (z)Hm (z) = c I2r , H

(34)

where c is a non-zero constant (a scale change in the reconstructed signal is allowed). 2

The modulation matrix is also called as the AC (alias component) matrix[8].

28

Wonkoo Kim and Ching-Chung Li

˜ G), ˜ the modulation Theorem 1. For two matrix filter pairs (H, G) and (H, ˜ matrices Hm (z) and Hm (z) are defined by     ˜ ˜ H(z) H(−z) ˜ m (z) := H(z) H(−z) . (35) Hm (z) := , H ˜ ˜ G(z) G(−z) G(z) G(−z) Then

˜ ∗m (z) = H ˜ ∗m (z)Hm (z) = c I2r , Hm (z)H

(36)

where c is a nonzero constant, is the necessary and sufficient condition for the ˜ G) ˜ to be biorthogonal and to ensure the two matrix filter pairs (H, G) and (H, perfect reconstruction. If these filter pairs generate multiscaling functions and multiwavelets, then they are biorthogonal. ˜ = H and G ˜ = G, and then For orthogonal filter pairs, we have H Hm (z)H∗m (z) = H∗m (z)Hm (z) = cI2r .

(37)

Hence, Hm (z) is paraunitary (lossless), i.e., unitary for all z on the unit circle.

4

Construction of Biorthogonal Multiwavelets

Plonka and Strela constructed biorthogonal Hermite cubic (piecewise cubic polynomial) multiscaling functions and multiwavelets using the cofactor method [9,10]. The coefficient matrix   −1 2 −1 −1 H(z) =

1 4(1 + z ) −2(1 − z )(1 + z ) 16 3(1 − z −1 )(1 + z −1 ) −1 + 4z −1 − z −2

(38)

generates Hermite cubic multiscaling functions, where det H(z) = (1+z −1 )4 /128. ˜ for dual functions is A possible choice of H   −1 −2 −3 −2 −3 1 z − 8 + 18z − 8 + z ˜ H(z) = 2z − 8 + 8z −2 − 2z −3 32

−3z + 12 − 12z + 3z −4z + 8 + 24z −1 + 8z 2 − 4z −3

By the biorthogonality conditions, we have  −1 −1 2 z ˜ G(z) = 16

and by cofactor method,  −1 G(z) =

−4(1 − z ) 6(1 − z −1 )(1 + z −1 ) −1 −1 −(1 − z )(1 + z ) 1 + 4z −1 + z −2

(39)

 (40)



1 1 + 8z + 18z −2 + 8z −3 + z −4 −1 − 4z −1 + 4z −3 + z −4 . −1 −3 −4 6 + 24z − 24z − 6z −4 − 8z −1 + 24z −2 − 8z −3 − 4z −4 32 (41)

The Hermite cubic multiscaling functions and multiwavelets generated by H and G are shown in Figure 4 (a)–(d). Their corresponding biorthogonal multiscaling functions and multiwavelets are shown in Figure 4 (e)–(h).

A Study on Preconditioning Multiwavelet Systems for Image Compression Hermite cubic multiscaling function 1

0.5 0.4 0.3 0.2 0.1 0 -0.1 -0.2 -0.3 -0.4 -0.5 0

1 0.8 0.6 0.4 0.2 0

0

0.5

1

1.5

2

(a) φ1 Multiscaling function dual to Hermite cubics 1

0.5 0 -0.5

-1

-0.5 0

0.5

1

Hermite cubic multiwavelet 1

1.5

(e) φ˜1

2

2.5

3

0.5 0.4 0.3 0.2 0.1 0 -0.1 -0.2 -0.3 -0.4 -0.5 0

Hermite cubic multiwavelet 2

1.2

2 1.5

1

1

0.8

0.5 0

0.6

-0.5

0.4

-1

0.2

0.5

1

1.5

2

0

-1.5

0

0.5

(b) φ2

1

-1

Hermite cubic multiscaling function 2

29

1

1.5

2

2.5

3

-2

0

0.5

(c) ψ 1

Hermite cubic multiscaling function 2

0.5

1

1.5

2

1 0.8 0.6 0.4 0.2 0 -0.2 -0.4 -0.6 -0.8 -1

Multiwavelet dual to Hermite cubics 1

0

0.5

(f) φ˜2

1

1.5

1

1.5

2

2.5

3

(d) ψ 2

2

2.5

Multiwavelet dual to Hermite cubics 2

3

0.4 0.35 0.3 0.25 0.2 0.15 0.1 0.05 0 -0.05 0

0.5

(g) ψ˜1

1

1.5

2

2.5

3

(h) ψ˜2

Fig. 4. Hermite cubics and their dual multiwavelets

5

Preconditioning Multiwavelet Systems

In this section we consider multiwavelet systems that analyze discrete data, and investigate how to precondition a multiwavelet system by prefiltering input data, which is not necessary for the case of single (or scalar) wavelet systems. 5.1

Prefilters and Postfilters

Consider the multiwavelet series expansion:  fj (t) := cTj,k φ(2j t − k)

(42)

k

From a given 1-D signal x[n], construct a vector-valued sequence x[n] by   x[nr]   .. x[n] :=  , r ≥ 1 .

(43)

x[nr + r − 1]

Let us define a prefilter Q(z), which maps a vector-valued sequence space onto itself, such that the coefficient vector sequence c0,k is obtained by filtering x[n]: c0 (z) = Q(z)x(z)

(44)

For any j ≤ 0, cj,k is decomposed to {cj−1,k , dj−1,k } by a layer of multiwavelet decomposition. Recursive multiwavelet decompositions down to a resolution level J < 0 give us a set of decomposed data sequences cJ,k and {dj,k }J≤j2?  >
? %  " "    " %% % "  "   $ $"      0, we have two real roots (without loss of generality we assume µ1 < µ2 ) √ b2 −4ac µ1 = −b+√2a . (B.8) b2 −4ac µ2 = −b− 2a

282

Weichuan Yu et al.

Here a < 0 (cf. (B.6)). Submitting µ1 and µ2 into (B.1-1) and (B.2-1) and further taking into account that x1 = a1 µ1 , x2 = a2 µ2 we solve a1 and a2  √  a = a(2ab2 +bb1 +b√1 b2 −4ac) 1 2 −4ac b2 −4ac−b b√ . (B.9)  a2 = a(2ab2 2 +bb1 −b√1 2 b2 −4ac) b −4ac+b b −4ac

References 1. Gabor, D.: Theory of communication. Journal of the IEE 93 (1946) 429–457 269 2. Lee, T. S.: Image representation using 2D Gabor wavelets. IEEE Trans. Pattern Analysis and Machine Intelligence 18(10) (1996) 959–971 269, 270, 272 3. Adelson, E. H., Bergen, J. R.: Spatiotemporal energy models for the perception of motion. Journal of the Optical Society of America 1(2) (1985) 284–299 269 4. Heeger, D. J.: Optical flow using spatiotemporal filters. International Journal of Computer Vision 1(4) (1987) 279–302 269, 277 5. Daugman, J. G.: Uncertainty relation for resolution in space, spatial frequency and orientation optimized by two-dimensional visual cortical filters. Journal of the Optical Society of America 2(7) (1985) 1160–1169 269 6. Koenderink, J., Doorn, A. V.: Representation of local geometry in the vision system. Biological Cybernetics 55 (1987) 367–375 269 7. Heitger, F., Rosenthaler, L., der Heydt, R. V., Peterhans, E., Kuebler, O.: Simulation of neural contour mechanisms: from simple to end-stopped cells. Vision Research 32(5) (1992) 963–981 269 8. Beauchemin, S., Barron, J.: The frequency structure of 1d occluding image signals. IEEE Trans. Pattern Analysis and Machine Intelligence 22 (2000) 200–206 269, 276 9. Grzywacz, N., Yuille, A.: A model for the estimate of local image velocity by cells in the visual cortex. Proc. Royal Society of London. B 239 (1990) 129–161 269, 270, 271 10. Bovik, A. C., Clark, M., Geisler, W. S.: Multichannel texture analysis using localized spatial filters. IEEE Trans. Pattern Analysis and Machine Intelligence 12(1) (1990) 55–73 270, 272 11. Bracewell, R. N.: The Fourier Transform and Its Applications. McGraw-Hill Book Company (1986) 271 12. Jain, A., Farrokhnia, F.: Unsupervised texture segmentation using Gabor filters. Pattern Recognition 24(12) (1991) 1167–1186 272 13. Manjunath, B., Ma, W.: Texture features for browsing and retrieval of image data. IEEE Trans. Pattern Analysis and Machine Intelligence 18(8) (1996) 837–842 272 14. Poggio, T., Girosi, F.: Networks for approximatation and learning. Proceedings of the IEEE 78 (1990) 1481–1497 274 15. Daugman, J.: Complete discrete 2-d Gabor transforms by neural networks for image analysis and compression. IEEE Trans. Acoustics, Speech, and Signal Processing 36(7) (1988) 274 16. Simoncelli, E. P., Farid, H.: Steerable wedge filters for local orientation analysis. IEEE Trans. Image Processing 5(9) (1996) 1377–1382 276 17. Yu, W., Daniilidis, K., Sommer, G.: Approximate orientation steerability based on angular Gaussians. IEEE Trans. Image Processing 10(2) (2001) 193–205 276

Skewness of Gabor Wavelets and Source Signal Separation

283

18. Fleet, D. J.: Measurement of Image Velocity. Kluwer Academic Publishers (1992) 276 19. Fleet, D., Langley, K.: Computational analysis of non-Fourier motion. Vision Research 34 (1994) 3057–3079 276 20. Shizawa, M., Mase, K.: A unified computational theory for motion transparency and motion boundaries based on eigenenergy analysis. In: IEEE Conf. Computer Vision and Pattern Recognition, Maui, Hawaii, June 3-6 (1991) 289–295 277 21. J¨ ahne, B.: Spatio-Temporal Image Processing. Springer-Verlag (1993) 277 22. Yu, W., Sommer, G., Daniilidis, K.: Skewness of Gabor wavelets and source signal separation. submitted to IEEE Trans. Signal Processing (2001) 277 23. Radon, J., translated by P. C. Parks: On the determination of functions from their integral values along certain manifolds. IEEE Trans. Medical Imaging 5(4) (1986) 170–176 277 24. Cardoso, J.: Source separation using higher order moments. In: IEEE International Conf. on Acoustics, Speech and Signal Processing. (1989) 2109–2112 279 25. Farid, H., Adelson, E. H.: Separating reflections from images using independent components analysis. Journal of the Optical Society of America 16(9) (1999) 2136–2145 279

The Application of the Wavelet Transform to Polysomnographic Signals M. MacCallum and A. E. A. Almaini School of Engineering, Napier University Edinburgh [email protected] [email protected]

Abstract. Polysomnographic (sleep) signals are recorded from patients exhibiting symptoms of a suspected sleep disorder such as Obstructive Sleep Apnoea (OSA). These non-stationary signals are characterised by having both quantitative information in the frequency domain and rich, dynamic data in the time domain. The collected data is subsequently analysed by skilled visual evaluation to determine whether arousals are present, an approach which is both time-consuming and subjective. This paper presents a wavelet-based methodology which seeks to alleviate some of the problems of the above method by providing: (1) an automated mechanism by which the appropriate stage of sleep for disorder observation may be extracted from the composite electroencephalograph (EEG) data set and (2) an ensuing technique to assist in the diagnosis of full arousal by correlation of wavelet-extracted information from a number of specific patient data sources (e.g. pulse oximetry, electromyogram [EMG] etc)

1

Introduction

Although sleep encompasses a third of the average person’s life, sleep and the disorders of sleep are poorly understood. Research suggests that sleep plays a restorative role in physiologic mechanisms and that the long-term disruption of sleep leads to disease and other degenerative disorders. [1]. The most common of these is Obstructive Sleep Apnoea (OSA), which is a progressive, life-threatening condition that affects a large percentage of the population. It is normally only diagnosed following a lengthy evaluation process which involves extensive recording and visual scrutiny of polysomnographic data. This process is both labour-intensive and error-prone [2]. The key objective of this work is to automate the above process by firstly producing wavelet-based tool, within MATLAB, which will establish the sleep stage. This is accomplished by establishing the presence of the K-Complex and Sleep Spindle and so confirming stage 2 of sleep, which is the principal region of evaluation. It is intended to then develop this tool to enable the detection of full arousal, the key indicator of OSA. Y. Y. Tang et al. (Eds.): WAA 2001, LNCS 2251, pp. 284-295, 2001.  Springer-Verlag Berlin Heidelberg 2001

The Application of the Wavelet Transform to Polysomnographic Signals

2

285

The Nature of Polysomnographic Signals

Polysomnographic, or sleep, signals are part of a composite physiological data which are characterised by being non-stationary and having information in both time and frequency domains. The physiological functions occurring during sleep extensive. They are, in essence, neuro-cardio-respiratory in nature Such signals gathered from patients whilst asleep within a controlled environment.

set the are are

The key signals in question are: 1. 2. 3.

EEG (electroencephalogram): neurological data collected from the scalp [3] EMG (electromyogram): muscular data collected from on and beneath the chin EOG (electrooculogram): eye movement data collected from the periphery of the left and right eyes

For the purpose of conventional analysis, the signal of primary importance is the EEG, although it may be correlated with others in the above set for more precise assessment of an event. The signal before amplification has an amplitude of between 10 to 100 µV and a frequency range of 0.5 to 40 Hz.

3

Sleep Architecture

Sleep is separated into two main regions of categorisation: Rapid-Eye Movement Sleep (REM) and Non-Rapid-Eye Movement Sleep (NREM). The sleep process is preceded with Stage W (wakefulness), which is followed by Stages 1, 2, 3 and 4 consecutively and collectively termed NREM sleep. Stage 5 is classified as REM sleep. The cycle stage 1 -5 repeats approximately every 90 minutes in a normal individual. For the purpose of assessment, Stage 2 sleep is of special interest. It is characterised by Sleep Spindles and K-Complexes along with mixed frequency, low voltage EEG signal activity. The EEG signal is also normally classified into frequency bands: • • • •

4

below 4Hz (δ band) between 4 and 8Hz (θ band) between 8 and 12Hz (α band) above 12Hz (β band)

The Nature of Obstructive Sleep Apnoea (OSA)

Sleep Apnoea Syndrome (SAS), first described in 1965, is a phenomenon characterised by excessive daytime sleepiness. It is classified as Obstructive , Central or Mixed [4]. Obstructive Sleep Apnoea (OSA) is by far the most common form of the aliment and is caused primarily by the collapse of the upper airway, resulting in

286

M. MacCallum and A. E. A. Almaini

diaphragmatic and chest wall movement without airflow. Individuals who have a narrower than average throat due to either genetic factors or obesity are thus more likely to occlude their upper airway during sleep giving rise to the repetitive arousal from sleep and repetitive hypoxaemia. It is this consequent sleep disruption and hypoxaemia which causes the daytime sleepiness and impaired cognitive function.

Falling asleep

Upper airway obstruction

Arousal from sleep

Breathing pause

Fig. 1. The Cycle of events in OSA

As illustrated in Figure 1, when the patient drifts off to sleep the muscles of the upper airway relax and loosen. Under these conditions, the upper airway can be obstructed completely producing a breathing pause or 'Apnoea' of at least 10 seconds in duration (meaning ‘want of breath’ from Greek) [5]. During such episodes, the patient will make continued efforts to breathe through the upper airway obstruction, but oxygen levels in the blood will fall. This results in the sufferer becoming hypoxic and the individual may exhibit heavy snoring. After between 10 and 90 seconds, the increased respiratory effort to clear the obstruction or the falling blood oxygen levels alert the brain, and produce a brief awakening, or 'arousal', from sleep. Arousal restores the muscle tone of the upper airway and breathing can subsequently recommence until the next drift into sleep, when the cycle may be repeated. The above cycle of disturbance may occur hundreds of times in the course of a night’s sleep. The normal pattern of sleep, progressing through stages of light sleep into deeper and restorative slow wave sleep and then into dreaming sleep, may be grossly disrupted by the frequent arousals to recommence breathing, resulting in the shallow and broken sleep pattern which is a major cause of the daytime problems experienced in OSA. Obstructive sleep apnoea is claimed to be an important cause of premature death and disability . There is increasing pressure to provide sleep services for the treatment of patients with sleep apnoea . Epidemiological evidence suggests that sleep apnoea causes vehicle and other workplace accidents . One of the most dramatic physiological consequences of OSA is the large rise in systemic blood pressure that occurs at the end of each apnoeic episode . Systolic blood pressure can increase by up

The Application of the Wavelet Transform to Polysomnographic Signals

287

to 100 mmHg, 300-400 times each night in severe cases. With regard to treatment, there is a paucity of robust evidence for the clinical and cost effectiveness of continuous positive airways pressure (CPAP) for most patients with sleep apnoea. [6]. In summary, OSA is a serious, progressive and potentially life-threatening breathing disorder. It is among the most common chronic disorders in humans with a prevalence of around 4% of the general population. It is more common in men than women and it is estimated that mild to moderate sleep apnoea goes undetected in some 90% of sufferers. Some studies indicate that it is associated with an increased risk of heart attack and stroke and is also linked to depression, irritability and learning difficulties.

5

Conventional Polysomnographic Signal Analysis

The gold standard for diagnosis and evaluation of sleep apnoea is overnight polysomnography [7]. This is an expensive and labour intensive procedure which requires the patient to remain overnight in a sleep laboratory. The scoring of the polygraph is based on the unit of the epoch, which is one page of a sleep record. In this case, this unit represents 30 seconds of recording time. For the purpose of scoring, each page is scrutinised in turn and assessed as a whole for its sleep stage. In some situations, the stages in the preceding and/or following pages influence the scoring of that page. The main evaluatory signal is the EEG, but EMG and EOG are also used in certain cases where sufficient uncertainty exists to warrant correlation. The human analyst, must thus scan the entire sleep record, manually scoring periods of absent or decreased airflow (apnoeas and hypopneas), whilst correlating discontinuities against the other sampled data, as shown in Figure 2. The repetitiveness and subjectiveness of the task also leads to inaccuracies and low interscorer agreement. The growing number of patients being examined for OSA has caused a strain on healthcare personnel and consequently a need has arisen for technological improvements to increase efficiency of diagnosis.

SCORED AROUSAL Fig. 2. Visually Scored Data with Arousal Shown

288

6

M. MacCallum and A. E. A. Almaini

Data Source

The Scottish National Sleep Centre within Edinburgh Royal Infirmary provided the data for this work. The EEG data was gathered from scalp-mounted transducers at C3 and C4 on the scalp as indicated in [3]. The data was recorded on a computerised polysomnographic system (Compumedics Inc., Melbourne, Australia), sampled at 125 Hertz and stored on optical disk. The data is encoded in EDF (European data Format). The data from 11 patients was supplied, ranging from normal through atypical to pathological OSA sufferers. Personnel at the Sleep Centre had manually scored this data and the tabulated results for both arousal events and sleep stages were supplied. For initial evaluation purposes, it was decided to focus on the data of a severe OSA patient, as the various events within the EEG data set for this individual were much easier to correlate by visual inspection

7

Automatic Detection Method for K-Complex and Sleep Spindle

7.1 Rationale As a first step towards an automated mechanism for detecting arousal within OSA sufferers, it is imperative that the appropriate stage of sleep be first established. This is of particular importance as the vast majority of visual scoring is conducted in Stage 2 NREM sleep. A number of automated methods have emerged for the purpose of sleep staging over the past twenty or so years [8]. Any method so created must also take into account the considerable variability of data both intra and inter-individual and the inherent stochastic nature of the EEG [9]. Thus the key characteristics of this sleep stage are: 1. 2.

Presence of Sleep Spindles Presence of K-Complexes

The difficulty in detecting the above parameters is compounded by EEG background activity. Thus some type of filtering technique must be utilised in the first instance. The signal processing method chosen for this investigation was the Wavelet Transform for the following reasons: 1. 2.

It is capable of extracting both time and frequency information Unlike the Short-Term Fourier Transform (STFT), it’s resolution varies with frequency

The above is especially useful when the signal under scrutiny has short duration, high frequency components and long duration ,low frequency components. 7.2 K-Complex This signal, which is central vertex in origin, is described as a biphasic wave swinging negative then positive going (or the reverse depending on the location of the probes

The Application of the Wavelet Transform to Polysomnographic Signals

289

on the skull). A typical K-Complex waveform is shown in Figure 3. The voltage measured from peak-to-peak should exceed 75 micro volts with a duration of between 0.5 and 5 seconds. It should also be at least twice the amplitude of the preceding one second of sleep activity.

Fig. 3. Typical K-Complex

Particular care must be taken not to confuse K complexes with slow (delta) waves, as shown in Figure 4. As a general rule, K complexes tend to occur in groups and runs. K complexes are often accompanied by a transientincrease in EMG activity, and this also helps to discriminate them from slow waves.

Fig. 4. Typical Delta Waves

7.3 Sleep Spindle A sleep spindle must be more than half a second in duration in order to be scored. Sleep spindles are small ‘bursts’ of brain activity and are more abundant during, and thus indicative of, Stage 2 sleep. A typical Sleep Spindle waveform is illustrated in Figure 5. The frequency range of sleep spindles is 12 – 16 Hz [10]. The range of duration 0.5 seconds. Sleep spindles are generated in the thalamus and are generally diffuse, but of highest voltage over the central regions of the head. The amplitude is normally less than 50 µV in the adult. One of the identifying EEG features of non-REM stage 2 sleep; may persist into non-REM stages 3 and 4; not seen in REM sleep.

290

M. MacCallum and A. E. A. Almaini

Fig. 5. Typical Sleep Spindle

7.4 Automatic Detection Algorithm Development The composite algorithm for arousal detection is shown in Figure 6. The wavelet packet transform, within the MATLAB software suite, was employed to extract the relevant frequency band, although high frequency decomposition of the composite EEG signal was not an intention. The main purpose of using the packet approach was to afford easier graphical visualisation of different frequency bands within the decomposed signal The frequency bands available for analysis are dependent on the sampling frequency of the signal, which is 125 Hz. Using the wavelet decomposition tree this means the analysis level is level 5 (for K-Complex) in order to extract the appropriate ( < 4 Hz δ band) frequency range. It was subsequently discovered that level 4 (10-16 Hz) was better suited for Sleep Spindle detection (i.e. 12-24 Hz β band)). It was decided to use, for initial analysis, the HAAR wavelet as it is the simplest orthogonal system and thus fastest, especially for signals with rapid transitions. A test epoch (Slst21.mat) was used as a benchmark for initial evaluation as it contains a clear and confirmed K-complex, as well as an additional borderline K-complex. K-complex Detection Algorithm

30 second Epoch

Arousal Detection Algorithm

Arousal Information

Sleep Spindle Detection Algorithm

Fig. 6. Data flow between algorithm components

The Application of the Wavelet Transform to Polysomnographic Signals

291

Detection System for K-Complex This was based o the following criteria • • • •

Minimum period: 0.5 seconds Maximum period: 5 seconds Minimum amplitude: 75µV Frequency range: below 4Hz (δ band)

Figure 3 shows that the key points of interest are the turning points (TPs), the location of which allows the duration and amplitude to be calculated. This results in the process sequence shown in Figure 7.

1

Wavelet Coefficient Extraction

TP Detection 2

K-Complex Detection 3

4

1.

Input signal

2.

δ band

3.

Turning points

4.

K-complexes

Fig. 7. Full detection sequence for K-Complex

From the previous flow diagram the K-complex detection block may be created: The main stages are: • • • • •

Check the position of the pointer is valid, i.e. 3 positions from the end of the array. Check the type is N, i.e. negative turning point. Set the duration and amplitude Check the duration and amplitude are within the required limits. Log K-complex

The algorithms for both turning-point and K-Complex were encoded within MATLAB using M-Files Detection System for Sleep Spindle As illustrated in Figure 6, for full establishment of the sleep stage (i.e. stage 2), it is necessary to ascertain the presence of the Sleep Spindle. This is accomplished in a manner similar to that for the K-Complex Detection criteria for the Sleep Spindle was taken as : • • • • •

•

Minimum period: 0.5 seconds Maximum period: 2 seconds Minimum amplitude: 10µV Maximum amplitude: 50µV Frequency range 12-16 Hz Six or seven distinct waves

292

M. MacCallum and A. E. A. Almaini

Read TP Data

Yes

TP pointer < length(tp)-3? No

Increment TP pointer

Increment KC pointer

Read TP record

Log K-complex Yes

Is type=N?

No

No

amplitude L

(10)

To make the mean-variance estimation of the wavelet coefficients serial of the scale j smallest, namely

~

σ 2j =ˆ E{(d j [n] − d j [n]) 2 } = min( R j (0), σ w2 }  σ w2 =  R j (0)

σ w2 < R j (0) σ w2 ≥ R j (0)

(11)

302

Wei Su et al.

R j (0) = σ c2 2γj

Since

σ w2 j ≤ log 2 2 γ σc 1

so when

and

σ ≥ R (0) , 2 w

2 j

let

σ w2 L = [ log 2 2 ] , γ σc 1

where [ ]

means reserving the integer part, and we have

j≤L j>L

 0 ~ d j [ n] =   yˆ j [n]

4

(12)

Simulation

In the signal estimation of statistic self-similarity process, to demonstrate the viability and the effectiveness of the Hilbert transform, we use computer simulation to rehabilitate the 1/f-type fractional signal. Figure 1 and figure 5 indicate Gaussian zero-mean 1/f-type fractional signal, which comes from random Weiestrass function, where the number of sample points is 1000, γ = 1.7 , and bases on Harr wavelet. Figure 2 indicates the received signal in Gaussian white noise, in which the fractional signal has a figure that x=0dB. Figure 3 indicates the Hilbert transform of the received signal. Figure 4 indicates the estimation of by the optimum threshold. Figure 6 indicates the signal by inverse Hilbert transform of the signal in figure 4, where M=2, and the error of signal’s estimation deta=0.5591. So the viability and the effectiveness of the Hilbert transform have been demonstrated. 4 2 0 -2 -4 0

100

200

300

400 500 600 Figure1.1/f Signal

700

800

900

1000

0

100

200

300 400 500 600 700 Figure2.1/f Signal in Noise

800

900

1000

5

0

-5

Wavelet Transform Method of Waveform Estimation

303

5

0

-5 0

100

200

300 400 500 600 700 800 Figure3. Signal through Hilbert Trans form

900

1000

0

100 200 300 400 500 600 700 800 900 Figure4. E stim ated S ignal by the Optim um Thres hold Technique

1000

0

100

900

1000

0

100 200 300 400 500 600 700 800 900 Figure6.E s tim ated Signal through Invers e Hilbert Trans form

1000

6 4 2 0 -2 -4

4 2 0 -2 -4 200

300

400 500 600 Figure5.1/f S ignal

700

800

4 2 0 -2 -4

5

Conclusion

In this paper, those splendid characters of the Hilbert transform let the processes that taking wavelet transform after taking Hilbert transform for the statistic self-similarity processes FBM [ B H (t ) ] become another processes, that firstly taking Hilbert transform for the wavelet function

φ (t )

and forming a new wavelet function

secondly taking the wavelet transform for

ψ (t ) ,

BH (t ) . Then, we use the optimum

threshold to estimate the Bˆ H (t ) embedded in additive white noise. Typical computer simulation results to demonstrate the viability and the effectiveness of the Hilbert transform in the signal’s estimation of the statistic self-similarity process. So this paper is the fundamental work, later we will take part in the estimation of complex signals.

304

Wei Su et al.

References 1.

2.

3. 4. 5.

6. 7. 8.

B. S. Chen and G. W. Lin, “Multiscale Wiener filter for the restoration of fractal signals: Wavelet filter bank approach”, IEEE Trans. Signal Processing, Vol. 42, No. 11, PP. 2972-2982,1994. B. S. Chen and W. S. Hou, “Deconvolution filter design for fractal signal transmission systems: A multiscale Kalman filter bank approach”, IEEE Trans. Signal Processing, Vol. 45, PP. 1395-1364, 1997. P. Flandrin, “On the spectrum of fractional Brownian motion”, IEEE Trans. Information Theotry, Vol. 35, No. 1, PP. 197-199, 1989. P. Flandrin, “Wavelet analysis and synthesis of fractional Brownian motion”, IEEE Trans. Information Theory, Vol. 38, No. 2, PP. 910-917, 1992. B. B. Mandelbrot and J. W. Van Ness, “Fractional Browrian motions, fractonal motions, fractional noises and applications” SIAM Rev., Vol. 10. No. 4, pp. 422437, 1968. G. W. Wornell, “A Karhunen-Loeve-Like Expansion for 1/f Processes Via Wavelets”, IEEE Trans, Information Theoty, V0l. 36, No. 4, PP. 859-861, 1990. Hong Ma, Michio Umeda, Wei Su, “Hilbert Transform of Non-stationary Stochastic Signal and Parameter Estimation”, to appear. Kesu Zhang, Hong Ma, Zhisheng You, Michio Umeda, “Wavelet Estimation of Non-stationary Fractal Stochastic Signals Using Optimum Threshold Technique”, to appear.

Multiscale Kalman Filtering of Fractal Signals Using Wavelet Transform* Juan Zhao1, Hong Ma1, Zhi-sheng You2, and Michio Umeda3 1

Dept of Mathematics, Sichuan University Chengdu 610064,China 2 Dept of Computer Science, Sichuan University Chengdu 610064,China 3 Dept of Information Engineering, Osaka Electro-Communication University Osaka, Japan

Abstract. A filter bank design based on orthonormal wavelets and equipped with a multiscale Kalman filter was recently proposed for signal restoration of fractal signals corrupted by external noise. In this paper, we give the corresponding parameters of the dynamic system and more accurate estimation. Comparisons between Wiener and Kalman filters are given. Typical computer simulation results demonstrate its feasibility and effectiveness.

1

Introduction

The family of 1/f stochastic processes constitutes an important class of models for different signal processing applications. Examples are geophysical and economic time series, biological and speech signals, noise in electronic devices, burst errors in communications, and recently, traffic in computer networks[4]. A typical model for these processes is the fractional Brownian motion (fBm), which is a Gaussian zeromean nonstationary stochastic process B H (t ) indexed by a parameter 00 σ [ n ] − σ [ n] = ∑ 2 j i =1,! p −1 p n + i / n + i + σ w 1 That is to say, dˆ [ n] is more accurate than dˆ [ n] in the mean-square sense . 2 j

1

2 j

j

Proof: Subtracting

j

d j [n − i ] by the both sides of the equation (6)and then doing

variance operation, yields

Multiscale Kalman Filtering of Fractal Signals Using Wavelet Transform

309

2

p

j n ,i +1

=p

j n −1,i

−2

where

pj + j i / n 2 2 E ( y j [ n] − φ j xˆ j [ n − 1]) 2 ( pn / n + σ w )

pij/ n E (dˆ j [n − 1, n − i ] − d j [ n − i])(φ j xˆ j [ n − 1] − y j [n]) p nj / n + σ w2

p nj,i +1 and p nj−1,i are the i + 1 th element of the diagonal of Pnj and the i th j

element of the diagonal of Pn −1 ,respectively. Substituting y j [ n] = d j [ n] + w j [n] into the above equation and considering the independence between xˆ j [ n − 1] d j [n] and w j [n] , it follows from (4) and(7) that

E ( y j [n] − φ j xˆ j [n − 1]) 2 = p nj / n + σ w2 E (dˆ j [n − 1, n − i] − d j [n − i])(φ j xˆ j [n − 1] − y j [n]) = pij/ n 2

So we obtain

p

j n ,i +1

=p

j n −1,i

pj − j i / n 2 i = 1, ! p − 1 , i.e. the mean-square pn / n + σ w

estimation error of d j [ n − i ] is deceasing. It follows from (5) and (9)that the theorem holds. Now we consider the estimation of the approximation coefficients using the sequence of approximation coefficients of the signal y (k )

y Ja [n] = a J [n] + wJa [n] a

where a J [n] and w J [n] are the sequences of approximation coefficients of the fBm and of the observation white noise process, respectively. We use a simple memoryless estimator a aˆ J [ n] = E{a J [ n] | y J [ n ]} =

Var ( aJ [ n]) a y [ n] 2 J Var (aJ [ n]) + σ w

(10)

which is optimal in the mean-square sense and the estimation error variance is

σ a2 [n] = E (a J [n] − aˆ J [n]) 2 =

Var (a J [n ])σ w2 Var (a J [n]) + σ w2

Then the mean-square error of the estimation of the process

E[( BH ( k ) − Bˆ H ( k ) ] = 1 ( 2

N0

m ( J ) −1

∑σ n=0

(11)

BH (k ) is given by

J m ( j ) −1

2 a

[n] + ∑ j =1

where m( j ) is the number of samples in the scale

∑σ

2 j

[n ])

(12)

n =0

j , k = 0 , ...... , N 0 − 1

310

3

Juan Zhao et al.

Simulations

We will estimate a fractal signal embedded in noise with different methods. Considering that Wiener filters use correlation function R j (0), ! , R j ( M − 1) and Kalman filters use R j (0), R j (1), !, R j ( p ) , we let p = M − 1 to guarantee that they use the same information. We consider 1000 samples of a fractal signal

BH (k ) embedded in additive Gaussian white noise with variance σ w2 =1. For the simulations, we have chosen Haar wavelet and used Weiestrass function to generate of the fBm with H = 0.35 and a filter bank corresponding to 3 scales. Kalman filters 2 ( p = 3) and Wiener filters ( M = 4) are used to estimate the fractal signal and the estimate errors are 0.3020 and 0.3044 ,respectively. 3 2 1 0 -1 -2 0

100

200

300 400 500 600 Figure 1. fBm with H=0.35

700

800

900

1000

100

200 300 400 500 600 700 800 900 Figure 2. fBm embedded in additive white noise

1000

6 4 2 0 -2 -4 0

Because the decay of the correlation heavily depends on R, i.e. the number of vanishing moments of wavelet[3], the estimated fractal signal will be more accurate when we use wavelet with large R under the same conditions. Now we make a comparison between Haar wavelet ( R = 1) and Daubechies5 wavelet wavelet ( R ≥ 2)

Multiscale Kalman Filtering of Fractal Signals Using Wavelet Transform

311

4 2 0 -2 -4

0

100

200 300 400 500 600 700 800 Figure 3. estimated fBm by Kalman filters 2

900

1000

0

100

200

900

1000

4 2 0 -2 -4

300 400 500 600 700 800 Figure 4. estimasted fBm by Wiener filters

( R ≥ 2) .We consider 2048 samples of a fractal signal embedded in additive 2

white noise with variance σ w =1 and calculate the theoretical mean square error values of Kalman Filters 1 ( p = 4 ), Kalman Filers 2 ( p = 4 )and Wiener filters( M = 5 ) corresponding to 3 scales. Table 1 Mean-Square Error for Different Values of the Parameter H and Different Methods

Haar wavelet

H 0.55 0.60 0.65 0.70 0.75 0.80 0.85 0.90 0.95

Kalman filter 1

0.5499 0.5551 0.5627 0.5726 0.5849 0.5994 0.6161 0.6345 0.6544

Kalman filter 2

0.5498 0.5543 0.5606 0.5683 0.5770 0.5862 0.5954 0.6040 0.6115

312

Juan Zhao et al.

Table 2 Mean-Square Error for Different Values of the Parameter H and Different Methods Daubechies5 wavelet

H 0.55 0.60 0.65 0.70 0.75 0.80 0.85 0.90 0.95

Kalman filter 1

Kalman filter 2

0.4610 0.4520 0.4442 0.4374 0.4315 0.4262 0.4216 0.4176 0.4141

0.4550 0.4454 0.4370 0.4297 0.4231 0.4173 0.4122 0.4076 0.4035

We observe that we can obtain more accurate estimate by Kalman filters with 2% improvement under the same conditions and the estimation errors in the table II are dramatically lower than the values shown in the table I using wavelet with large R. Furthermore, we consider the choice of the number of scales J and p in the AR model. We shall show some numerical results obtained from simulations. For the simulations, we have considered Kalman filters 2 with Haar wavelet and H=0.75.The results obtained are shown in the Table III and we find that large p can decrease the estimation error effectively. Table 3 Mean-Square Error for Different Values of

p

in the AR Model and the Number of

Scales J

p=1 J=11 p=2 J=11 p=3 J=11

4

0.5879 0.5795 0.5772

p=2 J=3 p=3 J=3 p=4 J=3

0.5805 0.5782 0.5770

Conclusions

A scheme for the estimation of fBm was developed on the basis of a bank of multiscale Kalman filters. It takes into account the correlation of the wavelet coefficients and the approximation coefficients in the wavelet expansion. In this paper, we propose the more accurate estimation based on [5]. Comparisons between Wiener and Kalman filters are given. Numerical results were shown on the wavelet and the minimum mean-square error for different values of the parameter H in the fBm. The theoretical results matched the ones obtained in the numerical simulations. It can be used to estimate signal in noise such as communication in radar, processing of biological signal etc.

Multiscale Kalman Filtering of Fractal Signals Using Wavelet Transform

313

References 1.

2.

3. 4.

5.

6.

7.

B.S. Chen and G.W. Lin, “Multiscale Wiener filter for the restoration of fractal signals: Wavelet filter bank approach,” IEEE Trans. Signal Processing.Vol. 42,N0.11,pp. 2972~2982,1994. B.S. Chen and W.S. Hou, “Deconvolution filter design for fractal signal transmission systems: A multiscale Kalman filter bank approch,”IEEE Trans. Signal Processing, Vol. 45,pp. 1395~1364, 1997. P. Flandrin “Wavelet analysis and synthesis of fractional Brownian motion ”IEEE Trans.Inform . Theory Vol . 38,No.2,pp . 910~917,1992. G. A. Hirchoren and C. E . D‘Attellis “Estimation of fractal signals using wavelets and filter banks,” IEEE Trans. Signal processing, Vol.46,No.6, pp.1624~1630, June 1998. G. A. Hirchoren and C. E . D‘Attellis “Estimation of fractal Brownian Motion with Multiresolution Kalman Filter Banks,“ IEEE Trans Signal Processing, Vol. 47, No.5,pp. 1431~1434, May 1999. B.B.Mandelbrot and J.W.Van Ness, “Fractional Browrian motions, fractional motions, fractional noises and applications,” SIAM Rev., Vol.10. No.4, pp.422~437, 1968. G.W. Wornell and A.V. Oppenheim, “Estimation of fractal systems from noisy measurements using wavelets,” IEEE Trans. Signal Processing, Vol, 40, No.3, pp.611~622, 1992.

General Analytic Construction for Wavelet Low-Passed Filters* Jian Ping Li1 and Yuan Yan Tang2 1

International Centre for Wavelet Analysis and Its Applications, Logistical Engineering University, Chongqing 400016, P. R. China [email protected], [email protected] 2 Department of Computer Science, Hong Kong Baptist University, Hong Kong [email protected]

Abstract. The orthogonal wavelet lowpassed filters coefficients with arbitrary length are constructed in this paper. When N=2k and

N = 2 k − 1 , the general analytic constructions of orthogonal wavelet filters are put forward, respectively. The famous Daubechies filter and many other wavelet filters are tested by the proposed novel method, which is very useful for wavelet theory research and many applications areas such as pattern recognition.

1

Introduction

The scaling equation is

ϕ(x)=∑ h(n)ϕ(2x-n) , n∈ Z

(1)

where h(n), n∈ Z, is very complex, but also the building function of multiresolution analysis is not easy for finding. Essentially, finding wavelet function or scaling function is equally finding filters coefficients of wavelet[1~12]. The conditions of orthonormal bases of compactly supported wavelets are Eq.(2) to Eq.(5) as following: 2 N −1

∑ h (i ) =

2,

(2)

i=0 N −1

N −1 2 h ( 2 i ) = ∑ ∑ h ( 2 i + 1) = 2 , i=0 i=0

*

(3)

This work was supported by the National Natural Science Foundation of China under the grant number 69903012.

Y. Y. Tang et al. (Eds.): WAA 2001, LNCS 2251, pp. 314-320, 2001.  Springer-Verlag Berlin Heidelberg 2001

General Analytic Construction for Wavelet Low-Passed Filters

h(0)   

h(1) h(2) .... h(2N-1) h(0) h(1) h(2) .... h(2N-1) ............................ h(0) h(1) h(2) ....

   h(2N-1)  N × 2 N

315

(4)

is row orthogonal matrix, and 2 N −1

∑ h (i )h (i ) = 1 .

(5)

i=0

2

Analytic Construction of Wavelet Filters when N=1,2

When N=1 the filter has only two coefficients, if we denote the parameter angle as α, from Eq.(5), we can suppose h(0)= cosα; h(1)= sinα. It is clear that, if Eq.(2) and Eq.(3) are satisfied, then α=

π 4

, here the wavelet

function is Haar-Wavelet. On the contrary, for arbitrary parameter angle α, Eq.(4) and Eq.(5) are naturally satisfied, but Eq.(2) and Eq.(3) are not satisfied. We will discuss the case of N=2 below. Theorem 1 When N=2, wavelet filter coefficients {h(0),h(1),h(2),h(3)} are subject to orthogonal wavelet conditions Eq.(2), Eq.(3), Eq.(4) and Eq.(5) if and only if there are parameters angles α,β such that

α+β=

π 4

,

h(0)= cosα cosβ; h(1)= sinαcosβ; h(2)=- sinαsinβ; h(3)= cosαsinβ. Especially, when α=

π 3

, β= −

π 12

,the wavelet coefficients are below

h(0)=0.482963 h(1)=0.836516 h(2)=0.224144 h(3)=-0.129410, which are Daubechies wavelet filter coefficients when N=2. Of course, from above formulae, infinite kinds of filters coefficients will be gotten. The proof of this theorem is shown in [12].

316

3 3.1

Jian Ping Li and Yuan Yan Tang

Analytic Construction of Wavelet Filters when N = 2 k − 1 Commonly Conclusion When

N = 2k − 1

First of all, we define decomposed rules. Rule 1 Strict Sequence Decomposition (SSD) Definition 1 The decompositions of cosine function cos(α+β) and sine function sin(α+β) are called strict sequence decomposition (SSD) if cos(α+β)=cosαcosβ-sinαsinβ, sin(α+β)=sinαcosβ+cosαsinβ . It is clear that, decomposed items of cosine function cos(α+β) form even coefficients such as h0,h2,..., and decomposed items of sine function sin(α+β) form odd coefficients such as h1,h3,...... Definition 2 The decompositions of cosine function cos(α+β) and sine function sin(α+β) are called inverse sequence decomposition (ISD) if cos(α+β)=-sinαsinβ+cosαcosβ, sin(α+β)=cosαsinβ+sinαcosβ . Rule 2 Parameter Angles Sequence Combination (PASC) Definition 3 Decomposition of parameter angles is called back combination way (BCW) if α+β+γ=α+(β+γ), α+β+γ+θ= α+(β+γ+θ) . ... Definition 4 Decomposition of parameter angles is called fore combination way (FCW) if α+β+γ=(α+β)+γ, α+β+γ+θ=(α+β+γ)+θ . ... In following subsection, we decompose cos(α1+α2+.....αk) and sin(α1+α2+ .....αk) in BCW and SSD for k>1, N = 2 k − 1 时，and denote their decomposed items as vectors as following C(k)={h(0),h(2),h(4),...h(2N-2)},

(6)

S(k)={h(1),h(3),h(5)...h(2N-1)},

(7)

respectively. Then the decomposed items for cos(α+α1+α2+.....αk) and sin(α+α1+ α2+ .....αk) , which contain k+1 parameter angles as following vectors C (k+1)={cosα C (k), -sinαS(k)},

(8)

S (k+1)={sinα C (k), cosα S(k)},

(9)

respectively. This result is very simple for us to prove.

General Analytic Construction for Wavelet Low-Passed Filters

317

Lemma 1 For arbitrary set of parameter angles α1, α2, ..... αk (k>1, N = 2 k − 1 ), if cos(α1+α2+.....αk) and sin(α1+α2+.....αk) are expanded in BCW and SSD, then (k)={h(0),h(2),h(4),...h(2N-2)}, (k)={h(1),h(3),h(5)...h(2N-1)}, are subject to the row orthogonal condition Eq.(4). Proof. A mathematics induction method will be used in this proof, the details of the proof is shown in [12]. Theorem 2 For any set parameter angles of α1,α2,....αk (k>1, N = 2 k − 1 ), cos(α1+α2+.....αk), and sin(α1+α2+.....αk) are decomposed in BCW and SSD, i.e., the decomposed items from Eq.(6) to Eq.(9). If the sum of all the parameter angles is

π 4

, then the decomposed items are wavelet filters coefficients which subject to the

orthogonal wavelet basis conditions such as Eq.(2) to Eq.(5). 3.2

Some Special Cases for N = 2 k − 1

Some details are discussed for some k ( such as k=3)parameters in this subsection. Please pay attention to the sequence of decomposition, combination, incorporation. First of all, we construct the wavelet filter coefficients for N = 2 k − 1 , where k=3. Decomposing cos(α+β+γ), sin(α+β+γ) in BCW and SSD, we have cos(α+β+γ)=cos(α+(β+γ)) = cosα cos(β+γ)-sinαsin(β+γ) = cosα cosβ cos γ- cosα sinβ sinγ-sinα sinβ cos γ-sinα cosβ sinγ , sin(α+β+γ) =sin(α+(β+γ)) =sinαcos(β+γ)+cosαsin(β+γ) =sinαcosβcosγ-sinαsinβsinγ+cosαsinβcosγ+cosαcosβsinγ . We denote the decomposed items of cos(α+β+γ) as even items according to their sequence, i.e., h(0)=cosα cosβ cosγ; h(2)=-cosα sinβ sinγ; h(4)=-sinαsinβcosγ; h(6) =-sinα cosβ sinγ. In the same reason, denote the decomposed items of sin(α+β+γ) as odd items according to their sequence, i.e., h(1)=sinαcosβcosγ; h(3)=-sinαsinβsinγ; h(5) = cosαsinβcosγ; h(7)=cosαcosβsinγ. α+β+γ=π/4. Then, h(0),h(1),h(2),h(3),h(4),h(5),h(6),h(7) construct wavelet filter coefficients for N=4.

318

Jian Ping Li and Yuan Yan Tang

Incorporating h(4) in h(2) and h(5) in h(3), respectively, i.e., h(2)=h(2)+h(4), h(3)=h(3)+h(5), we have h(0)=cosαcosβcosγ; h(1)=sinαcosβcosγ;

h(2)=-sin(α+γ)sinβ, h(3)=cos(α+γ)sinβ,

h(4)=-sinαcosβsinγ; h(5)=cosαcosβsinγ.

Then, h(0),h(1),h(2),h(3),h(4),h(5) construct six coefficients for N=3. This is nice a easy to prove. And when α=π/2.66295; β=-π/6.28518; γ=π/2 7792 , we can get Daubechies wavelet filter coefficients forN=3 as following h(0)= .332671, h(1)=.806892, h(2)=.459878, h(3)=-.135011, h(4)=-.085441, h(5)=.0352263. If we set γ=0, and throw off zero elements, we can get wavelet filter coefficients for N=2, i.e. , Theorem 1.

4 4.1

Analytic Construction of General Filters Definition of Decomposed Method

Recursion Decomposed Method: When N=1, cosα and sinα can not be decomposed, i.e., h(0)= cosα, h(1)=sinα.. When N>1, decomposing cos(α1+α2+.....αN) and sin(α1+α2+.....αN), marking their decomposed items as following vectors, respectively, C(N)={h(0),h(2),h(4),...h(2N-2)},

(10)

S(N)={h(1),h(3),h(5)...h(2N-1)}.

(11)

Then, the decomposed items of cos(α+α1+α2+.....αN) and sin(α+α1+ α2 + .....αN) are vectors below, respectively (12) C(N+1)={cosα(C(N),0)-sinα(0,S(N))} S(N+1)={sinα(C(N),0)+cosα(0,S(N))}

(13)

Lemma 2 Denote ∑C(N) and ∑S(N) as sum of C(N) and S(N), respectively, then

4.2

∑C(N)=cos(α1+α2+.....αN),

(14)

∑S(N)=sin(α1+α2+.....αN).

(15)

General Conclusion

Lemma 3 For any set of parameter angles such as α1, α2, ..... αN, the decomposed items of cos(α1+α2+.....αN) and sin(α1+α2+.....αN) in induction method based on Eq.(10) to Eq.(13) as following , respectively,

General Analytic Construction for Wavelet Low-Passed Filters

319

(N)={h(0),h(2),h(4),...h(2N-2)}, (N)={h(1),h(3),h(5),...h(2N-1)}. Then, the decomposed items are subject to row orthogonal condition Eq.(4). Theorem 3 For any set of parameter angles such as α1, α2 ....., αk, if sum of all parameter angles is

π 4

, then decomposed items of cos(α1+α2+.....αk) and

sin(α1+α2+.....αk) based on induction decomposed method from Eq.(10) to Eq.(13) are wavelet filter coefficients which subject to orthogonal conditions such as Eq.(2) to Eq.(5).

5

Conclusions

This paper introduced the novel method of selection and construction of wavelet basis, that is, unified analytic construction of wavelet filters based on trigonometric functions. Many traditional famous wavelet coefficients are special cases of the novel method, and it is more fast, more simple, more efficient, more advantageous than the traditional methods. We have designed a very efficient and advantageous software , which can calculate fast and simply wavelet filters coefficients with arbitrary parameter angles produced by random process. The algorithms and formulae in this paper show that how to adaptively choose wavelet basis or the best wavelet basis for any problems is very easy and fast. Our novel method will influence on studies of wavelet theory and its applications, and it is very useful for application of wavelet to pattern recognition.

References 1. 2. 3. 4. 5.

6.

Daubechies: Orthonormal bases of compactly supported wavelets. Comm. Pure & Appl. Math 41(1988) 909~996 Z. X. Chen: Algorithms and applications on wavelet analysis. Xi’an Jiaotong University Publishing House, Xi’an, P.R.China( 1998) 78~119 Q. Q.Qin, Z. K. Yang: Applied wavelet analysis, Xi’an Electronic Science and Technology University Publishing House, Xi’an:,.R.China( 1994)41~53 Wickerhauser M V: Adapted wavelet analysis from theory to software. New York : SIAM , (1994) 442~462 Vaidyanathan P P, Huong P Q: Lattice structures for optimal design and robust implementation of two channel perfect-reconstruction QMF banks. IEEE Trans. On ASSP,1(1998)81~94 Jian Ping Li: Wavelet analysis & signal processing-----theory, applications & software implementations, Chongqing Publishing House, Chongqing (1997)96~101,282~298

320

7.

Jian Ping Li and Yuan Yan Tang

Jian Ping Li, Yuan Yan Tang. Applications of wavelet analysis method. Chongqing University Publishing House, Chongqing, P.R.China(1999)72~91 8. Jian Ping Li: Studies of the theories and applications of vector product wavelet transforms and wavelet analysis.[Ph.D. Thesis].In: Chongqing University, P.R.China(1998) 9. Yuan Yan Tang, et al: Wavelet theory and its application to pattern recognition. Singapore: World Scientific( 1999) 10. Jian Ping Li, Yuan Yan Tang: A novel method on fast wavelet analysis algorithm(I). Computer Science,5 ( 2001) 11. Jian Ping Li, Yuan Yan Tang: A novel method on fast wavelet analysis algorithm(II). Computer Science, 6(2001) 12. Jian Ping Li, Yuan Yan Tang: Analytic construction of wavelet filters based on trigonometric functions. IEEE Trans. on Information Theory (to appear)

A Design of Automatic Speech Playing System Based on Wavelet Transform⋆ Yishu Liu, Jinyu Cen, Qian Sun, and Lihua Yang Department of Scientific Computing and Computer Applications Zhongshan University, Guangzhou 510275, P. R. China

Abstract. This paper introduces a novel approach to store speech words after cutting the signals and decomposing them through Mallat’s decompostion algorithm, and generate a speech phrase by connecting such word data and reconstructing it through Mallat’s reconstruction algorithm. This way, speech signals of good quality can be produced easily from a small library of compressed speech words. Keywords: speech signal processing, wavelet basis, Mallat’s algorithm.

1

Introduction

Automatic Speech Playing plays an important role in speech signal processing. It is widely applied in many modern business automatic processing, such as electronic business, ATA (automatic time announcing) system, banks’ ATM system, IP telephone card service, etc.. An automatic speech playing system consists of a speech library and a speech connecting algorithm. Generally speaking, a speech library must be as small as possible and it must be guaranteed that the library has great ability in generating every kind of phrases and sentences. In the light of different practical application, accordingly we can choose the most basic speech units (in Chinese they are Chinese words’ speech data) to build an speech library. In order to reduce the volume of the library, its data can be compressed provided the result is acceptable. In this paper, we extract single Chinese word speech’s main part, which is then decomposed twice by wavelet decomposition algorithm. An approximation of the original speech obtained this way is made an element of the basic speech library. The library so built is very small. In phrase/sentence generating, the basic speech units are put together in proper order, then form a playable speech signal through wavelet reconstruction algorithm. Wavelet reconstruction makes the signals smoother, so that the audio effect is good. And this algorithm is potent in phrase/sentence generating. As a significant breakthrough of Fourier Analysis, wavelet has attached much attention in many fields from applied mathematics to signal processing. The idea of multi-resolution analysis underlying wavelet theory makes it possible to get the signals at different scales whose lengths are reduced by half successively. By ⋆

This work was supported by the Foundation for University Key Teacher by the Ministry of Education of China, NSFC(19871095) and GPNSFC(990227).

Y. Y. Tang et al. (Eds.): WAA 2001, LNCS 2251, pp. 321–325, 2001. c Springer-Verlag Berlin Heidelberg 2001


322

Yishu Liu et al.

neglecting the details, we can get an approximation of the signal, whose length is greatly shortened but whose main characteristics remain. Some basic facts on wavelets will be stated in the next section. A textbook-like reference on speech signal processing can be found in [4].

2

Some Basic Facts on Wavelets

Wavelet analysis has become an effective mathematical tool to process signals locally on time and frequency, which was developed more than ten years ago. With a wavelet basis, the space L2 (R) can be decomposed into the orthogonal sum of a sequence of closed spaces. That means, any signal with finite energy can be expressed as the sum of a series of local frequency structures. Such a decomposition is convenient for signal compression and smoothing. In this section, some basic facts on wavelet theory used in this paper are stated without proofs. A textbook-like introduction on wavelet theory can be found in [1,2,3]. Let L2 (R) be the space of all the finite energy signals, i.e., 
 ∞ 2 2 |f (t)| dt < ∞ . L (R) = f (t) : −∞

The well-known Mutliresolution Analysis (MRA) is defined as follows. Definition 1 Let φ(x) ∈ L2 (R). The sequence of closed subspaces of L2 (R) which are defined by Vj = {φj,k (x) = 2j/2 φ(2j x − k), k ∈ Z},

(j ∈ Z),

is called an orthonormal Mutliresolution Analysis (MRA) of L2 (R) if the following there conditions are satisfied: 1) Vj ⊆ Vj+1 , (∀j ∈ Z);   2) j∈Z Vj = L2 (R), j∈Z Vj = {0}; 3) {φ(t − k)}k∈Z is an orthonormal basis of V0 . Then φ(t) is said to be the corresponding scaling funtion of the MRA. It can be easily derived that {φj,k (t) = 2j/2 φ(2j t − k)}k∈Z is an orthonormal basis of Vj . For any j ∈ Z, we let Vj = Vj−1 ⊕ Wj−1 , where Wj−1 is the orthogonal complement of Vj−1 in Vj . We can find a wavelet function ψ(t) ∈ W0 such that {ψj,k (t) = 2j/2 ψ(2j t − k)}k∈Z is an orthonormal basis of Wj . Wj+1 ⊕ Wj+2 ⊕ ....... Therefore, for any j ∈ Z, we have L2 (R) = Vj ⊕ Wj ⊕ It can thus be inferred that ∀f (t) ∈ L2 (R), f (t) = Aj + Dn , where n>j

   Aj = an (j) · φj,n , an (j) = f, φj,n n   Dj = dn (j) · ψj,n , dn (j) = f, ψj,n n

A Design of Automatic Speech Playing System Based on Wavelet Transform

323

Aj is said to be the approximation of f(t) at 2j scale, it reflects the main information of f(x). And Dj is said to be the details of f(t) between scales 2j+1 and 2j . From Vj = Vj−1 ⊕ Wj−1 , we know that φj−1,0 and ψj−1,0 can be expanded in {ϕj,n }n∈Z to obtain scale equations as follows:



g(n)φj,n (t) , h(n)φj,n (t) , ψj−1,0 (t) = φj−1,0 (t) = n

n

where 1 t h(n) = 2− 2 φ( ), φ(t − n) , 2

t 1 g(n) = 2− 2 ψ( ), φ(t − n) = (−1)1−n h(1 − n). 2

And therefore, S. Mallat’s decomposition and reconstruction algorithms are obtained: Algorithm 2 Decomposition:    ak (j − 1) = h(n − 2k)an (j) n  ,  dk (j − 1) = g(n − 2k)an (j)

(k ∈ Z).

n

Reconstruction:



g(k − 2n)dn (j − 1), ajk = h(k − 2n)an (j − 1) +

3

(k ∈ Z).

n

n

The Construction of Basic Speech Library

The authors of the paper recorded some speech words. After our surveying and analyzing their wave forms in MatLab, a conclusion was arrived at: the waves are all composed of smooth parts and steep ones. Such an example is shown in the left of Fig.1. It is the speech data of Chinese pronunciation “jiu” of “nine”. In order to extract the signal’s main information (the steep part), an algorithm is presented as follows: (l)

1. For each speech signal f[i] , where l=0,1,2,......and i=0, 1, 2,..., Nl − 1 (Nl is an integer), do (a) Divide the interval [0, Nl − 1] equally into M parts; (b) Calculate num[j], j=0, 1, 2,..., M-1, where num[j] denotes the number (l) (l) of f[i] which satisfies: i belongs to the j’th part and f[i] ≥ ε; (c) Find the maximum among num[j], j=0, 1,..., M-1. Suppose it is num[maxj]. To select j’s around maxj such that num[j] ≥ δ ×num[maxj] and all the selected j’s are adjoining. Suppose they are j0 , j1 , ..., jk . (l) (l) N (d) Let=g[i] = f[i+ N j ] , i=0, 1, 2,..., M (jk − j0 + 1) − 1. M

0

2. Put all g (l) together in proper order to form a new signal.

324

Yishu Liu et al.

1

1

0.8

0.8

0.6

0.6

0.4

0.4

0.2

0.2

0

0

−0.2

−0.2

−0.4

−0.4

−0.6

−0.6

−0.8

−0.8

−1

0

0.5

1

1.5

2

2.5

−1

3

0

0.5

1

1.5

2

2.5

4

3 4

x 10

x 10

Fig. 1. Left: speech signal of “jiu”, Right: the extracted and decomposed signal 1

0.8

0.6

0.4

0.2

0

−0.2

−0.4

−0.6

−0.8

−1

0

0.5

1

1.5

2

2.5

3 4

x 10

Fig. 2. The connected speech signal The signal extracted still has some redundance. To save speech data efficiently, we further decompose it twice with Mallat’s decomposition algorithm with respect to Daubechies 10. Then the result is stored into the library as a basic element. The right of Fig.1 is an example for Chinese word “jiu”.

4

Connection and Reconstruction of Speech Signals

When the system is assigned to play a phrase such as the Chinese phrase of nine dollars and two cents:“jiu yuan er jiao”, four words will be chosen to connect the phrase which is then reconstructed by Mallat’s algorithm with respect to Daubechies 10. It can be shown in mathematics that such a reconstruction can smooth the connection points of a signal to obtain a better audio effect.

A Design of Automatic Speech Playing System Based on Wavelet Transform

325

The experiment is shown in Fig. 2. It has been played and shown to be audio acceptable.

References 1. C. K. Chui. An Introduction to Wavelets. Academic Press, Boston, 1992. 322 2. I. Daubechies. Ten Lectures on Wavelets. Society for Industrial and Applied Mathemathics, Philadelphia, 1992. 322 3. S. Mallat. Wavelet Tour of Signal Processing. Academic Press, San Diego, USA, 2nd edition, 1998. 322 4. L. Rabiner and B. H. Juang. Fundamentals of Speech Recognition. Prentice-Hall International, Inc., 1993. 322

General Design of Wavelet High-Pass Filters from Reconstructional Symbol⋆ Lihua Yang1 , Qiuhui Chen1 , and Yuan Y. Tang2 1

2

Department of Scientific Computing and Computer Applications Zhongshan University, Guangzhou 510275, P. R. China Department of Computer Science, Hong Kong Baptist University

Abstract. For given reconstructional low-pass filters, the general solu˜ (ξ)M ∗ (ξ) = I for the construction of orthogtions of matrix equation M onal or biorthogonal wavelet filter banks are presented. Keywords: matrix equation, MRA, wavelets, filter.

1

Introduction

In the theory of wavelet Analysis, it is well-known that the key to construct an orthogonal wavelet base or a pair of biorthogonal wavelet bases from MRA (Multiresolution Analysis) is to design the filter banks {m ˜ µ (ξ), mµ (ξ) |µ ∈ Ed },with Ed being the set of all the vertices of [0, 1]d, such that ˜ (ξ) := (m M ˜ µ (ξ + πν))µ,ν∈Ed , satisfy:

M (ξ) := (mµ (ξ + πν))µ,ν∈Ed ,

˜ (ξ) = M ˜ (ξ)M ∗ (ξ) = I2d M ∗ (ξ)M

a.e. ξ ∈ T d .

(1) (2)

Usually, the question on the solutions of (2) can be described as follows: Question 1 Assume m0 (ξ), the filter function of a MRA, is given. We are ˜ µ (ξ) ∈ L∞ (T ) (µ ∈ Ed \{0}) such needed to construct m ˜ 0 (ξ), mµ (ξ), m that (2) holds. ˜ 0 (ξ), the filter functions of a pair of biorthogQuestion 2 Assume m0 (ξ) and m onal MRAs, are given. We need to construct mµ (ξ), m ˜ µ (ξ) ∈ L∞ (T ) (µ ∈ Ed \{0}) such that (2) holds. It is essentially the problem of matrix extension which can be solved by constructing a or a pair of particular solution(s), or constructing all the possible solutions. Up till now, many results have been developed on the construction of wavelet bases from MRAs mainly by constructing a or a pair of particular solution(s) of (2)(see [1,2,6,3]). In this paper, we present all analytic solutions of (2) based a special solution. We also design an algorithm to get a special solution for the matrix equation (2) and illustrate some examples to verify our results. ⋆

This work was supported by the Foundation for University Key Teacher by the Ministry of Education of China, NSFC(19871095) and GPNSFC(990227).

Y. Y. Tang et al. (Eds.): WAA 2001, LNCS 2251, pp. 326–330, 2001. c Springer-Verlag Berlin Heidelberg 2001 

General Design of Wavelet High-Pass Filters

2 2.1

327

Main Results The general solutions of matrix equation (2)

We first restate the Question 1 and 2 by polyphase factorization of filters. We ˜ µ,τ (ξ) the polyphase components of mµ (ξ), m ˜ µ (ξ) if call respectively mµ,τ (ξ), m

mµ,τ (2ξ)e−iτ ξ and m ˜ µ (ξ) = m ˜ µ,τ (2ξ)e−iτ ξ . mµ (ξ) = τ ∈Ed

τ ∈Ed

Let (ν0 , · · · , νs ) be a permutation of Ed with s = 2d − 1 and ν0 = 0 ∈ Ed . ˜ For simplicity, we denote m(ξ) := (m ˜ µ,ν (ξ))µ,ν∈Ed , m(ξ) := (mµ,ν (ξ))µ,ν∈Ed ,. Then the filters mµ , m ˜ µ and their polyphase components have the following relations ˜ (ξ) = m(2ξ)E(ξ) ˜ M (ξ) = m(2ξ)E(ξ), M (3)  −iν ·(ξ+πν ) s k . It is easy with Vandemonde matrix E(ξ) defined by E(ξ) = e j j,k=0 to conclude that (2) leads to ∗ ˜ 2d m(ξ)m (ξ) = I,

a.e. ξ ∈ T d .

(4)

˜ (ξ), M (ξ) satisfying (2) is equivalent It is an well-known that constructing M d ˜ to constructing 2πZ -periodic matrices m(ξ), m(ξ) satisfying (4). An equivalent discription of Question 1,2 is stated as follows: Question 1 Assume the first row of a matrix m(ξ) is given and satisfies  2 d ν∈Ed |m0ν (ξ)| = 0, a.e. ξ ∈ T . We are needed to construct the other d ˜ rows and the 2πZ − periodic matix m(ξ) such that (4) holds. ˜ Question 2 Assume the first rows of matrixes m(ξ) and m(ξ) are given and satisfy:
2d m ˜ 0,ν (ξ)m ¯ 0,ν (ξ) = 1, a.e. ξ ∈ T d . ν∈Ed

We are needed to construct the other rows such that (4) holds.

Let e0 be the column vector whose first element equals to 1 and others being 0s. The following notations will be used in the following theorem.

∆d (ξ) := |m0 (ξ/2 + πν)|2 = 2d |m0,ν (ξ)|2 ; (5) ν∈E d

˜d (ξ) := ∆




ν∈E d

ν∈E d

2

|m ˜ 0 (ξ/2 + πν)| = 2

2d/2 U (ξ)e0 =  mt (ξ)e0 ∆d (ξ)

2d/2 ˜ ˜ t (ξ)e0 U(ξ)e m 0 =  ˜d (ξ) ∆

d




ν∈E d

|m ˜ 0,ν (ξ)|2 .

(6)

a.e. ξ ∈ T d .

(7)

a.e.ξ ∈ T d

(8)

328

Lihua Yang et al.

Theorem 1. Suppose that 2πZ d -periodic functions {m0,ν (ξ)}ν∈E d satisfy  2 d and U (ξ) is a 2πZ d -periodic unitary maν∈Ed |m0,ν (ξ)| = 0 a.e. ξ ∈ T d ˜ trix satisfying (7). Then the 2πZ -periodic solutions m(ξ) and m(ξ) of (4) can be expressed as:     m(ξ) = m0,0 (ξ) · t· · m0,νt s (ξ) , a.e. ξ ∈ T d;  (c(ξ), A (ξ))U (ξ)  t (ξ)A−1 (ξ) −d/2 1 −c    ˜ = √2 U t (ξ), a.e. ξ ∈ T d.  m(ξ) ∆d (ξ) 0(2d −1)×1 2−d/2 ∆d (ξ) A−1 (ξ) (9) d -periodic functions { m ˜ (ξ)} d satisfying If there exist 2πZ 0,ν ν∈E  ˜ 0,ν (ξ)m ¯ 0,ν (ξ) = 1 a.e. ξ ∈ T d , then the solution of (4) with the 2d ν∈Ed m ˜ first rows of m(ξ) and m(ξ) being (m0,ν (ξ))ν∈Ed and (m ˜ 0,ν (ξ))ν∈Ed respectively can be expressed as follows:   m0,0 (ξ) · · · m0,νs (ξ)   , a.e. ξ ∈ T d ;   m(ξ) = (c(ξ), At (ξ)) U t (ξ) (10)   m ˜ 0,0 (ξ) · · · m ˜ 0,νs (ξ)  d  m(ξ) ˜ , a.e. ξ ∈ T =  (0(2d −1)×1 , 2−d A−1 (ξ))U t (ξ) where A(ξ) is a 2πZ d −periodic nonsingular matrix of order 2d − 1 and   ¯˜ 0,0 (ξ) m    .. c(ξ) = 2d/2 ∆d (ξ) [0, At (ξ)]U t (ξ)  . . ¯˜ 0,νs (ξ) m

˜ (ξ) is a 2πZ d -periodic unitary matrix satisfying (8), then Furthermore, if U m(ξ) can be rewritten as follows:  m0,0 (ξ) · · · m0,νs (ξ) (11) m(ξ) = ˜ t (ξ) . (0, At (ξ)L(ξ)) U where    ¯˜ (ξ) ¯˜ (ξ)e et U t (ξ) U L(ξ) = (0, I2d −1 ) U t (ξ) I2d − ∆d (ξ)∆˜d (ξ)U 0 0

0



I2d −1 (12) is a nonsingular matrix of order 2d −1. Particularly, if m ˜ 0,ν (ξ) = m0,ν (ξ) a.e. ξ ∈ Td (∀ν ∈ Ed ), we have L(ξ) = I2d −1 . 2.2

The Construction of Unitary Matrix U (ξ)

This section focuses on the construction of the 2πZ d - periodic unitary matrix U (ξ) which satisfies (7) for a given nonzero vector m0 (ξ) := (m0,0 (ξ), · · · , m0,νs (ξ))t . For simplicity, we denote 
 |m0ν (ξ)|2 = 2−d/2 ∆d (ξ). (13) m0 (ξ) := ν∈Ed

General Design of Wavelet High-Pass Filters

329

Theorem 2. Let m0 (ξ) = 0 and m00 (ξ) = |m00 (ξ)|e−iθ(ξ) a.e. ξ ∈ T . Then, U (ξ), which is defined by  m (ξ) 00  m0ν1 (ξ) 1  U (ξ) = .. m0 (ξ)  .

m0ν1 (ξ)

··· M1 (ξ)

m0νs (ξ)

m0νs (ξ)   , 

(14)

with −iθ(ξ)

M1 (ξ) =

e m0 (ξ) + |m00 (ξ)| −m0 (ξ)e

−iθ(ξ)



m0ν1 (ξ)m0ν1 (ξ)  ..  . m0νs (ξ)m0ν1 (ξ)

I2d −1 ,

··· ···

 m0ν1 (ξ)m0νs (ξ)  ..  . m0νs (ξ)m0νs (ξ)

is a unitary matrix satisfying (7). Furthermore, If there exists constant c > 0 such that m0 (ξ) > c (∀ξ ∈ T d ), then U (ξ) is smooth as m0 (ξ) and θ(ξ). 2.3

Examples

For simplicity, we denote x := e−iξ1 , y := e−iξ2 . It can be verified that the following polynomials    1 1 1 −1 1  y 1 1  2 3  1 −1 1   m0 (ξ1 , ξ2 ) = (1, x, x , x )  1 1 −1 1   y 2  8 −1 1 1 1 y3 satisfies |m0 (ξ1 , ξ2 )|2 + |m0 (ξ1 + π, ξ2 )|2 + |m0 (ξ1 , ξ2 + π)|2 + |m0 (ξ1 + π, ξ2 + π)|2 = 1. The unitary matrix in Theorem 2.2 is U (ξ1 , ξ2 ) :=  1 + x + y − xy 1 1  − x + y + xy 4  1 + x − y + xy −1 + x + y + xy

1 − x + y + xy 1 + x + y − xy 1 − x − y − xy −1 − x + y − xy

1 + x − y + xy 1 − x − y − xy 1 + x + y − xy −1 + x − y − xy

 −1 + x + y + xy −1 − x + y − xy   −1 + x − y − xy  1 + x + y − xy

330

Lihua Yang et al.

which leads to the following three high-pass filters     1 1 1 −1 1         1 −1 1 1 y     m1 (ξ1 , ξ2 ) = 18 (1, x, x2 , x3 )     −1 −1 1 −1   y 2     3     1 −1 −1 −1   y    1 1 −1 1 1        1 −1 −1 −1 y   m2 (ξ1 , ξ2 ) = 18 (1, x, x2 , x3 )   1 1 1 −1   y 2     3     −1 1 −1 −1   y    −1 −1 1 −1 1         −1 1 1 1 y  1 2 3       m3 (ξ1 , ξ2 ) = 8 (1, x, x , x )  1 1 1 −1   y 2     −1 1 −1 −1 y3

Since there are different choices for the unitary matrix A(x, y), we can also construct other high-pass filter for the given low-pass filter m0 (ξ1 , ξ2 ). For example, for the unitary matrix  0 √y2 0, A(x, y) =  0 xy x √ √y 0 − 2 2  √x

2

we can get another three high-pass filters corresponding to the low-pass filter m0 (ξ1 , ξ2 ). We omit the details here.

References 1. A. Cohen,I.Daubechies,and J. C.Feauveau, Biorthogonal bases of compactly supported wavelets, Comm. Pure Appl. Math., 45:485-560,1992. 326 2. K.Grochning, Analyse multi-echelle et bases d’ondelettes, Acad. Sci. Paris., Serie 1,305:13-17,1987. 326 3. R. Q. Jia and C. A. Micchelli, Using the refinement equation for the construction of pre-wavelets V:extensibility of trigonometric polynomials, Computing, Vol. 48, 61-72,1992. 326 4. D. X. Zhou, Construction of real-valued wavelets by symmetry, preprint. 5. S. Mallat, Review of Multifrequency Channel Decomposition of Images and Wavelet Models, Technical report 412, Robotics Report 178, New York Univ., (1988). 6. C. A.Micchelli and Yuesheng Xu, Reconstruction and decomposition algorithms for biorthogonal multiwavelets, Multidimensional Systems and Signal Processing 8, 31-69,1997. 326

Realization of Perfect Reconstruction Non-uniform Filter Banks via a Tree Structure Wing-kuen Ling and Peter Kwong-Shun Tam

Department of Electronic and Information Engineering The Hong Kong Polytechnic University Hung Hom, Kowloon, Hong Kong Hong Kong Special Administrative Region, China Tel: (852) 2766-6238, Fax: (852) 2362-8439 Email: [email protected]

Abstract. It is well known that a tree structure filter bank can be realized via a non-uniform filter bank, and perfect reconstruction is achieved if and only if each branch of the tree structure can provide perfect reconstruction. In this paper, the converse of this problem is studied. We show that a perfect reconstruction non-uniform filter bank with decimation ratio {2,4,4} can be realized via a tree structure and each branch of the tree structure achieves perfect reconstruction.

1

Introduction It is well known that the tree structure filter bank shown in figure 1b can be realized via a non-uniform filter bank shown in

figure 1a, and perfect reconstruction can be achieved if and only if each branch of the tree structure can provide perfect reconstruction [1-4]. However, is the converse true? That is, given any perfect reconstruction non-uniform filter shown in figure 1a, can it be realized via a tree structure shown in figure 1b? In general, a perfect reconstruction non-uniform filter bank cannot be realized by a tree structure [11]. This paper works on this problem. There are some advantages of realizing a non-uniform filter bank via a tree structure, such as reducing the filter length in the filters [5], and improving the computation complexity and implementation speed [5]. In section II, we show how a perfect reconstruction non-uniform filter bank can be converted to a tree structure filter bank. Some illustrative examples are demonstrated in section III. Finally, a conclusion is given in section IV.

2

Realization of Non-uniform Filter Bank Via a Tree Structure

Theorem 1 A non-uniform filter bank with decimation ratio { 2,4,4} achieves perfect reconstruction if and only if it can be realized via a tree structure and each branch of the tree structure achieves perfect reconstruction. Proof: Since the if part was well known [1-4], we only prove the only if part. A non-uniform filter bank shown in figure 1a achieves perfect reconstruction if and only if ∃c∈C and ∃m∈Z such that:

Y. Y. Tang et al. (Eds.): WAA 2001, LNCS 2251, pp. 331-335, 2001. c Springer-Verlag Berlin Heidelberg 2001

332

Wing-kuen Ling and Peter Kwong-Shun Tam  1 ⋅ H 0 (z )   2  0   1 2   ⋅ H 0 (z ⋅W )  2   0 

1 ⋅ H (z ) 4 1 1 ⋅ H (z ⋅W ) 4 1 1 ⋅ H1 (z ⋅ W 2 ) 4 1 ⋅ H (z ⋅ W 3 ) 4 1

1  ⋅ H (z )   4 2 c⋅ zm   1 ⋅ H 2 (z ⋅ W )    G0 (z )  0  .  4  ⋅  G1 ( z)  =    1 ⋅ H (z ⋅ W 2 )   G (z )  0    2    4 2 0     1 ⋅ H 2 (z ⋅ W 3 )  4 

(1)

This directly implies that:   H (z ⋅ W ) H 2 (z ⋅ W )    =0. det  1 3 3    H 1 (z ⋅ W ) H 2 (z ⋅ W ) 

(2)

  H1 (z ) H 2 ( z)     H (z ⋅ W ) H 2 (z ⋅ W )    =0,  = 0 ⇒ det  det  1 2 2  3 3    H 1 (z ⋅ W ) H 2 (z ⋅ W )    H 1 (z ⋅ W ) H 2 (z ⋅ W ) 

(3)

Since

hence  1 ⋅ H 0 (z )   2  0   1 2 ⋅ H  0 (z ⋅W )  2  0  

1 ⋅ H (z ) 4 1 1 ⋅ H 1( z ⋅ W ) 4 1 ⋅ H1 (z ⋅ W 2 ) 4 1 ⋅ H (z ⋅ W 3 ) 4 1

1  ⋅ H 2 (z )   4 c⋅ zm   1 ⋅ H 2 (z ⋅ W )    G0 (z )  0  4   ⋅  G1 ( z)  =    1 ⋅ H 2 (z ⋅ W 2 )   G (z )  0    2    4 0     1 ⋅ H 2 (z ⋅ W 3 )   4

  H1 (z ) H 2 ( z)   H 2 ( z)   and det  H 0 (z )  ≠ 0. ⇒ det    ≠ 0   H 0 (z ⋅ W 2 ) H 2 (z ⋅ W 2 )     H1 (z ⋅ W ) H 2 (z ⋅ W )  

(4)

The converse is also true. That is if:   H1 (z )   H (z ⋅ W ) H 2 (z ⋅ W )   H 2 ( z)     H 0 (z ) H 2 (z )   , ,   ≠ 0 and det  1 det    ≠ 0 det  ( 2 2   H (z ⋅ W 3 ) H (z ⋅ W 3 )  = 0 2    H 1 ( z ⋅W ) H 2 (z ⋅ W )   1   H 0 z ⋅ W ) H 2 (z ⋅ W ) 

then there exist G0 (z), G1(z) and G2 (z) such that:  1 ⋅ H 0 (z )   2  0   1 2   ⋅ H 0 (z ⋅W )  2   0 

1 ⋅ H (z ) 4 1 1 ⋅ H 1( z ⋅ W ) 4 1 ⋅ H (z ⋅ W 2 ) 4 1 1 ⋅ H (z ⋅ W 3 ) 4 1

1  ⋅ H (z )   4 2 c⋅ zm   1 ⋅ H 2 (z ⋅ W )    G0 (z )  0  .  4  ⋅  G1 ( z)  =    1 ⋅ H (z ⋅ W 2 )   G (z )  0    2    4 2 0     1 ⋅ H (z ⋅ W 3 )  4 2 

(5)

Let H (z ) = z −l ⋅ E (z 4 ) , for i=0,1,2, then from equation (2), there exist R(z), R’ (z) and R”(z) such that: ∑ i i ,l 3

l =0

E1,1 (z4)=R(z4)◊E1,0(z4) and E2,1 (z4)=R(z4)◊E2,0(z4) and

(6)

E1,3 (z4)=R’ (z4)◊E1,2 (z4) and E2,3(z4 )=R’ (z4)◊E2,2(z4 ) and

(7)

{R(z4 )=R’ (z4) or {E1,0(z4 )=R”(z4)◊E1,2(z4 ) and E2,0(z4)=R”(z4 )◊E2,2 (z4)}}.

(8)

But E1,0(z4 )=R”(z4)◊E1,2(z4) and E2,0 (z4)=R”(z4 )◊E2,2 (z4) contradict equation (4). Hence, we have R(z4)=R’ (z4). R(z4 )=R’ (z4), which implies:

Realization of Perfect Reconstruction Non-uniform Filter Banks via a Tree Structure H 2 (z ) E 2 ,0 (z 4 ) + z −2 ⋅ E 2, 2 (z 4 ) . = H1 (z ) E1, 0 (z 4 ) + z − 2 ⋅ E1, 2 (z 4 )

333

(9)

Hence, there exist F’ 1(z), F0(z), and F1 (z) such that H1 (z)=F’ 1(z)◊F0(z2 ) and H2 (z)=F’ 1(z)◊F1(z2 ), respectively. And the nonuniform filter bank shown in figure 1a can be realized via a tree structure shown in figure 1b. From equation (4), we have: E1,2 (z4)◊E2,0(z4 )-E1,0 (z4)◊E2,2(z4)π0 and

(10)

{E0,1(z4 )◊E2,0 (z4)-E0,0(z4 )◊E2,1 (z4)π0 or

(11)

E0,3 (z4)◊E2,2(z4 )-E0,2 (z4)◊E2,3(z4)π0 or E0,1 (z4)◊E2,2(z4 )+E0,3(z4)◊E2,0(z4 )-E0,0 (z4)◊E2,3(z4 )-E0,2 (z4 )◊E2,1(z4)π0}.

(12) (13)

Let F1 (z2) be the numerator of H2 (z)/H1(z), and F0 (z2) be the denominator of H2 (z)/H1(z), respectively. We have: F1 (z )     F (z ) , and 2 2 2 2 −1 det  0   = 2 ⋅ z ⋅ (E1,2 (z )⋅ E 2, 0 (z ) − E1,0 (z )⋅ E 2, 2 (z )) ≠ 0    F0 (− z) F1 (− z ) 

(14)

F1′(z )   2 ⋅ z −1 ⋅ (E 0 ,1 (z 4 ) ⋅ E 2 ,0 (z 4 ) − E 0 ,0 (z 4 )⋅ E 2 ,1 (z 4 )) + 2 ⋅ z −5 ⋅ (E 0 ,3 (z 4 )⋅ E 2 ,2 (z 4 ) − E 0 ,2 (z 4 )⋅ E 2,3 (z 4 ))   H (z ) = det  0 −2 4 4  H 0 (− z ) F1′(− z )  E 2 ,0 (z ) + z ⋅ E 2 ,2 (z )   2 ⋅ z −3 ⋅ (E 0 ,1 (z 4 )⋅ E 2 ,2 (z 4 ) + E 0 ,3 (z 4 )⋅ E 2 ,0 (z 4 ) − E 0 ,0 (z 4 )⋅ E 2,3 (z 4 ) − E 0 ,2 (z 4 )⋅ E 2,1 (z 4 )) + −2 4 4 E 2 ,0 (z ) + z ⋅ E 2 ,2 (z ) ≠ 0.

Hence, each branch of the tree structure achieves perfect reconstruction.

3

Illustrative Examples

3.1

Non-tree Structure Filter Bank

(15)




Consider an example of H0(z)=1+z -1, H1(z)=1-z-1, and H2(z)=z -2, respectively. Since H1 (z)/H2(z)=z 2◊(1-z-1), there does not exist F’ 1(z), F0(z), and F1(z) such that H1(z)=F’ 1(z)◊F0(z2) and H2(z)=F’ 1(z)◊F1(z2 ), respectively. Hence, this non-uniform filter bank cannot be realized via a tree structure. By theorem 1, this non-uniform filter bank does not achieve perfect reconstruction. It is worth to note that by converting the non-uniform filter bank to a uniform filter bank shown in figure 2 [6-10], perfect reconstruction can be achieved. However, G’ -1(z)πz-2◊G’ 0(z), this implies that the corresponding synthesis filter G0(z) shown in figure 1a is time varying.

3.2

Tree Structure Filter Bank

Consider another example with H0(z)=2◊(1+z -1 +z-2 +z-3), H1(z)=4◊(2+6◊z-1 +4◊z-2+12◊z-3), and H2 (z)=4◊(5+15◊z-1 +7◊z2

+21◊z-3), respectively. Since H1(z)/H2(z)=(2+4◊z-2)/(5+7◊z-2 ), there exists F’ 1(z), F0 (z), and F1(z) such that H1 (z)=F’ 1(z)◊F0 (z2) and

H2(z)=F’ 1(z)◊F1(z2), respectively. Hence, this non-uniform filter bank can be realized via a tree structure. It can be checked easily that each branch in the tree structure achieves perfect reconstruction. Hence, this non-uniform filter bank achieves perfect reconstruction.

334 4

Wing-kuen Ling and Peter Kwong-Shun Tam Conclusion In this paper, we show that a non-uniform filter bank with decimation ratio {2,4,4} achieves perfect reconstruction if and

only if it can be realized via a tree structure and each branch of the tree structure achieves perfect reconstruction. The advantage of realizing a non-uniform filter bank via a tree structure is to reduce the computation complexity and provide a fast implementation for a non-uniform filter bank [5].

Acknowledgement The work described in this paper was substantially supported by a grant from the Hong Kong Polytechnic University with account number G-V968.

References 1.

Vaidyanathan P. P.: Lossless Systems in Wavelet Transforms. IEEE International Symposium on Circuits and Systems,

2.

Soman K. and Vaidyanathan P. P.: Paraunitary Filter Banks and Wavelet Packets. IEEE International Conference on

3.

Sodagar I., Nayebi K. and Barnwell T. P.: A Class of Time-Varying Wavelet Transforms. IEEE International Conference on

4.

Soman A. K. and Vaidyanathan P. P.: On Orthonormal Wavelets and Paraunitary Filter Banks. IEEE Transactions on Signal

ISCAS, Vol. 1. (1991) 116-119.

Acoustics, Speech, and Signal Processing, ICASSP, Vol. 4. (1992) 397-400.

Acoustics, Speech, and Signal Processing, ICASSP, Vol. 3. (1993) 201-204.

Processing, Vol. 41, No. 3. (1993) 1170-1183. 5.

Vaidyanathan P. P.: Multirate Systems and Filter Banks. Englewood Cliffs, NJ: Prentice Hall, 1993.

6.

Hoang P. Q. and Vaidyanathan P. P.: Non-Uniform Multirate Filter Banks: Theory and Design. IEEE International Symposium on Circuits and Systems, ISCAS, Vol. 1. (1989) 371-374.

7.

Li J., Nguyen T. Q. and Tantaratana S.: A Simple Design Method for Nonuniform Multirate Filter Banks. Conference Record of the Twenty-Eight Asilomar Conference on Signals, Systems and Computers, Vol. 2. (1995) 1015-1019.

8.

Makur A.: BOT’ s Based on Nonuniform Filter Banks. IEEE Transactions on Signal Processing, Vol. 44, No. 8. (1996) 1971-1981.

9.

Li J., Nguyen T. Q. and Tantaratana S.: A Simple Design Method for Near-Perfect-Reconstruction Nonuniform Filter Banks. IEEE Transactions on Signal Processing, Vol. 45, No. 8. (1997) 2105-2109.

10. Omiya N., Nagai T., Ikehara M. and Takahashi S. I.: Organization of Optimal Nonuniform Lapped Biorthogonal Transforms Based on Coding Efficiency. IEEE International Conference on Image Processing, ICIP, Vol. 1. (1999) 624-627. 11. Akkarakaran S. and Vaidyanathan P. P.: New Results and Open Problems on Nonuniform Filter-Banks. IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP, Vol. 3. (1999) 1501-1504.

Realization of Perfect Reconstruction Non-uniform Filter Banks via a Tree Structure

x[n]

H 0 ( z)

↓2

↑2

G0 (z)

H 1 ( z)

↓4

↑4

G1 ( z)

H 2 (z)

↓4

↑4

G2 (z )

y[n]

(a) H 0 ( z)

F0 ( z)

x[n]

F1′(z )

↑2

G0 (z)

↓2

↑4

G1 ( z)

↓2

↑4

G2 (z )

↓2

y[n]

↓2

F1 (z ) (b)

Fig. 1. (a) Non-uniform filter bank (b) Tree structure filter bank

z 2 ⋅ H0 (z )

↓4

↑4

G−′1 (z )

H 0 ( z)

↓4

↑4

G0′ (z) y[n]

x[n]

H 1 ( z)

↓4

↑4

G1 ( z)

H 2 (z)

↓4

↑4

G2 (z )

Fig. 2. Realization of non-uniform filter bank via a uniform filter bank

335

Set of Decimators for Tree Structure Filter Banks Wing-kuen Ling and Peter Kwong-Shun Tam

Department of Electronic and Information Engineering The Hong Kong Polytechnic University Hung Hom, Kowloon, Hong Kong Hong Kong Special Administrative Region, China Tel: (852) 2766-6238, Fax: (852) 2362-8439 Email: [email protected]

Abstract. In this paper, we propose a novel method to test if a set of decimators can be generated by a tree structure filter bank. The decimation ratio is first sorted in an ascending order. Then we group the largest decimators with the same decimation ratio together and form a new set of decimators. A set of decimators can be generated by a tree structure filter bank if and only if by repeating the above procedure, all the decimators can be grouped together. Some examples are illustrated to show that the proposed method is simple and easy to implement.

1

Introduction Non-uniform filter banks have taken an important role in this decade and they are widely applied in the area of digital

image compression [3, 6, 7, 9, 13]. By realizing a non-uniform filter bank in a tree structure [1, 2, 4, 5, 10-12], the filter lengths in the filters can be reduced, improving the computation complexity and the implementation speed [15]. However, not all the non-uniform filter banks can be realized via a tree structure [5, 8, 10-12]. This paper is to propose a method to test if a set of decimators can be generated by a tree structure filter bank. In order to tackle this problem, a method to compute the number of combinations of sub-trees is proposed [8]. However, if the number of decimators is large, it is very complicated to compute the number of combinations of sub-trees. Also, this method is order dependent, which will give a wrong result by changing the order of the decimators in the set [8].

2

Proposed Algorithm

Theorem 1

Let the ordered set of decimators {n 0,º,n 0,n 1,º,n 1,º,n N-1,º,n N-1} be D , where n i>n j for i>j, and the multiplicity of n i in D be p i. By grouping the largest decimators with the same decimation ratio together and forming a new set of decimators, a set of decimators can be generated by a tree structure filter bank if and only if by repeating the above procedures, all the decimators can be grouped together. Proof:

Y. Y. Tang et al. (Eds.): WAA 2001, LNCS 2251, pp. 336-340, 2001. c Springer-Verlag Berlin Heidelberg 2001

Set of Decimators for Tree Structure Filter Banks

337

Consider the only if part first. If D can be generated by a tree structure filter bank, then there should be no branch coming out from the decimators n N-1. Otherwise, n N-1 is not the greatest number in D . Hence, by grouping some or all of the decimators with decimation ratio n N-1 together, the branch corresponding to the grouped decimators is removed. Suppose k N-1 decimators are grouped together, where 2£k N-1£p N-1, then the effective decimation ratio corresponding to the grouped decimators is n N-1/k N-1. And the new set of the decimators become {n 0,º,n 0,n 1,º,n 1,º,n N-2,º,n N-2,n N-1/k N-1,n N-1,º,n N-1}. If D can be generated by a tree structure filter bank, then ∃n i∈D such that n i=nN-1/k N-1. Hence, by repeating the above procedure, all the branches are removed and eventually there is only one decimator left in D , which is 1. And this proves the only if part. For the if part, since ∃n i ∈D such that n i=nN-1/k N-1, we can construct a sub-tree corresponding to those k N-1 channels. By repeating the above procedure, the non-uniform filter bank can be realized in the form of a tree structure. Hence, this proves the


if part and the theorem.

Some problems are: When should we group all of the decimators with decimation ratio n N-1 together, that is k N-1=pN-1? When should we group part of them together, that is k N-1n j for i>j, and the multiplicity of n i in D be p i.

Let the corresponding analysis filters and synthesis filters be

{G (z), L, G 0 ,0

0 , p0 −1

(z ),L, GN −1,0 (z ),L , GN −1 ,p

{

N −1

, respectively. −1 ( z )}

If there exists a set of filters H N′ −1 ( z), H N′ −1 ,k (z ),L , H N′ −1,k 0

K N −1 − 1

{H (z ),L , H 0 ,0

0 , p0 −1

( z), L, H N −1,0 ( z), L, H N −1, p

N −1 − 1

(z )} and

(z )}, where k iŒ[0 p N-1-1] for i=0,1,º,K N-1-1 and KN-1Œ[2 pN-1],

Y. Y. Tang et al. (Eds.): WAA 2001, LNCS 2251, pp. 341-346, 2001. c Springer-Verlag Berlin Heidelberg 2001

342

Wing-kuen Ling and Peter Kwong-Shun Tam

such that:


n N-1/KN-1ŒZ,

(1)

 n N −1  and, H N′ −1 (z ) ⋅ H N′ −1,ki  z KN −1  = H N −1,k i (z )    




 H N′ −1,k 0 (z )    H N′ −1,k 0 (z ⋅ WN −1 ) det  M    H N′ −1,k 0 z ⋅ W N−1 K N −1 −1 




(

where WN −1 = e

−

j⋅ 2 ⋅p K N −1

(2)

H N′ −1,k1 (z ) H N′ −1,k1 (z ⋅ WN −1 )

)

H N′ −1,k1

M K −1 z ⋅ WN −1 N −1

(

 H N′ −1,k K 1 −1 (z ) N−  , H N′ −1,k K 1 −1 (z ⋅ WN −1 )   N− ≠0  O M  K −1  L H N′ −1,k K 1 −1 z ⋅ W N−1 N −1   N−  L L

)

(

(3)

)

, then by a proper design of the synthesis filters, those KN-1 channels can be grouped together into one channel

with the analysis filter H N′ −1 (z ) and the decimator ↓n N-1/KN-1. Now, we have a new set of decimators and analysis/synthesis filters. Let the new set of decimators {n’ 0,º,n’ 0,º,n’ N’1

,º,n’ N’ -1} be D ’ and the multiplicity of n’ i in D ’ be p’ i. Let the corresponding analysis/synthesis filters be

{H

new

0 ,0

(z ),L , H new0 , p′ −1 (z ),L , H new N ′−1,0 (z ),L, H new N′−1, p ′

N ′− 1 − 1

0

( z )}

{G

and

new 0 ,0

( z), L, Gnew0 , p′ −1 ( z), L, Gnew N′−1 ,0 (z ),L , G newN′−1, p′

N ′ −1 − 1

0

(z )},

respectively. By repeating the above grouping procedure, if all the channels can be grouped together, and eventually only one channel is left, then the non-uniform filter bank can achieve perfect reconstruction via a tree structure. Theorem 1

A non-uniform filter bank can achieve perfect reconstruction via a tree structure if and only if all the channels can be grouped together by the above grouping procedure. Proof: The if part is proved in the above. Now, let's consider the only if part. Since the non-uniform filter bank can be realized by a tree structure, ∃n i∈D

{H ′

such that n i=nN-1/KN-1, and a set of filters

N −1

(z ), H N′ −1,k (z ),L , H N′ −1,k 0

K N −1 −1

(z )} such that

 n N −1  H N′ −1 (z ) ⋅ H N′ −1,ki  z KN −1  = H N −1,k i (z ) . But do those filters satisfy equation (3)? Or in other words, if some of the analysis filters in a    

sub-tree are linearly dependent, does there exist a set of synthesis filters such that the whole system still achieves perfect reconstruction?

Assume

 H N′ −1,k 0 (z )    H N′ −1,k 0 (z ⋅ WN −1 ) det  M    H N′ −1,k 0 z ⋅ W N−1 K N −1 −1 

(

H N′ −1,k1 (z)

)

H ′N −1,k1 (z ⋅ WN −1 ) M K −1 H N′ −1 ,k1 z ⋅ WN −1 N −1

(

H N′ −1,k K N −1 −1 (z )

L

H N′ −1,k K N −1 −1 (z ⋅ WN −1 ) O M K −1 L H N′ −1,k K 1 −1 z ⋅ WN −1 N −1

  and a ,   = 0 ∃G N −1,k 0 (z ),L , G N −1,k K N −1 −1 ( z)     

L

)

N−

(

)

non-zero transfer function T(z) such that:  n   H N′ −1 (z ) ⋅ H N′ −1, k  z K         n  Kn  K ′ −1 ( z ⋅ W ) ⋅ H N′ −1, k  z ⋅ H W N     M  n   n −1 K K K  H N′ −1 z ⋅ W ⋅ H N′ −1, k z ⋅W     N −1

N −1

)

0

H ′N −1 ( z ) ⋅ H ′N −1, k

L

N −1

1

N −1

N −1

N −1

N −1

0

(

 n  H ′N −1 (z ) ⋅ H ′N −1,k  z K      n  Kn K ⋅W H ′N −1 ( z ⋅ W ) ⋅ H ′N −1, k  z   M n  n −1 K K K  z ⋅W ⋅ H ′N −1, k z ⋅W   N −1

N− 1

0

N −1

N −1

N −1

N− 1

   

⋅ (K N −1 −1)

N −1

N −1

N −1

N −1

1

   

H N′ − 1

(

N −1

)

1

N −1

N −1

N −1

N −1

   

⋅ (K N −1 −1)

L

H ′N −1 ( z ⋅ W ) ⋅ H ′N −1,k

O   L H ′N −1 z ⋅ W  

(

K N −1 − 1

 z  

 nK z  

n N− 1 KN − 1

   

N −1

K N − 1 −1

N− 1

⋅W

n N −1 K N− 1

   

M K N − 1 −1

)⋅ H ′

N −1, k K

 nK z  

N −1 N −1

N −1 − 1

⋅W

n N− 1 ⋅( K N − 1 KN − 1

        −1)    

Set of Perfect Reconstruction Non-uniform Filter Banks via a Tree Structure

 G N −1, k ( z )   G ( )  N −1, k z = ⋅ M    ( z ) G N −1, k 0

1

K N −1 − 1

where W = e

Since

−

T ( z )  ,  0   M     0 

j⋅ 2⋅ p nN −1

343

(4)

.

 H N′ −1,k 0 (z )   H N′ −1,k 0 (z ⋅ WN −1 ) det   M    H N′ −1,k 0 z ⋅ W N−1 K N −1 −1 

(

  n   H N′ −1( z )⋅ H N′ −1, k  z K        n   n H N′ −1 (z ⋅ W ) ⋅ H N′ −1, k  z K ⋅ W K  det      M  n  n  H ′ z ⋅ W K −1 ⋅ H ′ ) N −1,k  z K ⋅W K   N −1(    N −1

(

H N′ −1,k K N − −1 (z ) 1 H N′ −1,k K −1 (z ⋅ WN −1 ) N −1 M K −1 L H N′ −1,k K −1 z ⋅ WN −1 N −1

L L O

)

N −1

 n  H N′ −1( z) ⋅ H N′ −1, k  z K      n  n H N′ −1 (z ⋅ W )⋅ H N′ −1, k  z K ⋅ W K   M n −1)   n  z K ⋅W K  H ′ (z ⋅ W K −1) ⋅ H ′ N −1 N −1,k    

(

N −1

N− 1

L

N −1

0

1

N −1

N− 1

N −1

N −1

0

N− 1

)

H N′ −1,k1 (z) H N′ −1,k1 (z ⋅ WN −1 ) M K −1 H N′ −1 ,k1 z ⋅ WN −1 N −1

N− 1

N −1

N −1

N− 1

   

N −1

N −1

N− 1

1

⋅( KN −1

0

N −1

N− 1

N− 1

N −1

N −1

N −1

   

L

)

H ′N −1( z) ⋅ H ′N −1, k H ′N −1( z ⋅ W ) ⋅ H ′N −1, k

O

⋅ ( K N − 1 −1)

1

 n N −1  , by letting z = z KN −1 , we have:  = 0     

  L H ′ (z ⋅ W K N −1  

N −1

⋅W K

 Kn z  

⋅W K

nN −1

N −1

K N − 1 −1

)⋅ H ′

N −1, k K N −1 −1

   

 nK z  

N− 1

M N −1 −1

 n z K  

N− 1

K N −1 − 1

N −1 N− 1

N −1

nN −1 N −1

   

⋅ ( K N −1

  .      = 0    −1)       

(5)

Let the matrix in equation (5) be H. By examining equation (5) and applying Cramer’ s rule to equation (4), we find that the determinants of the matrices by deleting the first row and any columns are zero. By the modulation principle, we find that the determinants of the matrices by deleting the last row and any columns are zero. Let the rank of the matrix by deleting the first row of H be r, and that of the matrix by keeping the first r+1 rows of H be H ′ = [h′ L h′ 0 K

column of H’ and h0,0 are the first r elements of the first row of H’ . Since

] = hh

0 ,0

N − 1 −1



S ,0

h0 ,1  , where h’ is the i th i  hS ,1 

 T (z )  g   0  , where ga is a vector containing the H ′⋅  a  =   gb   M     0 

first r synthesis filters, we have h0,0⋅ ga+h0,1⋅ gb=T(z) and hS,0⋅ ga+hS,1⋅ gb=0. This implies that (h0,1-h0,0⋅ hS,0-1⋅ hS,1)⋅ gb=T(z), and

[det([h ′ 0

L hr′−1

hr′ ]) det([h0′ L hr′−1

[

hr′+1 ]) L det( h0′ L hr′−1

]]

hK′ N −1 −1 ) ⋅ g b = T ( z) ⋅ det(hS ,0 ) = 0 , which contradicts the

assumption. Hence, if some of the analysis filters in a sub-tree are linearly dependent, there does not exist a set of synthesis filters such that the whole system achieves perfect reconstruction. This proves the only if part and the theorem.

3

Illustrative Examples

3.1

Uniform Filter Bank



Consider an M-channel uniform filter bank with analysis filters {H0(z),H1(z),º,HM-1(z)}. In this case, N=1 and n 0=p0=K0=M. By selecting H’ 0(z)=1, H’ 0,i(z)=Hi(z) , for i=0,1,º,M-1, this M-channel uniform filter bank can achieve perfect reconstruction via a tree structure if and only if:   H ′0 ,0 (z ) H ′0 ,1 (z )  H 0′,1 (z ⋅ W )  H 0′,0 ( z ⋅W ) det   M M   H ′ (z ⋅ W M −1 ) H ′ (z ⋅ W M −1 ) 0 ,1   0 ,0

H 0′,M −1 (z )   L  , L H 0′,M −1 (z ⋅ W )   ≠0  O M   L H 0′, M −1 (z ⋅ W M −1 ) 

(6)

344

Wing-kuen Ling and Peter Kwong-Shun Tam

where W = e

3.2

−

j⋅ 2⋅ p M

[14].

Perfect Reconstruction Dyadic Tree Structure Filter Bank

Consider the non-uniform filter bank shown in figure 1 [1, 2, 5]: F0 ( z) = H 1 ( z)

↓2

↑2

G0 (z)

F1 ( z) = H0 (z ) ⋅ H1 (z 2 )

↓4

↑4

G1 ( z) y[n]

x[n]

F2 ( z) = H 0 (z ) ⋅ H 0 (z 2 )⋅ H1 (z 4 )

↓8

↑8

G2 (z )

F3 (z ) = H 0 (z )⋅ H 0 (z 2 )⋅ H 0 (z 4 )

↓8

↑8

G3 ( z)

Fig. 1. Perfect reconstruction dyadic tree structure filter bank In this case, n 0=2, n 1=4, n 2=8, p 0=1, p 1=1, p 2=2 and N=3. By selecting Kj=2, H ′0 (z ) = 1 , H 1′(z ) = H 0 (z ) , H 2′ (z ) = H 0 (z )⋅ H 0 (z 2 ) , H ′j ,0 (z ) = H1 (z ) , and H ′j ,1 (z ) = H 0 (z ) , for j=0,1,2, this non-uniform filter bank can achieve perfect reconstruction via a tree structure

if and only if det  H 0 (z ) 

H1 (z )   [1, 2, 5].   ≠ 0   H 0 (− z ) H 1 (− z ) 

3.3

Perfect Reconstruction Tree Structure Filter Bank

Consider the non-uniform filter bank shown in figure 2: H 0 ( z ) = 1 + z −1

↓2

↑2

G0 (z)

H 1 ( z) = 5 − 5 ⋅ z −1 + 2 ⋅ z −2 − 2 ⋅ z −3 + z −6 − z −7 + 2 ⋅ z −8 − 2 ⋅ z −9 + z −10 − z −11

↓6

↑6

G1 ( z) y[n]

x[n]

H 2 (z ) = 2 − 2 ⋅ z −1 + z −2 − z −3 + 2 ⋅ z −6 − 2 ⋅ z −7 + 4 ⋅ z −8 − 4 ⋅ z −9 + 2 ⋅ z −10 − 2 ⋅ z −11

↓6

↑6

G 2 (z )

H 3 (z ) = z −6 − z −7 + 2 ⋅ z −8 − 2 ⋅ z −9 + z −10 − z −11

↓6

↑6

G3 ( z)

Fig. 2. Perfect reconstruction tree structure filter bank In this case, n 0=2, n 1=6, p 0=1, p 1=3 and N=2. By selecting K1=3, H 1′(z ) = 1 − z −1 , H 1′,0 (z ) = 5 + 2 ⋅ z −1 + z −3 + 2 ⋅ z −4 + z −5 , H 1′,1 (z ) = 2 + z −1 + 2 ⋅ z −3 + 4 ⋅ z −4 + 2 ⋅ z −5 , and H 1′,2 (z ) = z −3 + 2 ⋅ z −4 + z −5 [14], we can group the last three channels together into one

channel with the new analysis filter H 1′(z ) = 1 − z −1 and the decimator ↓2. Similarly, by selecting K0=2, H’ 0(z)=1, H’ 0,0(z)=1+z-1, and H’ 0,1(z)=1-z-1, this non-uniform filter bank can achieve perfect reconstruction via a tree structure.

Set of Perfect Reconstruction Non-uniform Filter Banks via a Tree Structure

3.4

345

Not Perfect Reconstruction Tree Structure Filter Bank Due to the Dependent Kernel

Consider the same non-uniform filter bank shown in figure 2 with H0(z) is changed to F(z) ◊(1-z-1), where F(z)=F(-z). The last three channels are grouped together with the same procedure as above, and we have two channels left with decimator ↓2, and the analysis filters are F(z)◊(1-z-1) and (1-z-1), respectively. Since det  H 0 (z ) 

1 − z −1    = 0 , we conclude that this non-uniform 1 − 1 + z −   H z ( )  0 

filter bank cannot achieve perfect reconstruction even through it can be realized via a tree structure.

3.5

Cannot Be Realized Via a Tree Structure Filter Bank Due to Structural Problem

Consider the same non-uniform filter bank shown in figure 2 with H1(z) changed to (1-z-1)◊F1(z) , H2(z) changed to (1-z1

)◊F2(z), H3(z) changed to (1-z-1)◊F3(z), where F1(z)/F2(z) and F2(z)/F3(z) are not rational functions of z2. In this case, the last three

channels cannot be grouped together. Hence, this non-uniform filter bank cannot be realized via a tree structure.

3.6

Incompatible Non-uniform Filter Bank

Consider an incompatible non-uniform filter bank [15] with the set of decimators {2,3,6}. Since p i=1, ∀i, there does not exist KjŒ[2 p j]. Hence, an incompatible non-uniform filter bank cannot be realized via a tree structure [15].

3.7

Compatible Non-uniform Filter Bank, But Cannot Be Realized Via a Tree Structure

Consider a non-uniform filter bank with the set of decimators {5,5,5,7,7,35,35,35,35}. In this case, n 0=5, n 1=7, n 2=35, p 0=3, p 1=2, p 2=4, and N=3. Since there does not exist K2Œ[2 p2] such that n 2/K2∈Z , this non-uniform filter bank cannot be realized via a tree structure.

4

Conclusion In this paper, we propose a novel method to test if a non-uniform filter bank can achieve perfect reconstruction via a tree

structure. The advantage of realizing a non-uniform filter bank via a tree structure is to reduce the computation complexity and provide fast implementation for non-uniform filter bank [14].

Acknowledgement The work described in this paper was substantially supported by a grant from the Hong Kong Polytechnic University with account number G-V968.

346

Wing-kuen Ling and Peter Kwong-Shun Tam

References 1.

Vaidyanathan P. P.: Lossless Systems in Wavelet Transforms. IEEE International Symposium on Circuits and Systems, ISCAS, Vol. 1. (1991) 116-119.

2.

Soman A. K. and Vaidyanathan P. P.: Paraunitary Filter Banks and Wavelet Packets. IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP, Vol. 4. (1992) 397-400.

3.

Bamberger R. H., Eddins S. L. and Nuri V.: Generalizing Symmetric Extension: Multiple Nonuniform Channels and Multidimensional Nonseparable IIR Filter Banks. IEEE International Symposium on Circuits and Systems, ISCAS, Vol. 2. (1992) 991-994.

4.

Sodagar I., Nayebi K. and Barnwell T. P.: A Class of Time-Varying Wavelet Transforms. IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP, Vol. 3. (1993) 201-204.

5.

Soman A. K. and Vaidyanathan P. P.: On Orthonormal Wavelets and Paraunitary Filter Banks. IEEE Transactions on Signal Processing, Vol. 41, No. 3. (1993) 1170-1183.

6.

Vaidyanathan P. P.: Orthonormal and Biorthonormal Filter Banks as Convolvers, and Convolutional Coding Gain. IEEE Transactions on Signal Processing, Vol. 41, No. 6. (1993) 2110-2130.

7.

Soman A. K. and Vaidyanathan P. P.: Coding Gain in Paraunitary Analysis/Synthesis Systems. IEEE Transactions on Signal Processing, Vol. 41, No. 5. (1993) 1824-1835.

8.

Kovaèeviæ J. and Vetterli M.: Perfect Reconstruction Filter Banks with Rational Sampling Factors. IEEE Transactions on Signal Processing, Vol. 41, No. 6. (1993) 2047-2066.

9.

Bamberger R. H., Eddins S. L. and Nuri V.: Generalized Symmetric Extension for Size-Limited Multirate Filter Banks. IEEE Transactions on Image Processing, Vol. 3, No. 1. (1994) 82-87.

10. Makur A.: BOT’ s Based on Nonuniform Filter Banks. IEEE Transactions on Signal Processing, Vol. 44, No. 8. (1996) 1971-1981. 11. Li J., Nguyen T. Q. and Tantaratana S.: A Simple Design Method for Near-Perfect-Reconstruction Nonuniform Filter Banks. IEEE Transactions on Signal Processing, Vol. 45, No. 8. (1997) 2105-2109. 12. Akkarakaran S. and Vaidyanathan P. P.: New Results and Open Problems on Nonuniform Filter-Banks. IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP, Vol. 3,. (1999) 1501-1504. 13. Omiya N., Nagai T., Ikehara M. and Takahashi S. I.: Organization of Optimal Nonuniform Lapped Biorthogonal Transforms Based on Coding Efficiency. IEEE International Conference on Image Processing, ICIP, Vol. 1. (1999) 624-627. 14. Vaidyanathan P. P.: Multirate Systems and Filter Banks. Englewood Cliffs, NJ: Prentice Hall, 1993. 15. Hoang P. Q. and Vaidyanathan P. P.: Non-Uniform Multirate Filter Banks: Theory and Design. IEEE International Symposium on Circuits and Systems, ISCAS, Vol. 1. (1989) 371-374.

Joint Time-Frequency Distributions for Business Cycle Analysis∗ Sharif Md. Raihan 1 , Yi Wen 2 , and Bing Zeng 1 1

Department of Electrical and Electronic Engineering The Hong Kong University of Science and Technology Clear Water Bay, Hong Kong, China 2

Department of Economics Cornell University Ithaca, NY 14853, USA

Abstract: The joint time-frequency analysis (JTFA) is a signal processing technique in which signals are represented in both the time domain and the frequency domain simultaneously. Recently, this analysis technique has become an extremely powerful tool for analyzing nonstationary time series. One basic problem in business-cycle studies is how to deal with nonstationary time series. The market economy is an evolutionary system. Economic time series therefore contain stochastic components that are necessarily time dependent. Traditional methods of business cycle analysis, such as the correlation analysis and the spectral analysis, cannot capture such historical information because they do not take the time-varying characteristics of the business cycles into consideration. In this paper, we introduce and apply a new technique to the studies of the business cycle: the wavelet-based time-frequency analysis that has recently been developed in the field of signal processing. This new method allows us to characterize and understand not only the timing of shocks that trigger the business cycle, but also situations where the frequency of the business cycle shifts in time. Applying this new method to post war US data, we are able to show that 1973 marks a new era for the evolution of the business cycle since World War II. Keywords: Wavelets, time-frequency analysis, business cycle, nonstationary time series, scalogram, and spectrum.

∗

This work has been supported by a grant, HKUST6176/98H, from the Research Grants Council of the Hong Kong Special Administrative Region, China. Y. Y. Tang et al. (Eds.): WAA 2001, LNCS 2251, pp. 347-358, 2001. c Springer-Verlag Berlin Heidelberg 2001

348

Sharif Md. Raihan et al.

I.

Introduction

The analysis of nonstationary signal cannot be accomplished by classical time domain representations such as correlation methods, or by frequency domain representations based on the Fourier transform [2]. To analyze business cycles that evolve over time, we need to develop a concept of time-frequency distribution that takes into account jointly and simultaneously the information of time and frequency. The business cycle, one of the most puzzling phenomena in capitalistic, free-market economies, has long been the central focus of macroeconomic researches. The biggest challenge to researchers in this field is to capture business cycle patterns that vary in nature across time. Economic time series contain stochastic components that are necessarily time dependent. Although time-frequency analysis has its origin almost 50 years ago [Gabor, 1946; Ville, 1948], significant advances occurred only in the last 15 years or so. Recently, time-frequency representations have become an extremely powerful tool for analyzing nonstationary signals in many fields: such as engineering, medical sciences, and astronomy, to name just a few. A number of articles have also been published to deal with applications in economics and finance [10]. So far, many alternative transforms have been developed to overcome the problems associated with classical spectral analysis, we introduce in this paper a new technique of time series analysis to business cycle studies: a joint time-frequency distribution based on the wavelet transform. This new technique enables us to capture the evolutionary aspects of the spectral distribution of the business cycle across time. In this paper, we compare the wavelet-based time-frequency analysis to a traditional approach based on the windowed Fourier transform. We show that the wavelet transform has many advantages over the traditional approach in that the wavelet transform has a beautiful property: its window size adjusts itself optimally to longer basis functions at low frequencies and to shorter basis functions at high frequencies. Consequently, it has sharp frequency resolution for low frequency movements and sharp time resolution for high frequency movements. Thus, the new method is capable of capturing simultaneously the time-varying nature of low frequency cycles and the frequency distribution of sudden and abrupt shocks in the original time series. The rest of the sections are organized as follows. Section II describes the windowed Fourier transform and spectrogram. Section III describes the wavelet transform and scalogram. Section IV explains the implementation of the wavelet transform when applying to actual data. Section V uses artificial signals to demonstrate the advantages of wavelet transform over the windowed Fourier transform. Section VI applies the wavelet-based time-frequency analysis to economic data. Finally, we conclude the paper in section VII. II.

The Windowed Fourier Transform and Spectrogram

Fourier transform (FT), most widely used classical representation, is a mathematical technique for transforming a signal from the time domain to the frequency domain.

Joint Time-Frequency Distributions for Business Cycle Analysis

349

However, in the transformation process, the time information of the signal is completely lost. When we look at the FT of a signal, we observe no information about when a particular event took place. For signals in which the time information is not important but the frequency contents are of primary interest, this limitation is of little consequence. Thus, Fourier analysis is useful for analyzing periodic and stationary signals whose moments do not change much over time. However, many interesting and important signals are not stationary and need to be analyzed in both time and frequency domain simultaneously. For many years, the representation of a signal in a joint time-frequency space has been of interest in the signal processing area, especially when one is dealing with timevarying nonstationary signals. Performing a mapping of a one-dimensional signal of time into a two-dimensional function of time and frequency is thus needed in order to extract relevant time-frequency information. We refer to several excellent review papers on distributions for the time-frequency (TF) analysis [3, 6]. A classical linear time-frequency representation, called the windowed Fourier transform (WFT), has been extensively used for nonstationary signal analysis since its introduction by Gabor [5]. The basic idea of WFT is to find the spectrum of a signal x(t ) at a particular time τ by analyzing a small portion of the signal around this time point. Specifically, the signal is multiplied by a window function w(t ) centered at time point τ , and the spectrum of the windowed signal, x(t ) w* (t − τ ), is calculated by ∞

WFTx (τ , ω ) = ∫ x(t ) w* (t − τ )e − jωt dt , −∞

(1)

where ω is the angular frequency and * denotes the complex conjugation. Because multiplication by a relatively short window w(t − τ ) effectively suppresses the signal outside a neighborhood around the analysis time point t = τ , the WFT is a ‘local’ spectrum of the signal x(t ) around τ . Spectrogram is the most familiar representation to obtain the energy distribution of the signal. The spectrogram of a signal x(t ) is defined as the squared magnitudes of the WFT: SPx (τ , ω ) =

∫

∞

−∞

2

x(t ) w* (t − τ )e − jωt dt .

(2)

The WFT has many useful properties [9], including a well-developed theory [1]. It is one of the most efficient methods in computation. But a crucial feature inherent in the WFT method is that the length of the window can be selected arbitrarily, but is fixed exogenously once the selection is made. To enhance the time information, therefore, one must choose a short window; and to enhance the frequency resolution, one must choose a long window, which means that the time information (nonstationarities) occurring within the window interval is smeared. The length of the window is therefore the main issue involved in practice.

350

Sharif Md. Raihan et al.

III.

The Wavelet Transform and Scalogram

In recent years, an alternative representation, called the wavelet transform, has been widely adopted in the literature [4, 7, 14, 17]. One major advantage afforded by wavelet transform is that the windows vary endogenously in an optimal way. With this transform one can process data at different resolutions. In order to isolate signal discontinues, for example, one would like to have some very short basis functions. At the same time, in order to obtain detailed frequency analysis, one would like to have some very long basis functions. A way to achieve this is to have short basis functions for high-frequency movements and long ones for low-frequency movements. This is exactly what can be achieved with the wavelet transform. WT have an infinite set of possible basis functions. Thus, wavelet analysis provides immediate access to information that can be obscured by other time-frequency methods such as Fourier analysis. The wavelet transform is defined as the convolution of a signal x(t ) with a wavelet function Ψ (t ) , called mother wavelet, shifted in time by a translation parameter τ , and dilated by a scale parameter a , as shown by the following equation

WTx (τ , a) =

1

a

∫

∞

−∞

 t −τ  x(t )Ψ *  dt ,  a 

(3)

where Ψ * (.) is the complex conjugate of the basic wavelet function Ψ (t ) , the parameter a is the scaling factor that controls the length of the analyzing wavelet; and τ is the translation parameter. The squared modulus of the wavelet transform, called scalogram, is defined as 2

SCALx (τ , a ) =

 t −τ  x(t )Ψ  dt ∫ −∞  a  a

1

∞

*

(4)

The wavelet transform of a signal depends on two parameters: scale (or frequency) and time. This leads to a so-called time-scale representation that provides a tool for the analysis of nonstationary signals [7, 14]. There is a dozen of wavelet function available, such as Morlet, Mexican hat, Haar, Shannon, Daubechies wavelet function, etc. The choice of the wavelet function depends on the specific application. With respect to time and frequency localization, the Haar and Shannon wavelets take opposite extremes. Having compact support in time, the Haar wavelet has poor decay in frequency, whereas the Shannon wavelet has compact support in frequency with poor decay in time. Other wavelets typically fall in the middle of these two extremes. In fact, having exponential decay in both the time and frequency domain, the Morlet wavelet has optimal joint time-frequency concentration [16]. The wavelet that is used for analysis of economics fluctuations in this paper is Morlet wavelet, which is a modulated Gaussian function with exponential decay property. It is defined as Ψ (t ) = e

−

t2 2a2

e j 2πft ,

(5)

Joint Time-Frequency Distributions for Business Cycle Analysis

351

where f is the modulation (frequency) parameter. The scale parameter a and the frequency parameter f are related to each other by the relationship: a = f0 / f ,

(6)

where f 0 is the central wavelet frequency. IV.

Implementations

In WFT, the signal is divided into small enough segments, where these segments of the signal can be assumed to be stationary. For this purpose, a window function w is chosen. The width of this window must be equal to the segment of the signal where its stationarity is valid. This window function is first placed at the beginning of the signal and the Fourier transform is performed. Then the window is shifted to a new location and another Fourier transform is computed. This procedure continues until the end of the signal is reached. The spectrogram is computed accordingly as the squared modulus of the windowed Fourier transform. The wavelet transform is done in a similar manner to the WFT. The signal is multiplied by a wavelet function and the wavelet transform is computed according to equation (3) for different values of the scale parameter (a ) at different time location (τ ) . Suppose x(t ) is the signal to be analyzed. The mother wavelet is chosen to serve as a prototype for all wavelets in the process. All the wavelets that are used subsequently are the stretched (or compressed) and shifted versions of the mother wavelet. The computation starts with a value of the scaling factor a = a1 , and the wavelet is placed at the beginning of the signal. Since the wavelet function has only finite time duration, it serves just like a window in the WFT. The constant 1 / a1 is for normalization purpose so that the transformed signal will have the same energy at every scale. Next, with the same scale a = a1 , the wavelet function is shifted to the next sample point, and the wavelet transform is computed again. This procedure is repeated until the wavelet reaches the end of the signal. The result is a sequence of numbers corresponding to the scale a = a1 .

Next, the scale factor is changed to a = a 2 , and the whole procedure described above is repeated. When the process is completed for all desired values of a , the result is an energy distribution of the original signal along the two-dimensional time-frequency space. V.

Applications to Test Signals

To show the effects and the advantages of wavelet-based time-frequency analysis over the traditional WFT based time-frequency analysis, we present scalograms and spectrograms of two test signals. The signals are of length 512 points each. The WFT uses a Hanning window, and the scalogram is obtained with the Morlet wavelet. The horizontal axis is time and the vertical axis is frequency in both scalograms and spectrograms respectively.

352

Sharif Md. Raihan et al.

The first test signal used is composed of two parts: the 1st part is a time-varying low frequency sinusoidal cycle, and the 2nd part is a constant high frequency cycle with some sample points gap in the middle of the signal. The signal is shown in the top window in Figure 1.a, and the power spectrum is shown in the left window in Figure 1.a. The central window in Figure 1.a shows that the scalogram is able to capture not only the frequency location of the time-varying low frequency cycle, but also the exact timing of the missing signals presented in the constant high frequency cycle. There is no energy distribution in the middle of the scalogram due to the missing data points in the high frequency cycle (notice the sharp breaking edges in the middle of the scalogram). WFT, on the other hand, is unable to simultaneously capture all the information adequately. With a short window (Figure 1.b), the time information with respect to the exact timing of the missing data points is captured, but the frequency location of the low frequency cycle is not localized at all along the frequency axis. With a large window (Figure 1.c), on the other hand, the frequency locations of the cycles are well localized along the frequency axis, but the exact location and timing of the missing data points are not very well captured or localized along the time axis. The second test signal showing in Figure 2.a (top window) is composed of sine waves whose frequency shifts periodically across time in the low frequency region. Along the sample, however, there are three sharp transitory impulses. The power spectrum of the test signal is shown in the left window of Figure 2.a. It is seen there that the power spectrum is completely silent about the time-varying nature of the cycle and about the white noise impulses. Instead, it shows that there are simultaneously several major cycles contained in the low frequency region. The central window in Figure 2.a, however, shows how remarkably the scalogram captures not only the time-varying nature of the low frequency cycle, but also the exact timing of the white noise impulses. Notice that the frequency of the shifting cycle is highly localized along the frequency dimension on one hand, and the timing of the frequency shift is also highly localized along the time dimension on the other hand. As a comparison, the spectrogram based on WFT is shown in Figure 2.b and Figure 2.c. We see there that the spectrogram either gives an imprecise frequency localization of the time-varying low frequency cycle when the window size is small enough to adequately capture the timing of the high frequency impulses (Figure 2.b), or misses the impulses entirely when the window size is large enough to capture adequately the frequency location of the time-varying low frequency cycle in the original signal (Figure 2.c). This is so because both the time and the frequency resolutions of WFT are fixed once the window length is fixed. In contrast, scalogram allows good frequency resolution at low frequencies and good time resolution at high frequencies. VI.

Application to Economic Data

Since Second World War, the US economy has experienced several important institutional changes. These institutional changes have likely had important impact on

Joint Time-Frequency Distributions for Business Cycle Analysis

353

the structure of the US economy. The US economy has also experienced several unprecedented shocks that may also have brought deep structural adjustment to the economy. The oil price shock during the early 70s, for example, could have resulted in a fundamental reorganization of the input-output structure in the economy, especially with regard to the energy-intensive industries. It is then of great interest to investigate whether these changes have also brought fundamental changes to the nature of the US business cycle. In particular, it is of great interest to know whether the old business cycles observed by economists almost half century ago are still alive, and whether new business cycles have emerged during those years of social changes and economic development. Applying the wavelet-based time-frequency transform to the growth rate of real GDP (1960:1 - 1996:3), we find that the US business cycle has the following defining features: 1)

Business cycles through out the sample period are concentrated mostly in the frequency region below 10 quarters per cycle. They are triggered mostly by external shocks.

2)

Business cycles become far more active during the 70s and 80s after the oil price shocks in the early 70s. The two most active business cycles occurred around 1974 and 1983, both are triggered apparently by external impulses. The periodicity of the two cycles is about 6 years per cycle.

3)

There exist business cycles that are not triggered by any external shocks to GDP, such as the 1991 business cycle. On the other hand, strong external shocks to GDP do not necessarily trigger business cycles, such as the shocks during 19771978.

Figure 3 shows the contour of energy distribution of the US GDP growth across time and frequency. The time series (top window) reveals very little about the frequency location of the cycles, while the spectrum (left window) reveals nothing about the timing of the different cycles. The scalogram (center), however, shows that there have been three major business cycles since 1960. The first occurred in 1961, triggered by a sharp external impulse during that year. The 1961 cycle has a frequency of 0.1 cycles per quarter (or 10 quarters per cycle) and is short lived (it lasted about one year). The second major cycle took place in 1973, apparently triggered by two impulses during 1972 and 1973, and was greatly intensified by another impulse near 1975. This business cycle lasted about 3-4 years and peaked at the frequency of about 0.04 cycles per quarter (or 25 quarters per cycle). The third major cycle occurred during 1982-1984, apparently triggered by a shock in 1982. This cycle lasted about 3 years and peaked also at a frequency similar to the 1973 cycle. The 1973 cycle and the 1982 cycle dominated all other business cycles since 1960. Notice that the cycle in 1991 is very mild compared to the three major cycles mentioned above. It is apparently not triggered by any external shocks to GDP. The scalogram also reveals that a major shock around 1977-1978 did not trigger any business cycle around that time. In addition, there is a short-lived business cycle in 1966 triggered by an external impulse that is not obvious or noticeable, however, in the original time series (see top window in Figure 3).

354

Sharif Md. Raihan et al.

We think that these findings are of great importance to the business cycle theory. They not only help us identify the important historical shocks that triggered the business cycle, but also provide important information regarding the evolution of the business cycle across time. If the business cycle is unstable over time, for example, then there is the need for finding a common propagation mechanism to explain that instability. Without exception, existing real business cycle models all predict a stable business cycle with the same characteristic frequencies. But the scalogram shows otherwise: business cycles come and go; they emerge at different frequencies and at different times; they are not at all alike. VII.

Conclusions

A new technique of nonstationary time series analysis based on joint time-frequency representation was proposed. Two popular time-frequency distributions, the wavelet transform and the windowed Fourier transform were compared for this purpose. Our analyses showed that the wavelet-based time-frequency analysis is superior to the Fourier transform based time-frequency analysis. Applying the wavelet-based analysis to economic data, we found that business cycles in the US have not been stable over time. In particular, business cycles became far more active since the oil price crisis in the early 70s. References

[1]

Allen J. B., Rabiner L. R., “A unified approach to short-time Fourier analysis and synthesis,” Proceedings of the IEEE, vol. 65, no. 11, 1977, pp. 1558-64. [2] Boashash B., “Theory, implementation and application of time-frequency signal analysis using the Wigner-Ville distribution,” Journal of Electrical and Electronics Engineering, vol. 7, no. 3, 1987, pp. 166-177. [3] Cohen L., “Time-frequency distributions – a review,” Proceedings of the IEEE, vol. 77, no. 7, 1989, pp. 941-981. [4] Daubechies I., “The wavelet transform, time-frequency localization and signal analysis,” IEEE Transactions on Information Theory, vol. 36, no. 5, 1990, pp. 961-1005. [5] Gabor D., “Theory of communication,” J. Inst. Elec. Eng., vol. 93, 1946, pp. 429457. [6] Hlawatsch F., Boudreaux-Bartels G. F., “Linear and quadratic time-frequency signal representation,” IEEE Signal Processing Magazine, 1992, pp. 21-67. [7] Kaiser G., “A friendly guide to wavelets,” Birkhauser, Boston, 1994. [8] Lin Z., “An introduction to time-frequency signal analysis,” Sensor Review, vol. 17, no. 1, 1997, pp. 46-53. [9] Nawab S. N., Quatieri T. F., “Short-time Fourier transform,” In Lim, J. S. and Oppenheim, A. V. (Eds), Advanced Topics in Signal Processing, Prentice-Hall, Englewood Cliffs, NJ, 1988. [10] Ramsey J., “The contribution of wavelets to the analysis of economic and financial data,” Phil. Trans. R. Soc. Lond. A (forthcoming), 1996. [11] Ramsey J., Zhang Z., “The analysis of foreign exchange rates using waveform dictionaries,” Journal of Empirical Finance, 4, 1997, pp. 341-372. [12] Ramsey J., Usikov D., Zaslavsky G., “An analysis of US stock price behavior using wavelets,” Fractals, vol. 3, no. 2, 1995, pp. 377-389.

Joint Time-Frequency Distributions for Business Cycle Analysis

355

[13] Rioul O., Flandrin P., “Time-scale energy distributions: a general class extending wavelet transforms,” IEEE Transactions on Signal Processing, vol. 40, no. 7, 1992, pp. 1746-57. [14] Rioul O., Vetterli M., “Wavelets and signal processing,” IEEE Signal Processing Magazine, 1991, pp. 14-38. [15] Stankovic L., “An analysis of some time-frequency and time-scale distributions,” Annals of Telecommunications, vol. 49, no. 9-10, 1994, pp. 505-517. [16] Teolis A., “Computational signal processing with wavelets,” 1964. [17] Vetterli M., Harley C., “Wavelets and filter banks: theory and design,” IEEE Transactions on Signal Processing, vol. 40, no. 9, 1992, pp. 2207-2232. [18] Wen Y., Zeng B., “A simple nonlinear filter for economic time series analysis,” Economics Letters, 64, 1999, pp. 151-160.

Figure 1.a: Scalogram contour with signal (top) and spectrum (left).

356

Sharif Md. Raihan et al.

Figure 1.b: Spectrogram contour with signal (top) and spectrum (left) (window = 13).

Figure 1.c: Spectrogram contour with signal (top) and spectrum (left) (window = 27).

Joint Time-Frequency Distributions for Business Cycle Analysis

357

Figure 2.a: Scalogram contour with signal (top) and spectrum (left).

Figure 2.b: Spectrogram contour with signal (top) and spectrum (left) (window = 7).

358

Sharif Md. Raihan et al.

Figure 2.c: Spectrogram contour with signal (top) and spectrum (left) (window = 21).

Figure 3: Scalogram contour with time series (top) and spectrum (left). U.S. GDP growth rate (1960:1 - 1996:3).

The Design of Discrete Wavelet Transformation Chip Zaidi Razak and Mashkuri Yaacob Faculty of Computer Science and Information Technology, University Malaya Kuala Lumpur

Abstract. In this paper, an explanation on the need for a special discrete wavelet transformation hardware is presented. The development processes that have been carried out which includes simulation (both in MATLABTM and SYNOPSYSTM) and synthesis which also used SYNOPSYSTM tools is described together with a discussion on specific design issues in the coordination of the entities.

1 Why Special Hardware? There are a number of reasons that can be quoted that prompted the work on the hardware design of a discrete wavelet transform: (i) (ii)

(iii)

There has been too little research in wavelet transformation hardware because most research activities are centered on software development; Of late, there has been an increased requirement on real-time processing as well as increasing data size. This situation occurs because of the current technology that tries to represent the reality of what we have in our life today. For example, in the 1980’s colours were represented by 16 bits but now colours are represented by 256 bits; The cost effectiveness factor, i.e. cost can be reduced if it involves many more processes. This is due to the fact that if software is used, there are actually two cost factors that must be considered. Firstly, it is the cost of software and secondly, it is the cost of the equipment (e.g. main processor). Eventually, the cost will rise steeply if the processing procedure involves a large data size, where repetitive processing is needed, hence the increase in the software cost. On the same count, if off-the-shelf processors are used, the hardware cost will also increase. So, with this special hardware, the overall cost (including software cost) can be reduced substantially.

At the same time, successful research conducted by Michael L. Hilton, Björn D. Jawert and Ayan Sengupta [1] have managed to achieve data compression using the wavelet tranformation method. Researchers like Ian K.Levy dan Roland Wilson [2] have concentrated on research in 3D wavelet compression for videos. Currently, there are numerous research activities on wavelet application and usage in various fields.

Y. Y. Tang et al. (Eds.): WAA 2001, LNCS 2251, pp. 359-368, 2001.  Springer-Verlag Berlin Heidelberg 2001

360

Zaidi Razak and Mashkuri Yaacob

2 What is Discrete Wavelet Transformation? Discrete Wavelet Transformation (DWT) is a method being used to analyze wavelet, an important characteristic in effective processing. It is produced from the recursive wavelet formula as shown below [3]:

Φ ( sl ) ( x ) = 2

−

s 2

Φ (2− s x − l )

(1)

s and l being the respective scalar and translation multipliers for the particular wavelet. However, for DWT, the wavelet analyzer that is being used is in a discrete form as presented below: M −1

φ ( x) = ∑ ckφ (2 x − k )

(2)

k =0

where the adding ratio is determined by the positive M value & the multiplier C is the wavelet’s constant. This processing technique involves the slicing of the signal such that it will be processed into equal sizes. All of these signals will be processed in a non-dependent way. One of the few specialities of PDW processing is that it can process signals in various resolutions. To enable this feature, a scalar function can be used. The scalar can be shown as below: N −2

W ( x) =

∑ (−1)

k

ck +1Φ (2 x + k )

(3)

k = −1

where W(x) is the scalar function [3,4] for wavelet analyzer

Φ,

and

ck is the

wavelet coefficient. To enable this function to work well, the coefficient must confirm these linear and quadratic prerequisites: N −1

∑c

k

=2

(4)

k =0

N −1

∑c c

k k + 2l

= 2δ l , 0

(5)

k =0

where δ is the delta function and l is the coordination index at the wavelet. The wavelet analyzer that was shown above is processed in analog form. To process it in discrete, the formula below is used: M −1

φ ( x) = ∑ ckφ (2x − k )

(6)

k =0

In the DWT implementation, an algorithm, named the Pyramid Algorithm [5] is used. The implementation of this algorithm must comply with one rule, i.e. the size of the signal must be of a factor of 2.

The Design of Discrete Wavelet Transformation Chip

361

X[n]

t[n]

r[n]

2

2

t[n]

r[n]

2

2

t[n]

r[n]

2

2

.

.

.

Fig. 1. Pyramid Algorithm

where t[n] and r[n] are the respective highpass and lowpass functions which can be expressed below [6]: ai =

bi =

1 2

N

∑c

2i − j +1

f j i=1,….,

j =1

1 N ∑ (−1) j+1 c j+2−2i f j 2 j =1

N 2

i=1,….,

(7)

N 2

(8)

where a is the highpass function and b is the lowpass function. All of these theories and equations that have been presented will be translated into basic design forms and later on into an integrated design.

3 Design Stage In the process of transfering the above formulae into hardware design, the method used is to divide the formula into fundamental forms and these are later integrated together to produce the whole process itself. There are 3 phases of development, i.e. the planning of the algorithm, VHDL programming including the simulation process, followed by the synthesis process that

362

Zaidi Razak and Mashkuri Yaacob

involves the optimization of the results. The optimization process is to determine whether or not the design that is being produced can be synthesised into a physical chip.

4 Algorithm Stage One of the algorithms that has been developed is the algorithm for the highpass function which is illustrated below: 1. counter1 = 2 * (i – 1) 2. counter2 = mod (counter1, data size) 3. counter3 = mod (counter1 + min( data size, length(l)) – 1), data size) 4. for n from counter2 until counter3 do 4.1. calculate index to get highpass value, mod(n – counter1, data size) 4.2. calculate multiplication between highpass value with subscript index + 1 and data with subscript n + 1 4.3. store result in variable, b 5. end for 6. Result tally in 4.3

The algorithm below is the algorithm for processing the decomposition. 1. 2. 3.

4. 5.

flag = 0 call function get_h to get highpass value from lowpass value while data size >= 2 3.1 value of highpass and lowpass will be set 3.2 if flag = 0 then 3.2.1 for I := 0 to ( data size /2) do 3.2.1.1 call lowpass, result copy to array d. 3.2.1.2 call highpass, result copy to array h. 3.2.1.3 copy value in d to array temp 3.2.1.4 Set flag = 1 3.2.2 end for. 3.2.3 copy value in array h to final array result w in descending way. 3.3 else 3.3.1 repeat process 3.2.1.1 and 3.2.1.2 but inputs for lowpass and highpass are the elements in array temp 3.3.2 intializing array temp. 3.3.3 repeat process 3.2.1.3. 3.3.4 repeat process 3.2.3 to get the next result of highpass 3.4 end if. 3.5 Data size = ( Data size /2) end while. end.

The Design of Discrete Wavelet Transformation Chip

363

5 The Determination of Design Characteristics The characteristics that are being heavily considered in the determination of the design are as follows: (i) (ii) (iii)

types and the format of the data used; techniques of processing whether parallel or serial; number of inputs and outputs.

For the type of data, the IEEE 754 data format is used. For this standard, the floating data is represented as below: S 0

EEEEEEE 1 7

FFFFFFFFFFFFFFFFFFFFFFFFFFFFFF 8 31

Fig. 2. IEEE standard for 32-Bit

where

S represents the sign bit E represents the exponential value F represents the mantissa value

In determining the technique of processing, serial processing type is chosen for the main processing component. In situations involving input of data and output of result, the parallel technique is chosen. The serial processing is needed because each existing sub-process needs an output from the previous sub-process. The situation can be clarified below: A

C B

Fig. 3. Data Dependency

In the above figure, process C needs an output from A and B to execute its operation. So, C has to to be placed on a ‘wait’ state, until A and B produce its respective outputs. Because of this ‘wait’ state, this process must be carried on in serial order.

6 Involved Entities The entities involved in producing this special hardware are listed below: (i) 2 memory modules, 16 X 32 bits in size to store input and output data. (ii) one ROM, 8 X 32 bits in size to store wavelet coefficients i.e. the daubechies-4 coefficients.

364

Zaidi Razak and Mashkuri Yaacob

(iii) (iv) (v) (vi) (vii)

one memory, 16 X 32 bits in size used as a temporary storage. one memory, 16 X 32 bits in size that is divided into 2 banks for storage of current value of coefficients. a latch that can allow 32 bits of data. a buffer that can allow the temporary storage of 32 bits of data. one controller that controls the overall work that has been carried out.

A special entity is used to coordinate the data. This entity uses active address to select its output pins. a1 activation pin

a2 a3 a4

Output address (a1 - a8)

a5 a6

Tim ing

a7 a8

Fig. 4. Data Selection Entity

7 The Simulation of Entity After the algorithm and the entity determination process are implemented, a simulation process, or more accurately a test for each of the entity is carried out. This is to ensure that each entity is able to process the input accurately before the integration process takes place. One of the results that can be generated by this simulation is the simulation contrived by the latch. It is shown below:

Fig. 5. Simulation Result for latch

To ensure that each entity will produce accurate results, a detailed knowledge of the entity’s behavior is needed. In the above figure, the latch entity will store data that

The Design of Discrete Wavelet Transformation Chip

365

has been stored when the activating pin is 0 and it will produce data when the activating pin is 1. Both pins produce an effect when the timer is 1. For the integration result of all entities, to produce the DWT, the required data is as below: [0.1708,0.5724,0.0314,0.8033,0.1000,0.1011, 0.1111, 0.9801,0.9999,0.3311,0.8900,0.1231, 0.7651,0.0001,0.5555,0.9999] The data selection is important to determine the process that has been carried out that can cover all the data’s characteristics. But before that, the processing range must be chosen. Below are the rules and conditions of the selection: (i)

Lower boundary input value This value is needed to determine that there will be no error produced if the input process is in the lower boundary, i.e. near to 0. In the research, this value is 0.0001.

(ii)

Upper boundary input value This value must be included in this work so that it can produce a result, particularly to check whether the process experiences any flow. The value is 0.9999.

(iii)

Intermediate value between the upper and lower boundary and the median For this criterion, the value chosen will validate that the process that has been carried out can be executed for all values in the existing value range.

Before these data can be used for simulation, these data must be changed into a floating point format. To ensure that the results produced from the simulation using SYNOPSYSTM are accurate, a simulation using MATLABTM is used. The simulation using MATLABTM is a process of validating the decomposition and the reconstruction of all the data that has been processed. The table below shows an output list derived for the MATLABTM simulation (all values have been changed into floating point values) and the outputs of the simulation by using SYNOPSYSTM. Table 1.

Num

1. 2. 3. 4. 5. 6. 7. 8. 9.

MATLABTM Simulation Output Compared to SYNOPSYSTM

MATLAB Simulation Result (Hexadecimal) 3F711D14 BEB837B4 3E905530 3F0624DC 3E177318 3E013C360 3F11205A 3E3DA510 3DA8C150

SYNOPSYS Simulation Result (Hexadecimal) 3FF1174A BEB821D8 3E906C86 3F062C50 3E178F8A 3EB3D888 3F1122C4 3E3DBAC5 3DA8D808

Output Data (Decimal) 1.883523 -0.359633 0.282078 0.524114 0.148008 0.351261 0.566937 0.185283 0.082443

366

Zaidi Razak and Mashkuri Yaacob

10. 11. 12. 13. 14. 15. 16.

BC498580 BF0DA512 BF0DF27C BE327BB0 3F256042 3EAA3070 3EC594AC

BC481330 BF0DA0E0 BF00EA62 BE3253E4 3F2565C8 3EAA477E 3EC59F2A

-0.012212 -0.553236 -0.503576 -0.174148 0.646084 0.332577 0.385980

The figure below shows how the entities are integrated to produce the above results. ACTUAL COEFF MEMORY

WORK COEFF MEMORY

TEMP WORK MEMORY

ACTUAL INPUT MEMORY

WORK INPUT MEMORY

MULTIPLIER

LATCH_2

ACCUMULATOR

ADDER

LATCH_1

MAIN CONTROLLER

RESULTMEMORY TEMP RESULT MEMORY

RESULR BUFFER

Fig. 6. Block Design for DWT

8 Synthesis of the Entity Each entity that has been produced will be optimized to ensure the production of the DWT chip. However, the validation process for the design must be carried out to ensure that the derived result do not change after the optimization process. Two figures below show the similarity of the outcome before and after the optimization process that took place at the buffer.

The Design of Discrete Wavelet Transformation Chip

367

Fig. 7. Input and Output Data Before Optimization

Fig. 8. Input and Output Data After Optimization

The synthesis process was done after all the results have gone through the validation process. Figure 9 below shows a sample of the logic gate diagram i.e output from the synthesis process.

9 Conclusion The output that has been obtained shows that the process of producing a special DWT hardware can be achieved. Moreover, the design can be programmed into FPGA chips and tested further for full functionality. Currently the existing designs are being improved with the aim of minimising silicon real estate as well as improving the processing speed of the transform functions.

References 1.

2.

3. 4.

Hiton, M.L, Jawerth, B.D, Sengupta, A.N, “Compressing Still And Moving Images With Wavelets,” Multimedia Systems, Vol 2, No. 3, pp. 218 – 227, April 1994. Levy. I.K., Wilson. R.: Three Dimensional Wavelet Transform Video Compression. IEEE International Conference on Multimedia Computing And System, Vol. 2. (1999) 924 – 928. Grasp. A.: An Introduction to Wavelets. IEEE Computational Science and Engineering, Vol.2, No.2. (1995) 2 Coffey. M.A., Etter. D.M.: Image Coding With The Wavelet Transform. IEEE Symposium Circuit And System, Vol. 2. (1995) 1110-1113

368

Zaidi Razak and Mashkuri Yaacob

Fig. 9. Gate Logic Diagram of PDW

5.

6.

Mallat. S.: A Theory for Multiresolution Signal Decompositions, the Wavelet Representation. IEEE Trans. Pattern Analysis And Machine Intelligence, Vol. 2, (1989) 674-693. Edwards. T.: Discrete Wavelet Transforms: Theory And Implementation. Research Report, Stanford University. (September 1991) 4.

On the Performance of Informative Wavelets for Classification and Diagnosis of Machine Faults H. Ahmadi1, R. Tafreshi2, F. Sassani2, and G. Dumont1 1

The Department of Electrical and Computer Engineering The University of British Columbia {ahmadi,guyd}@ppc.ubc.ca 2 The Department of Mechanical Engineering The University of British Columbia {tafreshi,sassani}@mech.ubc.ca

Abstract. This paper deals with an application of wavelets for feature extraction and classification of machine faults in a real-world machine data analysis environment. We have utilized informative wavelet algorithm to generate wavelets and subsequent coefficients that are used as feature variables for classification and diagnosis of machine faults. Informative wavelets are classes of functions generated from a given analyzing wavelet in a wavelet packet decomposition structure in which for the selection of best wavelets, concepts from information theory i.e. mutual information and entropy are utilized. Training data are used to construct probability distributions required for the computation of the entropy and mutual information. In our data analysis, we have used machine data acquired from a single cylinder engine under a series of induced faults in a test environment. The objective of the experiment was to evaluate the performance of the informative wavelet algorithm for the accuracy of classification results using a real-world machine data and to examine to what the extent the results were influenced by different analyzing wavelets chosen for data analysis. Accuracy of classification results as related to the correlation structure of the coefficients is also discussed in the paper.

1

Introduction

The objective of machine data analysis for diagnosis is to extract features that can be used for classification and diagnosis of different machine faults. In general, faults in machine data analysis are attributed to component faults or machine events; they are characterized by nonstationary signals associated with burst of high-energy events such as combustion or closing/opening of a valve. Several approaches have been proposed by several research groups for the analysis and diagnosis of machine faults and some have been applied successfully [4,5,6]. They include methodologies using time-frequency approaches, statistical analysis and application of wavelet-based Y. Y. Tang et al. (Eds.): WAA 2001, LNCS 2251, pp. 369-381, 2001.  Springer-Verlag Berlin Heidelberg 2001

370

H. Ahmadi et al.

signal processing methods[3,5]. Wavelets are considered to be highly suitable for the analysis of nonstationary transient signals as often observed in machine data. This paper deals with an application of statistical approach for feature extraction namely informative wavelet algorithm for machine diagnosis in which wavelet coefficients are used as feature variables for the classification and diagnosis of different machine faults. The study is aimed at the following. •

• • •

2

To utilize informative wavelet algorithm for the analysis of real-world machine data and to evaluate the performance of the algorithm for the accuracy of classification results To analyze the accuracy of results as influenced by different analyzing wavelets used in the algorithm To trace the accuracy of classification results to parameters of training data To examine correlation structure of the informative wavelets and coefficient matrix as determined by different parameters of the algorithm and to identify the manner they influence the classification results

Introductory Remarks about Informative Wavelets and Classification Algorithm

Informative wavelets are classes of functions generated from a given analyzing wavelet in a wavelet packet decomposition structure in which for the selection of ‘best’ wavelets, concepts from information theory i.e. mutual information and entropy are utilized. Entropy is a measure of uncertainty in predicting a given state of a system where a system state refers to different operating conditions such as normal or faulty operation. Computation of entropy requires evaluation of probabilities generated from training data and is supplied as inputs to the algorithm. An iterative process to generate informative wavelets is applied where at each stage, algorithm selects a wavelet from a library of orthogonal wavelets in a given wavelet packet signal decomposition structure which results in a maximal reduction of entropy, i.e. maximal reduction in uncertainty of predicting a given system state. Reduction in uncertainty is expressed in terms of mutual information derived from the joint probability distributions of the training data and coefficients. Following derivation describes the concept. M

H ( S ) = ∑ P(Si) log(P(Si)) i =1

H(S) indicates entropy of system where S1, S2, …, SM are the states of the given system with probability of occurrences given by P(S1), P(S2), …, P(SM). States of the system are observed by a measurement system with N possible outputs {T1, T2, …, TN} of a random variable T with a probability distribution P(T1), P(T2), …, P(TN). Mutual information between the states and measurements is defined as the difference between the uncertainty of predicting S before and after the observation of T:

On the Performance of Informative Wavelets for Classification and Diagnosis M

N

JS (ωγ ) = H ( S ) − H ( S / T ) = ∑∑ P(SiTj) log i =1 j =1

371

P(SiTj) P(Si)P(Sj)

Here H(S/T) and P(SiTj) indicate conditional entropy of state S given measurement T and joint probability distribution of S=Si and T=Tj, respectively. When a given state of a system is independent of a measurements, i.e. Js =0, a change in the state of the machine will not result in any change in the probability P(SiTj). The algorithm selects wavelets that results in a maximal reduction of uncertainty i.e. maximal Js(ω). In informative algorithm, measurements are wavelet coefficients obtained by projecting data onto a selected wavelet. This is done iteratively where at each stage, the residual signal is considered for signal expansion. These wavelets are referred to as informative wavelets. The iterative selection of the informative wavelets is much similar to matching pursue algorithm [2]. Wavelet coefficients are then used as feature variables and as inputs to a neural network classifier for classification and diagnostics (For further details please refer to ref. No 3). Following block diagram illustrates the main stages of the algorithm.

Fig. 1. Block Diagram of Informative Wavelet Algorithm

3

Design of the Experiment

For the experiment, machine data from a single cylinder reciprocating engine were utilized in which two types of faults were considered. We considered engine knock and intake loose valve condition each with varying intensity levels. The engine was a dual mode engine operating both on diesel fuel and natural gas mode. Data presented here mainly belong to diesel mode operation. Engine knock condition was generated by carefully adjusted load changes. Load changes were made in incremental steps of approximately 15% increase at each step. For loose valve condition, set of progressively increasing valve clearances were induced on the valve at each run. Three categories of data were collected simultaneously: 1-cylinder pressure measured through a connecting tube to the cylinder, 2-acceleration measured at a carefully chosen location at the cylinder head, 3- engine RPM. Other data were also collected for engine diagnosis including engine power, peak cylinder pressure, peak pressure angle, etc. At each run, data from sixteen consecutive cycle runs were acquired.

4

Data Used in the Study

Machine acceleration data at the intake valve closing and combustion events were utilized in our data analysis. Data were collected from consecutive cycles and were

372

H. Ahmadi et al.

used as training data. We considered three conditions of valve clearance namely normal, 0.006 in. and 0.012 in. clearances as well as three load conditions i.e. 18, 22, 25 HP. An initial review of data where mean values vs. standard deviation in each training data are examined, indicates that a certain degree of data clustering and class separation can be found in some of the data, though this could not be observed in all of our data runs. Separation of classes was more vivid in training data belonging to valve clearance conditions. Certain number of outliers was also observed in each class.

5

Parameters Chosen for Data Analysis

We considered the following as input parameters in our data analysis. •

•

•

•

• •

6

We used equal number of training data in all three classes where we had initially 30 training data in each class. At a later stage the number was modified when we examined the effect of changes in the training data. Initially, two classes consisting of normal and one faulty condition were considered. At a later stage, we considered three classes namely healthy and two faulty conditions with two different intensity levels. Most of the results illustrated in this paper belong to the latter. In informative algorithm, “number of informative wavelets” corresponds to the number of feature variables used for classification. In the absence of any á priori knowledge about a suitable number of feature variables, several values ranging from 1 up to 50 were initially considered. At a later stage, the number was confined to a smaller set ranging from 4 to 10. We considered wavelets from orthogonal as well as biorthogonal wavelet families. We considered Daubechies wavelets, mainly Db5, Db20, Db40 and Db45 as well as Coif5, Symlet5, Bior3.1, and Bior6.8. Standard multi layer perceptron was used for a neural network classifier. For a three-class data, five nodes of hidden layer were used for classification. We used 30 levels (bins) in quantification of coefficient and training data for construction of probability distributions.

Observations and Data Analysis Results

Informative wavelet algorithm is mainly a statistical approach for fault classification in which probability distributions of training data are utilized to generate wavelets for signal expansion and subsequently for classification. In this algorithm, wavelet coefficients carry statistical properties best matched to those of the training data. As such, classification results are determined jointly by the statistical properties of the given training data and the analyzing wavelet used for data expansion. Performance of the algorithm for the accuracy of the classification results is highly influenced by the extent of uniformity of data in each class as well as properties that are necessary for class separation. Separation of classes in coefficient domain follows a similar pattern

On the Performance of Informative Wavelets for Classification and Diagnosis

373

as those of training data where statistics of the coefficients are also influenced by the particular wavelet used for the analysis. Following observations were made in our data analysis. They are listed in three sections as described below. 6.1 Classification Results We examined wavelet coefficients generated by different analyzing wavelets as the main output of the algorithm for the classification of three valve clearance conditions and load changes. Mean vs. standard deviations of the coefficients of training data for three classes as well as histogram of the coefficients were also examined. A sample classification error results for several analyzing wavelets and for three load conditions are given (Fig 2). The errors of classification are below 5% of maximum error output of a NN classifier for most of the wavelets and were considered to be small enough and acceptable for an accurate classification of both valve clearances and load conditions. As shown in Fig 2, different errors were obtained using different wavelets. Bior3.1 performed superior to others and Symlet5 exhibited the largest error. Classification errors for load changes and knock detection varied and were influenced to a large extent by the extent of uniformity of the training data in all classes. We traced the differences in classification errors to several parameters of the algorithm as well as input data including correlation structure of coefficient matrix as described in sections below. Classification error for: Db2, Db5, Db20, Db45, Co1, Co5, Bi31 25

Percentage Error

20 15 10 5 0

1

2

3

4

5

6

7

Fig.2. Classification errors, three classes, three load conditions (healthy +, mild *, sever fault o, total error x). Error values from left to right: Db2, Db5, Db40, Db45, Bior3.1, Bior6.8, Sym5

6.2 Informative Wavelets A varied classification results were obtained in our data analysis using different wavelets. We tried to trace these different results to different input parameters of the algorithm such as training data, number of iterations and more significantly to the intermediate outputs of the algorithm such as informative wavelets as described below.

374

H. Ahmadi et al.

Nonorthogonal Signal Expansion. Informative wavelet algorithm is a nonorthogonal signal decomposition in which informative wavelets are in general correlated and a certain degree of redundancy is always observed in signal decomposition. Nonorthogonality of signal decomposition can be attributed to the iterative process in selecting informative wavelets where at each stage the residual signal is considered for signal expansion much similar to matching pursue algorithm [2]. In our data analysis, we examined deviation from orthogonality for several analyzing wavelets. It was observed that for most of the orthogonal analyzing wavelets, such as Daubechies wavelets or Coiflets, majority of the informative wavelets of the first few iterations generated by the algorithm, were near orthogonal as indicated by the inner product of the wavelets (Fig 3). Deviations from orthogonality increased as we moved to latter stages of the iterations, i.e. informative wavelets generated at the later iterations exhibited a higher correlation than those of the first few iterations. However for biorthogonal wavelets namely Bior3.1, deviations from orthogonality was even high at the initial iterations. A singular value decomposition of matrix composed of informative wavelets was examined to identify the extent of correlation among the informative wavelets along different principal axes (Fig. 3). As it can be seen, for Bior3.1, the first singular value is nearly twice that of Db5. The same observations were also made when covariance of the informative wavelet matrix was examined. We also examined an increase in the number of informative wavelets and the manner the correlation and singular values were influenced by such changes. It was observed that correlation was increased as additional informative wavelets were added. However, the extent of the increase differed for different wavelets. Biorthogonal wavelets showed the largest increase. Non-orthogonal Signal Expansion and Accuracy of Classification Results. The accuracy of the classification results as influenced by the nonorthogonality of the informative wavelets, were studied using correlation and singular values of the coefficient matrix. Following observations were made. • •

•

It was observed that correlation structure and singular values of coefficient matrix followed the same pattern as those of informative matrix. For Db family of orthogonal wavelets, it was observed that singular values of informative matrix of the first few iterations did not differ significantly from each other. It was also observed that for these wavelets, classification results were also not noticeably different from each other. However, singular values of wavelet matrix of higher iterations differed from each other for different analyzing wavelets of Db family. Biorthogonal wavelets namely Bior3.1, showed a higher degree of correlations among the informative wavelets than orthogonal. First singular value of the covariance matrix of the wavelets for Bior3.1 was observed to be nearly ten times that of majority of the orthogonal wavelets such as Db5, Db40. Classification results in majority of the cases considered here indicated a higher accuracy of classification results were obtained for Bior3.1 as compared with other wavelets under identical conditions.

On the Performance of Informative Wavelets for Classification and Diagnosis

375

Orthogonality in Db5 Informative Wavelet 1.5

(1)

1 0.5 0 -0.5

1

2 3 4 5 6 7 Orthogonality in Db40 Informative Wavelet

8

1

2 3 4 5 6 7 SVD of Db5 $ Db40 Informative Wavelet

8

1

2

8

1.5

(2)

1 0.5 0 -0.5 2

(3)

1.5 1 0.5 0

3

4

5

6

7

Fig. 3. Inner product of informative wavelets generated by Db40 (1), Bi3.1 (2), and singular values of corresponding coefficient matrices (3). Shown are first four informative wavelets (‘o’, ‘x’, ‘*’, ‘+’) from left to right and singular values (Db40, ‘o’, Bior3.1, ‘*’)

General Pattern of the Centroid of the Wavelets. A generally distinct patterns and clustering of centroid of informative wavelets were observed in our data analysis. For majority of the cases considered here, centroid of the wavelets of small translation index occurred at the high scale levels (Fig. 4). It was also observed that centroid of majority of wavelets, lie at the first half-length of the given signal as shown in Fig. 4 (top diagram). As a result centroid of wavelets with longer support (higher order) such as db45, were seen to exhibit this pattern more vividly than low order wavelets in identical data analysis situations. This could be confirmed when informative wavelets generated by Db5 was compared with those of Db45. It was also observed that the centroid of wavelets with large oscillation index, lie at higher scale levels. This follows from wavelet packet decomposition where wavelets with high frequency of oscillations occur at high scale levels.

376

H. Ahmadi et al.

Clustering in Translation-Modulation. A more vivid clustering pattern of centroid of wavelets was also observed in translation-modulation plane where centroid of majority of the wavelets lied on or near origin (Fig. 4). Sensitivity of Wavelets to Changes in Training Data. In our data analysis, it was observed that a small changes in training data caused a noticeable change in the informative wavelets. For example a small increase in the number of training data (e.g. simple repetition of data) caused a considerable change in several numbers of the informative wavelets. This could be attributed to the generation of informative wavelets by the algorithm where changes in the probability distribution of the coefficients change mutual information followed by changes in informative wavelets. In our data analysis using eight informative wavelets, often a change in half of the wavelets was observed. Mutual Information and Pattern of Oscillatory Behavior. Mutual information is used as a measure of reduction of uncertainty of the prediction; it contains information that is reflective of those of the input data. While mutual information generally follows a declining pattern with the number of iterations; however, considerable oscillatory patterns were observed in several cases in our data analysis. Oscillatory pattern as well as increasing mutual information (instead of declining), was mostly observed in biorthogonal wavelets such as Bior3.1 and to a lesser degree in Bior6.8. Several cases with mutual information at a constant level of unity during several consecutive iterations were also observed. Increase in Number of Iterations. In the current algorithm, number of informative wavelets (iterations) is chosen á priori as an input to the algorithm. It was observed that increasing number of iterations in a given data analysis does not alter informative wavelets derived from prior iterations. As a result no change was observed in the coefficient values already derived. The additional informative wavelets increased feature variables and thus expanding the dimension of feature space. Increase in Number of Iterations and Accuracy of Classification Results. In our experiments, we have used 5-10 iterations (informative wavelets) though higher iterations were also examined. It was observed that an increase in the number of iterations was not always accompanied by an increase in the accuracy of classification results. This could be traced to the correlation structure of wavelet coefficients and changes made by the added wavelets as discussed in the following section. 6.3 Observation on the Coefficients Wavelet coefficients are the feature variables in informative wavelet algorithm; they contain the necessary information about a given data for classification and fault diagnosis. They are determined by the given data as well as informative wavelets generated by the algorithm. Following observations were made on the coefficients.

On the Performance of Informative Wavelets for Classification and Diagnosis

377

Trans

100 50 0

Modul

100

0 Mutual Info

1

2

3

4

5

6

7

8

0

1

2

3

4

5

6

7

8

0

1

2

3

4

5

6

7

8

50

1 .5 1 0 .5 100 Trans

0

50 0

0

10

20

30 40 M o d ula ti o n

50

60

70

Fig.7. Centroids of informative wavelets. In top three diagrams, horizontal axes are scale levels

Correlation Structure of the Coefficient Matrix. Informative wavelet algorithm is a nonorthogonal signal decomposition; as a result a certain degree of redundancy of signal expansion and correlation will always remain among the coefficient values. We examined correlation structure of coefficient matrix under several analyzing wavelets and different number of iterations. We examined correlation of coefficients for different informative wavelets in a given data. Significant differences were observed in correlation of the coefficients for different wavelets. We used singular value decomposition of the coefficient matrix to identify the manner classification results were influenced by the correlation structure of the coefficients. Singular values of the coefficient matrix for several wavelets are shown in Fig. 5. It was observed that for orthogonal wavelets such as Db wavelets, no significant differences could be observed between first singular values of the coefficient matrix of the first few iterations. However differences were observed in singular values of latter stages of the iterations. We observed the same for Coiflet5, but for Symlet5 singular values were different. For coefficients generated by biorthogonal wavelets such as Bior3.1, often a large first singular value was observed as compared with other wavelets (Fig. 5). Accuracy of Classification. In majority of cases considered here, accuracy of the classification results was higher when singular values of the coefficient matrix were also high. We have shown result of classification errors for several wavelets derived under identical conditions of data analysis(Fig. 2). Classification errors were seen to be approximately the same for orthogonal Db wavelets and were generally lowest for Bior3.1. It was also observed that for wavelets within the same group (such as db family) and for a given data analysis, a higher accuracy of classification results was obtained when correlation between informative wavelets was also large.

378

H. Ahmadi et al.

All Classes

SVD of Coef Matrix, all Classes, Db20 & Bior3.1, Last 4 Info. Wvlts. 5

0

Class 1

2

4 Class 2

1.5

2

2.5

3

3.5

4

1

1.5

2

2.5

3

3.5

4

1

1.5

2

2.5

3

3.5

4

1

1.5

2

2.5

3

3.5

4

1 0

2 0 4

Class 3

1

2 0

Fig. 5. Singular values of coefficient matrix for three classes of load conditions (individual and all classes), Db20 (‘o’), Bior3.1 (‘*’) C oeffs: Bior 3.1 with 28 training signals in each class W1

0.5 0 -0.5

0

10

20

30

40

50

60

70

80

90

-0.5 10

10

20

30

40

50

60

70

80

90

0

10

20

30

40

50

60

70

80

90

0

10

20

30

40

50

60

70

80

90

0

10

80

90

W2

0.5

W3

0

0 -1

W4

0.5 0 -0.5 2 1 0

20 30 40 50 60 70 Sum Squared C oeffs for 8 informative wavelets

Fig. 6. Wavelet coefficients, four informative wavelets W1, …, W4 for Bior3.1. Bottom diagram is coefficient L2 norm of training data for three classes (1:28, 29:56, 57:84)

On the Performance of Informative Wavelets for Classification and Diagnosis

379

Number of Iterations. An increase in the number of iterations was followed by an increase in most of the singular values. For Bior3.1 with highly correlated coefficients, the number of iterations was found to be in general lower than those of other wavelets tried in this experiment for an equal size of classification error. Reduced Size of Coefficient Matrix. Cases were observed in which wavelet coefficients of the signal expansion by different informative wavelets, differed significantly from each other where for some wavelets they were negligibly small across all classes. For such cases, rows of coefficient matrix having large coefficient values were selected and used as feature variables and as input to the classifier resulting in an increased computational efficiency. We also considered singular value decomposition of informative wavelet matrix to reduce the size of coefficient matrix where highly correlated wavelets (and coefficients) could be removed (Fig. 6). Coefficient Values and Separation of Classes. For each of the training data L2 norm of the coefficient values were evaluated (Fig. 6). Mean squared values and standard deviations of different classes were also used as an index for class separation. It was found that classification results using mean squared values of different classes were nearly identical with those obtained by the algorithm and when classification error was small. More accurate classification results could be obtained when standard deviations were small. Sum squared coefficient values were also used to identify outliers in training data as discussed below. Uniform Coefficient Values and Identification of Outliers. Under a given wavelet, a relatively a uniform coefficient values were obtained for signal expansion of different training data in a given class. This was more pronounced when sum squared coefficient values as mentioned above, were examined. A uniform pattern in coefficient values were used for identifying outliers in a given training data where coefficients differed significantly the general pattern (Fig. 6). While coefficients are wavelet dependent, often such outliers could be seen consistently across several analyzing wavelets. This enabled us to remove outliers in the training data and improve the results of the classification.

7

Conclusions

In this paper results of an experimental study for an application of informative wavelet algorithm for classification and diagnosis of machine faults were presented. We have used several wavelets and different set of machine data. Effectiveness of the algorithm for the classification of two categories of faults namely excess valve clearance and knock conditions each with variable intensity levels were examined. Accuracy of results was studied under several parameters of the algorithm. We have used different wavelets from both orthogonal and biorthogonal family of wavelets. Several illustrative examples were presented in this paper. Some of the results of the study were discussed and are summarized as follows. •

In majority of the experimental runs in which we used different analyzing wavelets, satisfactory classification results were obtained when training data were

380

•

•

•

•

•

H. Ahmadi et al.

considered to be sufficiently uniform and sufficient number of training data was used for classification. However different classification errors were obtained for different wavelets. For biorthogonal wavelets often classification errors were found to be lower as compared with orthogonal wavelets. For load changes and knock condition, accuracy of results varied for different training data and different intensity levels of faulty conditions. It was observed that informative wavelets generated by the algorithm are highly sensitive to changes in number as well as minor changes in actual values of training data. While classification results remained almost unchanged by small changes in training data, informative wavelets and coefficient values changed significantly. This was attributed to the structure of the algorithm in which probability distribution of the coefficients can be influenced by minor changes in training data and more significantly during the selection of the informative wavelets. It was often observed that wavelet coefficients of a given signal differed for different informative wavelets. As a result, small coefficient values across all classes under a given informative wavelets, could be removed and a reduced size of feature variables could be obtained without a noticeable effect on the classification results. Different informative wavelets were derived using different analyzing wavelets. This lead to different correlation structure of the coefficient matrix measured by singular values. It was observed that while for orthogonal wavelets of Db family, first few large singular values were not significantly different from each other, often Bior3.1 showed the highest correlation( large singular values) for a given signal analysis. Accuracy of results was also superior under Bior3.1 in majority of cases as compared with other wavelets. In a given data analysis where informative wavelets were highly correlated, it was possible to reduce the number of informative wavelets and as such number of features by removing rows of correlated coefficients without significant changes in the classification results. Different coefficient values were obtained when different analyzing wavelets were used for data analysis. As a result correlation structure of the coefficient matrix was influenced by the use of different wavelets leading to different size of the reduced matrix and different number of feature variables in a classification problem.

Acknowledgement This experiment was made possible through grants supplied by NRC_IRAP and REM Technology of Port Coquitlam, BC. Authors wish to thank Dr. Howard Malm of REM Technology for a continued support during both data collection and data analysis.

On the Performance of Informative Wavelets for Classification and Diagnosis

381

References 1. 2. 3. 4.

5.

6.

7.

Coifman, R.R, Wickerhauser M.V.: Entropy-based Algorithm for Best Basis Selection. IEEE Transactions on Information Theory 38,713-718 (1992) Mallat, S., Zhang, Z: Matching Pursuit with Time Frequency Dictionaries. IEEE Trans. on Signal Processing 41,3397-3415 (1993) Bao Liu, Shih Fu Ling: On the Selection of Informative Wavelets for Machinery Diagnosis. Mechanical Systems and Signal Processing, Vol. 13, No 1 (1999) Samimy B. Rizzoni, G: Mechanical Signature Analysis using Time Frequency Signal Processing: Application to Internal Combustion Engine Knock detection. Proc. of IEEE, Vo. 84 No.9 (Sep. 1996) Zheng G.T, McFadden P.D.: A time-frequency Distribution for Analysis of Signal with Transient Components and its Application to Vibration Analysis. Trans. ASME, Vol 121 (Jul, 1999) Samimy B. et all: Design of Training data–based Quadratic Detectors with Application to Mechanical Systems. Proc. ICASSP-96, May 7-10, Atlanta, GA (1996) Daubechies, I: Ten lectures on Wavelets. Siam, Philadelphia, PA (1992)

A Wavelet-Based Ammunition Doppler Radar System S. H. Ong and A. Z. Kouzani School of Engineering and Technology, Deakin University Geelong, Victoria 3217, Australia

Abstract. Today’s state-of-the-art ammunition Doppler radars use the Fourier spectrogram for the joint time-frequency analysis of ammunition Doppler signals. In this paper, we implement the joint time-frequency analysis of ammunition Doppler signals based on the theory of wavelet packets. The wavelet-based approach is demonstrated on Doppler signals for projectile velocity measurement, projectile inbore velocity measurement and on modulated Doppler signal for projectile spin rate measurement. The wavelet-based representation with its good resolution in time and frequency and reasonable computational complexity as compared to the Fourier spectrogram is a good alternative for the joint time-frequency analysis of ammunition Doppler signals.

1

Introduction

Continuous wave (CW) Doppler radars [1,2] are used for measuring the velocity and spin rate of projectile(s). Joint time-frequency analysis (JTFA) of the Doppler signal is done to extract the velocity-time and/or spin rate-time information of the projectile(s) from the Doppler signal. The Fourier spectrogram (FS) is the current method used in today’s ammunition Doppler radar systems. The requirements of a digital signal processing (DSP) system for the JTFA of ammunition Doppler signals are: 1.

2.

The computation of the velocity or spin rate results has to be done immediately after each round is fired or at a later time. Thus, the DSP algorithm has to be of reasonable computational complexity if the results are needed immediately after a round is fired. The system must be able to give accurate velocity and spin rate results. To satisfy this requirement, the time-frequency representation must have good resolution in time and frequency.

The FS is able to meet the first requirement using algorithms based on the fast Fourier transform. However, the main limitation of the FS is the poor resolution in time or frequency of the representation. The objective of this paper is to implement the JTFA of ammunition Doppler signals using the wavelet packet transform. In Sect. 2, we present the FS and explain its main limitation. In Sect. 3 we review the ammunition Doppler radar system and describe the main applications of ammunition Doppler Y. Y. Tang et al. (Eds.): WAA 2001, LNCS 2251, pp. 382-392, 2001.  Springer-Verlag Berlin Heidelberg 2001

A Wavelet-Based Ammunition Doppler Radar System

383

radar systems. In Sect. 4, we introduce the best-basis wavelet packet transform (BBWPT). In Sect. 5, we compare the wavelet-based approach with the FS-based approach for the JTFA of ammunition Doppler signals. In Sect. 6, we conclude the paper.

2

The Fourier Spectrogram

To determine the properties of a signal at a particular time, we emphasize the signal at that time and suppress the signal at other times. This is done by multiplying the signal by a window function h(t) centered at time t, to produce the signal,

st (τ ) = s (τ ) h (τ − t ) .

(1)

The Fourier transform of st(τ) will reflect the distribution of frequency around time t,

St (ω ) =

1 2π

∫e

=

1 2π

∫e

− jωτ

st (τ ) dτ

(2)

− jωτ

s (τ ) h (τ − t ) dτ .

(3)

The energy density spectrum at time t is given by 2

PSP ( t , ω ) = St (ω ) =

1 2π

2

∫e

− jωτ

s (τ ) h (τ − t ) dτ .

(4)

At each time instant we get a different spectrum. The totality of these spectra gives the time-frequency representation PSP called the FS. To obtain good time localization a narrow window in the time domain h(t) is used. To obtain good frequency localization a narrow window in the frequency domain H(ω) is used. But both h(t) and H(ω) cannot be made arbitrarily narrow. There is an inherent trade-off between the time and frequency localization in the FS for a particular window. The amount of trade-off depends on the signal, window, time, and, frequency. The uncertainty principle quantifies these trade-off dependencies. The poor resolution in time or frequency is the main limitation of the FS method.

3

Ammunition Doppler Radar System

The ammunition Doppler radar is a versatile instrument used for the testing of ammunition. In this section, we review the ammunition Doppler radar system and state its main applications. A simplified block diagram of an ammunition Doppler radar system is shown in Fig. 1.

384

S. H. Ong and A. Z. Kouzani

The output of the CW transmitter at frequency f0 Hz is routed through a circulator to the antenna. The wave transmitted by the antenna propagates to and is scattered from the moving projectile. The wave is then received back at the antenna. The received wave has a frequency of (f0 - fd) Hz. The wave passes through the circulator to the receiver. At the front end of the receiver, a mixer heterodynes the two signals together to produce a Doppler signal of fd Hz. The A/D converter digitizes the Doppler signal and sends it to a computer whereby the JTFA is done. The subsections below describe three major applications of CW ammunition Doppler radar systems.

Circulator Antenna

f0

f0

CW transmitter

f0 - fd Projectile

Gun

f0 - fd

f0 (leakage) Mixer

fd

ADC

Computer

Fig. 1. Simplified block diagram of an ammunition Doppler radar system

Projectile Velocity Measurement. JTFA of the Doppler signal is done to obtain the instantaneous frequency (single projectile case) or frequencies (multiple projectiles case) from the Doppler signal. The instantaneous velocity v of the projectile is computed from the instantaneous Doppler frequency shift fd using the following relation [3]:

v = 0.5 × f d × λ

(5)

where λ is the wavelength. λ is given by

λ=

c f0

(6)

where c is the velocity of radio waves and f0 is the frequency of the transmitted wave. A 16k byte multicomponent Doppler signal for the velocity measurement of a saboted round (signal no. 1) is shown in Fig. 2. The broadband FS of this signal as shown in Fig. 3 uses a length 256 Hanning window. The narrowband FS of the signal as shown in Fig. 4 uses a length 1k byte Hanning window. The narrowband FS has a better frequency resolution compared to the broadband FS. However, the time resolution is degraded resulting in smearing of the frequency over the length of the window due to the nonstationary nature of the Doppler signal. The poor resolution in time or frequency of the FS affects the accuracy of the velocity results. This is the

A Wavelet-Based Ammunition Doppler Radar System

385

main drawback of the FS when it is used for the JTFA of ammunition Doppler signals. 2000 1500

Voltage (x 0.01 volt)

1000 500 Sabots 0 -500 -1000 -1500 -2000

0

2000

4000

6000

8000 10000 Time (x 8 us)

12000

14000

16000

Fig. 2. Signal no. 1 - Doppler signal for projectile velocity measurement

Projectile In-Bore Velocity Measurement. Projectile in-bore velocity measurement involves the study of the motion of projectile inside a gun barrel. The radar antenna is placed at the side of the gun muzzle. A reflector is placed at a certain distance down range of the gun and in-line with the barrel bore axis. This reflector reflects the transmitted waves from the antenna into the barrel bore. Upon firing, as the projectile moves inside the barrel, the reflected waves scattered from the projectile are reflected by the reflector and received back at the radar antenna. The in-bore velocity of the projectile vib is related to the Doppler frequency shift fd by the following relation:

vib = 0.5 × f d × λ fs × M c

(7)

where λfs is the free space wavelength and Mc is the mode correction factor. The mode correction factor is described by the following equation.

Mc =

λib = λ fs

1  1.841× c  1−    π × d × f0 

where d is the diameter of the barrel bore.

2

(8)

386

S. H. Ong and A. Z. Kouzani 4

x 10 6

5

Frequency (Hz)

Penetrator 4 Sabots 3

2

1

0 0

0.02

0.04

0.06 0.08 Time (sec)

0.1

0.12

Fig. 3. Broadband FS of signal no. 1 4

x 10 6

5

Frequency (Hz)

Penetrator 4 Sabots 3

2

1

0

0

0.02

0.04

0.06 Time (sec)

0.08

0.1

0.12

Fig. 4. Narrowband FS of signal no. 1

Fig. 5 shows a 4k byte monocomponent Doppler signal (signal no. 2) for projectile in-bore velocity measurement. The FS of this signal as shown in Fig. 6 uses a length 512 Hanning window. Projectile Spin Rate Measurement. The amplitude and phase of the Doppler signal are not affected by the spin of an axially symmetric projectile. If a slot is milled on the base of the projectile to make it axially asymmetric, this will result in modulation of the amplitude and phase of the Doppler signal. To obtain sufficient modulation, the width and depth of the slot have to be at least one quarter the wavelength of the transmitted signal [4]. A 16k byte modulated Doppler signal for projectile spin rate measurement (signal no. 3) is shown in Fig. 7. The broadband FS of this signal as shown in Fig. 8 uses a length 512 Hanning window. The projectile spin rate cannot

A Wavelet-Based Ammunition Doppler Radar System

387

be determined from Fig. 8 as the frequency resolution of the representation is insufficient. The narrowband FS of the signal as shown in Fig. 9 uses a length 1k byte Hanning window. In Fig. 9, we are able to identify three peaks in the FS: a major peak at frequency fd and two minor peaks at frequencies fd ± 2 × fs . fd is the Doppler frequency shift corresponding to the velocity of the projectile. fs is the spin rate of the projectile. 2000 1500

Voltage (x 0.01 usec)

1000 500 0 -500 -1000 -1500 -2000

0

500

1000

1500 2000 2500 Time (x 5 usec)

3000

3500

4000

Fig. 5. Signal no. 2 - Doppler signal for projectile in-bore velocity measurement 4

5

x 10

4.5 4

Frequency (Hz)

3.5 3 2.5 2

Bullet exit from muzzle

1.5 1 0.5 0 0

2

4

6

8 10 Time (sec.)

12

14

16

18 -3

x 10

Fig. 6. FS of the Doppler signal for projectile in-bore velocity measurement

388

S. H. Ong and A. Z. Kouzani

2000 1500

Voltage (x 0.01 volt)

1000 500 0 -500 -1000 -1500 -2000

0

2000

4000

6000 8000 10000 Time (x 5 usec)

12000

14000

16000

Fig. 7. Signal no. 3 - Modulated Doppler signal for projectile spin rate measurement 4

10

x 10

9 8

Frequency (Hz)

7 6 5 4 3 2 1 0 0

0.01

0.02

0.03

0.04 0.05 Time (sec)

0.06

0.07

0.08

Fig. 8. Broadband FS of the modulated Doppler signal for projectile spin rate measurement

4

Best-Basis Wavelet Packet Transform

The discrete wavelet transform (DWT) is implemented by Mallat’s pyramid algorithm [5]. Fig. 10 shows a 1-stage decomposition of a signal in the DWT. h[n] and g[n] are the impulse responses of the analysis low pass and high pass filters, respectively. Sn,k+1 is one of the vector spaces at level k + 1 onto which signals are projected. The outputs of both filters are downsampled by a factor of two. The filtering and downsampling operations split the vector space into two subspaces S2n,k and S2n+1,k. The

A Wavelet-Based Ammunition Doppler Radar System

389

signal is successively decomposed into lower resolution components, while the highfrequency components are not analyzed any further. The wavelet packet transform (WPT) however uses both the low and high frequency components. Fig. 11 shows a wavelet packet tree. There is a large but finite library of bases in the wavelet packet tree. The best basis can be extracted from this library based on some criterion. This is done using the best-basis algorithm [6]. The best-basis algorithm compares the entropy of the children to their parent entropy, starting from the bottom of the tree. If the parent entropy is smaller than the sum of the children entropy, then the parent entropy is retained. Else, the children entropy sum replaces the parent entropy. 4

10

x 10

9 8

Frequency (Hz)

7 Minor peaks

6 5

Major peak

4 3 2 1 0

0

0.01

0.02

0.03

0.04 Time (sec)

0.05

0.06

0.07

Fig. 9. Narrowband FS of the modulated Doppler signal for projectile spin rate measurement

↓2 Sn,k+1

g(n) h(n)

S2n+1,k ↓2

S2n,k

Fig. 10. 1-stage decomposition of a signal

5

Wavelet-Based versus FS-Based Approach

Fig. 12 shows the BB-WPT representation of signal no. 1 for projectile velocity

Fig. 11. A wavelet packet tree. The wavelet basis is shown in continuous line

390

S. H. Ong and A. Z. Kouzani

measurement. Vaidyanathan-Huong 24 (VH24) [7] wavelet is used and the WPT is computed up to resolution level 14. Comparing Fig. 12 with the corresponding FSs shown in Figs. 3 and 4, we can see the good resolution in time and frequency of the BB-WPT as compared to the FS representation. Fig. 13 shows the BB-WPT representation of signal no. 2 for projectile velocity inbore measurement. VH24 wavelet is used and the WPT is computed up to resolution level 12. The BB-WPT representation shown in Fig. 13 is in conformance with the corresponding FS shown in Fig. 6. Fig. 14 shows the BB-WPT representation of signal no. 3 for projectile spin rate measurement. VH24 wavelet is used and the WPT is computed up to resolution level 14. Due to the good time and frequency resolution of the BB-WPT representation, we are able to identify the main peak and the two minor peaks in Fig. 14. From the two minor peaks, the spin rate of the projectile can be determined. In the corresponding FSs shown in Figs. 8 and 9, the main peak and the two minor peaks can only be identified from the narrowband FS. 1 0.9

Frequency [x 62.5 kHz]

0.8 0.7

Penetrator

0.6 Sabots 0.5 0.4 0.3 0.2 0.1 0

0

0.1

0.2

0.3

0.4 0.5 0.6 Time [x 0.131072 s]

0.7

0.8

0.9

1

Fig. 12. BB-WPT representation of signal no. 1 for projectile velocity measurement 1 0.9

Frequency [x 100 kHz]

0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0

0

0.1

0.2

0.3

0.4 0.5 0.6 Time [x 0.02048 s]

0.7

0.8

0.9

1

Fig. 13. BB-WPT representation of signal no. 2 for projectile in-bore velocity measurement

A Wavelet-Based Ammunition Doppler Radar System

391

1 0.9

Frequency [x 100 kHz]

0.8 0.7 Minor peaks 0.6 0.5 0.4

Major peak

0.3 0.2 0.1 0

0

0.1

0.2

0.3

0.4 0.5 0.6 Time [x 0.08192 s]

0.7

0.8

0.9

1

Fig. 14. BB-WPT of signal no. 3 for projectile spin rate measurement

6

Conclusion

We have demonstrated the JTFA using the wavelet-based approach via the BB-WPT on three types of ammunition Doppler signals as follows: 1. 2. 3.

Multicomponent Doppler signal for projectile velocity measurement. Monocomponent Doppler signal for projectile in-bore velocity measurement. Modulated Doppler signal for projectile spin rate measurement. The main limitation of the FS is the poor resolution in time or frequency of the representation. This affects the accuracy of the velocity and/or spin rate results. On the other hand, due to the unique properties of wavelet packets, the BB-WPT representation exhibits better time and frequency resolution compared to the FS representation. The BB-WPT is suitable for practical implementation as the algorithm is of reasonable computational complexity when implemented based on Mallat’s pyramid algorithm and the best basis algorithm. In conclusion, the waveletbased approach is a good alternative to the FS-based approach for the JTFA of ammunition Doppler signals.

References 1. 2. 3. 4.

Skolnik, M. I., Ed.: Radar Applications. IEEE Press. (1987) 443 – 452 Whetton, C. P.: Industrial and Scientific Applications of Doppler Radar. Microwave J., Vol. 18 (Nov 1975) 39 – 42 Levanon, N.: Radar Principles. John Wiley and Sons, Inc. (1988) 1 – 18 Jens-Erik Lolck: Spin Measurements. Tenth International Symposium on Ballistics. Vol. I. San Diego California. (1987)

392

5.

6. 7.

S. H. Ong and A. Z. Kouzani

Mallat, S. G.: A Theory for Multiresolution Signal Decomposition: The Wavelet Representation. IEEE Trans. Patt. Anal. and Mach. Intell., 11(7) (1989) 674 – 693 Coifman R. R., Wickerhauser, M. V.: Entropy-Based Algorithms for Best-Basis Selection. IEEE Trans. Inform. Theory. 38 (2) (Mar 1992) 713 – 718 Vaidyanathan P. P., Phuong-Quan Huong: Lattice Structures for Optimal Design and Robust Implementation of Two-Channel Perfect-Reconstruction QMF Banks. IEEE Trans. Acoust., Speech, and Signal Proc. 36(1), (Jan 1988) 81 - 94

The Application of Wavelet Analysis Method to Civil Infrastructure Health Monitoring* Jian Ping Li1, Shang An Yan1, and Yuan Yan Tang2 1

International Centre for Wavelet Analysis and Its Applications, Logistical Engineering University, Chongqing 400016, P. R. China [email protected] 2 Department of Computer Science, Hong Kong Baptist University, Hong Kong [email protected]

Abstract. Wavelet analysis and its applications have become one of the fastest growing research areas in the recent years. This is in part attributed to the pioneering work by the researchers as well as practitioners in the field of signal processing. Morlet first coined down the term of wavelet analysis in early 1980s. Meyer developed a wavelet basis in 1986, which is best known today as Meyer basis. Later, Mallat and Meyer formulated a theory of multiresolution analysis theory, and subsequently, proposed the Mallat algorithm, making wavelet transform readily implementable with digital computers. In 1990s, advanced research and development in wavelet analysis have found numerous applications in such areas as signal processing, image processing, and pattern recognition with many encouraging results. Despite this fast growth in theories and applications, the theoretical development of wavelet transform itself is somewhat lagging behind as compared to its applications. Recently, a new method based on wavelet analysis and wavelet transform has been developed to process nonlinear and nonstationary time series data by Huang[1,2,3,4] and J. P. Li, Y. Y. Tang [5,6]. This novel method, consisted of wavelet transform, Hilbert Spectral Analysis and Empirical Mode Decomposition. It has been applied to a variety of geophysical and bio-engineering problems. The specific application to civil infrastructure health monitoring has been reported. The basic method and infrastructure health monitoring application will be discussed here.

1

Introduction

To safeguard the safety performance of a bridge, regular inspections are essential. At the present time, the inspection method is primarily visual: A technician has to go *

This work was supported by the National Nature Science Foundation of China under the grand number 69903012.

Y. Y. Tang et al. (Eds.): WAA 2001, LNCS 2251, pp. 393-397, 2001.  Springer-Verlag Berlin Heidelberg 2001

394

Jian Ping Li et al.

through a bridge to examine each member and certify its safety. This method is subjective and flawed for lacking of rigorous and objective standards. For example, for a deteriorating bridge from fatigue or aging, the damage is not clear-cut at any time. Therefore, any call is judgmental. Furthermore, it is not feasible to use this method for complicated bridge structures: there might be members located at positions too awkward to access; there might be too many members that would require too much time to inspect; and there might be damage too subtle to be detected visually. Because of these limitations, the visual inspection results are known to be not totally reliable; yet, we are forced to rely on it today. We think that an ideal inspection method will have to satisfy the following conditions: 1. 2. 3. 4.

2

Robust, objective, and reliable To be able to identify the existence of damages, To be able to locate the damages, To be able to determine the degree of the damages.

Wavelet Transform

The wavelet approach is essentially an adjustable window Fourier spectral analysis with the following general definition:

Wa ,b ( f (t ),ψ ) = a

−1 / 2

∫ f (t )ψ (

t −b )dt a

(1)

in which ψ (t ) is the basic wavelet function that satisfies certain very general conditions, a is the dilation factor and b is the translation of the origin. Although time and frequency do not appear explicitly in the transformed result, the variable 1/a gives the frequency scale and b, the temporal location of an event. An intuitive physical explanation of equation above is very simple: Wa ,b ( f (t ),ψ ) is the ‘energy’ of f(t) of scale a at t=b. Because of this basic form of at+b involved in the transformation, it is also known as affine wavelet analysis. For specific applications, the basic wavelet function, ψ (t ) , can be modified according to special needs, but the form has to be given before the analysis. In most common applications, however, the Morlet wavelet is defined as Gaussian enveloped sine and cosine wave groups with 5.5 waves. Generally, ψ (t ) is not orthogonal for different a for continuous wavelets. Although one can make the wavelet orthogonal by selecting a discrete set of a, this discrete wavelet analysis will miss physical signals having scale different from the selected discrete set of a. Continuous or discrete, the wavelet analysis is basically a linear analysis. A very appealing feature of the wavelet analysis is that it provides a uniform resolution for all the scales. Limited by the size of the basic wavelet function, the downside of the uniform resolution is uniformly poor resolution. Although wavelet analysis has been available only in the last ten years or so, it has become extremely popular. Indeed, it is very useful in analyzing data with gradual frequency changes. Since it has an analytic form for the result, it has attracted

The Application of Wavelet Analysis Method

395

extensive attention of the applied mathematicians. Most of its applications have been in edge detection and image compression. Limited applications have also been made to the time-frequency distribution in time series and two-dimensional images. Despite versatile as the wavelet analysis is, we find that there are some problems with its applications, if these problems can be solved completely, we believe that wavelet transform will enrich its theory, have new substantial content in signal representation & reconstruction direction choosing & optimization and open up some perspective applied areas of wavelet analysis. 1.

2.

3.

The first problem with the most commonly used Morlet wavelet is its leakage generated by the limited length of the basic wavelet function, which makes the quantitative definition of the energy frequency time distribution difficult. The second problem is its counterintuitive. Sometimes, the interpretation of the wavelet can also be counterintuitive. For example, to define a change occurring locally, one must look for the result in the high-frequency range, for the higher the frequency the more localized the basic wavelet will be. If a local event occurs only in the low-frequency range, one will still be forced to look for its effects in the high-frequency range. Such interpretation will be difficult if it is possible at all. The third problem with the difficulty of the wavelet analysis is its non-adaptive nature. Once the basic wavelet is selected, one will have to use it to analyze all the data. Since the most commonly used Morlet wavelet is Fourier based, it also suffers the many shortcomings of Fourier spectral analysis: it can only give a physically meaningful interpretation to linear phenomena; it can resolve the interwave frequency modulation provided the frequency variation is gradual, but it cannot resolve the intrawave frequency modulation because the basic wavelet has a length of 5.5 waves.

In spite of all these problems, wavelet analysis is still the best available nonstationary data analysis method so far, therefore, we will use it in this paper as a reference to establish the validity and the calibration of the Hilbert spectrum.

3

The State-of-the-Art Review

The approach of using dynamic response and vibration characteristics for structure damage identification is the theoretical foundation of instrumental safety inspection methods. It has also been mainstream of research for more than thirty fears. Doebling et al. have reviewed the available literature of this approach. The practical problems associated with this approach have also been reviewed by Farrar and Doebling and Felber. In principle, each structure should have its proper frequency of vibration under dynamic loading. The value of this proper frequency can be computed based on wavelet transform and the elasticity properties of the structure. Sound as this argument is, the instrument inspection has never worked successfully. The reasons are many:

396

1. 2. 3. 4.

Jian Ping Li et al.

First, there is the lack of the precision sensors to measure the detailed dynamic response of the structure under loading. Secondly, there is a lack of existing data of bridges to be used as a reference state. Thirdly, there is a lack of proper data processing methods to process the structural response. Finally, there is a lack of sensitivity of the structure in response to the local damage, because of the large built in safety factor.

A damage up to 50% of the cross-section can only result in a few percentages vibration frequency changes. Such a small frequency shift, when processed with the conventional methods, would be totally lost in the inevitable noise in all the real situation. In the final analysis, many of the difficulties can be alleviated, if the data processing method can be made more versatile to handle highly transient and nonlinear vibration data. The wavelet analysis and wavelet transform theory will provide the good health monitoring for civil infrastructure.

4

Conclusion

Based on our analysis, we can conclude the following: 1.

2.

3.

We can indeed extract the bridge vibration signal from the noisy load test condition. The test condition was more complicated than our numerical model, but it is also realistic. Our analysis has extracted the structurally and dynamically significant components without any difficulties. We have also established that a realistic bridge is sensitive to a live transient load as a test for the structural integrity. The bridge vibrations in different directions at different instant and under a transient load indeed are sensitive enough to serve as indicators for the structural integrity test. We have also determined the structurally weak spot based on the Wavelet Transform (WT), the Empirical Mode Decomposition (EMD), and the Hilbert Spectral Analysis (HSA). Thus established the feasibility of the Nondestructive Instrument Bridge Safety Inspection System.

The data analysis method based on WT, EMD, and HSA is the most unique and at the forefront of the research in the data analysis[5,6]. It utilized not only the nonlinear characteristic of the response to determine the damage, but also the transient properties of the load to determine the damage location. Then, the free vibration frequency can be used to determine the extent of the damage. Considering the low number of the sensors required, and the efficient way of utilizing the data, HHT present a new data analysis alternative for bridge damage identification.

The Application of Wavelet Analysis Method

397

References 1.

2. 3. 4. 5. 6.

Huang, N. E: US Provisional Application Serial Number 60/023,411,August 14 (1966) and Serial No. 60/023,822 filed on August 12 (1996), Patent allowed March 1999 Huang, N. E.Z, Shen and S. R. Long: Adv. Appl. Mech., 32 (1996),59~117 Huang, N. E. et al: Proc. Roy. Soc. Lond., A454, (1998) 903~995 Huang, K: 1998a: US Patent Application Serial No. 09-210693, filed December 14 (1998) Jian Ping Li: Wavelet analysis & signal processing: theory, applications and software implementations, Chongqing Publishing House, Chonqqing (2001) Jian Ping Li, Yuan Yan Tang: The applications of wavelet analysis method, Chonqqing University Publishing House, Chongqing (2001)



          
     
      
              

      

           !  "  # $%&'( ')**+*',% -.# $%&'( ',*'+%/,0  # 1 233

334
      5     5     $6(           3     1     7      3       .     


  4  .3                       +    14 1+3  7      +    14 1+       7      7      73              7         

 7  

        1   3




  


                     !" #$      %                 & '$   (  ) *              +$ ," -  $    .     .      -          .   $        !/" #$          $       0  #.  $    %     .       .$       )


*           
   
  ℘ ℘
$  
$ ℘ !℘
= Φ $  ≠ $

"℘



=
$

)!*

∀

 Φ             .      $  .       &   $      1
 () = [() ⋅  ()] ∗

+∞

∑ ( −  ⋅ ℑ ) $ 

 →−∞

     ∈℘ 

 () =     

ℑ = ()  () $ ∈℘


)+*

∈℘


δ              $  $    $      1
 [ ] = ([ ] ⋅  [ ]) ∗

+∞

∑ [ −  ⋅ ℑ ] $ 

 →−∞



Y. Y. Tang et al. (Eds.): WAA 2001, LNCS 2251, pp. 398-403, 2001. c Springer-Verlag Berlin Heidelberg 2001

Piecewise Periodized Wavelet Transform and Its Realization

399

   ∈℘    [ ] =     

ℑ = [ ]  [ ]  ∈℘


),*

∈℘





             -  .    1        )2


*   1 +∞

∀ ∈ℜ $ …
$  (   ) =  ⋅

∫
 () ⋅

∗





 → −∞

 −      

)3*

       )4


*   1 ∀∈Z Z $ …
$

 [  ] =  − 
⋅ 

+∞

∫ 8 () ⋅  ( ⋅  





− 

−

)

⋅   $

)5*

 → −∞

 

              $         )4


*   1 ∀ ∈Z Z $ …
$ [  ] =

+∞

∑
 [ ]⋅  [ ⋅  




 → −∞




]

−  ⋅  
$

)6*

 
  
               $  0   . $    2


$ 4


  4


     -- 7                   --- -  -8$             &$       8



  

        

       )4
*  .  .    .  3$ 5"$  4


   .  .   
    . .$     !  !. &    4


$                $     . . . .     . 794      !  !     2


      !

 

   



                

&     $           $          % $    6" #$ 


$           $        %  & '$ 1   ' ⋅π ≥  ≥ 9 $ ℘ =     ⋅ω    ' ⋅π  $ ℘ =    9 >  ≥ −   ⋅ω    ' ⋅π  $ ℘
=     >   ⋅ω    ' ⋅π  $ ℘ =     < −   ⋅ω  

400

Wing-kuen Ling and Peter Kwung-Shun Tam  ⋅π ≥  ≥

 ⋅ω  ⋅π   >  ≥ −

  ≠

⋅ω   


⋅⋅ω
⋅    () =  
⋅⋅ω
⋅   

 

ℜℜ  :$$$$  ∗  () = 
⋅  ∀∈

);*

  1  : $  

 (9  ) = ' ⋅ π ⋅  ⋅ ω  ⋅ 
⋅⋅⋅ω
⋅ δ   +  ⋅ ω     : $  

 (:  ) = ' ⋅ π ⋅  ⋅ ω  ⋅ 
⋅⋅⋅ω
⋅ δ   +  ⋅ ω   

 ( ) =  ( ) = 

)= 2

where ψ ( x ) ∈ L

a

∞

∫ f ( x)Ψ ( −∞

x−b )dx , a

(1)

( R) is the wavelet function,

ψ a ,b ( x ) = ψ (x)

1

1

ψ(

a

x−b ) , a, b ∈ R, a ≠ 0 , a

(2)

satisfies ∞

∫ψ ( x)dx = 0 .

(3)

−∞

We call a set of subspaces

{V j } j∈Z and a function ϕ (x) as an orthogonal multi-

scale analysis if the following conditions are satisfied (1) V j ⊂ V j −1 , ∀j ∈ Z ; −

(2)

∩V j = {0} ,

∪V

j∈Z

j∈Z

j

= L2 ( R) ;

(3)

ϕ ( x) ∈ V0 , and {ϕ ( x − n)}n∈Z

(4)

f ( x) ∈ V j ⇔ f (2 x) ∈ V j −1 .

Based on the conditions (3) and (4),

f ( x) ∈ V0 ⇔ f (2 − j x) ∈ V j ,

is the normalized orthogonal bases of V0;

406

Hou Zunze

−

j 2

{2 ϕ (2 − j x − n)}n∈Z constructs a set of normalized orthogonal bases in the space Vj. The function ϕ (x ) is called the scale function in the multi-scale analysis, and φ (x ) constructed from ϕ (x) is the wavelet function. Let {V j } j∈Z be a given multi-scale analysis, ϕ (x ) and φ (x ) are corresponding there is an equivalent relationship and the function family

scale and wavelet functions, respectively, for a given

J 1 ∈ Z and function

f ( x) ∈ V J1 , there is the decomposition

f ( x) = AJ1 f ( x) = ∑ C J1 ,k ϕ J1 ,k ( x) ,

(4)

Denote

< ϕ J1 ,k , ϕ J1 +1,m >= hk − 2 m ,

(5)

< φ J1 ,k , φ J1 +1,m >= g k − 2 m ,

(6)

for an integer J2>J1, there is

f ( x) = AJ1 f ( x) = AJ1 +1 f ( x) + D J1 +1 f ( x) = AJ1 + 2 f ( x) + D J1 + 2 f ( x) + D J1 +1 f ( x) =...... J2

= AJ 2 f ( x) +

∑D

j

f ( x) ,

(7)

j = J1 +1

where ∞

∑C

A j f ( x) =

j ,m

ϕ j ,m ,

(8)

φ j ,m

(9)

m = −∞ ∞

D j f ( x) =

∑d

j ,m

,

m = −∞

On the other hand, ∞

C j ,m =

∑h k = −∞

k −2m

C j −1,k ,

(10)

Wavelet Transform and Its Application to Decomposition of Gravity Anomalies

407

∞

d j ,m =

∑g

k −2 m

C j −1,k ,

(11)

k = −∞

j=J1+1, J1+2, ……, J2

3

Let

Method of Wavelet Multi-scale Decomposition of Gravity Anomalies

{V } j

j∈Z

be an one-dimensional multi-scale analysis and its scale function is

denoted by φ and wavelet function by ψ. Denote

{ }

2

V j = V j ⊗ V j , then V j

2

j∈Z

form a two-dimensional multi-scale analysis. The 2-D scale function is defined by Φ(x,y)= φ(x)φ(y) ,

(12)

and the 2-D wavelet functions are defined by

Let

Ψ1(x,y)= φ(x)ψ(y) ,

(13)

Ψ2(x,y)= ψ(x)φ(y) ,

(14)

Ψ3(x,y)= ψ(x)ψ(y) ,

(15)

2

f ( x, y ) ∈ V J1 , following the principle of multi-scale analysis, we have 3

f ( x, y ) = AJ1 f ( x, y ) = AJ1 +1 f ( x, y ) + ∑ D J1 +1 f ( x, y ) , ε

(16)

ε =1

where

∑c

AJ1 +1 f ( x, y ) =

m1 , m2 ∈Z

J1 +1, m1 , m 2

∑dε

D ε J1 +1 f ( x, y ) =

Φ J1 +1, m1 ,m 2 ,

(17)

Ψ ε J1 +1,m1 ,m 2 ,

(18)

hk2 − 2m2c J 1,k1,k2 ,

(19)

J1 +1, m1 , m 2

m1 , m2 ∈Z

where

c J 1 + 1,m1,m2 =

∑h

k1 − 2m1

k1,k2 ∈Z

d 1 J1 +1,m1 , m2 =

∑h

k1 − 2 m1

k1 , k 2 ∈Z

g k 2 − 2 m2 c J1 ,k1 ,k 2 ,

(20)

408

Hou Zunze

∑g

d 2 J1 +1,m1 , m2 =

k1 − 2 m1

hk 2 − 2 m2 c J1 ,k1 ,k 2 ,

(21)

k1 − 2 m1

g k 2 −2 m2 c J1 ,k1 ,k 2 ,

(22)

− k)dx

(23)

k1 , k 2 ∈Z

∑g

d 3 J1 +1,m1 ,m2 =

k1 , k 2 ∈Z

where

hk =

1 2

+∞

x

∫ φ(2)φ(x

−∞

g k = (−1)k − 1 h 1 − k

(24)

Equation (5) can be further decomposed to the step of J2-J1 as J2

f ( x, y ) = AJ 2 f ( x, y ) +

3

Dε ∑∑ ε

j

f ( x, y ) ,

(25)

j = J1 +1 =1

where

A j f ( x, y ) =

∑c m1 ,m2 ∈Z

D ε j f ( x, y ) =

j , m1 , m 2

∑dε

Φ j ,m1 ,m 2 ,

j , m1 , m 2

Ψ ε j ,m1 ,m 2 ,

(26)

(27)

m1 , m2 ∈Z

j=J1+1,…,J2 . By letting ∆g(x,y)=f(x,y), we have the shorten decomposition expression ∆ g = AJG + D1G + D2G +... + DJG

(28)

Where D1G is denoted by the first order wavelet detail of the gravity anomalies, D2G is denoted by the second order wavelet detail of the gravity anomalies, and DJG the J-th order wavelet detail, AJG is denoted by the J-th order approximation of the gravity anomalies.

4

Application of the Wavelet Multi-scale Decomposition[10-12]

The method mentioned above is applied to decomposition of the Bouguer gravity field of China and the free-air gravity anomalies of the East China Sea. The data of Bouguer gravity field of China are picked from the Bouguer gravity map of China with scale 1:4,000,000, compiled by the Institute of Geophysical and Geochemical Exploration of Ministry of Geology and Mineral Resources. This map shows the latest regional gravity measurements using a grid of 40×40km. The first order wavelet detail of the gravity mainly reflects the density inhomogeneity of the

Wavelet Transform and Its Application to Decomposition of Gravity Anomalies

409

upper crust. From this map, one can see the difference of the upper crust between the eastern and the western parts, with the boundary from Helan mountain to Qionglai mountain. The western part shows string inhomogeneity of density striking west to east, while in the eastern part the inhomogeneity is weak and disperse. The second wavelet detail reflects density inhomogeneity of both the upper and the middle crust and so looks similar to the first order. One can see the differences of density between Yangtze and Huanan terrains and between Northeast and Huabei terrains. The third wavelet detail of the gravity anomalies mainly reflects density variation in the lower crust. The fourth order wavelet detail of the gravity anomalies mainly reflects density of the uppermost mantle in the eastern part of China. The fourth order wavelet approximation of the Chinese gravity field shows the trend of Moho fluctuation and density variation of largest scales. The gravity anomalies of the East China Sea covers 694 thousand square kilometers, spanning the East China shelf basin, Tiaoyu I. Folded doming-up belt, Okinawa trough basin, Ryukyu folded doming-up area and Ryukyu trench etc from west to east. According to previous studies, Moho surface rises step by from 28km to 16km from west to east. The East China Sea shelf basin consists of a lot of sags, Xihu sag famed in the world is one of them. For such a large and complex area, we apply the wavelet multi-scale analysis to decompose the free-air gravity anomalies. The first order transform detail shows that small circles with diameter of 10km or so are distributed a lot in the shelf or its west, the field value changes between (-5~10) ×10-5m/s2. Second order transform detail shows that the number of the contour circle is less than first order transform detail and the size is large than it, the field value changes between (-5~5) ×10-5m/s2. The scope of gravity anomalies of first and second order is small, it is small scale-gravity anomalies. According to the theory of analysis and result, they shall be related to the inhomogeneity of rock density in shallow stratum and some survey errors. The third order transform detail shows that the range of contour circle is a few hundred square kilometers, high contour value is 6×10-5m/s2 and low is -4×10-5m/s2 in the shelf and its west, it is supposed to be related to sediment thickness compared with the sediment of seismic interpretation. The sediment thickness in high value area is getting thin, that of low value is getting thick, the lowest value presents the center of sedimentary. In the middle part of the fourth order detail map, a high value belt tends towards north-northeast, the contour of (6~10) ×10-5m/s2 are distributed like a string of beads in the belt. The form and range of the whole high value area is in accord with Tiaoyu I. folded doming-up belt. In the west of high value belt is East China Sea shelf basin, its field is (-6~6) ×10-5m/s2 and the area of high, low value circles is up to 1800km2, the form and range of -2×105 m/s2contour is consistent with the seismic sedimentary form and range. To the east of high value belt and parallel with it, there is a low value belt, the contours of (-6~-8) ×10-5m/s2 are distributed like a string of beads. The form and range of the whole low value area consistent with Okinawa trough basin. Viewing the whole map, high value reflects lifts and low value reflects sags, the relations are marked in the map. As described above, the fourth order transform detail mainly reflects the lift of the sedimentary basement in East China Sea and adjacent regions. The fourth order wavelet approximation reflects the Moho surface in the area.

410

5

Hou Zunze

Conclusion

As a new mathematical tool, the wavelet transform enjoys many properties that other conventional mathematical methods cannot have. Wavelet transform is a powerful tool for multiple decomposition of gravity field. Wavelet multi-scale analysis technique has been successful in gravity anomalies decomposition of China and the East China Sea, etc.

References 1.

Hou Zunze: Calculation of gravity anomalies for multi-layer density interface. Computing Techniques for Geophysical and Geochenical Exploration (in Chinese). 10 (1988) 129-132 2. Li Shixiong and Liu Jiaqi: Wavelet Transform and Foundation of Math (in Chinese). Beijing, Geology Press (1994) 3. Liu Guizhong and Di Shuangliang: Wavelet Analysis and Its Application (in Chinese). Xi'an, Xi'an Electronics University Press (1992) 4. Hou Zunze and Yang Wencai: An operational research on the wavelet analysis. Computing Techniques for Geophysical and Geochenical Exploration (in Chinese), 17 (1995) 1-9 5. Daubechies, I.: The wavelet transform, time-frequency localization and signal analysis. IEEE TRANS. On Information Theory, 36 (1990)961-1006 6. Daubechies, I.: Orthonormal bases of compactly supported wavelets. Communications on Pure and Applied Math. XII (1988) 909-996 7. Daubechies, I.: Ten lectures on wavelets. Society for Industrial and Applied Math., Philadelphia, Pennsylvania (1992) 8. Mallat, S. And W. L. Hwang: Singularity detection and processing with wavelets. IEEE TRANS. On Information Theory, 38 (1992) 617-643 9. Mallat, S.: Multifrequency channel decompositions of image sand wavelet models. IEEE TRANS. On Acoustics, Speech and Signal Processing, 37 (1989) 2091-2110 10. Hou Zunze and Yang Wencai: Two-dimensional wavelet transform and multiscale analysis of the gravity field of China. Chinese J. Geophysics (in Chinese), 40 (1997) 85-95 11. Hou Zunze and Yang Wencai: Decomposition of crustal gravity anomalies in China by wavelet transform. 30th International Geological Congress. Beijing, China (1996) 12. Hou Zunze, Yang Wencai and Liu Jiaqi: Multi-scale inversion of density distribution of the Chinese crust. Chinese J. Geophysics (in Chinese), 41 (1998) 642-651

Computations of Inverse Problem by Using Wavelet in Multi-layer Soil Wu Boying1, Liu Shaohui1, and Deng Zhongxing2 1

Mathematics Department of Harbin Institute of Technology 150001, Harbin, the People’s Republic of China [email protected] 2 College of Applied Science Harbin University of Science and Technology 150080, Harbin, the People’s Republic of China

Abstract. In this paper we study the usage of wavelet in inverse problem multiplayer soil.We put forward a function and prove it is a wavelet function. Then we do theory analysis in detail about the application in computing soil parameters. At the same time, we do numerical experiments with two and three levels soil structure. The results indicate the valid of method

1

Introduction

Along with the development of electric power system capacity, the value of failure current flowing into ground has increased greatly. So grounding system is very important to ensure device and workmen safe. In design of substation grounding system, estimation of many main parameters is relevant closely to soil structure. In earlier years, designation of grounding system is based on considering soil as mean medium and simplified formulas, but it is impractical. Subsequently along with the development of computer technology F.P.Dawalibi[1]-[2]and Takehiko Takahashi[3] studied deeply the multi-layer soil. F.P.Dawalibi’s model paid more attention to the application of mathematics methods and accurate of calculation than physical sense. TakehikoTakahashi utilized the concept of templet in geognosy. Templets are finite and grounding parameters changes according to location, hence it can obtain the approximately parameters. In general, research on multiplayer soil is based constant current field theory. When current flows soil, each point satisfies the Laplace equation. Solving the equation, we can get the representation of electronic potential, and then getting the representation of apparent resistance ρ (r ) . If we expand it by Taylor expand method, then getting the series representation. Furthermore, we can obtain the parameters of multi-layer soil structure by observational data and least square method. But that method has drawbacks, such as the complexity of representation of ρ (r ) , the convergence of series and so on. Author improved on that method in [4]. He made use of Simpson formula in calculating representation of ρ (r ) , and transforming the representation of ρ (r ) in computation of parameters in Y. Y. Tang et al. (Eds.): WAA 2001, LNCS 2251, pp. 411-417, 2001.  Springer-Verlag Berlin Heidelberg 2001

412

Wu Boying et al.

multi-layer soil. The apparent resistivity ρ (r ) is relative to kernel function in analysis of soil structure as following fashion:

B (λ ) [3]

1 2

+∞

ρ (r ) = ρ 1 + 4 ρ1 r ∫ ( B(λ ) − B( 12 λ )) J 0 (rλ )dλ 0

(1)

The rest of this paper is organized as follows. First, we will clarify the terminology used for wavelet analysis. Secondly, we introduce a function and prove it be a wavelet. Next, we apply the wavelet in multi-layer soil structure. Finally, we will do experiments to verify our method feasibility.

2

The Proof of Wavelet

Definition 2.1 Let

ψ ∈ L2 ∩ L1

and

ψ ( 0) = 0 ,

then defining the set of function

{ψ a ,b } as −

1

ψ a ,b ( x ) = a 2 ψ (

x−b ), b ∈ R, a ∈ R − {0} a

We call them continuous wavelets generated by wavelet wavelet satisfying ψˆ (0)

ψ . Sometimes ones call

= 0 as base wavelet.

Definition 2.2 Let ψ be a base wavelet, 2.1. For

(2)

ψ a,b

is the continuous wavelet in definition

2

f ∈ L , wavelet transformation the signal or function f is defined as Wf (a, b) =< f ,ψ a ,b >= a

−

1 2

∫

R

For the sake of existence inverse transformation, admissibility condition, namely

f ( x)ψ (

x−b )dx a

ψ ∈ L2 ∩ L1

(3)

must be agree with

2

ψˆ (ω ) Cψ = 2π ∫ dω < ∞ R ω then

ψ is

(4)

admissible. According reference [6], one hand if admissibility condition

ψˆ (ω ) = 0 also holds, on the other hand, ψ (ω ) ≤ C (1 + ω ) −1−α , then ψ must be admissible.

holds, then

Definition 2.3 Suppose

if

ψˆ (ω ) = 0

f ( x) ∈ L2 [0,+∞) , wavelet ψ is defined as

holds, and

Computations of Inverse Problem by Using Wavelet in Multi-layer Soil

413

 xe − x cos( x), when x ≥ 0 0, when x < 0 

(5)

ψ ( x) =  Definition 2.4 Defining

{ψ a ,b } as −

1 2

ψ a ,b ( x ) = a ψ (

x−b ), b ∈ R, a ∈ [0,+∞) a

(6)

where ψ is defined in Definition 2.3. Our next work is to prove ψ be a wavelet. It is obvious that ψˆ (0) = 0 .That we prove ψ be a wavelet is equivalent to prove ψ agree with admissibility condition by term of discussion above. Propersition 2.1 Existing constan C and α > 0 make ψ ( x ) satisfy

ψ ( x) ≤ C (1 + x ) −1−α Definition 2.5 If function f { f } ⊆ [0,+∞), {ψ a ,b } is defined by (6)

Wf (a, b) =< f ,ψ a ,b >= a

−

1 2

∫

R

f ( x)ψ (

Theorem 2.1 According definition above, for all +∞

+∞

0

−∞

∫ ∫

Wf (a, b)Wg (a, b)

x−b )dx ,where α > 0 a

(7)

f , g ∈ L2 [0,+∞) We have

da db = Cψ < f , g > a2

(8)

Meantime, have inverse formula

f ( x) =

1 Cψ

+∞

+∞

0

−∞

∫ ∫

Wf (a, b)ψ a ,b ( x)

dadb a2

(9)

2

where

Cψ = 2π ∫

+∞

0

ψˆ (ω ) dω ω

Note1 we evaluate the value of fourier transform of wavelet and constant well known know

∫

+∞

0

2

xe − ax cos(bx)dx =

2

Cψ .It is

a −b (a > 0) . So ones are easy to (a 2 + b 2 ) 2

414

Wu Boying et al.

1

ψˆ (ω ) =

(ω 10 + 4ω 8 + 8ω 6 + 32ω 4 + 16ω 2 + 64) 2 ×ω (ω 4 + 4) 2 2π 1

ω ω2 +4 = 4 2π (ω + 4) 1

Note that

1 (π + 1) . 16

Cψ =

2 −

α

γ

Theorem 2.2 If ψ satisfies C1: ψˆ (ω ) ≤ C ω (1 + ω ) 2 , α > 0, γ > α + 1 , [7]

where

C

∑ ψˆ (2

−k

is

a

constant,

and

if,

2

for

ω ) ≥ α > 0 , then there must be b0

all

ω ≠ 0 ,ψˆ satisfies

C2:

making ψ k ,n ( x ) constitute frame

k =Z 2

of L [0,+∞ ) . So In order to prove wavelet defined in this paper constituting frame , we only need to verify the condition C1.First of all, the condition C2 is obvious. Next ,we verify condition C1. It is easy to show

there

exist

ω ω2 + 4 know ψˆ (ω ) = ,so only to 4 2π ω + 4 C > 0, α > 0 and γ > α + 1 and constant 1

γ

1 ω ω2 +4 α 2 − ≤ C ω (1 + ω ) 2 holds. And this is obvious Those 4 2π ω + 4 constants exist indeed, for example, C = 1, α = 1, γ = 2.1 > α + 1 . The last issue is to compute the dual wavelet in practical application. We adopt the method as Daubechies’s[6]. For convenience of discussion, we only select the first term in approximation. We have calculated the frames, and the specific values are 0.359 and 0.375.

3

Application in Multi-layer Soil

In introduction , we can transform the formulas into evaluating integral of

∫

∞

0

ρ (r ) J 0 (λr )dr .

Now, we evaluate this integration by the wavelet in

Computations of Inverse Problem by Using Wavelet in Multi-layer Soil

section

∫

∞

0

ρ (r ) J 0 (λr )dr . ρ (r ) =

section 2.Suppose

415

Now, we evaluate this integration by the wavelet in

∑ρ

ψ mn (r ) ,then we have:

mn

m , n∈Z

ρ ,ψ

Theorem 3.1 The definitions of

∫

∞

0

ρ (r ) J 0 (λr )dr

∑ ρ mn 2

=

m −1 2

m , n∈Z

+ e n (1−i ) −

as above, then :

∑

 n (1+i ) 1+ i e  (1 + i ) 2 + (2 m λz ) 2

[

1− i

[(1 − i)

ρ mn 2

2

m −1 2

+ ( 2 m λz ) 2

∫

− ne n (1−i ) 2

1

− ne n (1+i )

[(1 + i)

2

1

[(1 − i)

2

+ (2 m λ ) 2

]

1

2

+ (2 m λ ) 2

  2 

m

2 n

0

m , n∈Z n ≥0

]

3

]

3

ψ mn (r ) J 0 (λr )dr

holds. Proof: Because the space is limited, We only explain the main thought.

∫

∞

0

∞

ρ (r )J 0 (λr )dr = ∑ ρ mn ∫ ψ mn (r ) J 0 (λr )dr 0

mn

We devide the right term of this formulation into two parts, then computing them respectively. In computational process, the key is to compute

∫

∞

0

ψ mn (r ) J 0 (λr )dr .

Subsequently, we compute it. According to the lipschitz quadrature formula of Bessel function, namely

∫

∞

0

∫

∞

0

e −ar J 0 (λr )dr =

re −ar J 0 (λr )dr =

a ( a 2 + λ2 )

3 2

1 2

a + λ2

, we can obtain

. Then we use it and get the theorem 3.1.

Note 2. In the theorem although including complex numbers, it is obvious that the complex is conjecture ,therefore, the resulting is a real number. Note 3. Because we know only values at those discrete points, and its interval is [0,200]. This is a bounded function by expertise knowledge, but its value is large. And wavelet diminishes at infinite together with applied goal, we think instead of thinking, namely,transforming the interval [0,200] into interval [0,2] makes the approximation more exact.

416

4

Wu Boying et al.

Numerical Experiment and Conclusion

In order to verify our method , we do experiments with three-layer soil structure. The soil’s structure is three layers, we select h1 = 5m, h2 = 15m,

ρ1 = 1000Ωm, ρ 2 = 2000Ωm, ρ 3 = 1000Ωm,

the Figure 2,3and 4 show the

numerical results, unit is 100 Ωm . In this paper we construct a function and prove it be wavelet. Then we apply it in inverse problem of soil structure and make theory analysis in details. In the last, we do experiments to verify our method, the results indicate our method is feasible and valid. The design of substation grounding system is based on the analysis of soil structure, and it’s the degree of analysis is the key issue. But wavelet’s applications in this research area are very rare. So there are many open issues to study and research, we hope our work can advance this aspect research work.

Fig. 1. The apparent resistivity in three levels

Fig. 2. The numerical solution

Fig. 3. The relative error of three levels

References 1. 2.

3.

F. Dawalibi, C. J. Blattner. Earth Resistivity Measurement interpretation techniques. IEEE T-PAS.103(1984) 374-384 F. Dawalibi, N. Barbeito. Measurement and Computations of the performance of grounding system buried in multiplayer soil. IEEE Transactions on power Delivery. 6(1991) 1483-1490 T. Takahashi, T. Kawase. Analysis of apparent resistivity in a multi-layer earth structure. IEEE. T-PWRD. 5(1990) 604-612

Computations of Inverse Problem by Using Wavelet in Multi-layer Soil

4. 5. 6. 7.

417

Jiang Gao, the inverse problem of multi-layer soil structure, dissertation of master’s degree in Peking University 2000 Li Zhongxin, simulative computation of substation grounding based on complex image method. Dissertation of doctor’s degree in Tsinghua University, 1999 1-24 I. Daubechies. Ten lectures on wavelets. SIAM, 1992 53-107 C. K. Chui. An introduction to wavelets, Academic press, 1992:86-98

Wavelets Approach in Choosing Adaptive Regularization Parameter⋆ Feng Lu, Zhaoxia Yang, and Yuesheng Li Department of Scientific Computing and Computer Applications Zhongshan University, Guangzhou 510275, P. R. China

Abstract. In noise removal by the approach of regularization, the regularization parameter is global. Constructing the variational model min
f − g
2L2 (R) + αR(g),g is in some wavelets space. Through the g

wavelets pyramidal decompose and the different time-frequency properties between noise and signal, the regularization parameter is adaptively chosen, the different parameter is chosen in different level for adaptively noise removal. Keywords: Sobolev space, wavelet, noise, adaptive.

1

Wavelets and Discrete Equivalent Norm of Sobolev Space

The model of noisy image is: f = f0 + η

(1)

where f0 is original clean image,η is Guassian noise. Our task is to restore the original image f0 as possible. The regularization approach is always adopted to solve these problems, we consider the variational problems of the form: min
f − g
2L2 (R2 ) + αR(g) g

(2)

where g ∈ X; X ⊂ L2 (R2 ) X can be chosen as Sobolev space, Besov space ,Lipschitz space and so on, the sobolev space is chosen as X in this paper. α is regularization parameter that determines the trade-off between goodness the fit to the measured data, and the amount of regularization done to the measured image. In (2), the parameter is global, that the regularization parameter is the same number everywhere. In reference [4], the regularization parameter is chosen as a changeable number with the different gradient in some image. In [2] and [3], to choose the proper parameter, the Besov spaces of minimal smoothness can be embedded in L2 (R), and can get the discrete wavelets equivalent norm. ⋆

This work is supported by Natural Science Foundation of Guangdong (9902275), Foundation of Zhongshan University Advanced Research Centre.

Y. Y. Tang et al. (Eds.): WAA 2001, LNCS 2251, pp. 418–423, 2001. c Springer-Verlag Berlin Heidelberg 2001


Wavelets Approach in Choosing Adaptive Regularization Parameter

419

we can easily construct the two dimensional wavelets from one dimensional wavelets ψ and scale function φ by setting for x := (x1 , x2 ) ∈ R, ψ (1) (x1 , x2 ) := ψ(x1 )φ(x2 ); ψ (2) (x1 , x2 ) := φ(x1 )ψ(x2 ); ψ (3) (x1 , x2 ) := ψ(x1 )ψ(x2 ); If we let Ψ := {ψ (1) , ψ (2) , ψ (3) }, then the set of functions ψj,k (x) := 2k ψ(2k x − j)ψ∈Ψ,k∈Z,j∈Z 2 forms an orthonormal
basis for L2 (R2 ), that is, for every f ∈ L2 (R2 ), there are coefficients cj,k,ψ := R2 f (x)ψj,k (x)dx such that  cj,k,ψ ψj,k f= j∈Z 2 ,k∈Z,ψ∈Ψ


f
2L2(R2 ) =



c2i,k,ψ

(3)

j∈Z 2 ,k∈Z,ψ∈Ψ

In reference [2], the discrete equivalent norm of Sobolev Space is:   22βk |cj,k,ψ |2
f
2W β (L2 (R2 )) ≈

(4)

k≥0 j∈Z 2 ψ∈Ψ

where β is the smoothness order of the Sobolev Space. It is an excellent property that a Space Norm can be expressed by the discrete sequence, especially the wavelets coefficient sequence, it makes many problems easier largely.

2

Variational Model and Its Wavelets Solution

From previous work of regularization approach, we can choose the model as follow: (5) min{
f − g
2L2 (R) + α
g
2W 2 (L2 (D)) } g

2

where α > 0,W SobolevSpace with two-order smoothness.  (L2 (D) represents Let: f = j,k,ψ cj,k,ψ Ψj,k , g = j,k,ψ dj,k,ψ Ψj,k , From (4),(5) can be expanded as:  (|cj,k,ψ − dj,k,ψ |2 + α · 24k |dj,k,ψ |2 )) (6) j,k,ψ

In reference [6],Donoho points out that for the spectrum analysis of a noisy real image, the spectrum corresponding with the noise is quite small, while the spectrum corresponding with the original image is quite large. (See Figure 1) It means that the ”energy” of the noisy image is always ”concentrate” on the original image. Because of the wavelets’ better property of Locality in both time and frequency domain, the wavelets can concentrate the energy, that is, in wavelets transform domain, the energy of original image concentrate on some highlight

420

Feng Lu et al.

lines, while almost zeros else where. But for the noise, it is quite different. The wavelets coefficients corresponding with noise is always small, even almost zeros, in every level in wavelets transform domain, and its distribution is quite uniform in all levels. So it is a new way to choose the regularization parameter not as a constant, but changeable with the wavelets coefficients.

Fig. 1. Left:Original Image,

Right:Wavelets Coefficients

We can construct the new variational model with changeable parameter:  (|cj,k,ψ − dj,k,ψ |2 + α(cj,k,ψ ) · 24k |dj,k,ψ |2 )) (7) j,k,ψ

where α(t) > 0,t ∈ {cj,k,ψ } is the wavelets coefficient, W 2 (L2 (D) represents SobolevSpace with two-order smoothness. Here, the regularization is not a constant, but a changeable variable with wavelets coefficients. From this model, we can handle different level with wavelets decomposition with different regularization, when the wavelets coefficient is large, choosing the regularization parameter small for containing more original image , when the wavelets coefficient is small, choosing the parameter large for removing the noise much. So, we can get the regularization image adaptively which containing the information of original image more and removing the noise as well. Hence, two conditions must be satisfied for choosing regularization parameter: (1) lim α(t) = 0 t→∞

(2) lim α(t) = 1 t→0

Wavelets Approach in Choosing Adaptive Regularization Parameter

421

In practice, because the wavelets coefficients corresponding with the noise is quite small, we choose function α(t) with decaying rapidly. For example: 2

α(t) := e−t , α(t) :=

2

Fig. 2. Left:α(t) := e−t

1 (1 + t2 )

Right:α(t) :=

1 (1+t2 )

In reference [7], the formula of window size of decaying function is:  ∞ 1 x2 |α(x)|2 dx} { △α :=
w
2 −∞ Let α(t; m, s) := mα( st ), to meet the practices, we can change the Support Set and Amplitude through choosing the proper m, s. For every j, k, ψ, each term of (7) |cj,k,ψ − dj,k,ψ |2 + α(cj,k,ψ ) · 24k |dj,k,ψ |2 ≥ 0

(8)

Hence, one minimizes (7) just by minimizing separately over dj,k,ψ : |cj,k,ψ − dj,k,ψ |2 + α(cj,k,ψ ) · 24k |dj,k,ψ |2 for each j, k and ψ. Let:s := cj,k,ψ , v := dj,k,ψ ,and supposing v ≤ s, (8) can be reduced to: F (v) := |s − v|2 + α(s) · 24k v 2

(9)

Calculating the derivation of F (v) for v, we can get the minimizer of (9): v=

s 1 + α(s) · 24k

(10)

422

Feng Lu et al.

After calculating (10) for all the wavelets coefficients of all levels, we can get the new wavelets coefficients from regularization processing. Hence, we can get the restored image by wavelets reconstruction.

3

Experiments

An image Bird.bmp is adopted in the experiment, we choose the Haar wavelets 2 and α(t; m, s) := mα( st ) = me−(t/s) .

Fig. 3. Left:Original Image of Bird.bmp, white noise, variance δ 2 = 18

Right:Nosiy image with Gaussian

Fig. 4. Left:Restored image with removing two first level of wavelets coefficients, Right:Restored image with adaptive approach, where m = 0.8; s = 10

Wavelets Approach in Choosing Adaptive Regularization Parameter

4

423

Conclusion

Using the approach of adaptive changeable regularization parameter in image restoration, it is more flexible to choose the model. We can choose the spaces with more smoothness order which have powerful ability in noise removal, at the same time, choosing changeable regularization function to containing more details and removing more noise.

References 1. A. N. Tikhonov and Vasiliy Y. Arsenin Solution of ill-posed problems, V. H. Winston & Sons Press, 1997; 2. R. A. Devore Fast wavelet techniques for near-optimal image processing, IEEE Military Communications Conference Record, 1992, P1129-1135; 418, 419 3. R. A. Devore, Image compression through wavelet transform coding, IEEE Transactions on Information Theory, vol. 38, 1992, P719-746; 418 4. Adaptive regularized constrained least squares image restoration, IEEE trans. on Image Processing, 1999, P1191-1203; 418 5. I. Daubechies, Ten lecture on wavelets CBMSNSF Series in Applied Math #61, SIAM, Pub1., Philudelphia, 1992; 6. Donoho D. L. De-noising by soft-thresholding, IEEE Trans. on Information Theory, 1993, 41(3); 419 7. Chui C. K. An Introuduction to wavelets, Xi’an Jiaotong Univ. Press, 1994. (in chinese) 421

DNA Sequences Classification Based on Wavelet Packet Analysis* Jing Zhao1, Xiu Wen Yang1, Jian Ping Li1, and Yuan Yan Tang2 1

International Centre for Wavelet Analysis and Its Applications, Logistical Engineering University, Chongqing 400016, P. R. China [email protected] 2 Department of Computer Science, Hong Kong Baptist University, Hong Kong [email protected]

Abstract. The classification of two types of DNA sequences is studied in this paper. 20 sample artificial DNA sequences whose types have been known are given to recognize the types of other DNA sequences. Wavelet packet analysis is used to extract the features of the sample DNA sequences.

1

Introduction

Each DNA sequence is a permutation of 4 codes: a, t, c and g. Studying the structure characters of DNA sequences is one of the most important problems in Bioinformatics. In this paper, the classification of two types of DNA sequence which are Exon and Intron, is studied by means of wavelet packet analysis. We have 20 artificial DNA sequence samples whose types have been known, in which No.1-10 are Exons(type A) and No. 11-20 are Introns(type B). All of the lengths of these 20 samples are about 110. Wavelet packet analysis is used to extract the features of the sample DNA sequences and to recognize the types of other DNA sequences.

2

Changing DNA Sequence to Number Sequence

In order to study DNA sequence with wavelet packet decomposition, we make every code of one DNA sequence correspond to one number as following:

*

This work was supported by the National Natural Science Foundation of China under the grand number 69903012 and 69682011.

Y. Y. Tang et al. (Eds.): WAA 2001, LNCS 2251, pp. 424-429, 2001.  Springer-Verlag Berlin Heidelberg 2001

DNA Sequences Classification Based on Wavelet Packet Analysis

 0 . 25  0 .5  yi =   0 . 75  1

425

xi ='a' xi =' g ' xi = 'c'

(1)

xi = 't'

where xi is the i-th code of the DNA sequence. In this way, one DNA sequence x is changed to a number sequence y. And the number sequence y can been seen as a onedimensional signal.

3

Performing Wavelet Packet Decomposition

The wavelet packet analysis is a generalization of wavelet decomposition that offers a richer range of possibilities for signal analysis. In wavelet packet analysis, the details as well as the approximations can be split. It is easy to generate wavelet packets by using an orthogonal wavelet. We start with the two filters of length 2N,denoted h(n) and g(n), corresponding to the wavelet. They are respectively the reversed versions of the low-pass decomposition filter and the high-pass decomposition filter divided by 2 . Now we define the sequence of wavelet packets Wn(x) (n=0,1,2,…) by: 2 N −1

W2 n ( x) = 2 ∑ h(k )Wn (2 x − k ) k =0 2 N −1

(2)

W2 n +1 ( x) = 2 ∑ g (k )Wn (2 x − k ) k =0

where

W0 ( x) = φ ( x) is the scaling function and W1 ( x) = ψ ( x) is the wavelet

function. Here, for the corresponding number sequence y of each sample DNA sequence x, we compute its wavelet packet decomposition for the original Daubechies3 wavelet at level 3. Because the sampling number 110 of sequence y is small, we increase its sampling number to 10 times as the original sampling number by computing linear interpolation by every 0.1 before performing wavelet packet decomposition.

4

Reconstructing Wavelet Packet Coefficients

Now we compute the reconstruct signals of the wavelet packet coefficients we got by performing wavelet packet decomposition.

426

Jing Zhao et al.

y37

y36

y35 y34

y33 y32 y31

y30

y

1 0.5 0 20 1 0 0.50 0 -0.5 0.10 0 -0.1 0.10 0 -0.1 0.050 0 -0.05 0.050 0 -0.05 0.10 0 -0.1 0.050 0 -0.05 0

200

400

600

800

1000

1200

200

400

600

800

1000

1200

200

400

600

800

1000

1200

200

400

600

800

1000

1200

200

400

600

800

1000

1200

200

400

600

800

1000

1200

200

400

600

800

1000

1200

200

400

600

800

1000

1200

200

400

600

800

1000

1200

Fig. 1. The number sequence y and its reconstruct signals of the wavelet packet coefficients

For example, the corresponding number sequence y of DNA sequence x='aggcacggaaaaacgggaataacggaggaggacttggcacggcattacacggaggacgaggtaaaggaggcttg tctacggccggaagtgaagggggatatgaccgcttgg' and its reconstruct signals of the wavelet packet coefficients are shown in Figure 1, where y30, y31, y32, y33, y34, y35, y36, y37 respectively represents the reconstruct signal of AAA3, DAA3, ADA3, DDA3, AAD3, DAD3, ADD3, DDD3.

5

Computing the Total Energy of Every Reconstruct Signal

The corresponding total energy of signal y3j(j=0,1,2,…, 7) is as following: 2

n

E3 j = ∫ y3 j (t ) dt = ∑ y jk

2

j=0,1,2,… , 7

(3)

k =1

where

y jk ( j=0,1,2,…, 7, k=1,2, …, n) represent the numerical value of the

reconstruct signal y3j at discrete points. The total energies of the corresponding number sequences of the 20 sample DNA sequences are shown in Table 1.

DNA Sequences Classification Based on Wavelet Packet Analysis

427

Table 1. The total energies of the corresponding number sequences of the 20 sample DNA sequences

1 2 3 4 5 6 7 8 9 10

11 12 13 14 15 16 17 18 19 20

E30 19.0235 0.1638 19.4844 0.2220 18.0249 0.1842 20.6853 0.1878 19.4273 0.2244 18.2110 0.1364 19.1787 0.1594 20.1857 0.1504 20.3459 0.1328 20.4854 0.2717 24.1353 0.2135 24.3336 0.1871 25.4134 0.1970 24.7997 0.2023 26.7044 0.1987 23.7204 0.2103 21.5479 0.2070 24.9260 0.2133 26.5463 0.2083 26.8730 0.1912

E31 1.3277

E32 0.2574

E33 E34 E35 E36 0.0913 0.1065 0.3539

E37 0.1683

1.2462

0.3148

0.4194

0.1206

0.0982

0.1786

1.3403

0.2709

0.3714

0.1009

0.0982

0.1640

1.6786

0.2926

0.4110

0.1032

0.1220

0.1878

1.1339

0.3232

0.4199

0.1219

0.1056

0.1890

1.1456

0.2268

0.3093

0.0733

0.0923

0.1364

1.3396

0.2692

0.3605

0.0903

0.1102

0.1717

1.2496

0.2498

0.3361

0.0861

0.1046

0.1641

0.9324

0.2327

0.3030

0.0715

0.0968

0.1461

1.0851

0.3876

0.5042

0.1482

0.1314

0.2016

1.7968

0.3697

0.5483

0.1481

0.2076

0.2174

1.4035

0.2676

0.4055

0.1032

0.1164

0.1726

1.4337

0.3379

0.4696

0.1231

0.1771

0.1975

1.5487

0.3036

0.4183

0.1082

0.1234

0.1850

1.4602

0.2979

0.4144

0.1058

0.1242

0.1789

1.7926

0.3871

0.5460

0.1506

0.2172

0.2256

1.6151

0.3338

0.4828

0.1291

0.1733

0.1990

1.8311

0.3747

0.5506

0.1501

0.2079

0.2200

1.6933

0.3057

0.4361

0.1165

0.1267

0.1939

1.5491

0.3545

0.5023

0.1419

0.2078

0.2034

428

6

Jing Zhao et al.

Extracting Features of the Sample DNA Sequences

The sample DNA sequences 1-10 are belong to type A and 11-20 are belong to type B. From Table 1 we can see the features of the sample DNA sequences as following: 1. 2.

Energy E30 contains the main energy of the corresponding number signal of the DNA sequence. Energy E30 of type A and type B has an outstanding difference.

For type A, the mean of E30 is 19.5052 and the maximum number is 20.6853. For type B, the mean of E30 is 24.9000 and the minimum number is 21.5479. So E30 of type A is obviously smaller than that of type B. Let AEmax represent the maximum number of E30 of the sample DNA sequence of type A, BEmin represent the minimum number of E30 of the sample DNA sequence of type B, YE30 represent E30 of the corresponding number signal of a DNA sequence X whose type is unknown. From above discussion, we get the classification regulation: X belongs to type A, if YE30 ≤ AEmax; X belongs to type B, if YE30 ≥ BEmin; X belongs to type A, if AEmax ≤ YE30 ≤ BEmin, and YE30-AEmax ≤ BEmin-YE30; X belongs to type B, if AEmax ≤ YE30 ≤ BEmin, and YE30-AEmax ≥ BEmin-YE30.

7

Experiments

Here we have another 20 artificial DNA sequences and 182 natural DNA sequences whose types have been known. Now we try to recognize the types of these DNA sequences using the given 20 sample DNA sequences and the classification regulation. 7.1 Classification of 20 Artificial DNA Sequences The lengths of the 20 given artificial DNA sequences whose serial number are from 21 to 40 are about 110, almost the same with those of the 20 sample DNA sequences. So, as the same with the 20 sample DNA sequences, for the corresponding number signal of each DNA sequences, we increase its sampling number to 10 times as the original sampling number by computing linear interpolation by every 0.1 before performing wavelet packet decomposition. Here, because AEmax =20.6853 and BEmin =21.5479, we could recognize the types of the 20 artificial DNA sequences. In the 20 DNA sequences, only one has been recognized as a wrong type. So the successful rate of the classification regulation for artificial DNA sequences, whose lengths are about the same with those of the sample DNA sequences, is 95%.

DNA Sequences Classification Based on Wavelet Packet Analysis

429

7.2 Classification of 182 Natural DNA Sequences The lengths of the 182 given natural DNA sequences are from 1061 to 21246. So these natural DNA sequences are much longer than the sample DNA sequences. In order to compare their energies, we must let them have about the same lengths. So, in order to recognize the type of each natural DNA sequences, at first we must do linear interpolation for the corresponding number signal of every sample DNA sequence so that its length become the same with the natural DNA sequence. Secondly, we perform wavelet packet decomposition for this natural DNA sequences and all of the sample DNA sequences to get AEmax, BEmin and E30. At last, we recognize the type of this natural DNA sequence by the classification regulation. For all of the given 182 natural DNA sequences, 47 DNA sequences are recognized as wrong types, 135 DNA sequences are recognized as right types. The successful rate is 74%. In order to find the reason that the successful rate for natural DNA sequences is lower than that of artificial DNA sequences, we analyzed the results of 182 natural DNA sequences. We see that when the length of DNA sequences becomes longer than 8000, the successful rate decreases obviously. This may be explained as that when the DNA sequence is much longer than the sample DNA sequences, the information of sample DNA sequences become not enough to recognize the given DNA sequence.

References 1.

2.

Yuan Yan Tang, Lihua Yang, Jiming Liu,Hong Ma: Wavelet Theory and Its Application to Pattern Recognition, World Scientific Publishing Co.Pte.Ltd, Singapore (2000) Dazhi Meng: Construction and Simplified Model of DNA Sequences, Mathematics in Practices and Theory, 1(2001) 54-58

The Application of the Wavelet Transform to the Prediction of Gas Zones* Xiu Wen Yang1, Jing Zhao1, Jian Ping Li1, Jing Liu2, and Shun Peng Zeng2 1

International Centre for Wavelet Analysis and Its Applications, Logistical Engineering University, Chongqing 400016, P.R.China [email protected] 2 Chongqing Petroleum College, Chongqing 400042, P.R.China Abstract. An accurately evaluate about the zone number and position of the gas zone is put forward in this paper. It provides the reliable basis for developing natural gas through synthetically analyzing the result of carrying on wavelet de-noising and wavelet package denoising disposal simultaneously to the density porosity curve and neutron porosity curve. If there is natural gas in the void of underground reservoir, it can increase the density well logging porosity φ D and decrease the compensation neutron porosity φCNL . As long as overlapping these two kinds of porosity curves we can directly determine the zone meeting the condition of φ CNL < φ D is that one containing gas. While because of noise signals are contained in most of the well logging traces, small saw teeth will take place in the curves caused by some factors. Though this phenomenon is independent of the character of the zone, either of the explanation of the single curve or the two overlapping curves may run into obstacle, and makes the evaluation lack of accuracy. So it is obviously important to control the noise of the well logging traces. Despite a few ways existing for a long time in low frequency filter on the well logging traces, the rate of distinguish of the curves are reduces after dispose, so we can not explain gas zones effectively. Wavelet analysis, which has extensive application on the aspect of signal analysis and graph disposal, is a new developing branch of mathematics in recent years and achieves noticeable success. In this paper, in order to remove the signal noise of φ CNL and φ D , the one-dimension method of wavelet denoising and wavelet package denoising is used to achieve the purpose of prediction of parameters about gas zone.

*

This work was supported by the National Natural Science Foundation of China under the grand number 69903012 and 69682011.

Y. Y. Tang et al. (Eds.): WAA 2001, LNCS 2251, pp. 430-434, 2001.  Springer-Verlag Berlin Heidelberg 2001

The Application of the Wavelet Transform to the Prediction of Gas Zones

1

431

The Principle of Wavelet Transform

The one-dimension signal-de-noising disposal is the one of the important applications of the analyses of wavelet denoising and wavelet package denoising. The basic principle is as follow: A basic model of si containing noise signal:

si = f i + σ zi

i = 0,1,!, n − 1

fi is the real signal, the part of noise is zi , which is often called Gauss vacant noise N (0,1) , σis the noise grade. The purpose of removing noise is to reduce the value of noise part and recover the real signal fi. The Steps of Wavelet De-noising: 1.

2.

3.

The decomposition of the one-dimension wavelet signal Choose a wavelet of Sym8 and decide the number of layers of wavelet decomposition N=5, then decompose the one-dimension signal for the N zone. To quantitatively determine the threshold of high frequency coefficient We adopt the principle of maximum and minimum to choose the threshold, as can achieve the minimum of the maximum mean square error. Quantitatively dispose high frequency coefficient of every zone according to the soft threshold from first to fifth layer. To recompose the one-dimension wavelet According to the low frequency coefficient of the fifth zone and the high frequency coefficient after being modified from first to fifth zone we can calculate the recomposition of one-dimension wavelet.

The idea of denoising by using wavelet package is as nearly same as that of wavelet denoising. The only difference between them, allowing wavelet package subdivide and quantitatively determine the threshold of parts of both low and high frequency simultaneously, lies in the more complicate and more flexible analysis way the wavelet package provides. The steps of wavelet package de-noising: The decomposition of one-dimension wavelet package. 1. 2. 3.

4.

To choose a wavelet of Sym8 and decide the zone of wavelet decomposition N=5, then decompose the one-dimension signal for the N zone wavelet package. To compute the optimum tree (namely determine the optimum wavelet package base).The optimum tree is computed based on the minimum entropy criterion. To quantitatively determine the threshold of wavelet package decomposition coefficient We adopt the principle of maximum and minimum to choose the threshold and quantitatively decide the threshold of each wavelet package decomposition coefficient, especially the low frequency decomposition coefficient. To recompose the wavelet package.

432

Xiu Wen Yang et al.

According to the fifth zone wavelet package decomposition coefficient and the quantitatively disposed coefficient, the signal wavelet package can be recomposed.

2

An Example

We take a well logging trace of carbonate section in an oil field as the example. 1.

Collect materials for well logging traces such as Microlog, Compensated neutron log, Densilog, Spontaneous potential log, Gamma-rag reading, well diameter, etc.

2.

Calculate the density porosity

3.

φD =

ρ ma − ρ b ρ ma − ρ f

by using the density curve material,

ρb

is the reading of Bulk density of fermation,

ρf

is the Density of fluid in the void.

ρ ma is the density of matrix and

Plot the original curve overlap involving two curves of density porosity

φD

and

φ

neutron porosity CNL (fig. 1). According to the curve overlap and infiltrative zone in corresponding section, because of the noise in the curves we can only find a gas zone, called A zone which is at the position from 2678 to 2690 meters. Whereas, the accurate position of the bottom and top interfaces of the gas zone can not be made sure. Moreover the two sites, one of which is 2635 to 2640 meters and another 2660 to 2675 meters, are not easy to be determined whether they are gas zone or not. Note: In Fig. 1, Fig. 2, Fig. 3, blue curve indicate density porosity curve, red line indicate neutron porosity curve. 4.

5.

On the basis of the fig. 1 (original curve), through wavelet denoising disposal to the density porosity curve and neutron porosity curve (fig. 2 wavelet denoising disposal curve), we can directly find two zones. The position of top interface of one called A is still inexplicit. Although another zone called B is 2659 to 2672 meters, the position from about 2635 to 2640 meters is hard to be determined whether it is gas zone or not. According to fig. 1, through wavelet package de-noising disposal to the density porosity curve and neutron porosity curve (fig.3 wavelet package de-noising disposal curve), we can obviously find there are three gas zones which are A from 2677 to 2691 meters, B from 2659 to 2672 meters and C from 2636 to 2641 meters respectively. The result of B from wavelet denoising and wavelet package denoising is very consistent. If the results of the interface of one or two zones of three have small difference between two ways above, we can use the average of them to make the result more accurate and reasonable.

The Application of the Wavelet Transform to the Prediction of Gas Zones

3

433

Conclusions

The influence of noise can cause inaccuracy in predicting the bottom and top interfaces by directly using the original well logging trace (fig.1) which contains lots of “small saw teeth”, etc noises independent of the character of the zone, even can omit some zones. The way of wavelet denoising keeps more main formation in the original curve but it cannot predict accurately the gas zones. Only by analyzing the result from wavelet package denoising and wavelet de-noising can we distinctly distinguish the positions of the three zones and get the result coinciding with practice. 2700

2690

2680

2670

2660

2650

2640

2630 0

0.05

0.1

0.15

0.2

0.25

0.3

0.35

0.4

0.45

Fig. 1. original curve

2700 2700

2690 2690

2680 2680

2670 2670

2660 2660

2650

2650

2640

2640

2630

2630

0

0.05

0.1

0.15

0.2

Fig. 2. wavelet denoising curve

0.25

0

0.05

0.1

0.15

0.2

0.25

0.3

0.35

Fig. 3. wavelet package denoising

References 1.

Jian Ping Li: Wavelet analysis and signal processing: theory, applications & software implementations, Chongqing Publishing House, Chongqing (2001)

434

2.

3.

Xiu Wen Yang et al.

Hu Canghua: Base on matlab systematic analysis & design—wavelet analysis, Xian University of electronic science and tecanology publishing house, Xian, China( 2001) Ding Ciqian: Geophysical well logging, Oil university publishing house, Beijing,China( 1989)

PARAMETERIZATIONS OF M-BAND BIORTHOGONAL WAVELETS Zeyin Zhang and Daren Huang

Abstract. In this paper, we consider the structure of compactly supported wavelets. And we prove that any wavelet matrix (the polyphase matrix of the scaling filter and wavelet filters) can be factored as the product of fundamental biorthgonal matrices and a constant valued matrix.

1. Introduction Fixed an integer m ≥ 2. A compactly supported function ϕ ∈ L2 (R) is an m-band scaling function if there exists a finite length sequence {h0k } such that X h0k ϕ(m · −k), ϕ(x) = k

the z-transform

X

h0k z −k

k

is a Laurent polynomial which is called scaling filter of ϕ. Let ϕ(x) ˜ ∈ L2 (R) be another compactly scaling function with Laurent polynomial scaling filter X gk0 z −k . k

The pair of ϕ and ϕ˜ is said to be a biorthogonal pair if Z ϕ(x)ϕ(x ˜ − k)dx = δ0,k R for k ∈ Z, where δ0,0 = 1, and δ0,k = 0 if k ∈ Z \ {0}. Corresponding to the biorthogonal scaling functions, there exist compactly supported wavelets X ψi (x) = hik ϕ(m · −k) k

1991 Mathematics Subject Classification. Primary 42C15, 46A35, 46E15. Key words and phrases. Wavelet, polyphase matrix, Parameterizations, Filter bank.

Y. Y. Tang et al. (Eds.): WAA 2001, LNCS 2251, pp. 435-447, 2001. c Springer-Verlag Berlin Heidelberg 2001

436

Zeyin Zhang and Daren Huang

with finite length coefficient sequences {hik } for 1 ≤ i ≤ m − 1, and wavelets X ψ˜i (x) = gki ϕ(m ˜ · −k) k

with finite length sequences {kgki } for 1 ≤ i ≤ m − 1, such that the family {mj/2 ψi (mj · −k), j, k ∈ Z, 1 ≤ i ≤ m − 1} and the family {mj/2 ψ˜i (mj · −k), j, k ∈ Z, 1 ≤ i ≤ m − 1} are biorthogonal bases in L2 (R). Now we introduce the polyphase Laurent polynomials X j (1.1) Hi,j (z) = hmk+i z −k k

(1.2)

Gi,j (z) =

X

j gmk+i z −k

k

for 0 ≤ i, j ≤ m − 1. Let (1.3)

H(z) = (Hi,j )0≤i,j≤m−1 ,

G(z) = (Gi,j )0≤i,j≤m−1 .

By the biorthogonality we get (1.4)

G∗ (z −1 )H(z) = mIm

and the first column vectors of H(1) and G(1) is (1, 1, . . . , 1)∗ . G∗ (z −1 )= G(¯ z −1 )∗ , Here and hereafter, for a matrix or vector A, A∗ denote the Hermite transpose of A, Im is an m square identity matrix. The theoretical work of orthogonal wavelets was done in the late eighties [1, 2, 4-6, 11, 15] and the framework of biorthogonal wavelets was established in the early nineties [3, 8, 10]. The invention of the polyphase decomposition is one of the reasons why multirate filter banks processing became practically attractive. It is valuable not only in practical design and actual implementation of filter banks, but also in theoretical study[14]. Actually with the polyphase decomposition, P. P. Vaidyanathan and his colleagues [9, 13 ] derive factorizations of paraunitary matrices and apply such factorizations to design quadrature mirror filter (QMF) banks for digital signal processing problems. P. N. Heller, H. L. Resnikoff, and R. O. Wells, Jr. [7, 12] use the polyphase decomposition to develop a parametrization theory of compactly supported orthonormal wavelets. The purposes of this paper is to factorize A pair of matrices H(z) and G(z) satisfying (1.4) into some simple building block. The building block used in this paper are of the form Im − P + P z ±1 , where P is an one order idempotent matrix. This

Parameterizations of M-Band Biorthogonal Wavelets

437

paper is organized as follows. in section 2, we give a some definition and lemmas for the later use. then discuss parameterizations of dual Laurent polynomial pairs (section 3), and derive parametric decomposition of biorthogonal wavelet filter matrix(section 4), At last some final remarks are given (section 5) 2. some lemmas For the convenience in the following we give some definitions. Definition 1. The pair (H(z), G(z)) of matrices consist of polyphase Laurent polynomials (1.1) and (1.2) of scaling filter and wavelet filters defined as in (1.3) is said to be a biorthogonal wavelet matrix pair. H(z), G(z) are said to be biorthogonal wavelet matrices. Now we consider a pair of Laurent polynomials vectors α(z) and β(z) with vector valued coefficients, α(z) = αs z −s + αs+1 z −s−1 + · · · + αk z −k β(z) = βp z −p + βp+1 z −p−1 + · · · + βq z −q with αi , βj ∈ Rm , for s ≤ i ≤ k, p ≤ j ≤ q. Let Vα be a subspace of Rm spanned by {αi , s ≤ i ≤ k}, Vβ be a subspace of Rm spanned by {βj , p ≤ j ≤ q}. Definition 2. We say (α(z), β(z)) of Laurent polynomial vectors is a dual pair if α∗ (z −1 )β(z) = m where α∗ (z) = α(¯ z )∗ . Now if we rewrite (1.3) into (2.1)

H(z) = (α0 (z), α1 (z), . . . , αm−1 (z))

and (2.2)

G(z) = (β0 (z), β1 (z), . . . , βm−1 (z))

then (2.3)

αi∗ (z −1 )βj (z) = mδi,j , 0 ≤ i, j ≤ m − 1

by (1.4), so (αi (z), βi (z)), 0 ≤ i ≤ m − 1 are m dual pairs of Laurent polynomial vectors. Lemma 1. Let U = Vα ∩ Vβ⊥ , W = Vβ ∩ Vα⊥ . If (α(z), β(z)) is a dual pair of Laurent polynomial vectors, then the difference spaces Vα ⊖ U , Vβ ⊖ W are adjoint. Here and hereafter, we say two subspaces V1 , V2 ⊆ Rm are adjoint if there exist a basis {αi }k1 of V1 and a basis {βi }k1 of V2 such that αi∗ βj = δi,j ,

1 ≤ i, j ≤ k.

438

Zeyin Zhang and Daren Huang

Proof. Let k be the rank of matrix (αi∗ βj )s≤i≤k,p≤j≤q . Then there exist a k order invertable block (αi′ ∗ βj′ )1≤i,j≤k of matrix (αi∗ βj ) and such that k X ′∗ (2.4) αi∗ βj = am,j αm βj , p ≤ j ≤ q; m=1

and

(2.5)

αi∗ βj

=

k X

′ bi,m αi∗ βm ,

s ≤ i ≤ k.

m=1

P By (2.4) and (2.5),P it is easy to verify that {αi − kl=1 bi,l αl′ , s ≤ i ≤ k} ⊂ U , and {βj − al,j βl′ , p ≤ j ≤ q} ⊂ W , therefore Vα ⊖ U is the subspace spanned by {αi′ }ki=1 , Vβ ⊖ W is the subspace spanned by {βi′ }ki=1 . To prove Vα ⊖ U and Vβ ⊖ W are adjoint, it is need to prove that there are two bases of the two subspaces respectively which are biorthogonal. In fact, let {β¯j′ }k1 be defined by (β¯1′ , β¯2′ , · · · , β¯k′ ) = (β1′ , β2′ , · · · , βk′ )A−1 where A = (αi′ ∗ βj′ )1≤i,j≤k . Then {αi′ }ki=1 and {β¯j′ }k1 are bases of Vα ⊖ U and Vβ ⊖ W respectively, and they are biorthogonal, the proof is completed. Definition 3. Under the condition in Lemma 1. The dual order of dual pair (α(z), β(z)) of Laurent polynomial vectors is defined as the dimension of Vα ⊖ U . For a subspace V ⊆ Rm , define V ⊥ = {α ∈ Rm ; αβ ∗ = 0, ∀β ∈ V }. By the argument in Lemma 1, we see that the dual order of (α(z), β(z)) is equal to the rank of matrix (αi∗ βj )s≤i≤k,p≤j≤q . By the result of Lemma 1, we have Lemma 2. Let k be the order of dual pair (α(z), β(z)) of Laurent polynomial vectors is k, then there exist birothogonal bases α1′ , . . . , αk′ ∈ Vα and β1′ , . . . , βk′ ∈ Vβ satisfying ∗

αi′ βj = δi,j , 1 ≤ i, j ≤ k such that (2.6)

(

P P ˜izi α(z) = ki=1 Hi (z)αi′ + i α Pk P β(z) = i=1 Gi (z)βi′ + j β˜j z j

where α ˜ i ∈ Vα ∩ Vβ⊥ and β˜j ∈ Vβ ∩ Vα⊥ , and Hi , Gi are Laurent polynomials.

Parameterizations of M-Band Biorthogonal Wavelets

439

In Lemma 2, if k = 1, then H1 (z) = cz −n , G1 (z) = mc z −n for a nonzero constant c and an integer n. Especially if (α(z), α(z)) is a dual pair of Laurent polynomial vectors, by the fact Vα ∩ Vα⊥ = {0}, we have Lemma 3. If Let k be the dual order of (α(z), α(z)).There exist α1 , . . . , αk ∈ Vα such that

αi∗ βj = δi,j , 1 ≤ i, j ≤ k and α(z) =

k X

Hi (z)αi

i=1

where Hi , 1 ≤ i ≤ k are Laurent polynomials. A matrix P ∈ Rm×m is said idempotent if P 2 = P . for a given matrix Q ∈ Rm×m and a subspace V ⊆ Rm , define P V = {P α; α ∈ V }. For a subspace V ⊆ Rm , Q ∈ Rm×m is said to be an annihilator on V , if QV = {0}. Denoted by N (V ) the set of all annihilators on V . 3. Parameterizations of dual pair of Laurent polynomial vectors with one rank idempotent matrices Theorem 1. If (α(z), β(z)) is a dual pair of Laurent polynomial vectors, then there exist one rank idempotent matrices P1 , P2 , . . . , Pd with that Pi ∈ N (Vβ⊥ ), Pi∗ ∈ N (Vα⊥ ), 1 ≤ i ≤ d such that α(z) = Vd (z)Vd−1 (z) · · · V1 (z)δ(z) ∗ β(z) = Vd∗ (z)Vd−1 (z) · · · V1∗ (z)γ(z) where (δ(z), γ(z)) is a one order dual pair of Laurent polynomial vectors, Vδ ⊆ Vα , Vγ ⊆ Vβ and Vi (z) = Im − Pi + Pi z −1 , 1 ≤ i ≤ d. Let P be a one order idempotent matrix, that is, there exist u, v ∈ Rm , u∗ v = 1 such that P = uv ∗ . Define (3.1)

V (z) = Im − P + P z −τ , τ ∈ {−1, +1}

then V ∗ (z) = Im − P ∗ + P ∗ z −τ , so (3.2)

V (z)V (z −1 ) = 1,

det(V (z)) = z −τ

we will say that the matrix V (z) of the form (3.1) as primitive biorthogonal matrix.

440

Zeyin Zhang and Daren Huang

Proof of Theorem 1. Let k be the dual order of (α(z), β(z)). By Lemma 2, we can represent α(z) and β(z) in the form (2.6)with coefficient Laurent polynomials Hi , Gi , 1 ≤ i ≤ k. Now write r X T −n ηi z −i , (3.3) (H1 (z), H2 (z), . . . , Hk (z)) = z 0

T

here and hereafter, for a vector α, α denote the transpose of α, n is an integer, ηi ∈ Rk , i = 0, 1, . . . , r. The scheme of the proof is to decrease the length r + 1 to 1 recursively. If r = 0, the length of (3.3) is just one, there is need do nothing. Assume r ≥ 1. case 1. η0 , ηr is independent. Let (b1 , b2 , . . . , bk )T = ηr and u (a1 , a2 , . . . , ak )T = ∗ , ηr u where u = (η0∗ η0 )ηr − (η0∗ ηr )η0 . Define k k X X ′ β= ai βi , α = bi αi′ , 1

1

then β ∈ Vβ and α ∈ Vα . Let P = αβ ∗ , then P is a one order idempotent matrix, and P ∈ N (Vβ⊥ ), P ∗ ∈ N (Vα⊥ ). Define V (z) = Im − P + P z −1 , then V (z) is a primitive biorthogonal wavelet matrix. And define α′ (z) = V (z −1 )α(z),

β ′ (z) = V ∗ (z −1 )β(z),

it follows that (α′ (z), β ′ (z)) is a dual pair of Laurent polynomial vectors, Vα′ ⊆ Vα , Vβ ′ ⊆ Vβ and α(z) = V (z)α′ (z), Note that ′

α (z) =

k X

Hi′ (z)αi′ +

1

where

β(z) = V ∗ (z)β ′ (z).

(H0′ (z), H1′ (z), . . . , Hk′ (z))T = z −n (η0 +

X

α ˜ i z −i

u∗ η1 u∗ ηr−1 −r+1 ηr )z ). η +· · ·+(η +η − r r r−1 u∗ ηr u∗ η r

Thus the length of (H0′ (z), H1′ (z), . . . , Hk′ (z))T is decreased by 1.

Parameterizations of M-Band Biorthogonal Wavelets

441

case 2. η0 , ηr are dependent. Write Laurent polynomials Gi , 1 ≤ i ≤ k in (2.6) as s X T −n1 (G1 (z), . . . , Gk (z)) = z γs z −i , 0

= 0. we only consider η0∗ γs = = = = 0 or then ∗ ηr γs = 0, for another is similar. There exist an l such that ηi∗ γs = 0, 0 ≤ i ≤ l − 1 and ηl∗ γs 6= 0. Let ηl = (c1 , c2 , . . . , ck )T and γs = (d1 , d2 , . . . , dk )T Define k k X X ′ α= di βi′ , ci αi , β = η0∗ γs

ηr∗ γs

η0∗ γ0

ηr∗ γ0

1

1

∗

then α ∈ Vα and β ∈ Vβ . Now if we set P = γαβ ∗ , then P is a one rank s ηl ⊥ ∗ ⊥ idempotent matrix, P ∈ N (Vβ ), P ∈ N (Vα ). Define

V (z) = Im − P + P z −n = (Im − P + P z −1 )n then it is a power of primitive biorthogonal wavelet matrix, and define α′ (z) = V (z −1 )α(z),

β ′ (z) = V ∗ (z −1 )α(z).

It follows that (α′ (z), β ′ (z)) is a dual pair of Laurent polynomial vectors, Vα′ ⊆ Vα , Vβ ′ ⊆ Vβ and α(z) = V (z)α′ (z), Note that α′ (z) =

k X 1

where

β(z) = V ∗ (z)α′ (z).

Hi′ (z)α′ +

X

α ˜ i z −i

(H1′ (z), H2′ (z), . . . , Hk′ (z))T = z −n ((η0 + ηn ) + η1 z −1 + · · · + ηr z −r ) thus the length of (H1′ (z), H2′ (z), . . . , Hk′ (z))T is the same as the length of (3.3), but η0 + ηn and ηr are independent,which transform to condition in the case 1. Recursively proceeding in this fashion, we decrease the length of (3.3) to 1, that is α(z) = Vd (z)Vd−1 (z) · · · V1 (z)δ(z), ∗ β(z) = Vd∗ (z)Vd−1 (z) · · · V1∗ (z)γ(z),

442

Zeyin Zhang and Daren Huang

where

k X

δ(z) = z −n

ci αi +

1

and γ(z) =

k X

G′i (z)βi +

1

X

α ˜ i z −i

X

β˜i z −i ,

therefore (δ(z), γ(z)) is a dual pair of Laurent polynomial vectors with one order, and Vδ ⊆ Vα , Vγ ⊆ Vβ . The proof is completed. In the following, we consider the parameterizations of one order dual pair of Laurent polynomial vectors. Theorem 2. Let (α(z), β(z) be a dual pair of Laurent polynomial vectors. If the dual order is one, then there exist idempotent matrices Pi , 1 ≤ i ≤ d with rank one, and Pi ∈ N (Vβ⊥ ), Pi∗ ∈ N (Vα⊥ ), 1 ≤ i ≤ d such that α(z) = z −k Vd (z)Vd−1 (z) · · · V1 (z)α(1) ∗ (z) · · · V1∗ (z)β(1) β(z) = z −k Vd∗ (z)Vd−1 where k is an integer, and

Vi = Im − Pi + Pi z −τi , τi ∈ {1, −1}, i = 1, 2, · · · , d. Proof. Since (α(z), β(z)) is a one order dual pair of Laurent polynomial vectors with order one, then by Lemma 2, we have ½

α(z) = αk−r z −k+r + · · · + αk z −k + · · · + αk+s z −k−s β(z) = βk−p z −k+p + · · · + βk z −k + · · · + βk+q z−k − q

where αi∗ βj = mδi,k δj,k for k − r ≤ i ≤ k + s, k − p ≤ j ≤ k + q. Define Uj (z) = Im − Qj + Qj z −1

Vi (z) = Im − Pi + Pi z −1 ,

as primitive biorthogonal wavelet matrix, where Pi = (

k X

αl )(

l=k−i

and Qj = ( Then Pi , Qi ∈

k+j X

αl )(

k X

βl ) ∗ , 0 ≤ i ≤ r − 1

l=k−i k+j X

βl )∗ ,

l=k−r l=k−r ⊥ ∗ ∗ N (Vβ ), Pi , Qi ∈ N (Vα⊥ ),

1 ≤ j ≤ s − 1. 0 ≤ i ≤ r − 1, 1 ≤ j ≤ s − 1.

Parameterizations of M-Band Biorthogonal Wavelets

Define

s−1 Y

α ˜ (z) =

r

Uj (z)Vr (z)

˜ β(z) =

r−1 Y

Vi (z −1 )α(z),

r−1

1

and define

0 Y

443

Vi∗ (z −1 )(Vr∗ (z)r )

1 Y

Uj∗ (z)β(z).

s−1

0

˜ It follows that (˜ α(z), β(z)) is a dual pair, Vα˜ ⊆ Vα , Vβ˜ ⊆ Vβ , and α(z) =

r−1 Y

Vi (z)(Vr (z −1 )r )

β(z) =

Uj∗ (z −1 )Vr∗ (z −1 )r

Note that α ˜ (z) =

0 Y

Vi∗ (z)α(z).

r−1

1

αn′ z −n ,

Uj (z −1 )α(z),

s−1

0

s−1 Y

1 Y

where n = k + s, and αn′ =

Pk+s k−r

αi and

′ ′ ˜ z r−n + · · · + βn′ z −n + · · · + βn+s z −n−s β(z) = βn−r with that αn′ ∗ βi′ = mδn,i . ˜ The next step is to factor β(z). Define

Vi′ (z) = Im − Pi′ + Pi′ z −1 ,

Uj′ (z) = Im − Q′j + Q′j z −1

as primitive biorthogonal wavelet matrices, where n X βj′ )∗ , 0 ≤ i ≤ r Pi′ = αn′ ( n−i

and

Q′j

=

n+j X

βl′ )∗ ,

αn′ (

1 ≤ j ≤ s − 1.

n−r N (Vβ⊥ ), Pi′∗ , Q′∗ j

∈ N (Vα⊥ ), 0 ≤ i ≤ r, 1 ≤ j ≤ It follows that Pi′ , Q′j ∈ ′ ′ s − 1. And Pi , Qj are idempotent matrices with rank one. Note that 1 Y

r Ui′∗ (z)(Vr′∗ (z) )

s−1

where β˜n+s =

r−1 Y

˜ Vi′∗ (z −1 )β(z) = β˜n+s z −n−s ,

0

P

j

0 Y r−1

βj′ and

r Vi′ (z −1 )Vr′ (z)

s−1 Y 1

Ui′ (z)˜ α(z) = α ˜ n z −n−s .

444

Zeyin Zhang and Daren Huang

By the fact of biorthogonal matrix (3.2), we obtain α(z) = z −n−s Vd (z)Vd−1 (z) · · · V1 (z)γ, ∗ β(z) = z −n−s Vd∗ (z)Vd−1 (z) · · · V1∗ (z)δ. At last letting z = 1 above to get γ = α(1), δ = β(1). Together with Theorem 1 and Theorem 2, we get

Theorem 3. For any dual pair of Laurent polynomial vectors α(z) and β(z), we have α(z) = z −n Vd (z)Vd−1 (z) · · · V1 (z)α(1), ∗ β(z) = z −n Vd∗ (z)Vd−1 (z) · · · V1∗ (z)β(1). Where n is an integer, and there exist one rank idempotent matrices Pi satisfying Pi ∈ N (Vβ⊥ ), Pi∗ ∈ N (Vα⊥ ) such that Vi (z) = Im −Pi +Pi z −τi , with that τi ∈ {1, −1}, 1 ≤ i ≤ d.

4. Parameterizations of biorthogonal wavelet matrix Theroem 4. If H(z) and G(z) is a pair of biorthogonal wavelet matrices, then there exist one rank idempotent matrices Pi and integers ki , i = 1, 2, · · · , m such that H(z) = Vd (z)Vd−1 (z) · · · V2 (z)V1 (z)diag(z −k1 , z −k2 , · · · , z −km )H(1), and ∗ (z) · · · V2∗ (z)V1∗ (z)diag(z −k1 , z −k2 , · · · , z −km )G(1), G(z) = Vd∗ (z)Vd−1

where Vi (z) = Im − Pi + Pi z τi , τi ∈ {1 , −1 }, 1 ≤ i ≤ d. Proof: Writing H(z), G(z) in the form as (2.1)and (2.2) respectively, then (2.3)holds, (αi , βi ), 0 ≤ i ≤ m − 1 are dual pairs of Laurent polynomial vectors. By theorem 3, for the dual pair (α0 (z), β0 (z)) of Laurent polynomial vectors, there exist primitive biorthogonal matrices V0,1 (z), V0,2 (z), · · · , V0,d1 (z) and an integer k such that α0 (z) = z −k1 V0,d1 (z)V0,d1 −1 (z) · · · V0,1 (z)α0 (1), ∗ ∗ ∗ β0 (z) = z −k1 V0,d (z)V0,d (z) · · · V0,1 (z)β0 (1) 1 1 −1 where k1 and d1 are non-negative integers. Define

H1 (z) = V1 (z −1 )V2 (z −1 ) · · · Vd1 (z −1 )H(z) and G1 (z) = V1∗ (z −1 )V2∗ (z −1 ) · · · Vd∗1 (z −1 )G(z), then (H1 (z), G1 (z)) is a pair of biorthogonal wavelet matrices and H(z) = Vd1 (z)Vd1 −1 (z) · · · V1 (z)H1 (z),

Parameterizations of M-Band Biorthogonal Wavelets

445

G(z) = Vd∗1 (z)Vd∗1 −1 (z) · · · V1∗ (z)G1 (z). It follows that

and

¡ ¢ H1 (z) = z −k1 α0 (1), α1,1 (z), · · · , α1,m−1 (z)

¡ ¢ G1 (z) = z −k1 β0 (1), β1,1 (z), · · · , β1,m−1 (z) .

By the biorthogonality we get (4.1)

α0 (1) ∈ Vβ⊥1,k , β0 (1) ∈ Vα⊥1,k , 1 ≤ k ≤ m

and ∗ β1,i (z −1 )β1,j (z) =

© m, if i = j 0, if i 6= j.

therefore, (β1,1 (z), β1,1 (z)) is a dual pair of Laurent polynomial vectors. By Theorem 3, there exist primitive biorthogonal matrices V1,1 (z), V1,2 (z), · · · , V1,d2 (z) such that α1,1 (z) = z −k2 V1,d2 (z)V1,d2 −1 (z) · · · V1,1 (z)α1,1 (1), ∗ ∗ ∗ β1,1 (z) = z −k2 V1,d (z)V1,d (z) · · · V1,1 (z)β1,1 (1), 2 2 −1

where k2 and d2 are non-negative integers. Define H2 (z) = V1,1 (z −1 )V1,2 (z −1 ) · · · V1,d2 (z −1 )H1 (z) and ∗ ∗ ∗ G2 (z) = V1,1 (z −1 )V1,2 (z −1 ) · · · V1,d (z −1 )G1 (z) 2

then (H2 (z), G2 (z)) is a pair of biorthogonal wavelet matrices and H1 (z) = V1,d2 (z)V1,d2 −1 (z) · · · V1,1 (z)H2 (z), ∗ ∗ ∗ G1 (z) = V1,d (z)V1,d (z) · · · V1,1 (z)G2 (z). 2 2 −1

Note the fact (4.1) and Theorem 3, we get ¡ ¢ H1 (z) = z −k1 α0 (1), z −k2 α1,1 (1), · · · , α1,m−1 (z)

and

¡ ¢ G1 (z) = z −k1 β0 (1), z −k2 β1,1 (1), · · · , β1,m−1 (z) .

Proceeding in the same fashion, we get primitive biorthogonal matrices Vi,j (z), j = 1, 2, · · · , di , 1 ≤ i ≤ r

446

Zeyin Zhang and Daren Huang

with nonnegative integers di , 1 ≤ i ≤ r and integers ki , 1 ≤ i ≤ m such that H(z) = Vr−1,dr (z)Vr−1,dr −1 (z) × · · · Vr−1,1 (z) · · · V1,d2 (z)V1,d2 −1 (z) × · · · V1,1 (z)Vd1 (z)Vd1 −1 (z) · · · V1 (z) × diag(z −k1 , z −k2 , · · · , z −km )U and ∗ ∗ (z) × (z)Vr−1,d G(z) = Vr−1,d r −1 r ∗ ∗ ∗ · · · Vr−1,1 (z) · · · V1,d (z)V1,d (z) × 2 2 −1 ∗ (z)Vd∗1 (z)Vd∗1 −1 (z) · · · V1∗ (z) × · · · V1,1

diag(z −k1 , z −k2 , · · · , z −km )W. By taking z = 1 above we get U = A(1) and W = B(1), therefore U, W is a pair of constant-valued biorthogonal wavelet matrices. Especially, for orthogonal wavelet matrices, by using Lemma 2 and the similarly procedure as above, we have Theroem 5. If H(z) is an orthogonal wavelet matrix, then there exist symmetric idempotent matrices Pi with rank one and integers ki , i = 1, 2, · · · , m such that H(z) = Vd (z)Vd−1 (z) · · · V2 (z)V1 (z)diag(z −k1 , z −k2 , · · · , z −km )U, where U is a constant-valued orthogonal wavelet matrix, Vi = Im − Pi + Pi z −1 , 1 ≤ i ≤ d.

5. Final remark 1. In [12], H. L. Resnikoff, J.Tian and R. O. Wells. Jr discussed the parameterizations and parameterizations in biorthogonal wavelet space, they proved that any biorthogonal wavelet matrix pair can be decomposed into four components: an orthogonal component, a pseudo identity matrix pair, an invertible matrix and a constant matrix. The result is modified into theorem 3 in this paper: Any biorthogonal wavelet matrix pair can be decomposed into two parts: an biorthogonal components V (z) and an constant matrix H.

Parameterizations of M-Band Biorthogonal Wavelets

447

2. It was proved in [12] that any constant matrix in Theorem 4 can be decomposed into ¶ µ 1 0 ˜ H H= 0 U

˜ = (γ0 , γ1 , . . . , γm−1 ) with that γ0 = (1, 1, . . . , 1) , where H r m (0, · · · , 0, −m + i, 1, · · · , 1 )T . γi = | {z } (m − i)(m − i + 1) | {z } i−1 terms

m−i terms

for 1 ≤ i ≤ m − 1, and U is an (m − 1) × (m − 1) nonsingular constantvalued matrix. References [1] Bi N., Dai X. and Sun Q., Construction of compactly supported M-band wavelets, Appl. Comp. Harmonic Anal. 6(1999), pp.113-131. [2] Chui C. K. and Lian J., Construction of compactly supported symmetric and antisymmetric orthogonal wavelets with scale=3, Appl. Comput. Harmonic Anal., 2(1995), pp.21-51. [3] Cohen A., Daubechies I. and Feauveau J. C., Biorthogonal basis of compactly supported Wavelets, Commun. Pure Appl. Math., 45(5)(1992), pp.485-560. [4] Daubechies I., Ten lectures on wavelets, SIAM, Philadelphia, PA, 1992. [5] Han B., Symmetric orthogonal scaling functions and wavelets with dilation factor 4, Adv. Compt. Math., 8(1998), pp.221-247. [6] Heller D. N., Rank m wavelets with n vanish moments, SIAM J. Matrix Anal. 16(2)(1994), pp.502-519. [7] Heller P. N., Resnikoff H. L. and Wells R. O. Jr., Wavelet matrices and the representation of discrete functions, in Wavelet- A Tutorial in theory and applications, C. K. Chui (ed.), Academic Press, Inc.(1992), 15-50. [8] Ji H. and Shen Z., Compactly supported (bi)orthogonal wavelets generated by interplatory refinable functions, Adv. Comput. Math., 111999, pp.81-104. [9] Soman A. K., Vaidyanathan P.P. and Nguyen T.Q., Linear phase paraunitary filter banks: theory, factorization and designs, IEEE Trans. Signal Processing 41(1993), pp.3480-3496. [10] Soardi P., Biorthogonal M-channel compactly supported wavelets, Constr. Approx., 16(2000), pp.283-311. [11] Sun Q. and Zhang Z., M-Band scaling function with filter having vanishing moments two and minimal length, J. Math. Anal. 222(1998), pp.225-243. [12] Resnikoff H. L., Tian J. and Wells R. O. Jr, An algebraic structure of orthogonal wavelet space, Appl. Comput. Harmon. Anal., 8(2000), pp. 223–248. [13] Vaidyanathan P. P., Multi-rate systems and filter banks, Prentice-Hall, Englewood Cliffs, NJ, 1993. [14] Vetterli M. and Herley C., Wavelet and filter banks: Theory and design, IEEE Trans. Acounst. Speech SignaL Processing, 40(1992), pp. 2207-2232. [15] Welland G. V. and Lundberg M., Construction of compact p-wavelets, Constr. Approx. 9(1993), pp.347-370.