Land Cover Classification of Remotely Sensed Images: A Textural Approach 3030665941, 9783030665944

Citation preview

S. Jenicka

Land Cover Classification of Remotely Sensed Images A Textural Approach

Land Cover Classification of Remotely Sensed Images

S. Jenicka

Land Cover Classification of Remotely Sensed Images A Textural Approach

S. Jenicka Information Technology Sethu Institute of Technology Pulloor, Tamil Nadu, India

ISBN 978-3-030-66594-4 ISBN 978-3-030-66595-1 https://doi.org/10.1007/978-3-030-66595-1

(eBook)

© The Editor(s) (if applicable) and The Author(s), under exclusive license to Springer Nature Switzerland AG 2021 This work is subject to copyright. All rights are solely and exclusively licensed by the Publisher, whether the whole or part of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation, broadcasting, reproduction on microfilms or in any other physical way, and transmission or information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now known or hereafter developed. The use of general descriptive names, registered names, trademarks, service marks, etc. in this publication does not imply, even in the absence of a specific statement, that such names are exempt from the relevant protective laws and regulations and therefore free for general use. The publisher, the authors, and the editors are safe to assume that the advice and information in this book are believed to be true and accurate at the date of publication. Neither the publisher nor the authors or the editors give a warranty, expressed or implied, with respect to the material contained herein or for any errors or omissions that may have been made. The publisher remains neutral with regard to jurisdictional claims in published maps and institutional affiliations. This Springer imprint is published by the registered company Springer Nature Switzerland AG The registered company address is: Gewerbestrasse 11, 6330 Cham, Switzerland

This book is dedicated to my Lord and savior Jesus Christ who lives in my heart forever

Foreword

I am delighted to write the foreword for the book titled Land Cover Classification of Remotely Sensed Images – A Textural Approach. Texture is fundamental in solving many image processing related problems. The book describes a wide range of texture feature extraction techniques that have evolved over the years. The book discusses in detail about how land cover classification can be performed using the texture feature extraction techniques described earlier. The advantage of reading this book is that the reader is exposed to relevant Matlab source codes along with the description of each method. I have known the book’s author, Dr. S. Jenicka, since a long time. She has written it precisely based on her research experience and expertise gained through guiding her research scholars. Dr. Jenicka has organized and described each chapter so well as to help the reader understand the two domains – Remote Sensing and Digital Image Processing – associated with the application land cover classification. The book is immensely informative, providing practical texture based solutions to land cover classification. The book will serve as a good teaching tool for professors, a guide to researchers, and a lecture note for students. I am hopeful that this book will be an eye-opener to those who wish to learn texture based image processing and remote sensing applications. Through careful

vii

viii

Foreword

reading, the reader can also gain knowledge and insight into solving several other related remote sensing applications apart from land cover classification.

Date : 20. I0.2020

Dr. V. Thirupathi, M.E., Ph.D., Dean of the Agricultural Engineering College and Research Institute Tamilnadu Agricultural University Kumulur – 621712, Lalgudi (Tk), Trichy Dt., Tamil Nadu, India Email: [email protected] / [email protected]

Preface

Scope of the Book The scope of the book is “texture based image processing” concepts applied in “land cover classification of remotely sensed images.” It also is a subdomain of digital image processing and remote sensing. The book serves as a handy Matlab implementation guide to researchers and students interested in the specified area.

Whom Is This Book For? The book is immensely useful for computer science and civil engineering undergraduates and postgraduates who plan to do research or project work in digital image processing or in particular, satellite image processing. It can serve as a guide to those who narrow down their research to processing remotely sensed images. It addresses a wide range of texture models and classifiers. The book not only guides but also aids the reader in implementing the concepts through the Matlab source codes listed. In short, the book will be helpful for growing academicians to gain expertise in their area of specialization and students who aim at gaining in-depth knowledge through practical implementations. The exercises given under texture based segmentation (excluding land cover classification exercises) can serve as lab exercises for the undergraduate students who learn texture based image processing.

ix

x

Preface

Highlights of the Book • The book is aimed at exposing the readers to texture and land cover classification. • The focus of this book is to enable the readers to implement existing texture based land cover classification methods and develop new tailor-made methods. • To stay focused on the aim, the book discusses remote sensing, texture, classifiers, and procedures for performing texture based segmentation of a stitched image and land cover classification of a remotely sensed image. • The book addresses the issues and challenges in texture based segmentation and land cover classification. • The book contains the Matlab source codes with descriptive comments for a wide range of texture models, texture segmentation methods, and land cover classification.

Assumptions The book assumes that the reader has acquired basic knowledge in Matlab and digital image processing.

Why Write This Book? As Matlab has been widely used as an implementation tool for doing research or project work related to digital image processing, books that serve as a step-by-step guide with worked-out source codes relevant to the scope of this book are only a few in number. The proposed book can definitely bridge the existing gap by catering to the needs of the reader. The book includes those topics that have not been addressed in the existing books so far. The approach and style of writing is simple and tailormade to kindle the interests of the reader.

Practical Issues For working out the practical source codes listed in Matlab, the reader requires “image processing” and “neural networks” toolboxes in Matlab. Pulloor, Tamil Nadu, India

S. Jenicka

Acknowledgments

First and foremost, I would like to thank my Lord and savior Jesus Christ, without whom this book would not have been possible. This book is the record of my research experience. I wish to thank my parents Mr. Sam.G. Andrews and Mrs. Glory Kamalabhai for bringing me up and educating me to write this book. I wish to specially thank my husband Er. A. Divyanathan, who is my real strength and source of motivation. His enthusiasm and support greatly pushed me to write this book. I would like to thank my daughters D. Agnes Rose and D. Christina Grace for their love and support. Despite their busy academic schedule, they took special interest and encouraged me to complete this book. I am greatly indebted to my mentor and guide Dr. A. Suruliandi, professor in the Department of Computer Science and Engineering at Manonmaniam Sundaranar University, Tirunelveli, Tamil Nadu, for teaching me the concepts behind my research. I am grateful to Dr. V. Thirupathi, Dean, Tamil Nadu Agricultural University, for writing the foreword to this book. I would like to thank Ms. Doris Bleier, Publishing Editor, Earth Sciences, Geography and Environment, Springer Nature, and the honorable reviewers for giving their constructive suggestions that greatly contributed in improving the quality of this book. I wish to thank the project coordinator and his team for monitoring the production process. I thank the overall Springer Nature publishing team for providing a very good publishing experience.

xi

Contents

1

Introduction to Remote Sensing . . . . . . . . . . . . . . . . . . . . . . . . . . . .

1

2

Introduction to Texture and Related Work . . . . . . . . . . . . . . . . . . . .

17

3

A Few Existing Basic and Multivariate Texture Models . . . . . . . . . .

37

4

Supervised Texture-Based Segmentation Using Basic Texture Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

73

Texture-Based Segmentation Using ULBP with Supervised and Unsupervised Classifiers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

89

5 6

Land Cover Classification of Remotely Sensed Images Using Textural Approaches . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111

7

Texture-Based Segmentation and Land Cover Classification Using Hidden Markov Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 137

8

Overview of Spatial Data Analysis and Other Land Cover Classification Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 155

9

Research Questions and Challenging Issues for Further Research on Land Cover Classification Using Textural Approaches . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 165

Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 173

xiii

About the Author

S. Jenicka completed her undergraduate course in computer science and engineering at Thiagarajar College of Engineering, Madurai, Tamil Nadu, India, in 1994. Later she completed her postgraduation in the same discipline in 2009 at Manonmaniam Sundaranar University, Tirunelveli, Tamil Nadu. Jenicka received her doctoral degree in computer science and engineering in 2014. Her research work was on “Texture based classification of remotely sensed images.” Jenicka’s interests include satellite image processing and texture analysis. This book is the offspring of the expertise gained by her through research work. She has notable online conference and journal publications. Jenicka has nearly 13 years of teaching experience at reputed institutions.

xv

Chapter 1

Introduction to Remote Sensing

Abstract This chapter explores types of satellites, remote sensing process, sensors, types of remotely sensed images, basics of land cover classification, and remote sensing applications. Under the resolution characteristics of remotely sensed images, spatial, spectral, radiometric, and temporal resolution are defined and discussed. The land covers and typical spectral reflectance characteristics of land cover classes are also described. Keywords Reflectance characteristics · Resolution characteristics · Remotely sensed images · Remote sensing applications · Sensors · Land cover · Land cover properties

1.1

Introduction

The first chapter focuses on introducing the basics of remote sensing to the readers. A reader has to be aware of the terminologies associated with remote sensing before he starts implementing useful applications. As the remotely sensed data used for experimentation in this book was captured by an Indian remote sensing satellite, some terminologies associated with the satellites have been explained by keeping Indian satellites in mind. The chapter is organized as follows. Section 1.2 introduces the reader to the basics of remote sensing. Section 1.3 describes the types of remotely sensed images and features extracted for analysis. Section 1.4 discusses the resolution characteristics of remotely sensed imagery. Section 1.5 exposes readers to the basics of land cover classification. Section 1.6 describes various remote sensing applications. By the end of this chapter, the reader will be familiar with the following: • Understand the basic terminologies in remote sensing. • Understand the role of sensors in capturing remotely sensed images. • Analyze the resolution characteristics of remotely sensed images and reflectance characteristics of land cover classes.

© The Author(s), under exclusive license to Springer Nature Switzerland AG 2021 S. Jenicka, Land Cover Classification of Remotely Sensed Images, https://doi.org/10.1007/978-3-030-66595-1_1

1

2

1 Introduction to Remote Sensing

• Identify the remotely sensed data suitable for land cover classification and its specifications. • Understand land covers and classification criteria. • Understand how texture improves the accuracy of estimates of land cover properties.

1.2

Basics of Remote Sensing

Remote sensing is the science of acquiring information about the earth’s surface without being in contact with it (Lillesand and Kiefer 1994). It includes sensing the earth’s surface and recording the reflected energy. Subsequently the recorded information is processed and analyzed to get useful inferences.

1.2.1

Satellite and Launch Vehicle

Satellite A satellite is an artificial object placed into orbit round the earth or another planet for collecting useful information. These satellites are called artificial satellites in contrast to the natural satellites like the moon. The path followed by a satellite is referred to as its orbit. Satellites can revolve around the earth in two different orbits. They are geosynchronous and polar orbits. A geosynchronous satellite revolves in a geosynchronous orbit, with an orbital period same as the earth’s rotation period. Geosynchronous satellites remain permanently in the same area of the sky, as viewed from a particular ground station on earth. A satellite on a geosynchronous orbit flies around 36,000 km above the earth’s surface in high earth orbit. Geosynchronous satellites are useful for a variety of applications such as weather forecasting, communication, and global positioning. Satellites on polar orbits called polar orbiting satellites circle the planet on a near-polar inclination maintaining an altitude of at least 700 km in low earth orbit. Polar orbiting satellites are useful in photography and mapping. Launch Vehicle A launch vehicle or carrier rocket is a rocket used to carry a payload from the earth’s surface into outer space (https://www.isro.gov.in/ launchers). Expendable launch vehicles are designed for one-time use. They usually separate from their payload and may break up during atmospheric re-entry. Reusable launch vehicles, on the other hand, are designed to be recovered intact and used again for subsequent launches. In India, the polar satellite launch vehicle (PSLV) is the first operational launch vehicle of Indian space research organization (ISRO). It places satellites in polar and sun synchronous orbits. Geosynchronous satellite launch vehicle (GSLV) is a launch vehicle commonly used for launching communication satellites. It places satellites in geosynchronous orbits.

1.2 Basics of Remote Sensing

1.2.2

3

Types of Satellites

Earth Observation Satellites These satellites are specifically designed to observe earth from the orbit intended for uses such as environmental monitoring, meteorology, map making, etc. In India, Indian remote sensing (IRS) system launched its first remote sensing satellite IRS-1A in 1988 (https://www.isro.gov.in/spacecraft/earthobservation-satellites). With 11 satellites in operation, the data obtained from IRS system is used for several applications covering agriculture, water resources, urban development, mineral prospecting, environment, forestry, drought and flood forecasting, ocean resources, and disaster management. The RESOURCESAT-1 satellite was launched into the polar sun synchronous orbit (altitude of 817 km) by PSLV-C5 launch vehicle on October 17, 2003, with a design life of 5 years. RESOURCESAT1 is also called IRS-P6. It is also the tenth satellite of ISRO in IRS series. The remotely sensed image used as the experimental data in this book was captured by IRS-P6. Research Satellites Research satellites measure fundamental properties of outer space such as magnetic fields, the flux of cosmic rays and micrometeorites, properties of celestial objects, etc. that are difficult to observe from the earth. Communication Satellites Communication satellites are satellites stationed in space for the purpose of telecommunication. Modern communication satellites typically use geosynchronous orbits. Weather Satellites These meteorological satellites provide continuous, up-to-date information about large-scale atmospheric conditions such as cloud cover and temperature profiles. Navigational Satellites A satellite navigation system is a system of satellites that provide autonomous geo-spatial positioning with global coverage.

1.2.3

Remote Sensing Process

The basic principles underlying the remote sensing process (Lillesand and Kiefer 2015) are described as follows. A remote sensing satellite has an energy source which illuminates the target of interest. As the energy travels from its source to the target, it interacts with the atmosphere it passes through. The interaction takes place again as the energy travels from the target to the sensor. Once the energy makes its way to the target through the atmosphere, it interacts with the target depending on the properties of both the target and the radiation. After the energy has been scattered by or emitted from the target, a remote sensor is used to collect and record the electromagnetic radiation. The energy recorded by the sensor has to be transmitted in electronic form to a processing station where the data are processed into an image. The processed image is interpreted to extract information about the target which was illuminated.

4

1.2.4

1 Introduction to Remote Sensing

Sensors

The sun is a natural source of energy or radiation. The sun provides a very convenient source of energy for remote sensing. The sun’s energy is either reflected or absorbed and then reemitted. Remote sensing systems which measure naturally available energy are called passive sensors. Passive sensing takes place only when the sun is illuminating the earth. Active sensors, on the other hand, provide their own energy source for illumination. The sensor emits radiation which is directed toward the target to be investigated, and the radiation reflected from that target is detected and measured. Three types of sensors were used in IRS P6 satellite, and they are a mediumresolution four-band linear imaging self-scanning sensor (LISS-III), a coarseresolution four-band advanced wide field sensor (AWiFS), and a high-resolution three-band linear imaging self-scanning sensor (LISS-IV). As a satellite revolves around the earth, the sensor sees a certain portion of the earth’s surface. The area imaged on the surface is referred to as the swath. Imaging swaths for spaceborne sensors generally vary between tens and hundreds of kilometers wide. Table 1.1 shows the IRS P6 sensors with the corresponding electromagnetic (EM) bands used for capturing data, the wavelength range of each EM band, spatial resolution, swath width, revisit period, and their applications. The remotely sensed data from LISS-IV sensor has a spatial resolution of 5.8 m. The remotely sensed image used as experimental data in this book was formed by combining the three bands provided by LISS-IV sensor of IRS P6 satellite. Table 1.1 Sensors in IRS P6, their characteristics, and their applications

Sensor LISSIII

LISSIV

AWiFS

Bands Green (B2) Red (B3) Near IR (B4) Shortwave IR (B5) Green (B2) Red (B3) Near IR (B4) Green (B2) Red (B3) Near IR (B4) Shortwave IR (B5)

Wavelength Range (in μm) 0.52–0.59 0.62–0.68 0.77–0.86

Spatial Resolution 23.5 m

Swath width 141 km

Revisit period 24 days

0.52–0.59 0.62–0.68 0.77–0.86

5.8 m

23.9 km

5 days

Vegetation discrimination, detailed level land cover mapping

0.52–0.59 0.62–0.68 0.77–0.86

56 m

740 km

5 days

Regional scale vegetation monitoring

Application District level land cover mapping

1.55–1.7

1.55–1.7

Adapted from https://www.nrsc.gov.in/sites/default/files/pdf/brresourcesat1.pdf

1.3 Types of Remotely Sensed Images and Features Extracted for Analysis

5

A few other sensors used in remote sensing applications are LIDAR and RADAR. LIDAR is an acronym for light detection and ranging, an active imaging technology. Pulses of laser light are emitted from the sensor, and energy reflected from the target is detected. Lidar is used effectively for measuring heights of land features such as forest canopy height relative to the ground surface, and water depth relative to the water surface. RADAR is an acronym for radio detection and ranging. RADAR systems are active sensors which emit microwave radiation in a series of pulses from an antenna. When the energy reaches the target, some of the energy is reflected back toward the sensor. The backscattered microwave radiation is detected, measured, and timed. The time required for the energy to travel to the target and return to the sensor determines the distance or range to the target. By recording the range and magnitude of the energy reflected from all targets, a two-dimensional image of the surface can be produced.

1.3 1.3.1

Types of Remotely Sensed Images and Features Extracted for Analysis Types of Remotely Sensed Images

Remotely sensed images are classified into multispectral, hyperspectral, and ultraspectral images. A multispectral image may have three to ten spectral bands. For example, the multispectral sensor LISS-IV captures multispectral images which are composed of three bands such as green, red, and near infrared (NIR). Multispectral images have a wide range of applications in agriculture, urban planning, earth science, etc. A hyperspectral image may have bands in the order of hundreds up to a thousand. The hyperspectral images have numerous narrow bands. The hyperspectral images are useful in environmental monitoring, biotechnology, oil and gas exploration, etc. The ultra-spectral images have thousands or more bands. They are useful in high precision applications.

1.3.2

Types of Features Extracted from Remotely Sensed Images for Analysis

Feature of a remotely sensed image is a piece of information (possibly spectral values) relevant to solve a computational task related to a satellite image processing application. Feature can be of type texture, shape, and color.

6

1 Introduction to Remote Sensing

1.3.2.1

Texture

Textural features quantify texture in the center pixel of a local neighborhood of a remotely sensed image from the gray scale variations of the center pixel with the neighbor pixels. A detailed discussion on texture and literature survey on texture analysis is given in Chap. 2 of this book.

1.3.2.2

Shape

Shape features extract the outline of the area of interest in a remotely sensed image with the help of morphological operations and structural elements of various shapes like line, disk, cube, rectangle, square, sphere, etc. Ding et al. (2018) proposed a morphological building index (MBI) for extracting buildings from high-resolution remotely sensed images. They used a building extraction framework based on MBI and image segmentation techniques, spectral constraint, shadow constraint, and shape constraint and achieved correctness rate over 86%.

1.3.2.3

Color

Color refers to the relative shades of colors in an image. Tone or color is a fundamental property of an image which is directly related to the light reflected by the objects in the image. Like texture and shape, color information in a remotely sensed image is also a useful feature. A RGB image (color image) can be formed by combining the spectral values of three suitable bands in a remotely sensed image. For example, the simplest color feature, RGB histogram, can be found for an area of interest in the RGB image from the color values in each band. Yang and Newsam (2012) performed geographic image retrieval from a land use land cover ground truth dataset using simple statistics (mean and variance), texture (texture features based on Gabor filter), and color features (color histograms from three color spaces). The properties like invariance, robustness, density, and efficiency were evaluated. Out of the three color spaces RGB, hue lightness saturation (HLS), and CIE Lab experimented, color histograms from HLS color space gave optimal performance.

1.4

Resolution Characteristics of Remotely Sensed Imagery Data

The resolution characteristics of remotely sensed imagery are defined (Jensen 1996) and described as follows.

1.4 Resolution Characteristics of Remotely Sensed Imagery Data

1.4.1

7

Spatial Resolution

Remotely sensed images are composed of a matrix of picture elements called pixels which are the smallest units of an image. Image pixels represent a certain ground area. For example, the LISS-IV sensor has a spatial resolution of 5.8 m. It implies that each pixel in the multispectral image when displayed at full resolution represents an area of (5.8
5.8) sq.m on the ground.

1.4.2

Spectral Resolution

Spectral resolution describes the ability of a sensor to resolve fine wavelength intervals. Broad classes, such as water and vegetation, can be separated using the visible and NIR bands having broad wavelength ranges. For example, the LISS-IV sensor uses visible (in particular green and red) and NIR bands for capturing images. Other specific classes such as different rock types are not distinguishable using these broad wavelength ranges and require comparison at fine wavelength ranges. Thus, a sensor with higher spectral resolution is required.

1.4.3

Radiometric Resolution

The radiometric resolution of an imaging system describes the sensitivity of the sensor to detect subtle differences in energy emitted, reflected, or backscattered from the target. Imagery data are represented by positive digital numbers which vary from 0 to a selected power of 2. The maximum number of digital numbers available depends on the number of bits used in representing the energy recorded. So if a sensor uses 8 bits to record the data, its radiometric resolution is 8, and the image captured by it will have digital numbers ranging from 0 to 255 (28 ¼ 256). For example, the LISS IV sensor has a radiometric resolution of 7 bits.

1.4.4

Temporal Resolution

Revisit period refers to the length of time taken by a satellite to complete one entire orbit cycle. The temporal resolution of a remote sensing system refers to how often it records imagery of a particular area. So the absolute temporal resolution of a remote sensing system is equal to the revisit period of the satellite which is usually several days. However, because of some degree of overlap in the imaging swaths of adjacent orbits of a satellite, some areas of the earth tend to be imaged frequently. So, the

8

1 Introduction to Remote Sensing

actual temporal resolution of a sensor is lesser than the absolute temporal resolution. For example, the temporal resolution of LISS-IV sensor is 5 days.

1.5 1.5.1

Basics of Land Cover Classification Land Covers

Land cover refers to the surface cover on the ground. In National Land Cover Database 2011 (NLCD 2011) (https://www.mrlc.gov/data/legends/national-landcover-database-2011-nlcd2011-legend), land cover classes have been divided into eight major classes. They are described below. 1. Water class Open water and perennial ice/snow subclasses fall under water class. Areas of open water with less than 25% cover of vegetation or soil are designated as open water. Perennial ice/snow refers to areas characterized by a perennial cover of ice and/or snow, greater than 25% of total cover. 2. Developed class Areas with the constructed materials and vegetation characterize developed class. Open space, low intensity developed area, medium intensity developed area, and high intensity developed area subclasses fall under developed class. Open space contains impervious surfaces (constructed materials) less than 20% of total cover. Low intensity developed area contains impervious surfaces that account for 20–49% of total cover. Medium intensity developed area contains impervious surfaces that account for 50% to 79% of total cover. High intensity developed area contains impervious surfaces that account for 80% to 100% of total cover. 3. Barren class Barren land subclass falls under barren class. In barren land, vegetation accounts for less than 15% of total cover. 4. Forest class Areas dominated by trees greater than 5 meters tall and greater than 20% of total vegetation cover characterize forest class. Deciduous, evergreen, and mixed forests subclasses fall under forest class. In deciduous forest, more than 75% of the tree species shed foliage simultaneously in response to seasonal change. In evergreen forest, more than 75% of the tree species maintain their leaves all year. In mixed forest, neither deciduous nor evergreen species are greater than 75% of total tree cover. 5. Shrubland class Dwarf shrub and shrub subclasses fall under shrubland class. Dwarf shrub is often co-associated with grasses, sedges, herbs, and non-vascular vegetation. Shrub includes true shrubs, young trees in an early successional stage or trees stunted from environmental conditions.

1.5 Basics of Land Cover Classification

9

6. Herbaceous class Areas dominated by herbaceous vegetation, greater than 80% of total vegetation, characterize herbaceous class. Grass land, sedge, lichens, and moss subclasses fall under herbaceous class. 7. Planted/cultivated class Pasture/hay and cultivated crops subclasses fall under planted/cultivated class. In pasture/hay, pasture/hay vegetation accounts for greater than 20% of total vegetation. In cultivated crops, crop vegetation accounts for greater than 20% of total vegetation. 8. Wetlands class Areas where the soil or substrate is periodically saturated with or covered with water characterize wetlands class. Woody wetlands and herbaceous wetlands subclasses fall under wetlands class. Woody wetlands characterize areas where forest or shrubland vegetation accounts for greater than 20% of vegetative cover. Herbaceous wetlands characterize areas where perennial herbaceous vegetation accounts for greater than 80% of vegetative cover.

1.5.2

Spectral Reflectance Characteristics of Land Cover Classes

Having been exposed to various land covers, let us analyze the EM spectrum and the reflectance characteristics of various land covers. The regions of the EM spectrum useful for remote sensing are visible spectrum and infrared. Blue, green, and red are the primary colors of the visible spectrum. The wavelength of the visible region ranges from 0.45 μm to 0.69 μm. The next portion of the spectrum of interest is the infrared (IR) region which covers the wavelength range from approximately 0.76 μm to 12.5 μm. The infrared region can be subdivided into two types such as the reflected IR and the thermal IR. The reflected IR is a combination of NIR and Mid IR (MIR). The radiation in the reflected IR covers wavelengths from approximately 0.76 μm to 2.35 μm. The thermal IR region is quite different from the visible and reflected IR portions as it is emitted from the earth’s surface in the form of heat. The thermal IR falls after MIR. The wavelengths of the EM signals are shown in Table 1.2. From the typical spectral reflectance curves of the land cover classes (Fig. 1.1) such as vegetation, water, and soil (Swain and Davis 1978), the reflectance of water is seen only in the visible spectrum. The reflectance of clear water is low. However, the reflectance of water is maximal at the blue end of the spectrum and decreases as the wavelength increases. Coastal water mapping involves drawing the boundary between land and water along the coast. Blue band is useful in coastal water mapping. Vegetation can be distinguished from the other types of land covers in the optical/ NIR bands. In vegetation, light absorption by leaf segments is high in the visible region. Chlorophyll pigments (Huete 2004) absorb blue and red wavelengths for

10

1 Introduction to Remote Sensing

Table 1.2 Wavelength ranges of signals in EM spectrum Signals in EM spectrum Blue

Wavelength (in μm) (0.45–0.52) μm (0.52–0.6) μm (0.63–0.69) μm (0.76–0.9) μm

Green Red Near IR(NIR)

Mid IR(MIR)

(1.55–1.75) μm (10.4–12.5) μm (2.08–2.35) μm

Thermal IR Mid IR(MIR)

Principal applications Coastal water mapping, soil/water discrimination, and forest type mapping Vegetation discrimination and vigor assessment Plant species differentiation from chlorophyll absorption Determination of vegetation types, vigor, and biomass content, for delineating water bodies, and for soil moisture discrimination Indicative of vegetation moisture content and soil moisture. Useful to differentiate snow from clouds Useful in vegetation stress analysis, soil moisture discrimination, and thermal mapping applications Useful for discrimination of mineral and rock types. Also sensitive to vegetation moisture content

From Lillesand and Kiefer 1994

Dry bare soil (Gray-brown) Vegetation (Green) Water (Clear)

Reflectance (%)

60

40

20

0 0.4

0.6

0.8

1.0

1.2

1.4

1.6

1.8

2.0

2.2

2.4

2.6

Wavelength (µm)

Fig. 1.1 Typical spectral reflectance curves for vegetation, water, and soil. (Adapted from Swain and Davis 1978)

photosynthesis and reflect green wavelength, and that is why healthy vegetation appears green. Chlorophyll pigment is also present in algae living in surface water. The carotene and xanthophyll pigments absorb blue wavelength and reflect green and red wavelengths. Leaf pigment and cellulose in vegetation transmits and reflects NIR wavelength. This is the cause for such a high reflectance of vegetation in the NIR region of the curve. The sharp rise in reflectance of vegetation between red and NIR is called red edge. The red edge portion shifts to longer or shorter wavelengths

1.5 Basics of Land Cover Classification

11

depending on the chlorophyll content in the leaves, and so this property serves as an indicator of vegetation stress. It can be explained as follows. Healthy vegetation shows high reflectance in red region and low reflectance in NIR region, whereas stressed vegetation shows low reflectance in red region and high reflectance in NIR region. Among plant nutrients, nitrogen is one of the most important factors in maximizing the crop yields, soil fertility, and economic returns to farmers. The shift to longer wavelength in red edge portion is also an indicator of fall in nitrogen content leading to stressed vegetation (NRSC/ISRO 2010). Forest type mapping involves mapping various forest types like coniferous, deciduous, evergreen, etc. in a study area. The reflectance of soil goes high in MIR. Soil reflectance increases with increasing wavelength depending on the composition of the soil. So reflectance of soil in MIR band provides discrimination of mineral and rock types. Stressed vegetation can also be detected and analyzed using reflectance in thermal IR band.

1.5.3

Estimation of Land Cover Properties Using Spectral Reflectance and Spectral Indices

Spectral indices are useful in estimating land cover properties (Xue and Su 2017). Many vegetation indices are dominantly used in identifying vegetation. The most used vegetation index is the normalized difference vegetation index (NDVI), and it was proposed by Rouse Jr. et al. (1974). It is calculated using “Eq. 1.1.” NDVI ¼

ðNIR  RedÞ ðNIR þ RedÞ

ð1:1Þ

where NIR and Red are the reflectances in the respective bands. It also is an excellent qualitative measure of the vigor and density of vegetation. For vegetation, NDVI value is between 0 and 1. Leaf area index (LAI) (Sripada et al. 2005) is an important vegetation biophysical parameter associated with plant biological, environmental, and agronomic processes (Lu et al. 2018). It is defined as the area of single-sided leaves per area of soil (Zhang et al. 2012). Its green amount is also linked to vegetation photosynthesis, transpiration, and energy balance (Kucharik et al. 1998; Córcoles et al. 2013). Chlorophyll content is an essential vegetation biochemical property that controls over the amount of solar radiation and CO2 that a leaf absorbs for photosynthesis. Therefore, it provides valuable information about vegetation physiological status (Blackburn 2004; Darvishzadeh et al. 2010). Biomass is also a biophysical parameter of vegetation, and it is defined as the organic matter used for energy production or in various industrial processes as raw material for a range of products. The reflectance values in NIR band help in determining biomass.

12

1 Introduction to Remote Sensing

Normalized difference water index was proposed by McFeeters (1996) and is defined as in “Eq. 1.2.” The NDWI is used for estimating leaf water content using which drought can be predicted and precision farming can be implemented (NRSC/ ISRO 2010). The NDWI takes any value from 1 to 1. High NDWI implies high moisture content and low NDWI implies water stress. NDWI ¼

Green  NIR Green þ NIR

ð1:2Þ

Soil adjusted vegetation index (SAVI) was proposed by Huete (1988) to improve the sensitivity of NDVI to soil background. It is defined as in “Eq. 1.3.” SAVI ¼

ðNIR  RÞ
ð1 þ LÞ ðNIR þ R þ LÞ

ð1:3Þ

where L is the soil conditioning index. L takes any value from 0 to 1. In experiments, L is determined from the environmental conditions. When L is 0, SAVI equals NDVI. It is equal to 1 when the degree of vegetation coverage is high. A chromophore is a parameter or substance (chemical or physical) that significantly affects the shape and nature of a soil spectrum. A given soil sample consists of a variety of chromophores, which vary with environmental conditions. In soils, three major chemical chromophores (Ben-Dor 2002) can be roughly categorized as follows: minerals (mostly clay and iron oxides), organic matter (living and decomposing), and water (solid, liquid, and gas phases). Soil moisture is defined as the amount of water soil can hold. The reflectance of soil in MIR also provides soil moisture discrimination. The physical properties of soil are soil color, soil texture (refers to the relative proportions of sand, silt, and clay in soil), structure, soil moisture, surface roughness, etc. Soil color is an important parameter that allows the diagnosis of soil types and their properties, as well as the detection of changes affecting ecosystems like erosion, salinization, and alkalization (NRSC/ISRO 2010). A detailed investigation by Escadafal (1993) revealed that for measuring soil color, reflectance values of blue, green, and red bands are used.

1.5.4

Estimation of Land Cover Properties Using Texture

A land cover property map can be constructed from the reflectance values in various bands. Similar to finding land cover properties from imagery spectral information, texture features can be calculated from the spatial scales of ground features to improve the accuracy of estimating land cover properties (Colombo et al. 2003; Sarker and Nichol 2011). Colombo et al. (2003) used the IKONOS image with 1 m spatial resolution as input to create LAI map for a study area. Field measurements of LAI for different vegetation types were made. Seven spectral vegetation indices (SVI) were found from the imagery spectral data. Through analysis, the overall

1.6 Remote Sensing Applications

13

relationship between the field measurements of LAI and the measurements of LAI from spectral indices for different vegetation types was found inadequate for mapping. So the relationship between LAI-SVI was improved by going for SVI-texture combination and in some cases SVI-geostatistical information combination. Sarker and Nichol (2011) used high-resolution optical data from ALOS AVNIR-2 sensor as input data to estimate biomass. They used spectral reflectance and simple spectral band ratio, commonly used vegetation indices, texture parameters, and ratio of texture parameters. Through analysis, it was found that spectral reflectance and simple spectral band ratio explained only 58% of variability in the field of data, whereas texture features explained 76% of variability in the field of data. Results indicated that texture features provided better biomass estimation, and the results could be further improved by going for the ratio of texture parameters which combined texture and ratio. Boutsoukis et al. (2019) proposed an overall method for canopy height (a biophysical parameter of forest) estimation from texture analysis of a single 2D multispectral image. An object-oriented classification approach was followed to segment land cover objects. Global texture descriptors were found for each land cover object. A multi-resolution training set was used to provide height discrimination among land cover objects. The results were promising when compared against a reference LIDAR-derived canopy height model.

1.5.5

Land Cover Classification

Land cover classification involves classifying the landscape into land covers based on its features. Digital image processing techniques are vital in performing land cover classification. Remotely sensed image of a landscape and its ground truth obtained through field visit are needed for performing land cover classification and subsequent validation of the classified result. This book aims at developing tailormade Matlab applications that use textural approach for performing land cover classification. A detailed literature survey on texture-based land cover classification is listed in Sections from 2.4 to 2.7.

1.6

Remote Sensing Applications

Remote sensing applications have been categorized into 16 types depending on the domain where they are applied (NRSC/ISRO 2010). The domains are agriculture (crop type classification, crop monitoring and assessment, and crop parameter estimation), land use land cover (land use/cover classification, change detection), forest and vegetation (forest cover assessment, forest type mapping, biomass assessment, and biodiversity assessment), soil and land degradation (soil mapping, land degradation mapping, soil moisture estimation), urban and regional planning, water resources management (surface water mapping and monitoring, flood disaster

14

1 Introduction to Remote Sensing

monitoring and management, and evapotranspiration analysis), geosciences (geological mapping, lithological mapping, and geomorphological mapping), mineral exploration, geoenvironmental studies, geoengineering studies, groundwater (groundwater estimation), oceans (coastal zone management), atmosphere (atmospheric parameters estimation), cyclones, flood disaster management, agricultural drought monitoring and assessment, landslides, earthquakes, and forest fire monitoring. Some applications are described in the following subsections.

1.6.1

Land Use Land Cover (LULC)

Land use refers to the purpose the land serves such as recreation, wildlife habitat, or agriculture. Land use studies in remote sensing facilitate the removal of urban encroachment, depletion of forests, etc. The knowledge derived from land cover classification (see Sect. 1.6) of remotely sensed images is vital to monitor and develop strategies for natural resource conservation (for related literature see Sections from 2.4 to 2.7).

1.6.2

Change Detection

Change detection is a technique used in remote sensing to compare, determine, and evaluate the changes in a particular land cover between two or more time periods. One can also assess the development of a city from the changes in the land terrain. Zhu (2017) divided the Landsat time series change detection algorithms (based on the mathematical approach used for detecting change) into six major categories including thresholding, differencing, segmentation, trajectory classification, statistical boundary, and regression. The algorithms were reviewed based on two categories, namely, change target (knowing what is changing) and change agent (understanding the cause of change), and significant conclusions were arrived at and reported.

1.6.3

Agriculture

Agricultural applications of remote sensing include the following. Useful inferences can be obtained from images of crops starting from initial to final stages of cultivation such as mapping of soil types, crop type classification, crop condition assessment, and crop yield estimation. Kussul et al. (2017) classified crop types of 19 multi-temporal scenes acquired by Landsat-8 and Sentinel-1A RS satellites pertaining to a test site in Ukraine. They concluded that convolutional neural

References

15

network (CNN) outperformed multilayer perceptron (MLP) and yielded classification accuracy over 85%.

1.6.4

Mineral Exploration

Geologists use optical and radar remote sensing for mineral exploration. The mineral exploration application requires in addition to the information extracted about the land covers from remotely sensed images, information on surface topography. In order to identify the potential fluoride contamination ground water zones in West Bengal, Thapa et al. (2017) collected digital elevation model (DEM) data from CartoDEM with a resolution of 2.5 m and prepared elevation map and drainage density map. They also collected geology and geomorphology map, fault and lineament map, and hydrogeological map. Different thematic maps were prepared using GIS software. The weighted overlay analysis was carried out to generate the final potential fluoride contamination zones. Exercises 1. Consider a Worldview-2 image, and find out its spatial, spectral, radiometric, and temporal resolution. 2. Prepare a list of satellite sensors in the increasing order of spatial resolution. 3. Find out the specifications of the satellite images (e.g., its type, resolution, etc.) required to extract objects (like buildings, road, trees, etc.). 4. Find out how B2 (green), B3 (red), and B4 (NIR) of LISS-IV image can be combined to produce a RGB image. 5. Find out the specifications of the satellite image required to extract and classify different types of rock and soil. 6. Identify various land covers in the district where you live. 7. Prepare a list of biophysical and biochemical properties of the major land cover types such as vegetation, water, and soil. 8. Prepare a list of spectral indices available in remote sensing.

References Ben-Dor E (2002) Quantitative remote sensing of soil properties. Adv Agron 75:173–244 Blackburn GA (2004) Wavelet decomposition of hyperspectral reflectance data for quantifying photosynthetic pigment concentrations in vegetation. In: Proceedings of the XXth ISPRS Congress; Commission, vol 7, pp 12–23 Boutsoukis C, Manakos I, Heurich M, Delopoulos A (2019) Canopy height estimation from single multispectral 2D airborne imagery using texture analysis and machine learning in structurally rich temperate forests. Remote Sens 11(23):2853 Colombo R, Bellingeri D, Fasolini D, Marino CM (2003) Retrieval of leaf area index in different vegetation types using high resolution satellite data. Remote Sens Environ 86(1):120–131

16

1 Introduction to Remote Sensing

Córcoles JI, Ortega JF, Hernández D, Moreno MA (2013) Estimation of leaf area index in onion (Allium cepa L.) using an unmanned aerial vehicle. Biosyst Eng 115(1):31–42 Darvishzadeh R, Atzberger C, Skidmore A, Schlerf M (2010) Retrieval of vegetation biochemicals using a radiative transfer model and hyperspectral data. ISPRS Ding Z, Wang XQ, Li YL, Zhang SS (2018) Study on building extraction from high-resolution images using Mbi. Int Arch Photogramm Remote Sens Spat Inf Sci 42(3) Escadafal R (1993) Remote sensing of soil color: principles and applications. Remote Sens Rev 7 (3–4):261–279 https://www.isro.gov.in/launchers https://www.mrlc.gov/data/legends/national-land-cover-database-2011-nlcd2011-legend https://www.nrsc.gov.in/sites/default/files/pdf/brresourcesat1.pdf Huete A (1988) Huete, AR A soil-adjusted vegetation index (SAVI). Remote Sens Environ 25:295–309 Huete AR (2004) Remote sensing for environmental monitoring. In: Environmental monitoring and characterization. Academic, pp 183–206 Jensen JR (1996) Introductory digital image processing: a remote sensing perspective. Prentice Hall, Upper Saddle River Kucharik CJ, Norman JM, Gower ST (1998) Measurements of branch area and adjusting leaf area index indirect measurements. Agric For Meteorol 91(1–2):69–88 Kussul N, Lavreniuk M, Skakun S, Shelestov A (2017) Deep learning classification of land cover and crop types using remote sensing data. IEEE Geosci Remote Sens Lett 14(5):778–782 Lillesand T, Kiefer RW (1994) Remote sensing and image interpretation. Wiley, Chichester Lillesand TM, Kiefer RW (2015) Remote sensing and image interpretation, 7th edn. Wiley, New York Lu B, He Y, Liu HH (2018) Mapping vegetation biophysical and biochemical properties using unmanned aerial vehicles-acquired imagery. Int J Remote Sens 39(15–16):5265–5287 McFeeters SK (1996) The use of the Normalized Difference Water Index (NDWI) in the delineation of open water features. Int J Remote Sens 17(7):1425–1432 NRSC/ISRO (2010) Ebook on remote sensing applications Rouse JW Jr, Haas RH, Schell JA, Deering DW (1974) Monitoring vegetation systems in the great Plains with Erts, vol 351. NASA Special Publication, Washington, DC, p 309 Sarker LR, Nichol JE (2011) Improved forest biomass estimates using ALOS AVNIR-2 texture indices. Remote Sens Environ 115(4):968–977 Sripada RP, Heiniger RW, White JG, Weisz R (2005) Aerial color infrared photography for determining late-season nitrogen requirements in corn. Agron J 97(5):1443–1451 Swain PH, Davis SM (eds) (1978) Remote sensing: the quantitative approach. McGraw-Hill, New York Thapa R, Gupta S, Kaur H (2017) Delineation of potential fluoride contamination zones in Birbhum, West Bengal, India, using remote sensing and GIS techniques. Arab J Geosci 10(23):1–18 Xue J, Su B (2017) Significant remote sensing vegetation indices: a review of developments and applications. J Sens 2017 Yang Y, Newsam S (2012) Geographic image retrieval using local invariant features. IEEE Trans Geosci Remote Sens 51(2):818–832 Zhang B, Wu D, Zhang L, Jiao Q, Li Q (2012) Application of hyperspectral remote sensing for environment monitoring in mining areas. Environ Earth Sci 65(3):649–658 Zhu Z (2017) Change detection using landsat time series: a review of frequencies, preprocessing, algorithms, and applications. ISPRS J Photogramm Remote Sens 130:370–384

Chapter 2

Introduction to Texture and Related Work

Abstract This chapter explores texture, texture model, and properties of texture in image processing. It exposes standard texture dataset to the reader which is vital for the validation of any new texture model. It also introduces texture analysis applications where the reader can contribute useful applications by the end of reading this book. This chapter also exposes readers to texture-related literature that includes survey of texture models, structural texture models, statistical texture models, spectral texture models, model-based texture models, fuzzy logic-based texture models, combined approach models (where in addition to texture other features like shape and color are jointly used), classifiers, and distance measures. Keywords Texture · Texture model · Texture analysis · Texture dataset · Literature on texture models · Literature on classifier · Literature on distance measures

2.1

Introduction

As the focus of the book is to impart knowledge in implementing texture-based image processing concepts in remote sensing applications, it is necessary to introduce texture to the readers. The chapter is organized as follows. Section 2.2 describes the basics of texture, and Sect. 2.3 describes various texture analysis applications. As the reader will be at this point familiar with the basics of remote sensing and texture, a detailed survey of literature related to texture analysis is carried out. Section 2.4 discusses the survey papers on texture models. Section 2.5 discusses the literature on the existing texture models falling under different types. Section 2.6 explores the literature on classifiers, and Sect. 2.7 explores the distance measures applied in literature on texture analysis. By the end of this chapter, the reader will be familiar with the following: • • • •

Define texture and a texture model. Identify the presence and type of texture in a digital image. Understand the role of standard texture datasets. Be aware of the real-world applications where texture analysis is applied.

© The Author(s), under exclusive license to Springer Nature Switzerland AG 2021 S. Jenicka, Land Cover Classification of Remotely Sensed Images, https://doi.org/10.1007/978-3-030-66595-1_2

17

18

• • • •

2

Introduction to Texture and Related Work

Understand and survey the literature on texture models. Understand the basic idea and chronological history behind a texture model. Classify any texture model into its type. Identify the classifiers and distance measures applied in texture-based applications.

2.2 2.2.1

Basics of Texture Texture

Texture is characterized by tonal primitive properties as well as spatial relationships between the spectral values in an image (Haralick 1979). Texture defines the properties of an image that characterize coarseness, regularity, and smoothness (Gonzalez and Woods 2002). Patterns can be recognized using texture as each pattern exhibits different surface characteristics or combinations of coarseness, smoothness, and regularity.

2.2.2

Texture Model

A texture model tries to quantify the texture property in a local neighborhood of an image. The spatial relationship of pixels in a local neighborhood is captured by finding a number that characterizes the gray scale variations of the center pixel with the neighbor pixels.

2.2.3

Properties of Texture

Textures can be weak, strong, and constant textures. A texture image is composed of the basic unit called the texture element or texture primitive. Weak textures have less spatial relationships between primitives in contrast to strong textures which have regular spatial relationships among primitives. Texture in an image region has a constant texture if a set of its local properties in the region is constant, slowly changing, or approximately periodic. Textures are coarse and fine in nature. Coarse textures consist of larger texture primitives and lower spatial frequencies than fine textures which consist of relatively smaller texture primitives and higher spatial frequencies. With respect to human visual perception, texture defines properties (Fig. 2.1) like coarseness, contrast, directionality, line likeness, regularity, and roughness (Asendorf and Hermes 1996; Gibson and Bridgeman 1987). Contrast refers to the luminance difference projected between two adjacent portions in an image. In directional textures, the gray scale variations are oriented

2.3 Texture Analysis in Image Processing Applications

19

Fig. 2.1 Properties of texture

in a particular angle (like 0 , 45 , 90 , 135 , etc.). Regular texture has a periodic arrangement of texture primitives. Rough texture exhibits surface roughness. Random textures have random orientation of texture primitives. A texture image is said to have line likeness if it is composed of lines.

2.2.4

Standard Texture Dataset

Standard texture dataset is essential for validating a texture model. Any proven texture model should be capable of recognizing the texture classes present in the texture dataset. The Brodatz book (Brodatz 1966) consists of a variety of textures, both small and large grained. The Brodatz texture database consists of the photographic prints of the textures which were digitized (as gray scale images of 640  640 size with 0 to 255 possible gray levels) and histogram equalized. It is a benchmark database that has 111 textures named as D4, D9, etc. where the numbers refer to the page numbers in the Brodatz album.

2.3

Texture Analysis in Image Processing Applications

Apart from various applications in remote sensing, texture analysis-based methods are widely used in other image processing applications like face recognition, image enhancement, fingerprint recognition, content-based image retrieval, pattern classification, pattern segmentation, and edge detection. Face recognition involves identifying the face of a person from the facial details. Texture features (Rivera et al. 2012) have been proved to capture the facial details effectively. Image enhancement involves improving the quality and information content in the original image. Rubel

20

2

Introduction to Texture and Related Work

et al. (2016) proposed discrete cosine transform (DCT)-based filtering for denoising and enhancing the texture images. Fingerprint recognition involves recognizing the identity of a person by matching two fingerprints. Cao and Jain (2018) proposed a texture-based approach to improve latent fingerprint recognition accuracy. Contentbased image retrieval (CBIR) involves reading a query image and retrieving the closely matching images from the image database. Texture feature extraction provides accurate pattern matching (Kumar and Mohan 2019) in CBIR. Pattern classification involves classifying the class label of a test pattern based on its features. Texture features (Ojala et al. 2000) yield promising results in pattern classification. Pattern segmentation involves partitioning the input image into meaningful regions. Pixels in the input image are partitioned or grouped based on the similarities of their texture features (Arivazhagan and Ganesan 2003). Edge detection involves detecting the edges in an input image. A point lies on an edge if there is a marked change in image intensity in the neighborhood of the point. So filtering can be performed in the neighborhood of a point. As texture features capture the spatial relationship in a neighborhood, they are useful in detecting edges (Laws 1980a, b).

2.4

Survey Papers on Texture Models

Many survey papers on texture models are found in literature. Weszka et al. (1976) studied the performances of the three texture analysis algorithms on 54 aerial photographic terrain samples belonging to 9 land use classes and extended the experiment to 180 LANDSAT imagery samples belonging to 3 geological terrain types. The algorithms make use of features based on the Fourier power spectrum, second-order gray level statistics, and first-order statistics of gray level differences. It was found that the Fourier features performed poorly while the other feature sets performed comparably. Conners and Harlow (1980) concluded that co-occurrence method performed better when compared with run-length difference, gray level difference density, and power spectrum. A survey of texture segmentation and feature extraction methods (Reed and Dubuf 1993) were performed with emphasis on techniques developed since 1980 for unsupervised segmentation applications. They concluded that various methods for texture feature extraction had been proposed previously, but the texture analysis problem remained difficult and became subject to intensive research. Pal and Pal (1993) analyzed several fuzzy set approaches and neural network methods for segmentation of gray level images, but they suggested that it was difficult to find a single quantitative index because many factors like homogeneity, contrast, compactness, continuity, and psychovisual perceptions were to be taken into account. Ojala et al. (1996) analyzed the performance of some texture measures used in various applications and some new promising approaches proposed in the recent past. For classification, a method based on Kullback discrimination of sample and prototype distributions was used. The classification results for single features with one-dimensional feature value distributions and for pairs of complementary features with two-dimensional distributions

2.5 Papers on Texture Models Used for Characterizing Images

21

were presented. Randen and Husoy (1999) reviewed many major filtering approaches to texture feature extraction and performed a comparative study. Filtering approaches reviewed were Law’s masks, ring/wedge filters, dyadic Gabor filter banks, wavelet transforms, wavelet packets and wavelet frames, quadrature mirror filters, discrete cosine transform, Eigen filters, optimized Gabor filters, linear predictors, and optimized finite impulse response filters. The features were computed as the local energy of the filter responses. The effect of the filtering was highlighted, keeping the local energy function and the classification algorithm identical for all the approaches. For reference, comparisons with two classical non-filtering approaches such as co-occurrence (statistical) and autoregressive (model based) features were considered. A ranking of the tested approaches based on the extensive experiments was presented. Suruliandi and Jenicka (2015) performed a comparative study of a range of statistical and spectral models applied in land cover classification and found multivariate local texture pattern (MLTP) proposed by Suruliandi (2009) to outperform other texture models based on classification accuracy and kappa statistics obtained for the classified results. Ratajczak et al. (2019) proposed a novel dataset of land cover patches and carried out a comparative study of texture features and classification algorithms. They evaluated 12 texture features (including local binary pattern (LBP) and its variants) and applied them on 4 optimized classifiers and 5 deep convolutional neural networks (DCNN). A novel feature named light combination of local binary pattern (LCoLBP) was found efficient from the study.

2.5

Papers on Texture Models Used for Characterizing Images

Texture models are divided into structural, statistical, spectral, model-based, fuzzy logic-based, and combined approach models. A detailed survey of literature is done on each type of texture model in the following subsections.

2.5.1

Structural Texture Models

Structural models describe texture by its primitives and placement rules. Van Gool et al. (1985) discussed about the problem of texture segmentation after some essential preprocessing steps. Haralick (1979) suggested that region-based (structural) models were more realistic in that they treated texture as a layout of regions, as called for by structure of the physical texture. Julesz (1986) suggested identifying the basic elements of image texture called texture elements or texels, obtaining the texel model, and performing texel-based segmentation. Rao (1990) recorded that a texture region was represented by its symbolic representation from which a quantitative reconstruction of the original texture could be obtained. The technique found was

22

2

Introduction to Texture and Related Work

useful in describing complex flow visualization pictures. Textures were considered as primitives called tokens. Tuceryan and Jain (1990) proposed a texture segmentation algorithm based on Voronoi tessellation of the tokens. The algorithm computed feature vectors based on Voronoi polygon followed by probabilistic token labeling to identify regions of texture. The algorithm successfully segmented texture images. Mirmehdi and Petrou (2000) generated a multi-scale representation of the texture image by a multiband smoothing algorithm for segmentation of neighbor image textures. Initial segmentation was achieved by applying a clustering algorithm to the image at the coarsest level of smoothing. The process was then propagated through finer levels of smoothing until a full segmentation was achieved at the highest level of resolution. There were several attempts to mathematically define the notion of textons. Zhu et al. (2005) modeled texture as a superposition of Gabor base functions, which were in turn generated by a user-specified vocabulary of textons. Todorovic and Ahuja (2009) used a texture segmentation algorithm to identify the modes of the probability density function in a texture image because the texture image contained a large number of statistically similar texels which gave rise to the modes in the probability density functions. Experiments demonstrated that the approach led to competitive performance relative to the state-of-the-art methods. Kumar and Manimegalai (2020) proposed a near lossless compression technique for images where repeated fractal patterns were stored to compress the image. The proposed approach gave significant improvements in terms of compression ratio, peak signal to noise ratio (PSNR), and other compression parameters.

2.5.2

Statistical Texture Models

Haralick et al. (1973) proposed the gray level co-occurrence matrix (GLCM) method for textural information extraction. These co-occurrence matrices with 14 statistical texture measures provided promising results. These features represented the texture properties of the image with the given displacement vector. Users could calculate several co-occurrence matrices in different directions, for example, 0 , 45 , 90 , and 135 . Laws’ texture energy measures (Laws 1980a) and their variants proposed required simple convolution and moving window updating techniques, and they were invariant to changes in illumination, contrast, and orientation. These methods do not require any preprocessing techniques. The texture energy measures developed by Laws had the advantage of being able to discriminate well between the different textures and were quick (Laws 1980b) and easy to compute. Goldfarb (1984) concluded that the universality, flexibility, and the ability to connect intrinsically low-level process that selected the primitives for the pattern representation with the higher level recognition process made the newly proposed analytical computational model superior to other contemporary models. According to Haralick and Shapiro (1985), clustering was grouping in measurement space, while segmentation tried to do groupings in spatial domain. Unser (1986) used histograms of sum and difference between gray levels of neighbor pixels of a local region for pattern characterization.

2.5 Papers on Texture Models Used for Characterizing Images

23

Bovik et al. (1990) used complex Gabor filters for pattern description. Instead of using a fixed set of filters, they applied a simple peak finding algorithm to the power spectrum of the image in order to determine the radial frequencies of the appropriate Gabor filters. He and Wang (1990) proposed the “texture spectrum” based on texture units, which characterized the local texture information in all eight directions. Texture Spectrum method was applied to texture feature extraction (1991a), texture classification (Wang and He 1990a; He and Wang 1992), texture analysis (Wang and He 1990b; He and Wang 1990), and textural filtering (He and Wang 1991b). The texture spectrum was widely mentioned and reported in the literature. Texture spectrum could be directly used for image classification and image analysis applications. A multivariate rotation-invariant simultaneous autoregressive (RISAR) model (Mao and Jain 1992) was introduced which was based on the circular autoregressive (CAR) model. Experiments showed that the multivariate RISAR model outperformed the CAR model in texture classification. Integrating the information extracted from multi-resolution simultaneous autoregressive models gave much better performance than single resolution methods in both texture classification and texture segmentation. Topi et al. (2000) proposed multi-predicate local binary pattern operator for texture classification. Ojala et al. (2000) proposed local binary pattern (LBP) for pattern description of gray level images. They (2001) proved that the proposed local binary pattern operator was gray scale and rotation invariant. The LBP texture descriptor played an important role in classification of texture images. The classification accuracies of LBP and its derivatives were found better in many applications. Turtinen et al. (2003) suggested that the combination of LBP and self-organizing map (SOM) gave better performance when log likelihood distance was used as a dissimilarity measure instead of Euclidean distance. Ahonen et al. (2004) proposed a novel facial representation for face recognition based on LBP features. In the work, they compared four different texture features for describing the appearances of local regions. The experimental results clearly showed and confirmed the validity of using LBP for face description. Their results showed that with suitable descriptors, the recognition rate of the proposed approach exceeded the recognition rates of principal components analysis (PCA) algorithm. Local binary patterns and texton histograms with the proposed extension were found to be the most suitable descriptors for face representation. Lucieer et al. (2005) proposed multivariate local binary pattern (MLBP) for texture-based classification of remotely sensed images. In MLBP, nine pattern units incorporating the cross relations between bands were added to form the feature vector of the color image. Benčo and Hudec (2007) proposed color GLCM extending GLCM, a texture representation for gray level images to color images. Liao and Chung (2007) proposed advanced local binary pattern (ALBP) for texture classification which was calculated by taking the single minimum value obtained through repeated left circular shift operation of LBP pattern unit. It was proved that ALBP characterized the local and global texture information and was robust in discriminating texture. Tan and Triggs (2007) introduced local ternary patterns for face recognition combining the strengths of robust illumination normalization, local texture-based face representations, and distance transform-based matching metrics. It was also shown that replacing local

24

2

Introduction to Texture and Related Work

histogramming with a local distance transform-based similarity metric further improved the performance of LBP or local ternary pattern-based face recognition. The pattern descriptor was found suitable for face recognition under difficult lighting conditions. Suruliandi and Ramar (2008) proposed local texture pattern (LTP) for gray level images. The model used three discrete output levels such as 0, 1, and 9 unlike LBP that used only two discrete output levels such as 0 and 1. It also assigned unique pattern labels to patterns. The model gave promising results for texture-based classification and segmentation. The LTP descriptor developed for gray level images was later extended to remotely sensed images as MLTP descriptor (Suruliandi 2009). Seng et al. (2008) proposed local Moran index (local spatial statistics for remotely sensed images of Mangrove) to improve the accuracy of classification which was found effective. Liao et al. (2009) proposed dominant local binary pattern (DLBP) for texture classification of standard textures by finding the dominant patterns which contributed to 80% of pattern occurrences. The method used histograms of dominant patterns for pattern description. Raju et al. (2010) proposed local derivative pattern (LDP) that captured pattern in face with first order, second order, and third order micro patterns. The LDP descriptor captured pattern unit in different angles. Rivera et al. (2012) proposed local directional number (LDN) texture measure for face recognition and obtained promising results. Jenicka and Suruliandi (2014) proposed multivariate discrete local texture pattern (MDLTP) with four discrete output levels for capturing texture features in remotely sensed images and obtained promising classified results. They (2015) also proposed multivariate ternary pattern (MTP) with three discrete output levels and found support vector machine (SVM) to produce better classification accuracy than the chosen classifiers taken for experimentation. Kumar and Mohan (2019) proposed a novel texture model called local mean differential excitation pattern (LMDeP) for CBIR which proved to be robust to illumination variations and noise.

2.5.3

Spectral Texture Models

Zhang et al. (2000) found Gabor wavelet-based texture representation to be very useful for content-based image retrieval. Zhang et al. (2002) proposed a new method using circular Gabor filters (CGF) for rotation invariant texture segmentation. They modified the traditional Gabor function into a circular symmetric version. Arivazhagan and Ganesan (2003) performed texture segmentation of gray level images using wavelet transform. They reported that co-occurrence features obtained from wavelet coefficients of image provided good texture discrimination. Arivazhagan et al. (2006) also used rotation invariant mean and variance features computed from Gabor coefficients of image for performing texture segmentation of gray level images and got promising results. Mao et al. (2012) presented an efficient and practical approach for automatic, unsupervised object detection and segmentation in two-texture images based on the concept of Gabor filter optimization. The entire process occurred within a hierarchical framework and consisted of object

2.5 Papers on Texture Models Used for Characterizing Images

25

detection, coarse segmentation, and fine segmentation. The proposed approach, being unsupervised and automatic, had the potential utility in machine vision applications that dealt with two-texture images. Wu et al. (2010) carried out the study of detecting land cover changes in remotely sensed data for environmental monitoring using the two types of texture measures such as Laws’ masks and Gabor filters. Murugan et al. (2010) proved that Gabor principal components analysis (Gabor PCA)-based method for face recognition outperformed PCA-based Eigenface method. Chen et al. (2017) proposed a method for hyperspectral image classification using texture features from Gabor filters and deep learning convolutional neural network, and it provided competitive results. Cid et al. (2017) proposed a method for 3D solid texture classification. They proposed and compared three novel local alignment criteria for higher order 3D Riesz wavelet transforms. The alignment methods provided better accuracy than the accuracies previously reported on the experimental database.

2.5.4

Model-Based Texture Models

Baum and Petrie (1966) proposed hidden Markov model (HMM) and described how statistical inference can be derived for probabilistic functions of finite state Markov chains. A hidden Markov model-based method (Younis et al. 2007) was introduced for co-segmentation of MRI and MRSI data of the brain. The technique demonstrated the ability of HMM to handle the co-analysis of MRI and MRSI for the purpose of improving the accuracy of MRI segmentation as well as the quantification of brain metabolites. The technique involving parallel HMMs that separately analyzed brain MRI and MRSI data and combined segmentation results demonstrated better accuracy and faster segmentation times when MRI and MRSI data were analyzed separately compared to the co-analysis of combined MRI and MRSI data. Leite et al. (2011) proposed a HMM-based method for classifying crop types based on the plant phenology for different crop classes. The proposed method classified based on analyzing the spectral profiles of different crop types over a sequence of temporal images. Song et al. (2019) proposed an unsupervised color texture segmentation algorithm using a multi-scale region level Markov random field model (MsRMRF) for segmenting color texture images. The experiments showed that the proposed method showed competitive performance with the state-of-the-art methods. Jenicka and Suruliandi (2016) proposed a distributed algorithm for performing land cover classification of remotely sensed image. As the testing phase of HMM was distributed among four parallel workers, the algorithm achieved reasonable speedup. Zhang et al. (2019b) used HMM for gesture recognition. In the paper they used two new shape features which were used for training HMM. After identifying gestures hierarchically, they used wavelet texture energy features for improving recognition rate.

26

2.5.5

2

Introduction to Texture and Related Work

Fuzzy Logic-Based Texture Models

All real-world pattern classification applications involve uncertainty stressing the need for the application of soft computing techniques. Jiji and Ganesan (2005) proposed fuzzy-based approach that performed segmentation through decisionmaking based on local and global fuzzy texture description and produced promising results. Chatzichristofis and Boutalis (2008) proposed the extraction of a new low level feature that combined, in one histogram, neighbor and texture information called fuzzy neighbor and texture histogram (FCTH). The proposed feature was suitable for accurately retrieving images even in distortion cases such as deformations, noise, and smoothing. Iakovidis et al. (2008) developed the fuzzy local binary pattern (FLBP) descriptor for texture characterization of B scan ultrasound images. Chen and Ji (2010) compared four commonly used fuzzy analytical methods for remote sensing digital image classification, i.e., fuzzy c-means, semi-supervised fuzzy cluster labeling, fuzzy nearest neighbor, and object-oriented fuzzy classifiers. Results showed that the fuzzy labeling approach produced the highest quality, which was followed by the object-oriented fuzzy classifier. Srinivasan et al. (2011) proposed fuzzy local texture pattern (FLTP) for feature extraction of gray level images. They used fuzzy membership values instead of discrete values 0 and 1 for calculating fuzzy texture descriptor. In FLBP (Iakovidis et al. 2008) and FLTP (Srinivasan et al. 2011), it was emphasized that the fuzzy-based descriptors performed better than their basic descriptors. Shankar et al. (2011) proposed a wavelet feature-based supervised scheme for fuzzy classification of land covers in multispectral remote sensing images. The proposed scheme was developed in the framework of wavelet-fuzzy hybridization, a soft computing approach. Four different fuzzy classifiers were chosen for the purpose and evaluated using different wavelet features. Further, the performance of the Biorthogonal3.3 (Bior3.3) wavelet was observed to be superior to other wavelets. The Bior3.3 wavelet in combination with fuzzy product aggregation reasoning-rule outperformed all other methods. Potentiality of the proposed soft computing approach in isolating various land covers was evaluated both visually and quantitatively using indices like measure of homogeneity and Xie-Beni measure of compactness and separability. Reddy et al. (2017) proposed a novel face recognition integrated matrix known as fuzzy-based texture unit and GLCM. The novel method addresses the challenges of face recognition like high dimensionality of features extracted in existing texture models, presence of noise in face images, and face images with expression and illumination variation. Jenicka (2019) proposed multivariate ternary fuzzy pattern for land cover classification of remotely sensed images.

2.6 Papers on Classifiers Applied in Texture-Based Study

2.5.6

27

Combined (Texture, Shape, and Color) Approach Models

Choi et al. (2011) proposed new neighbor local texture features, i.e., neighbor local Gabor wavelets (CLGWs) and neighbor local binary pattern (CLBP), for the purpose of face recognition. The proposed neighbor local texture features were able to exploit the discriminative information derived from spatio-chromatic texture patterns of different spectral channels within a certain local face region. The features maximized the complementary effect of neighbor and texture information. Lee et al. (2011) proposed a novel face descriptor based on neighbor information called LCVBP for face recognition. The proposed LCVBP consisted of two discriminative patterns such as neighbor norm patterns and neighbor angular patterns. Experimental results showed that the proposed LCVBP feature was able to yield promising results for face images with challenges. As many feature extraction techniques are available, Gupta et al. (2016) selected optimal features for land cover classification using SAR data. For optimal feature selection, they examined four types of features, namely, polarimetric features, texture features, color features, and wavelet features. Separability index of each of the land cover class (urban, water, bare soil, short vegetation, and tall vegetation) with the rest of the classes was computed. Through separability index, they found optimal features for segregating each land cover class. Naive Bayes classifier was used for classifying SAR data with the optimal features, and high classification accuracy was achieved.

2.6

Papers on Classifiers Applied in Texture-Based Study

Many classification algorithms have been used in papers. Classification algorithms are responsible for finding a separating plane that keeps dissimilar samples apart and similar samples together. The classifier should also minimize error and separate outliers which lead to over fitting prior to classification. Among many classification algorithms used, Fuzzy c-means (FCM) (Dunn 1973; Bezdek 1981), fuzzy k-Nearest Neighbor (Keller et al. 1985), SVM (Vapnik 1982; Cortes and Vapnik 1995), relevance vector machine (RVM) (Tipping 2000), SOM (Kohonen 1982), adaptive neuro-fuzzy inference system (ANFIS) (Jang 1993), extreme learning machine (ELM) (Huang 2003; Huang et al. 2004, 2006), and fuzzy-based classifiers (Nedeljkovic 2004) are reported in recent literature. Cover and Hart (1967) proposed k-NN classifier. Ojala et al. (2000) used k-Nearest Neighbor (k-NN) classifier for texture-based classification in most of his papers. The supervised, unsupervised, and semi-supervised classification of remotely sensed images were performed with several fuzzy classifiers including Fuzzy k-NN (Keller et al. 1985), and promising results were obtained. Jenicka (2018) proposed a land cover classification algorithm using multivariate binary threshold pattern (MBTP) and fuzzy k-NN classifier and obtained better results. Texture classification of remotely sensed images was

28

2

Introduction to Texture and Related Work

performed using SVM (Hermes et al. 1999). It was suggested that SVM was more suitable for heterogeneous samples for which only a few number of training samples were available. Automatic image categorization using low-level features is a challenging research topic in remote sensing application. Ge et al. (2008) performed an effective feature mapping through a chosen metric distance function. Sparse SVM was adopted to dramatically reduce the regions that were needed to classify images. It had been shown that it could produce better results than the k-NN for supervised classification. Turtinen et al. (2003) performed texture classification of gray level images using SOM. It was suggested that the combination of LBP and SOM gave better performance when log likelihood distance was used as a dissimilarity measure instead of Euclidean distance. Nedeljkovic (2004) concluded that the supervised classification of remotely sensed images could be performed with fuzzy classifiers and the classification accuracy obtained was better than the pixel-based classifiers like maximum likelihood classifier. It was clear that the learning speed of the feed forward neural networks was slow because of the slow gradient-based learning algorithm in which the parameters were tuned iteratively. Unlike these traditional implementations, a new learning algorithm called ELM (Huang 2003; Huang et al. 2004, 2006) for single hidden layer feed forward neural networks (SLFN) was proposed which randomly chose the input weights and analytically determined the output weights of SLFN. In theory, the algorithm tended to provide the best generalization performance at extremely fast learning speed. Güler and Übeyli (2005) applied the ANFIS model for classification of electroencephalogram (EEG) signals. Decision-making was performed in two stages. The first stage included the feature extraction using the wavelet transforms (WT), and in the second stage ANFIS was trained with the back propagation gradient descent method in combination with the least squares method. The performance of the ANFIS model was evaluated in terms of the training performance and classification accuracies, and the results confirmed that the proposed ANFIS model had the potential in classifying the EEG signals. Lu and Weng (2007) performed a detailed survey of various classification algorithms including pixel-based, sub-pixel-based, parametric, non-parametric, and hard and soft classification algorithms. They summarized that the success of an image classification algorithm depended on the availability of highquality remotely sensed imagery, the design of a proper classification procedure, and analyst’s skills. Demir and Erturk (2007) presented a hyperspectral image classification method based on RVMs. Support vector machine-based approaches had been proposed for hyperspectral image classification. The RVM-based approach had merits like high relevance rate and fast learning time for the classification of hyperspectral images. These merits made the RVM-based hyperspectral classification approach more suitable for real time applications. Ghosh et al. (2011) proposed a context-sensitive technique for unsupervised change detection in multi-temporal remote sensing images. They had used two fuzzy clustering algorithms, namely, FCM and Gustafson–Kessel clustering (GKC) algorithms. Hybridization of FCM and GKC with two other optimization techniques, genetic algorithm (GA) and simulated annealing (SA), was made to further enhance the performance. Results were compared and found to be more superior to the results obtained from the

2.7 Papers on Distance Measures Applied in Texture-Based Study

29

existing Markov random field (MRF) and neural network-based algorithms. Sharma et al. (2011) recorded that fuzzy-based classifiers were fast and the accuracy depended on the type of member function used to model the input features. Tanyildizi (2012) substantiated that the performance of the proposed hybrid 2D and semi 3D texture feature coding method (TFCM) and ELM methodology for classification of 2D and 3D neighbor textures was better than the performance of wavelet feature extractor and an ANFIS classifier. Zhang et al. (2019a) attempted to unify the remote sensing classification of land cover (LC) and land use (LU), where previously each had been considered only separately. The proposed joint deep learning (JDL) method provided a general framework within which the pixelbased multilayer perceptron (that conducts LC classification) and the patch-based convolutional neural network (that conducts conditional LU classification based on LC probabilities) provided mutually complementary information to each other such that both were refined in the classification process through iteration. The proposed method provided high classification accuracy for very fine spatial resolution (VFSR) imagery.

2.7

Papers on Distance Measures Applied in Texture-Based Study

Several distance measures are applied in literature on texture analysis. The non-parametric tests or statistical distance measures used in the texture studies are designed to capture differences between samples without assuming the distribution or any prior knowledge of the samples. A distance measure should be able to discriminate and magnify the pattern differences that exist between different textures. The concept of distance between two probability distributions was initially developed by Mahalanobis (1936). Bhattacharyya (1943) proposed Bhattacharyya distance as a measure of divergence between two probability distributions. Shannon’s concept of information-theoretic entropy and its generalization known as the Kullback and Leibler relative entropy (Kullback 1959) or the divergence measure between two probability distributions had also been used in several texturebased applications. Ojala et.al (1996) used Kullback Leibler (KL) distance for texture-based classification of standard textures. Later, they (2000) used G Statistics log likelihood distance measure as the dissimilarity measure for comparing two one-dimensional histograms representing feature vectors. Sokal and Rohlf (1987) proposed dissimilarity measures for classification. Puzicha et al. (1997) proposed and examined nonparametric statistical tests to define similarity and homogeneity measures for textures. The statistical tests were applied to the coefficients of images filtered by a multi-scale Gabor filter bank. The similarity measures were found useful for texture-based image retrieval and segmentation conducted on Brodatz-like micro textures (Brodatz 1966) and a collection of real-world images. Rubner et al. (2001) conducted an empirical evaluation of dissimilarity measures for neighbor and texture

30

2

Introduction to Texture and Related Work

information and concluded that the selection of distance measure was dependent on the application under consideration. A thorough quantitative performance evaluation had been presented for distribution-based image dissimilarity measures. It was demonstrated how the selection of a measure, based on large scale evaluation, substantially improved the quality of classification, retrieval, and unsupervised segmentation of neighbor and texture images. Groenen et al. (2006) applied the objective function-based fuzzy clustering algorithms using a generalized distance function. In particular, the extension of the fuzzy c-means algorithm to the parametric Minkowski distance function and to the root of the squared Minkowski distance function was analyzed. Jenicka and Suruliandi (2011) performed a comparative study to find suitable distance measures for land cover classification of LISS IV remotely sensed image and found that Bhattacharyya and chi squared distances provided better classification accuracy, maximal intra-cluster distance, and minimal inter-cluster distance. Yang and Newsam (2012) performed geographic image retrieval using a variety of distance measures like Euclidean, Bhattacharyya, chi-square, correlation, cosine, inner product, intersection, L1, L2, and Earth Mover’s Distance (EMD). Exercises 1. Reason how texture is quantified by finding the gray scale variations in a local neighborhood of an image. 2. Find out a list of real-world applications where only texture-based approaches can be used for getting better results. 3. Find out other standard texture datasets (apart from Brodatz) available online for experimentation. 4. Find out how the Matlab command “nlfilter” works and how it can be used for performing neighborhood operation in an image. 5. Identify who was the pioneer in proposing a texture model. 6. Trace and list the challenges recorded in the literature on texture models and how they are addressed in later work. 7. Prepare a table listing the names of texture models falling under the types structural, statistical, and spectral. 8. Trace and find out the specific distance measures commonly used in texture-based applications. 9. List out the classifiers most recently used in literature on texture-based applications.

References Ahonen T, Pietikainen M, Hadid A, Maenpaa T (2004) Face recognition based on the appearance of local regions. In: Proceedings of the 17th international conference on Pattern Recognition, 2004. ICPR 2004, vol 3. IEEE, pp 153–156 Arivazhagan S, Ganesan L (2003) Texture segmentation using wavelet transform. Pattern Recogn Lett 24(16):3197–3203

References

31

Arivazhagan S, Ganesan L, Priyal SP (2006) Texture classification using Gabor wavelets based rotation invariant features. Pattern Recogn Lett 27(16):1976–1982 Asendorf G, Hermes T (1996) Textures: an approach for new abstract description language. In: Human vision and electronic imaging, vol 2657. International Society for Optics and Photonics, pp 98–106 Baum LE, Petrie T (1966) Statistical inference for probabilistic functions of finite state Markov chains. Ann Math Stat 37(6):1554–1563 Benčo M, Hudec R (2007) Novel method for color textures features extraction based on GLCM. Radioengineering 16(4):64–67 Bezdek JC (1981) Objective function clustering. In: Pattern recognition with fuzzy objective function algorithms. Springer, Boston, pp 43–93 Bhattacharyya A (1943) On a measure of divergence between two statistical populations defined by their probability distributions. Bull Calcutta Math Soc 35:99–109 Bovik AC, Clark M, Geisler WS (1990) Multichannel texture analysis using localized spatial filters. IEEE Trans Pattern Anal Mach Intell 12(1):55–73 Brodatz P (1966) Textures: a photographic album for artists and designers. Dover, New York Cao K, Jain AK (2018) Latent fingerprint recognition: Role of texture template. In: 2018 IEEE 9th international conference on biometrics theory, applications and systems (BTAS). IEEE, pp 1–9 Chatzichristofis SA, Boutalis YS (2008) Fcth: fuzzy color and texture histogram-a low level feature for accurate image retrieval. In: 2008 ninth international workshop on Image Analysis for Multimedia Interactive Services. IEEE, pp 191–196 Chen W, Ji M (2010) Comparative analysis of fuzzy approaches to remote sensing image classification. In: 2010 seventh international conference on Fuzzy Systems and Knowledge Discovery, vol 2. IEEE, pp 537–541 Chen Y, Zhu L, Ghamisi P, Jia X, Li G, Tang L (2017) Hyperspectral images classification with Gabor filtering and convolutional neural network. IEEE Geosci Remote Sens Lett 14 (12):2355–2359 Choi JY, Ro YM, Plataniotis KN (2011) Color local texture features for color face recognition. IEEE Trans Image Process 21(3):1366–1380 Cid YD, Müller H, Platon A, Poletti PA, Depeursinge A (2017) 3D solid texture classification using locally-oriented wavelet transforms. IEEE Trans Image Process 26(4):1899–1910 Conners RW, Harlow CA (1980) A theoretical comparison of texture algorithms. IEEE Trans Pattern Anal Mach Intell (3):204–222 Cortes C, Vapnik V (1995) Support-vector networks. Mach Learn 20(3):273–297 Cover T, Hart P (1967) Nearest neighbor pattern classification. IEEE Trans Inf Theory 13(1):21–27 Demir B, Erturk S (2007) Hyperspectral image classification using relevance vector machines. IEEE Geosci Remote Sens Lett 4(4):586–590 Dunn JC (1973) A fuzzy relative of the ISODATA process and its use in detecting compact wellseparated clusters, pp 32–57 Ge QZ, Ling ZC, Qiong L, Hui XX, Zhang G (2008) High efficient classification on remote sensing images based on SVM. Int Arch Photogramm Remote Sens Spat Inf Sci 37:1011–1014 Ghosh A, Mishra NS, Ghosh S (2011) Fuzzy clustering algorithms for unsupervised change detection in remote sensing images. Inf Sci 181(4):699–715 Gibson JJ, Bridgeman B (1987) The visual perception of surface texture in photographs. Psychol Res 49(1):1–5 Goldfarb L (1984) A unified approach to pattern recognition. Pattern Recognit 17:575–582 Gonzalez RC, Woods RE (2002) Digital image processing Groenen PJ, Kaymak U, van Rosmalen J (2006) Fuzzy clustering with Minkowski distance functions. Rotterdam Econometric Institute, p 24 Güler I, Übeyli ED (2005) Adaptive neuro-fuzzy inference system for classification of EEG signals using wavelet coefficients. J Neurosci Methods 148(2):113–121

32

2

Introduction to Texture and Related Work

Gupta S, Kumar S, Garg A, Singh D, Rajput NS (2016) Class wise optimal feature selection for land cover classification using SAR data. In: 2016 IEEE International Geoscience and Remote Sensing Symposium (IGARSS). IEEE, pp 68–71 Haralick RM (1979) Statistical and structural approaches to texture. Proc IEEE 67:786–804 Haralick RM, Shapiro LG (1985) Image segmentation techniques. Comput Vis Graph Image Proc 29:100–132 Haralick RM, Shanmugam KK, Dinstein I (1973) Texture features for image classification. IEEE Trans Syst Man Cybern 8(6):610–621 He D-C, Wang L (1990) Texture unit, texture spectrum, and texture analysis. IEEE Trans Geo-Sci Remote Sens 28(1):509–513 He DC, Wang L (1991a) Texture features based on texture spectrum. Pattern Recogn 24 (3):391–399 He DC, Wang L (1991b) Textural filtering based on the texture spectrum. Pattern Recogn 24:1187–1195 He DC, Wang L (1992) Unsupervised textural classification of images using the texture spectrum. Pattern Recogn 25(3):247–255 Hermes L, Frieauff D, Puzicha J, Buhmann JM (1999) Support vector machines for land usage classification in Landsat TM imagery. In: IEEE 1999 International Geoscience and Remote Sensing Symposium. IGARSS’99 (Cat. No. 99CH36293), vol 1. IEEE, pp 348–350 Huang G-B (2003) Learning capability and storage capacity of two-hidden layer feedforward networks. IEEE Trans Neural Netw 14(2):274–281 Huang GB, Zhu QY, Siew CK (2004) Extreme learning machine: a new learning scheme of feedforward neural networks. In: Proceedings of International Joint Conference on Neural Networks (IJCNN), Budapest, Hungary, vol 2529(2), pp 985–990 Huang GB, Zhu QY, Siew CK (2006) Extreme learning machine: theory and applications. Neurocomputing 70(1):489 Iakovidis DK, Keramidas EG, Maroulis D (2008, June) Fuzzy local binary patterns for ultrasound texture characterization. In: International conference image analysis and recognition. Springer, Berlin/Heidelberg, pp 750–759 Jang JS (1993) ANFIS: adaptive-network-based fuzzy inference system. IEEE Trans Syst Man Cybern 23(3):665–685 Jenicka S (2018) Land cover classification using the proposed texture model and fuzzy k-NN classifier. In: Optimization techniques for problem solving in uncertainty. IGI Global, pp 226–261 Jenicka S (2019) Sugeno fuzzy-inference-system-based land cover classification of remotely sensed images. In: Environmental information systems: concepts, methodologies, tools, and applications. IGI Global, pp 1247–1283 Jenicka S, Suruliandi A (2011) Empirical evaluation of distance measures for supervised classification of remotely sensed image with modified multivariate local binary pattern. In: 2011 international conference on Emerging Trends in Electrical and Computer Technology. IEEE, pp 762–767 Jenicka S, Suruliandi A (2014) A textural approach for land cover classification of remotely sensed image. CSI Trans ICT 2(1):1–9 Jenicka S, Suruliandi A (2015) Comparison of soft computing approaches for texture based land cover classification of remotely sensed image. Res J Appl Sci Eng Technol 10(10):1216–1226 Jenicka S, Suruliandi A (2016) Distributed texture-based land cover classification algorithm using hidden Markov model for multispectral data. Surv Rev 48(351):430–437 Jiji GW, Ganesan L (2005) Unsupervised segmentation using fuzzy logic based texture spectrum for MRI brain images. In: WEC (5), pp 155–157 Julesz B (1986) Texton gradients: Thetextonn theory revisited. Biol Cybern 54:245–251 Keller JM, Gray MR, Givens JA (1985) A fuzzy k-nearest neighbor algorithm. IEEE Trans Syst Man Cybern (4):580–585

References

33

Kohonen T (1982) Self-organized formation of topologically correct feature maps. Biol Cybern 43 (1):59–69 Kullback S (1959) Information theory and statistics. Wiley, New York Kumar RS, Manimegalai P (2020) Near lossless image compression using parallel fractal texture identification. Biomed Signal Process Control 58:101862 Kumar GS, Mohan PK (2019) Local mean differential excitation pattern for content based image retrieval. SN Appl Sci 1(1):46 Laws K (1980a) Textured image segmentation. Ph.D. dissertation, University of Southern California Laws K (1980b) Rapid texture identification. SPIE Image Process Missile Guid 238:376–380 Lee SH, Choi JY, Ro YM, Plataniotis KN (2011) Local color vector binary patterns from multichannel face images for face recognition. IEEE Trans Image Process 21(4):2347–2353 Leite PBC, Feitosa RQ, Formaggio AR, da Costa GAOP, Pakzad K, Sanches IDA (2011) Hidden Markov models for crop recognition in remote sensing image sequences. Pattern Recogn Lett 32 (1):19–26 Liao S, Chung AC (2007) Texture classification by using advanced local binary patterns and spatial distribution of dominant patterns. In: 2007 IEEE International Conference on Acoustics, Speech and Signal Processing-ICASSP’07, vol 1. IEEE, pp 1–1221 Liao S, Law MW, Chung AC (2009) Dominant local binary patterns for texture classification. IEEE Trans Image Process 18(5):1107–1118 Lu D, Weng Q (2007) A survey of image classification methods and techniques for improving classification performance. Int J Remote Sens 28(5):823–870 Lucieer A, Stein A, Fisher P (2005) Multivariate texture-based segmentation of remotely sensed imagery for extraction of objects and their uncertainty. Int J Remote Sens 26(14):2917–2936 Mahalanobis PC (1936) On the generalized distance in statistics. Proc Nat Inst Sci India 12:49–55 Mao J, Jain AK (1992) Texture classification and segmentation using multiresolution simultaneous autoregressive models. Pattern Recogn 25(2):173–188 Mao C, Gururajan A, Sari-Sarraf H, Hequet E (2012) Machine vision scheme for stain-release evaluation using Gabor filters with optimized coefficients. Mach Vis Appl 23(2):349–361 Mirmehdi M, Petrou M (2000) Segmentation of color textures. IEEE Trans Pattern Anal Mach Intell 22(2):142–159 Murugan D, Arumugam S, Rajalakshmi K, Manish T (2010) Performance evaluation of face recognition using Gabor filter, log Gabor filter and discrete wavelet transform. Int J Comput Sci Inform Technol 2(1):125–133 Nedeljkovic I (2004) Image classification based on fuzzy logic. Int Arch Photogramm Remote Sens Spat Inf Sci 34(30):3–7 NRSC/ISRO (2010) Ebook on remote sensing applications Ojala T, Pietikäinen M, Harwood D (1996) A comparative study of texture measures with classification based on featured distributions. Pattern Recogn 29(1):51–59 Ojala T, Pietikainen M, Maenpää T (2000) Gray scale and rotation invariant texture classification with local binary patterns. In: Proceedings of sixth European conference on Computer Vision, Dublin, Ireland, vol 33, pp 43–52 Ojala T, Pietikainen M, Maenpää T (2001) A generalized local binary pattern operator for multiresolution gray scale and rotation invariant texture classification. In: Advances in pattern recognition, vol 2013. ICAPR, pp 399–408 Pal NR, Pal SK (1993) A review on image segmentation techniques. Pattern Recogn 26 (9):1277–1294 Puzicha J, Hofmann T, Buhmann JM (1997) Non-parametric similarity measures for unsupervised texture segmentation and image retrieval. In: Proceedings of IEEE Computer Society conference on Computer Vision and Pattern Recognition. IEEE, pp 267–272 Raju USN, Kumar AS, Mahesh B, Reddy BE (2010) Texture classification with high order local pattern descriptor: local derivative pattern. Glob J Comput Sci Technol

34

2

Introduction to Texture and Related Work

Randen T, Husoy JH (1999) Filtering for texture classification: a comparative study. IEEE Trans Pattern Anal Mach Intell 21(4):291–310 Rao A (1990) A taxonomy for texture description and identification. Springer, New York Ratajczak R, Crispim-Junior CF, Faure É, Fervers B, Tougne L (2019) Automatic land cover reconstruction from historical aerial images: an evaluation of features extraction and classification algorithms. IEEE Trans Image Process 28(7):3357–3371 Reddy AM, Krishna VV, Sumalatha L, Niranjan SK (2017) Facial recognition based on straight angle fuzzy texture unit matrix. In: 2017 International Conference on Big Data Analytics and Computational Intelligence (ICBDAC). IEEE, pp 366–372 Reed TR, Dubuf JH (1993) A review of recent texture segmentation and feature extraction techniques. CVGIP Image Underst 57(3):359–372 Rivera AR, Castillo JR, Chae OO (2012) Local directional number pattern for face analysis: face and expression recognition. IEEE Trans Image Process 22(5):1740–1752 Rubel A, Lukin V, Uss M, Vozel B, Pogrebnyak O, Egiazarian K (2016) Efficiency of texture image enhancement by DCT-based filtering. Neurocomputing 175:948–965 Rubner Y, Puzicha J, Tomasi C, Buhmann JM (2001) Empirical evaluation of dissimilarity measures for color and texture. Comput Vis Image Underst 84(1):25–43 Seng CY, Inbaraj S, Sun Z (2008) Local spatial statistics for remotely sensed image classification of mangrove. Int Arch Photogramm Remote Sens Spat Inf Sci 37 Shankar BU, Meher SK, Ghosh A (2011) Wavelet-fuzzy hybridization: feature-extraction and landcover classification of remote sensing images. Appl Soft Comput 11(3):2999–3011 Sharma M, Gupta R, Kumar D, Kapoor R (2011) Efficacious approach for satellite image classification. J Electr Electron Eng Res 3(8):143–150 Sokal RR, Rohlf FJ (1987) Introduction to bio statistics, 2nd edn. W.H Freeman & Co, New York Song X, Wu L, Liu G (2019) Unsupervised color texture segmentation based on multi-scale regionlevel Markov random field models. Компьютерная оптика 43(2) Srinivasan EM, Ramar K, Suruliandi A (2011) Texture analysis using local texture patterns: a fuzzy logic approach. Int J Pattern Recognit Artif Intell 25(05):741–762 Suruliandi A (2009) A study on classification of remotely sensed multispectral mages-a textural approach. Ph.D dissertation work submitted to Manonmaniam Sundaranar University, Tamil Nadu, India Suruliandi A, Jenicka S (2015) Texture-based classification of remotely sensed images. Int J Signal Imag Syst Eng 8(4):260–272 Suruliandi A, Ramar K (2008) Local texture patterns- a univariate texture model for classification of images. In: ADCOM-2008, 16th international conference, December 14–17 Tan X, Triggs B (2007) Enhanced local texture feature sets for face recognition under difficult lighting conditions. In: International workshop on analysis and modeling of faces and gestures. Springer, Berlin/Heidelberg, pp 168–182 Tanyildizi E (2012) A hybrid color texture image classification method based on 2D and semi 3D texture features and extreme learning machine. Przeglad Elektrotechniczny 11 Tipping ME (2000) The relevance vector machine. Adv Neural Inf Proces Syst:652–658 Todorovic S, Ahuja N (2009) Texel-based texture segmentation. In: 2009 IEEE 12th International Conference on Computer Vision. IEEE, pp 841–848 Topi M, Matti P, Timo O (2000) Texture classification by multi-predicate local binary pattern operators. In: Proceedings 15th International Conference on Pattern Recognition. ICPR-2000, vol 3. IEEE, pp 939–942 Tuceryan M, Jain A (1990) Texture segmentation using Voronoi polygons. IEEE TPAMI 12 (2):211–216 Turtinen M, Mäenpää T, Pietikäinen M (2003) Texture classification by combining local binary pattern features and a self-organizing map. In: Scandinavian conference on image analysis. Springer, Berlin/Heidelberg, pp 1162–1169 Unser M (1986) Sum and difference histograms for texture classification. IEEE Trans Pattern Anal Mach Intell 8(1):118–125

References

35

Van Gool L, Dewaele P, Oosterlinck A (1985) Texture analysis anno 1983. Comput Vis Graph Image Process 29(3):336–357 Vapnik VN (1982) Estimation of dependences based on empirical data, Addendum 1. Springer, New York Wang L, He DC (1990a) Texture classification using texture spectrum. Pattern Recogn 23 (8):905–910 Wang L, He DC (1990b) A new statistical approach for texture analysis. Photogramm Eng Remote Sens 56(1):61–66 Weszka J, Dyer C, Rosenfeld A (1976) A comparative study of texture measures for terrain classification. IEEE Trans Syst Man Cybern 6:269–285 Wu X, Yang F, Lishman R (2010) Land cover change detection using texture analysis. J Comput Sci 6(1):92 Yang Y, Newsam S (2012) Geographic image retrieval using local invariant features. IEEE Trans Geosci Remote Sens 51(2):818–832 Younis AA, Soliman AT, John NM (2007) Co-segmentation of MR and MR spectroscopy imaging using hidden Markov models. In: 2007 IEEE/NIH life science systems and applications workshop. IEEE, pp 188–191 Zhang D, Wong A, Indrawan M, Lu G (2000) Content-based image retrieval using Gabor texture features. IEEE Trans Pami 13 Zhang J, Tan T, Ma L (2002) Invariant texture segmentation via circular Gabor filters. In: Object recognition supported by user interaction for service robots, vol 2. IEEE, pp 901–904 Zhang C, Sargent I, Pan X, Li H, Gardiner A, Hare J, Atkinson PM (2019a) Joint deep learning for land cover and land use classification. Remote Sens Environ 221:173–187 Zhang L, Zhang Y, Niu L, Zhao Z, Han X (2019b) HMM static hand gesture recognition based on combination of shape features and wavelet texture features. In: International Conference on Wireless and Satellite Systems. Springer, Cham, pp 187–197 Zhu SC, Guo CE, Wang Y, Xu Z (2005) What are textons? IJCV 62(1–2):121–143

Chapter 3

A Few Existing Basic and Multivariate Texture Models

Abstract This chapter describes a wide range of basic statistical texture models like local binary pattern (LBP), ternary pattern (TP), gray level co-occurrence matrix (GLCM), texture spectrum (TS), discrete local texture pattern (DLTP), and local derivative pattern (LDP). It also discusses multivariate statistical texture models like multivariate local binary pattern (MLBP), multivariate ternary pattern (MTP), color GLCM (CGLCM), and multivariate discrete local texture pattern (MDLTP). Furthermore wavelet and Gabor wavelet-based texture representation for gray scale and multispectral images are described. In the chapter appendix, the Matlab source codes of the texture models (described earlier in the chapter) are listed. The reader is equipped to explore and implement new ideas (related to texture model) after reading this chapter. Keywords Local binary pattern · Gray level co-occurrence matrix · Multivariate local binary pattern · Wavelet-based texture representation · Gabor wavelet-based texture representation

3.1

Introduction

The focus of this chapter is to familiarize the readers to the implementation details of some basic and multivariate texture models. Many basic statistical texture models have been proposed for gray scale images in literature. In Sect. 3.2, a wide range of basic statistical texture models are described. In literature, some basic statistical texture models have been extended for multispectral images. In Sect. 3.3, the extended multivariate texture models are described. Many texture-based basic and multivariate spectral texture models have also been proposed in literature. The wavelet and Gabor wavelet-based texture representation for gray scale and multispectral images fall under the category of spectral texture models. The gray scale versions of wavelet and Gabor wavelet texture representation are described in Sect. 3.2, while the multispectral versions of wavelet and Gabor wavelet texture representation are described in Sect. 3.3. The chapter appendix lists

© The Author(s), under exclusive license to Springer Nature Switzerland AG 2021 S. Jenicka, Land Cover Classification of Remotely Sensed Images, https://doi.org/10.1007/978-3-030-66595-1_3

37

38

3 A Few Existing Basic and Multivariate Texture Models

the Matlab codes of the existing basic and multivariate texture models described in the chapter. By the end of this chapter, the reader will be able to do the following: • Understand why do we need a texture model and its significance. • Understand how a texture descriptor is found for a local neighborhood. • Understand how global histograms are formed, its bin requirement, and what its implications are with respect to time complexity. • Implement any existing statistical or spectral texture model.

3.2 3.2.1

Existing Basic Texture Models Local Binary Pattern

The LBP spatial descriptor proposed by Ojala et al. (2001) extracts texture in a 3
3 local neighborhood of a gray scale image. Let the gray scale values in the neighborhood be gc, g0. . . gP1 where gc corresponds to the gray scale value of the center pixel, P corresponds to the total number of surrounding neighbors (which is 8 in a 3
3 neighborhood), and gi (i ¼ 0, . . . , P  1) corresponds to the gray scale value of the ith neighbor. The spatial relationship between the center pixel and one of its neighbor pixels is defined in “Eq. 3.1.”
s ð xÞ ¼

1

if x  0

0

if x < 0

ð3:1Þ

If the signed gray scale value difference between the center pixel and a neighbor pixel is 0 or positive, then s(x) is 1. Otherwise s(x) is 0. Each s(x) value is attached a binary weight clockwise from the top left position. The sum of the products of s(x) and its corresponding binary weight gives the conventional LBP. The conventional LBP is calculated as in “Eq. 3.2.” LBP ¼

P1  X  s gp  gc 2i

ð3:2Þ

i¼0

A sample calculation of the conventional LBP in a 3
3 neighborhood is given in “Eq. 3.3.” In “Eq. 3.3,” the top left s(x) value is 1 because the gray scale value of the neighbor (123) is greater than the gray scale value of the center pixel (120).

3.2 Existing Basic Texture Models

2

123 110

6 6 117 120 4 130 125

113

39

3

2

1 0

0

1 1

1

7 6 6 135 7 5 ! 40 128

3

2

20

21

22

26

25

24

6 7 7 6 17 5 ! 42

3

7 23 7 5

ð3:3Þ

¼ 1*20 þ 1*23 þ 1*24 þ 1*25 þ 1*26 ¼ 121 The “Program 1” in chapter appendix lists the Matlab code for computing conventional LBP. The authors introduced the concept of uniform patterns and it was observed that certain patterns seemed to exhibit the fundamental properties of texture, and these patterns were called uniform because they had at most two, one-tozero or zero-to-one transitions in the circular binary code. For example, in “Eq. 3.3,” there are 4 such transitions in the circular binary code “10011110.” So uniformity measure, U is defined as in “Eq. 3.4.” U ¼ jsðgP1  gc Þ  sðg0  gc Þj 7   X    s gp  gc  s gp1  gc  þ

ð3:4Þ

p¼1

Patterns with a U value less than or equal to two are designated as “uniform.” The rotation invariant uniform pattern code for any uniform pattern is calculated by simply counting the ones in the circular binary code computed earlier. All other patterns are labeled as miscellaneous and collapsed into one value 9. The mathematical function for finding rotation invariant uniform LBP (ULBP) is given in “Eq. 3.5.” 8 7 > < P sg  g  p c ULBP ¼ p¼0 > : 9

U < 1 if gi < gc  Δg   Pðgi , gc Þ ¼ 0 if gc  Δg  gi  gc þ Δg >   : 1 if gi > gc þ Δg

ð3:6Þ

where Δg is a small positive value which is set to express the closeness of neighboring pixel with the center pixel. The value p(gi, gc)stands for the discrete output level assigned to ith pixel in the neighborhood. The discrete output levels 1, 0, and 1 characterize the neighborhood pixel relation. Concatenation of these discrete output levels in a neighborhood gives us a pattern unit. A sample calculation of the pattern string with Δg ¼ 3 is given in “Eq. 3.7.” 2

123

6 4 117 130

110

113

3

2

0

120

7 6 135 5 ! 4 0

125

128

1

1 1 1

3

7 1 5 ¼ 0  1  111110ðPattern UnitÞ ð3:7Þ 1

The total number of patterns considering all combinations of three discrete output levels with the number of pixels in the neighborhood (P) equal to 8 will be 38. This will lead to increase in the number of bins required when these local patterns are accumulated to characterize the global regions. In order to reduce the number of possible patterns, a uniformity measure (U ) is introduced as defined in “Eqs. 3.8 and 3.9.” It corresponds to the number of circular spatial transitions between the discrete output levels like 1, 0, and 1 in the pattern string. U ¼ jsðgP1  gc Þ  sðg0  gc Þj 7   X    s gp  gc  s gp1  gc  þ

ð3:8Þ

p¼1

where
sðx, yÞ ¼

1 if jx  yj > 0 0 if otherwise

ð3:9Þ

Patterns for which U value is less than or equal to three are considered uniform, and others are considered non-uniform patterns. The gray scale TP for a 3
3 local region is derived as in “Eq. 3.10.” The value PS stands for the sum of all positive discrete output levels, and NS stands for the absolute sum of all negative discrete output levels. To each pair of (NS, PS) values, a unique TP value is obtained from the lookup table “L” for all uniform patterns, and 46 will be assigned for non-uniform patterns.

3.2 Existing Basic Texture Models Table 3.1 Lookup table of TP texture model

41

NS/PS 0 1 2 3 4 5 6 7 8

0 1 10 18 25 31 36 40 43 45

1 2 11 19 26 32 37 41 44 0

2 3 12 20 27 33 38 42 0 0

3 4 13 21 28 34 39 0 0 0

4 5 14 22 29 35 0 0 0 0

5 6 15 23 30 0 0 0 0 0

6 7 16 24 0 0 0 0 0 0

7 8 17 0 0 0 0 0 0 0

8 9 0 0 0 0 0 0 0 0

Table 3.2 TP texture model extended to different sizes of neighborhood Size of neighborhood (3
3) (5
5) (7
7)

Number of points in the neighborhood 8 16 24


TPðX Þ ¼

Total number of possible patterns 73 145 217

Total number of uniform patterns 46 118 190

LðNS, PSÞ U < 0 if gi < gc E ðiÞ ¼ 1 if gi ¼ gc > : 2 if gi > gc

ð3:18Þ

E(i) denotes the discrete output level assigned to the ith pixel. As there are three discrete output levels such as 0, 1, and 2, tertiary weights are assigned to each E(i).

44

3 A Few Existing Basic and Multivariate Texture Models

The formula in “Eq. 3.19” is used for finding texture spectrum, TU. It will result in 38 ¼ 6561 possible texture spectrum values. TU ¼

8 X

E ðiÞ  3i1

ð3:19Þ

i¼1

A sample calculation of texture spectrum is given in “Eq. 3.20.” 2

123

6 6 117 4 130

110

120

3

2

2

0

1

2

2

1

120

7 6 6 135 7 5 ! 40

125

120

3

2

30

31

32

36

35

34

6 7 7 6 27 5 ! 43

3

7 33 7 5

ð3:20Þ

¼ 2*30 þ 1*32 þ 2*33 þ 1*34 þ 2*35 þ 2*36 ¼ 2090 The “Program 6” in chapter appendix lists the Matlab code for computing texture spectrum in a 3
3 neighborhood of a gray scale image. As before, texture spectrum histogram provides global description within a window of a gray scale image.

3.2.5

Discrete Local Texture Pattern

The texture model DLTP proposed by Jenicka and Suruliandi (2014) extracts local texture information in a 3
3 neighborhood of a gray scale image. As before, the spatial relationship between the center pixel (gc) and one of its neighbor pixels (gi) is described in “Eq. 3.21.” 8 1 if gi < ðgc  nÞ > > > < 0 if ðg  nÞ  g  g c i c P ð gi , gc Þ ¼ > 1 if g < g  ð g þ n Þ > c i c > : 9 if gi > ðgc þ nÞ

ð3:21Þ

where “n” is the threshold which is set to express the closeness of the neighboring pixel with the center pixel. The value p(gi, gc) stands for the discrete output level assigned to the ith pixel in the neighborhood. The discrete output levels are fixed numerically to 1, 0, 1, and 9 to assign unique pattern values during individual summation of positive and negative values later. The discrete output levels characterize the neighborhood pixel relation. Concatenation of these discrete output levels in a neighborhood gives us a pattern unit. The sample calculation of pattern unit for n ¼ 5 is shown in “Eq. 3.22.”

3.2 Existing Basic Texture Models

2

206

6 6 203 4 212

194

201

3

2

45

1

1

0

9

9

1

201

7 6 6 198 7 5 ! 41

210

202

3

7 07 5 ! 1  1 0 0 1 9 9 1ðPattern unitÞ ð3:22Þ

The total number of patterns considering all combinations of four discrete output levels with the number of pixels in the neighborhood (P) equal to 8 will be 48. This will lead to increase in the number of bins required when these local patterns are accumulated to characterize the global regions. In order to reduce the number of possible patterns, a uniformity measure (U ) is introduced as defined in “Eqs. 3.24 and 3.25.” It corresponds to the total number of circular spatial transitions between the discrete output levels like 1, 0, 1, and 9 in the pattern unit. Patterns for which U value is less than or equal to three are considered uniform, and others are considered non-uniform patterns. The gray scale DLTP for a 3
3 local region is derived as “Eq. 3.23.” As before in finding TP, the value PS stands for the sum of all positive discrete output levels, and NS stands for the absolute sum of all negative discrete output levels (Eq. 3.26). To each pair of (NS, PS) values, a unique DLTP value is obtained from the lookup table “L” for all uniform patterns, and 166 will be assigned for non-uniform patterns.
DLTP ¼

LðNS, PSÞ U3 166 Otherwise

ð3:23Þ

where U ¼ jsðgP1  gc Þ  sðg0  gc Þj P1   X    s gp  gc  s gp1  gc  þ

ð3:24Þ

P¼1

where
sðx, yÞ ¼

1 0

if jx  yj > 0 if otherwise

ð3:25Þ

and PS ¼

P1 X

pð gi , gc Þ

if pðgi , gc Þ  0

i¼0

NS ¼

P1 X i¼0

ð3:26Þ pð gi , gc Þ

if pðgi , gc Þ < 0

46

3 A Few Existing Basic and Multivariate Texture Models

The lookup table (L) provides unique pattern values to the different combinations of NS and PS values, and it can be generated using the Matlab “Program 7” listed in chapter appendix. The maximum negative sum (NS) is 8 (by taking the absolute value leaving negative sign) as there can be eight 1’s. The maximum positive sum (PS) is 72 as there can be eight 9’s. So the size of the lookup table is (8
72). All the entries in the table are filled sequentially starting from 1 to 165 which characterize the unique pattern labels. Zero entries in the lookup table show that those patterns will never occur. This scheme produces 165 uniform patterns, and other non-uniform patterns are assigned 166 as the miscellaneous pattern label. The “Program 8” in chapter appendix lists the Matlab code for computing DLTP. The DLTP texture model characterizes local texture in a 3
3 neighborhood, and DLTP histogram provides global texture description within a window of a gray scale image. The DLTP histogram requires 166 bins.

3.2.6

Local Derivative Pattern

Local derivative pattern texture model was proposed by Raju et al. (2010). Given an image I(V ), the first-order derivatives along 0 , 45 , 90 , and 135 directions are denoted as I 0α ðV Þ where α ¼ 0 , 45 , 90 , and 135 . Let V0 be the center point in I(V ), and Vi where i ¼ 1, . . . 8 be the neighboring pixels around V0 as shown in “Eq. 3.27.” The four first-order derivatives at V ¼ V0 for 0 , 45 , 90 , and 135 , respectively, are given in “Eq. 3.28.” 2

V1

6 4 V8 V7

V2 V0 V6

V3

3

7 V4 5 V5

ð3:27Þ

I 00 ðV 0 Þ ¼ I ðV 0 Þ  I ðV 4 Þ

I 045 ðV 0 Þ ¼ I ðV 0 Þ  I ðV 3 Þ

I 090 ðV 0 Þ ¼ I ðV 0 Þ  I ðV 2 Þ I 0135 ðV 0 Þ ¼ I ðV 0 Þ  I ðV 1 Þ

ð3:28Þ

The second-order directional LDP, LDP2α ðV 0 Þ in α direction at V ¼ V0, is defined in “Eq. 3.29.” LDP2α ðV 0 Þ ¼

0       

f I α ðV 0 Þ, I 0α ðV 1 Þ , f I 0α ðV 0 Þ, I 0α ðV 2 Þ , . . . f I 0α ðV 0 Þ, I 0α ðV 7 Þ , f I 0α ðV 0 Þ, I 0α ðV 8 Þ ð3:29Þ

where f(.,.) is a binary coding function determining the types of local pattern transitions. It encodes the co-occurrence of two derivative directions at different neighboring pixels as given in “Eq. 3.30.”

3.2 Existing Basic Texture Models

f





I 0α ðV 0 Þ, I 0α ðV i Þ


¼

0 1

47

if I 0α ðV i ÞI 0α ðV 0 Þ > 0 if I 0α ðV i ÞI 0α ðV 0 Þ  0

where i ¼ 1, 2 . . . 8

ð3:30Þ

Finally, the second-order LDP is defined as the concatenation of the four 8-bit directional LDPs as given in “Eq. 3.31.”    

LDP2 ðV Þ ¼ LDP2α ðV Þj α ¼ 0 , 45 , 90 , 135

ð3:31Þ

The second-order LDP represents the texture information in a local neighborhood of a gray scale image by a one dimensional vector of size (1
32). The “Program 9” in chapter appendix lists the Matlab code for computing LDP.

3.2.7

Wavelet-Based Texture Representation

Wavelet-based texture feature extraction method (Arivazhagan and Ganesan 2003) is a frequency domain (spectral) method which extracts the frequency components of a 2-dimensional (2D) gray scale image through down sampling it to the required scale. The 2D Stationary Wavelet Transform (SWT) is a translation invariant version of 2D Discrete Wavelet Transform (DWT). Texture features of a gray scale image of size “M
N” are extracted by decomposing the image using 2D stationary wavelet transform. Through the first level of decomposition applied on the gray scale image f(x, y), the wavelet coefficients f 0LL ðx, yÞ , f 0HL ðx, yÞ , f 0LH ðx, yÞ , and f 0HH ðx, yÞ are obtained. The f 0LL ðx, yÞ (where L denotes low frequency) wavelet coefficient represents one approximate component, while f 0HL ðx, yÞ , f 0LH ðx, yÞ , and f 0HH ðx, yÞ (where H denotes high frequency) wavelet coefficients represent three detail components. The statistical features like mean (μ), standard deviation (σ), and energy (E) of the wavelet coefficients ( f0(x, y)) are found using the formulae given in “Eqs. 3.32, 3.33 and 3.34.” μ¼

M N 1 XX 0 f ðx, yÞ MN x¼1 y¼1

vffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi u M X N u 1 X 2 σ¼t ð f 0 ðx, yÞ  μÞ MN x¼1 y¼1 vffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi u M N uX X 0 E¼t f ðx, yÞ2 x¼1 y¼1

ð3:32Þ

ð3:33Þ

ð3:34Þ

48

3 A Few Existing Basic and Multivariate Texture Models

The statistical feature values (x) thus obtained are normalized using the formula given in “Eq. 3.35.” xNormalized ¼

x  min max  min

ð3:35Þ

where “min” and “max” are the minimum and maximum values of the statistical features. The normalized statistical features found are concatenated to get global feature vector of the gray scale image as in “Eq. 3.36.” Waveletglobal ¼ ½μ, σ, E

ð3:36Þ

The “Program 10” in chapter appendix lists the Matlab code for computing wavelet texture features.

3.2.8

Gabor Wavelet-Based Texture Representation

Zhang et al. (2000) proposed texture feature representation using Gabor wavelet and proved it to be useful for content-based image retrieval. Gabor wavelet transform can break up a gray scale image into its frequency components. The 2D Gabor wavelet filter can be represented as a complex sinusoidal signal modulated by a Gaussian kernel function. It is capable of down sampling an image to a desired number of scales and orientations. So texture features obtained using Gabor wavelet are translation and rotation invariant. If f(x, y) is the gray scale image of size “M
N,” then the convolution of the image f(x, y) and a Gabor kernel (g(x, y)) gives the augmented Gabor feature vector f 0(x, y) as in “Eq. 3.37.” f 0 ðx, yÞ ¼ f ðx, yÞ  gðx, yÞ

ð3:37Þ

If a Gabor filter bank with “a” scales and “b” orientations is convolved with the gray scale image, the Gabor coefficients represented as f 011 ðx, yÞ ,. . . f 0ab ðx, yÞ are obtained. The Gabor wavelet texture features of the gray scale image are obtained by finding the statistical features like mean (μ), standard deviation (σ), and energy (E) of Gabor coefficients ( f0(x, y)) using “Eqs. 3.32, 3.33 and 3.34.” The statistical feature values (x) thus obtained are normalized using the formula given in “Eq. 3.35.” The normalized statistical features found are concatenated to get the global feature vector of the gray scale image as in “Eq. 3.38.” Gaborglobal ¼ ½μ, σ, E

ð3:38Þ

The “Program 11” in chapter appendix lists the Matlab code for computing Gabor wavelet texture features.

3.3 Existing Multivariate Texture Models

3.3 3.3.1

49

Existing Multivariate Texture Models Multivariate Local Binary Pattern

The LBP texture measure characterizes the pattern in the local neighborhood of a gray scale image. The multispectral image is obtained as follows. Three most suitable multispectral bands from a satellite sensor are chosen and combined to form a RGB image. To extend LBP for multispectral images with three bands, Lucieer et al. (2005) proposed the MLBP descriptor. The authors find 9 cross relations in the 3
3 neighborhoods of the three bands (RGB) in the following way. Three 3
3 matrices (neighborhoods) describe the local texture in each of the three bands R, G, and B individually (denoted as RR, GG, and BB). For example, the RR neighborhood is shown in “Eq. 3.39” where gcR represents the gray scale value of the center pixel in R band and g0R, g1R . . ., g7R represent the gray scale values of the neighbor pixels in the same band. Six more 3
3 matrices describe the local texture of the cross relation of each band with other bands (RG, RB, GR, GB, BR, and BG). For example, the GR neighborhood is shown in “Eq. 3.40.” The arrows in “Eqs. 3.39 and 3.40” indicate how GR cross relation is found by placing the center pixel of R band in place of the center pixel of G band in GG neighborhood. 2

gR0 6 R RR neighbourhood is4 g7 gR6

2

gR1 ! gRc

3 gR2 7 gR3 5

gR5

gR4

gG 0

gG 1

gG 6

gRc ↵ gG 5

6 GR neighbourhood is4 gG 7

gG 2

ð3:39Þ

3

7 gG 3 5 gR4

ð3:40Þ

The MLBP descriptor is defined in “Eq. 3.41” where “i” ranges from 0 to 7. It computes s(x) values in all nine “3
3” matrices formed through finding cross relations between bands and performs addition. The sum obtained is called MLBP, and it can have values in the range of 0 to 72.

MLBP ¼

P1 X i¼0

     B  R R s gRi  gRc þ s gG i  gc þ s gi  gc þ    G   B  G G s gRi  gG i þ s gi  gc þ s gi  gc þ      B  B B s gRi  gBc þ s gG i  gc þ s gi  gc þ

ð3:41Þ

The MLBP descriptor describes the texture in a local neighborhood of a RGB image. The global description within a window of a RGB image is provided by MLBP histogram with 72 bins. The “Program 12” in chapter appendix lists the Matlab code for computing MLBP.

50

3.3.2

3 A Few Existing Basic and Multivariate Texture Models

Multivariate Ternary Pattern

The MTP texture model proposed by Jenicka and Suruliandi (2015) is a multispectral extension of the basic TP descriptor. Three TP descriptors describe the local texture in each of the three bands (RGB) individually. For example, the TPRR descriptor is found as shown in “Eq. 3.42.” Considering the cross relations of each band with other bands as in MLBP, six more TP descriptors are found. For example, the TPGR descriptor is found as shown in “Eq. 3.43.” Nine TP descriptors thus found are arranged in a 3
3 matrix as in “Eq. 3.44,” and again TP is found which gives MTP. 2

TPRR

gR0 6 ¼ TP4 gR7 gR6

2

TPGR

gG 0 6 G ¼ TP4 g7 gG 6

2

gR1 ! gRc

3 gR2 7 gR3 5

gR5

gR4

gG 1 gRc ↵

3 gG 2 7 gG 3 5

gG 5

gR4

TPRR 6 GR MTP ¼ TP4 TP

TPRG TPGG

BR

BG

TP

TP

3 TPRB 7 TPGB 5

ð3:42Þ

ð3:43Þ

ð3:44Þ

BB

TP

The MTP descriptor describes texture in the local neighborhood of a RGB image. The MTP histogram describes the global texture within a window of a RGB image. The MTP histogram requires only 46 bins. The “Program 13” in chapter appendix lists the Matlab code for computing MTP.

3.3.3

Color GLCM

Homogeneity is one of the 14 Haralick’s texture measures related to contrast and is defined in “Eq. 3.12.” Color GLCM (CGLCM) was proposed by Benco and Hudec (2007) for GLCM-based feature extraction in RGB color space. Nine matrices showing cross relations between three bands of a color image were formed as explained previously in MLBP (see Sect. 3.3.1). The homogeneity measures of nine matrices were found and concatenated to obtain the feature vector (CGLCM) as in “Eq. 3.45.” For example, GLCMGR hom denotes the homogeneity value found for the cross relation matrix obtained by replacing the center pixel of G band with the center pixel of R band.

3.3 Existing Multivariate Texture Models

51

 RG RB CGLCM ¼ GLCMRR hom , GLCMhom , GLCMhom , GG GB GLCMGR hom , GLCMhom , GLCMhom , BG BB GLCMBR hom , GLCMhom , GLCMhom

ð3:45Þ

The CGLCM descriptor represents the global texture in a RGB image. The CGLCM global feature is a row vector of 9 elements as in “Eq. 3.45.” The “Program 14” in chapter appendix lists the Matlab code for finding CGLCM.

3.3.4

Wavelet-Based Texture Representation for Multispectral Images

The procedure followed (Arivazhagan and Ganesan 2003, Suruliandi and Jenicka 2015) to extract the wavelet-based texture features of a multispectral image is as follows. The multispectral image is converted from RGB image format to HSI (Hue-Hu, Saturation-S, and Intensity-I) image format because the bands “Hu” and “S” of the multispectral image have rich energy and color content that help in classifying the patterns precisely. These two bands are subsequently used for forming the global feature vector of the sub image (see Eq. 3.36) within the multispectral image as follows. The global feature vectors of “Hu” and S bands are defined in “Eqs. 3.46 and 3.47.” Waveletglobal,Hu ¼ ½μ, σ, EHu

ð3:46Þ

Waveletglobal,S ¼ ½μ, σ, ES

ð3:47Þ

The global feature vectors of “Hu” and S bands are further concatenated to represent the global feature vector of the sub image as in “Eq. 3.48.”  Waveletglobal ¼ Waveletglobal,Hu , Waveletglobal,S

ð3:48Þ

The “Program 15” in chapter appendix lists the Matlab code for finding the wavelet features of multispectral images.

3.3.5

Gabor Wavelet-Based Texture Representation for Multispectral Images

As in the previous section where texture features are extracted from a multispectral image using wavelet transform, a similar procedure is adapted (Zhang et al. 2000,

52

3 A Few Existing Basic and Multivariate Texture Models

Suruliandi and Jenicka 2015) for extracting texture-based Gabor wavelet features also. The multispectral image is converted from RGB image format to HSI image format, and the feature vectors (see Eq. 3.38) of “Hu” and “S” bands (Gaborglobal, Hu and Gaborglobal, S) are defined in “Eqs. 3.49 and 3.50.” Gaborglobal,Hu ¼ ½μ, σ, EHu

ð3:49Þ

Gaborglobal,S ¼ ½μ, σ, ES

ð3:50Þ

The feature vectors of “Hu” and “S” bands are concatenated to obtain the global feature vector (Gaborglobal) of the sub image within a multispectral image as given in “Eq. 3.51.”  Gaborglobal ¼ Gaborglobal,Hu , Gaborglobal,S

ð3:51Þ

The “Program 16” in chapter appendix lists the Matlab code for finding Gabor wavelet features of multispectral images. Exercises 1. In LBP, reason how incorporation of uniformity measure (U ) improves pattern representation. 2. Compare and contrast DLTP and TP. 3. Find ULBP for a 5
5 neighborhood assuming the same uniformity condition. 4. Implement and find a modified MLBP that sums up the ULBP values found for the nine cross relations of a 3
3 neighborhood. Discuss how many bins are required by the model. 5. Implement and find GLCM for an image of size “16
16” with values ranging from 1 to 10. Discuss about the size of the GLCM in such a case. 6. Spectral methods return augmented frequency coefficients by convolving the frequency domain filter with the input image in multiple scales and orientations. Justify how the multi-resolution approach is significant in texture analysis. 7. The LBP texture model is illumination, scaling, and rotation invariant. Justify from literature. 8. Implement in Matlab local directional number (LDN) texture measure proposed by Rivera et al. (2012) for face recognition in Matlab. 9. Implement and find augmented contourlet and shearlet transform features of a sub image in Matlab.

Appendix

Appendix Matlab Codes of Some Basic and Multivariate Texture Models (a) LBP Program1: Finding Conventional LBP function [LBP]=book_lbp(p) % Input: Let p(1:3,1:3) be the gray scale values of pixels in a 3x3 % neighborhood % Output: Local binary pattern value of the center pixel in the 3x3 % neighborhood (LBP) % Let f(1:8) hold the gray scale values of 8 neighbors in the % 3x3 neighborhood f(1)=p(1,1); f(2)=p(1,2); f(3)=p(1,3); f(4)=p(2,3); f(5)=p(3,3); f(6)=p(3,2); f(7)=p(3,1); f(8)=p(2,1); % Let gc hold the gray scale value of the center pixel in the % neighborhood gc=p(2,2); s(1:8)=0; % Assigning discrete output levels 0 and 1 to neighbor pixels for(i=1:8) if (f(i)< gc) s(i)=0; else s(i)=1; end end disp(s) % the built-in function ‘bi2de’ performs binary to decimal % conversion considering the right most bit as the most significant % bit. LBP=bi2de(s,'right-msb'); end Output: p=[123 110 113 117 120 135 130 125 128]; function [LBP]=book_lbp(p) s=[ 1 0 0 1 1 1 1 0] LBP=121

53

54

3 A Few Existing Basic and Multivariate Texture Models

Program 2: Finding ULBP function [x]=book_ULBP(p) % Input: Let p(1:3,1:3) be the gray scale values of pixels in a 3x3 % neighborhood % Output: Local binary pattern value of the center pixel in the 3x3 % neighborhood (LBP) % Let f(1:8) hold the gray scale values of 8 neighbors in the % 3x3 neighborhood f(1)=p(1,1); f(2)=p(1,2); f(3)=p(1,3); f(4)=p(2,3); f(5)=p(3,3); f(6)=p(3,2); f(7)=p(3,1); f(8)=p(2,1); % Let gc hold the gray scale value of the center pixel in the % neighborhood gc=p(2,2); s(1:8)=0; % Assigning discrete output levels 0, 1 and 9 to neighbor pixels for i=1:8 if (f(i)>=gc) s(i)=1; else s(i)=0; end end disp(s) z=abs(s(8)-s(1)); for(j=2:8) z=z+ abs(s(j)- s(j-1)); end if (z