A Comprehensive Guide to Neural Network Modeling 1536184667, 9781536184662

Citation preview

COMPUTER SCIENCE, TECHNOLOGY AND APPLICATIONS

A COMPREHENSIVE GUIDE TO NEURAL NETWORK MODELING

No part of this digital document may be reproduced, stored in a retrieval system or transmitted in any form or by any means. The publisher has taken reasonable care in the preparation of this digital document, but makes no expressed or implied warranty of any kind and assumes no responsibility for any errors or omissions. No liability is assumed for incidental or consequential damages in connection with or arising out of information contained herein. This digital document is sold with the clear understanding that the publisher is not engaged in rendering legal, medical or any other professional services.

COMPUTER SCIENCE, TECHNOLOGY AND APPLICATIONS Additional books and e-books in this series can be found on Nova’s website under the Series tab.

COMPUTER SCIENCE, TECHNOLOGY AND APPLICATIONS

A COMPREHENSIVE GUIDE TO NEURAL NETWORK MODELING

STEFFEN SKAAR EDITOR

Copyright © 2020 by Nova Science Publishers, Inc. All rights reserved. No part of this book may be reproduced, stored in a retrieval system or transmitted in any form or by any means: electronic, electrostatic, magnetic, tape, mechanical photocopying, recording or otherwise without the written permission of the Publisher. We have partnered with Copyright Clearance Center to make it easy for you to obtain permissions to reuse content from this publication. Simply navigate to this publication’s page on Nova’s website and locate the “Get Permission” button below the title description. This button is linked directly to the title’s permission page on copyright.com. Alternatively, you can visit copyright.com and search by title, ISBN, or ISSN. For further questions about using the service on copyright.com, please contact: Copyright Clearance Center Phone: +1-(978) 750-8400 Fax: +1-(978) 750-4470

E-mail: [email protected].

NOTICE TO THE READER The Publisher has taken reasonable care in the preparation of this book, but makes no expressed or implied warranty of any kind and assumes no responsibility for any errors or omissions. No liability is assumed for incidental or consequential damages in connection with or arising out of information contained in this book. The Publisher shall not be liable for any special, consequential, or exemplary damages resulting, in whole or in part, from the readers’ use of, or reliance upon, this material. Any parts of this book based on government reports are so indicated and copyright is claimed for those parts to the extent applicable to compilations of such works. Independent verification should be sought for any data, advice or recommendations contained in this book. In addition, no responsibility is assumed by the Publisher for any injury and/or damage to persons or property arising from any methods, products, instructions, ideas or otherwise contained in this publication. This publication is designed to provide accurate and authoritative information with regard to the subject matter covered herein. It is sold with the clear understanding that the Publisher is not engaged in rendering legal or any other professional services. If legal or any other expert assistance is required, the services of a competent person should be sought. FROM A DECLARATION OF PARTICIPANTS JOINTLY ADOPTED BY A COMMITTEE OF THE AMERICAN BAR ASSOCIATION AND A COMMITTEE OF PUBLISHERS. Additional color graphics may be available in the e-book version of this book.

Library of Congress Cataloging-in-Publication Data Names: Skaar, Steffen, editor. Title: A comprehensive guide to neural network modeling / Steffen Skaar, editor. Description: New York : Nova Science Publishers, [2020] | Series: Computer science, technology and applications | Includes bibliographical references and index. | Identifiers: LCCN 2020035191 (print) | LCCN 2020035192 (ebook) | ISBN 9781536184662 (paperback) | ISBN 9781536185423 (adobe pdf) Subjects: LCSH: Neural networks (Computer science) | Food--Drying. | Food industry and trade. | Water quality management. Classification: LCC QA76.87 .A255 2020 (print) | LCC QA76.87 (ebook) | DDC 006.3/2--dc23 LC record available at https://lccn.loc.gov/2020035191 LC ebook record available at https://lccn.loc.gov/2020035192

Published by Nova Science Publishers, Inc. † New York

CONTENTS Preface Chapter 1

Chapter 2

Chapter 3

Chapter 4

Index

vii Application of Artificial Neural Networks (ANNS) Modelling in Drying Technology of Food Products: A Comprehensive Survey Raquel Guiné, Iman Golpour, Maria João Barroca and Mohammad Kaveh Application of Artificial Neural Networks in the Food Engineering Ana Jurinjak Tušek, Davor Valinger, Maja Benković, Jasenka Gajdoš Kljusurić and Tamara Jurina Artificial Neural Networks as a Chemometric Tool in Analysis of Biologically Active Compounds Strahinja Kovačević, Milica Karadžić Banjac, Sanja Podunavac-Kuzmanović and Lidija Jevrić River Water Quality Modelling Using Artificial Intelligence Techniques Eda Göz, Erdal Karadurmuş and Mehmet Yüceer

1

57

83

123 155

PREFACE As artificial neural networks have been gaining importance in the field of engineering, this compilation aims to review the scientific literature regarding the use of artificial neural networks for the modelling and optimization of food drying processes. The applications of artificial neural networks in food engineering are presented, particularly focusing on control, monitoring and modeling of industrial food processes. The authors emphasize the main achievements of artificial neural network modeling in recent years in the field of quantitative structure– activity relationships and quantitative structure–retention relationships. In the closing study, artificial intelligence techniques are applied to river water quality data and artificial intelligence models are developed in an effort to contribute to the reduction of the cost of future on-line measurement stations. Chapter 1 - Drying of foods has been used to preserve food and agricultural products since immemorial times. However, still nowadays it assumes a prominent place among food processing technologies applied industrially to extend shelf life of foods. Although having some important advantages, like the reduction in water activity and subsequent minimization of degradation reactions of biological, chemical or enzymatic nature, reduction in size for transportation and storage or avoidance of refrigeration

viii

Steffen Skaar

systems during transportation and storage, it is also true that drying brings high energy costs and some possible undesirable changes in quality parameters. Hence, the optimization of drying processes is of the utmost importance to minimize energy costs and maximize quality. Mathematical modelling in food process engineering allows important savings, while also guaranteeing the safety of industrial plants and workers, and finally achieving ultimate quality of the dried foods. Because artificial neural networks (ANNs) have been gaining importance in the context of many problems in the fields of engineering, among others, this chapter aims to do a review of scientific literature about the use of artificial neural networks to modelling and optimization of food drying processes. Finally, opportunities and restrictions of the ANNs technique for drying process simulation, optimization, and control are achieved to guide future R&D in this area. Chapter 2 - Artificial neural networks (ANNs) are of great interest because of their ability to solve problems connected to interpretation of results obtained by various analytical methods. These results sometimes differ from the ordinary form in term of vast number of results for one measurement. Examples of those results include Near Infrared Spectroscopy (NIRs) spectra or results that have to be in a specific interval. ANNs are composed of group of nonlinear regression and discrimination statistical methods and are often used for their ability of visualization and prediction which is based on their learned and trained knowledge. Use of ANNs has been widely studied since they correspond to computational systems that aim to imitate some properties of biological neurons. Basically, the ANN is a system which corresponds to human brain in term of neurons that are linked by synaptic connections. The neurons are divided into i) incoming; which are stimulated by external environment, ii) internal or hidden neurons and iii) output neurons; which provide communication to the outside system. There are a lot of advantages of ANNs such as: use for nonlinear and nonparametric modeling, stability (with enough data) and high noise tolerance. Due to their characteristics, ANNs have found wide areas of application, from finance and medicine, over geology and physics to food engineering. In this chapter, the application of ANNs in food engineering will be presented. According to available novel literature, ANNs have been used in

Preface

ix

food engineering for control, monitoring and modeling of industrial food processes. Furthermore, ANNs are used for recognition, detection, classification as well as for the search of patterns, prediction of on-line parameters, image processing and optimization. Why and how ANNs are applied is explained in this chapter using examples from food/beverage matrices. Chapter 3 - Prediction of biological activity and molecular properties of newly synthesized compounds and compounds with certain biological effects is a quite complex task. It requires high-quality and relevant data, as well as sophisticated chemometric methods for data analysis and extraction of the most important information from the data set. Artificial neural networks (ANNs) have become very popular in chemometric modeling of biological activity and chromatographic retention behavior of biologically active compounds in the past decades. ANNs have been widely utilized for regression purposes in Quantitative Structure–Activity Relationship (QSAR) and Quantitative Structure–Retention Relationship (QSRR) modeling as a non-linear regression method. Their efficacy and usefulness in prediction of various molecular features have been proven in numerous scientific studies whose aim was to establish high-quality models which served as chemometric guidelines for selection of the most prominent compounds. The present review chapter emphasizes the main achievements of ANN modeling in recent years in the field of QSAR and QSRR analysis of various biologically active molecules and points out its advantages and disadvantages. The applicability domain of ANNs has also been discussed, including the necessary data transformation prior to modeling. One of the main goals of this chapter is to present in a simple way contemporary developments in the application of ANNs in the design of biologically active compounds, as well as in the prediction of their physicochemical and biological properties, which are crucial for their further application in biological research. Chapter 4 - Water pollution has become a major issue in rivers. The possibility of a pollutant to be discharged to the river as municipal and industrial waste is an important problem for those using water from rivers.

x

Steffen Skaar

Nevertheless, due to the rapid population growth in the world and the irresponsible use of water resources, the world will face a serious lack of water in the near future. Therefore, the water resources of the future must be preserved very well to leave healthy and enough water for next generations. In order to prevent river water pollution, river water quality should be constantly monitored and evaluated. This way, information on the status of water quality may be obtained, and river basin management planning may be carried out. For this purpose, measurement at points can be made, or online monitoring stations can be established on river basins. According to the collected data, management actions may be created for how waterways function and how pollutants affect evaluation. In addition to this effect, seasonal changes and long-term trends must be taken into consideration. Artificial intelligence (AI) techniques have been used recently in many engineering fields. The most widely used ones among AI techniques are artificial neural networks (ANN). These are followed by support vector regression (SVR), least squares support vector regression (LS-SVR), least squares support vector machine (LS-SVM) and fuzzy logic. In the past 15 years, extreme learning machine (ELM) and its types have been used in development of many forecasting models. The statistical accuracy of classical models is commonly poor because natural systems tend to be complex and nonlinear for deterministic modelling methods. AI techniques provide a fast and flexible means of creating models for estimation of river water quality. In recent years, AI techniques have shown exceptional performance as regression tools, especially when used for pattern recognition and function estimation. In this study, AI techniques will be applied to river water quality data, and AI models will be developed. The data were collected from an on-line measurement station that was established on the Yeşilırmak River in Amasya/Turkey. In the selected region, two different measurement stations were built at a distance of about 28 km. Twelve parameters as luminescent dissolved oxygen (LDO), pH, conductivity, nitrate nitrogen (NO3-N), ammonium nitrogen (NH4-N), total organic carbon (TOC), chloride, orthophosphate, temperature, turbidity, suspended solid and flow rate were measured at five-minute intervals at these stations. Specifically, two different models as DO prediction and TOC

Preface

xi

prediction models were developed with five different approaches. These approaches were artificial neural network (ANN), support vector regression, least squares support vector regression, extreme learning machine and kernel extreme learning machine. Model performances were evaluated with some performance indices. This study is a state-of-the-art study due to the fact that parameters that are expensive to measure can be predicted from parameters that are cheaper to measure. For this reason, it will contribute significantly to reducing the cost of on-line measurement stations planned to be established in the future.

In: A Comprehensive Guide … Editor: Steffen Skaar

ISBN: 978-1-53618-466-2 © 2020 Nova Science Publishers, Inc.

Chapter 1

APPLICATION OF ARTIFICIAL NEURAL NETWORKS (ANNS) MODELLING IN DRYING TECHNOLOGY OF FOOD PRODUCTS: A COMPREHENSIVE SURVEY Raquel Guiné1,*, Iman Golpour2, Maria João Barroca3,4 and Mohammad Kaveh5 1

CERNAS-IPV Research Centre, Polytechnic Institute of Viseu, Campus Politécnico, Viseu, Portugal 2 Department of Mechanical Engineering of Biosystems, Urmia University, Urmia, Iran 3 Research Unit in Molecular Chemistry-Physics, Department of Chemistry, University of Coimbra, Portugal 4 Coimbra College of Agriculture, Polytechnic Institute of Coimbra, Portugal 5 Department of Agricultural Machinery, University of Mohaghegh Ardabili, Ardabil, Iran

*

Corresponding Author’s Email: [email protected].

2

Raquel Guiné, Iman Golpour, Maria João Barroca et al.

ABSTRACT Drying of foods has been used to preserve food and agricultural products since immemorial times. However, still nowadays it assumes a prominent place among food processing technologies applied industrially to extend shelf life of foods. Although having some important advantages, like the reduction in water activity and subsequent minimization of degradation reactions of biological, chemical or enzymatic nature, reduction in size for transportation and storage or avoidance of refrigeration systems during transportation and storage, it is also true that drying brings high energy costs and some possible undesirable changes in quality parameters. Hence, the optimization of drying processes is of the utmost importance to minimize energy costs and maximize quality. Mathematical modelling in food process engineering allows important savings, while also guaranteeing the safety of industrial plants and workers, and finally achieving ultimate quality of the dried foods. Because artificial neural networks (ANNs) have been gaining importance in the context of many problems in the fields of engineering, among others, this chapter aims to do a review of scientific literature about the use of artificial neural networks to modelling and optimization of food drying processes. Finally, opportunities and restrictions of the ANNs technique for drying process simulation, optimization, and control are achieved to guide future R&D in this area.

Keywords: drying technology, process control and modelling, ANNs modelling

INTRODUCTION TO THE DRYING OF FOODS The changes in consumers’ attitudes and the life expectancy of the population have increased the consumption of processed and convenience foods. Furthermore, the consumption of fresh, highly nutritious foods, such as fruits and vegetables, is generally below the recommended daily levels for normal diets (Huang and Zhang, 2012). Their acceptability can be increased if they are minimally processed, giving place to food products that closely resemble the fresh material and with high quality, conveniently packaged and with a longer shelf life. Among the processes to extend the

Application of Artificial Neural Networks …

3

preservation of fresh foods that are used in the food industry, drying is the most widely used method, although it may significantly affect the quality of the final products (Mujumdar, 2006). The drying processes extend the shelf life of foods by reducing the water content and, thus, inhibiting enzymatic modifications and microbial and reducing the rate of chemical reactions in the materials’ matrix. Besides preservation, other important advantages of drying are the reduction in size and weight, minimizing packaging, facilitating transport, reducing storage space, allowing their use during offseason and avoiding the need of refrigeration systems. Finally, the drying process increases the food diversity and the alternative ways of consuming food (Guiné et al., 2013; Mota et al., 2010; Zielinska et al., 2019). The drying process is one of the most important methods for industrial food preservation that can be applied to a wide variety of foods (fruits, vegetables, meat, fish, cereals, industrial by-products, etc…) producing stable products (powders) suitable for snacks, cereals, cake mixes and soups, promoting the development of new food products to satisfy the always increasing demand of novel foods (Huang and Zhang, 2012). However, the acceptance of processed or minimally processed food products by consumers is highly dependent on quality and nutritional attributes as well as on their organoleptic characteristics. Moreover, consumers now demand high-value food products that are healthier and have high nutritional value.

THE NEED TO CONTROL DRYING PARAMETERS Drying processes have a great impact on the product’s structural properties (e.g., shrinkage, porosity, volume, density, pore size distribution and surface area) and also impacts on its physicochemical properties (e.g., texture, colour, nutritive value and appearance) of most food products (Nguyen et al., 2018; Yang et al., 2020; Zielinska and Markowski, 2010). Drying is a complex process with unsteady heat and mass transfer that allows physical or chemical transformations, which can affect product quality (Chandramohan and Talukdar, 2014). Thus, to preserve the desired

4

Raquel Guiné, Iman Golpour, Maria João Barroca et al.

quality of the dried foods it is essential an adequate selection of the drier and the optimization of the drying process. There is no simple way to classify dryers since they can work in different modes of operation, pressure, heat input type, number of stages or state of the material in the dryer, among others. Even though they may present some differences, driers can be classified in one of the following categories: (i) direct contact between the heating medium (hot air) and wet material (food); (ii) indirect contact where heat transfer takes place from an external medium, such as condensing steam; and (iii) heated by dielectric or microwave energy (Siddiq and Uebersax, 2018). Drying technology includes methods that start with the simple use of solar energy and evolve to current technologies, which include, among others, convective stove, tray drying, spray drying, drum drying, lyophilisation, osmotic dehydration, extrusion, fluidization, and the use of microwaves and radio frequency, which can be used as a single preservation technology or in conjugation with others in hurdle technology. (VegaMercado et al., 2001). Solar energy is a friendly environmental energy source, that is frequently used for drying fruits and vegetables and, still up to recent decades, solar drying has become popular particularly in regions characterized by a hot and dry climate. The oldest, traditional and cheapest process of food drying is the open sun drying, but this method has a lot of disadvantages as the materials are spread over the ground or on mats or trays and, therefore, the process has a strong dependency on the weather conditions and the food material is easily contaminated by dirt, damaged by animals, birds or insects and other environmental hazards derived from exposure to direct sun radiation. The spoilage of dried foods due to contamination by the action of insects, rodents and microorganisms allied to the deterioration during storage due to non-uniform drying cause additional losses in the direct solar drying of foods (Anderson and Westerlund, 2014). Furthermore, the drying time can be longer than 10 days (Chandramohan, 2020). To reduce the drawbacks of open sun drying, the indirect type solar dryer (natural convection dryer and forced convection dryer) can be used as an alternative (Kumar et al., 2016). Dryer selection depends on many

Application of Artificial Neural Networks …

5

parameters, but the most dominant ones affecting the drying rate are air temperature and flow rate, followed by solar radiation, type of product, initial moisture content and total mass of the product. To increase the drying rate and to preserve the quality of the products after drying, pre-treatment of the products can be used (El-Sebaii and Shalaby, 2013; Vijayan et al., 2016). Among the various types of dryers and construction details, natural convection dryers have an easy construction and require lower cost than forced convection dryers, but have no control over drying rate, unlike forced convection dryers. Forced convection of indirect type solar dryer with double/triple pass and collector with reflector promise better performance, higher drying rate and final food quality (Kumar et al., 2016; Lingayat et al., 2020). Moreover, the use of heat or thermal storage systems is useful for continuing the drying process after sunset, providing a shortening of the drying process (Kant et al., 2016). Although solar energy has been used from ancient times to dry foods, the application of the method as a unit operation in food processing led to the use of other sources of energy, namely, fossil fuel, electricity, natural gas and biomass (Lingayat et al., 2020). Moreover, concerning the optimization of the process in terms of final food quality and energy consumption, the evolution of drying technology can be divided into four generations (Varzakas and Tzia, 2015; Vega-Mercado et al., 2001).

Generation I The first generation includes cabinet and bed type dryers (such as tray, stove, rotary flow conveyor and tunnel) that use hot air flowing over an extensive area of the product to remove its water from the surface. These types of dryers, with several commercial alternatives, are mostly suitable for solid materials such as vegetables and fruits, sliced or chunked, and grains (Chandramohan, 2020). These driers are one of the most energy-consuming food preservation processes, but their main disadvantage focuses on the reduction of quality of dried foods compared with the fresh ones. High temperatures during hot-air drying have a great influence on the physical

6

Raquel Guiné, Iman Golpour, Maria João Barroca et al.

structure of the final food (such as a decrease in porosity, reduction in volume and increase in stickiness (Michailidis and Krokida, 2013).

Generation II The second generation of dryers includes spray dryers and drum dryers that are used mainly for dehydration of purees and slurries, to produce flakes and powders that are intended to further rehydration. Spray drying involves both particle formation through liquid atomization and drying by feed (solution, suspension, emulsion or paste) into a hot drying medium. The drying would be controlled through the properties of the product (such as its total soluble solids) and air input conditions (flow and temperature) to obtain products which rehydrate quickly and disperse/dissolve without the formation of lumps (Pandidurai and Vennila, 2018). The short drying time and low temperature allow the drying of products extremely heat-sensitive. However, this process is associated to the high cost and low thermal efficiency. Drum dryers are one of the most energy-efficient methods in which drying takes place on the outer cylindrical surface of one or two slowly rotating drums. As the exposure to high temperatures (120170 ºC) is limited to a few seconds, the method can be applied even in the cases of heat-sensitive foods. Energy efficiencies in these dryers may range between 70% and 90%, when compared to 40–60% for hot air drying (Michailidis and Krokida, 2013; Varzakas and Tzia, 2015).

Generation III The third generation includes drying methods such as freeze and osmotic drying, which facilitate water removal by methods not based on water vaporization, but rather on other principles, namely, sublimation and movement due to osmotic pressure differential. These dryers include technologies that involve the immersion of food products in a hypertonic

Application of Artificial Neural Networks …

7

solution (sugar, salt, sorbitol or glycerol) as osmotic dehydration or the food freezing before the drying by direct sublimation of the ice under reduced pressure (freeze dehydration). Osmotic dehydration has received greater attention in recent years as an important complementary treatment and food preservation technique in the processing of dehydrated foods, since it presents some benefits such as reducing the damage of heat to the flavour, colour and inhibiting the enzymatic browning. Moreover, the use of this treatment in combination with freeze, convective, vacuum, microwave or infrared drying is the best way to decrease energy costs and to improve the quality of the product (Bhatta et al., 2020; Ciurzyńska et al., 2016; Haseley and Oetjen, 2018; Khan, 2012; Miguel Landim et al., 2016; Ramya and Jain, 2017). Freeze-drying (lyophilisation) involves water removal by sublimation at low temperatures and under a high vacuum and, consequently, it has been one of the most useful methods for producing high quality products. Although during freeze-drying the glass transition temperature is a critical point to the product quality, it’s monitoring ensures adequate product structural properties and moisture content (Oyinloye and Yoon, 2020). However, its application is still not widespread due to the required high upfront capital and operating costs. Nevertheless, the protection of sensitive properties such as colour, appearance, texture, porosity, aroma, phenolic compounds and the nutritional value of foods may compensate its high operating costs as well as the long drying time (Oyinloye and Yoon, 2020; Shofian et al., 2011; Wang et al., 2006).

Generation IV The fourth generation refers to methods that have been recently developed in the area of dehydration technology, involving fluidization, high vacuum and the use of radio frequency (RF), ultrasound, microwaves, refractance window and the hurdle approach (Li et al., 2010; Xie et al., 2020; Zielinska et al., 2019). The use of these methods promotes the increase of the drying rate, the interaction of the drying atmosphere with the inner molecules of food, the reduction of energy consumption and the extended

8

Raquel Guiné, Iman Golpour, Maria João Barroca et al.

preservation of bioactive compounds. The applications of these technologies are dependent on the properties of the raw materials and of the intended quality of the dried products. However, due to the low extensive chemical changes and energy savings, the microwave and RF have been attracting growing interest in dehydration of different food products (Li et al., 2010; Vega-Mercado et al., 2001; Wang et al., 2020; Xie et al., 2020; Zielinska et al., 2019). Even though the aforementioned generations are still considered viable ways for the food drying processes, the current trend of food drying is now focusing on the development of novel hybrid methods trying to combine the advantages of two or more individual techniques in order to provide the best trade-off between quality and energy efficiency. The optimization of process conditions and product characteristics is pivotal for drying efficiency, allowing minimizing costs and maximizing quality. The ANNs are powerful tools to help achieve these objectives, and have been used extensively for modelling and optimization of many food processing technologies, including drying (Barroca et al., 2017; Guiné et al., 2019; Guiné et al., 2019).

THE PRINCIPLES OF ANN MODELLING AND ALGORITHMS Drying is an extremely complex thermal process during which unsteady heat and moisture transfer happen simultaneously. The use of mathematical models to describe the drying processes is particularly useful because it allows a better understanding of these operations. Hence, the perception and modelling of the complicated relationships in the process of drying is very important in the food industry (Guine et al., 2017). Diagnosing the available relationships between the input and output parameters in a process by application of statistical, numerical, prevalent mathematical and analytical methods is very complex or even infeasible. Anyway, intelligent approaches

Application of Artificial Neural Networks …

9

such as artificial neural networks (ANNs), as promising alternatives, can be applied for solving the nonlinear and complex problems of ill-defined processes like drying technology by use of previous identified data.

Model of a Biological Neuron Scientists have proven that the human brain is based on a structural unit of its complex biological neural network, consisting of an extremely complex system of about 1011 nervous cells named neurons and 104 connections per neuron (Ladstätter and Garrosa, 2008). It is worth mentioning that these neurons can be found in different types, based on the sizes and shapes, in various segments of the human brain. ANNs are known as the connectionist models due to the connections occurred among signalprocessing units called the neurons (Shanmuganathan, 2016). Figure 1 shows the basic components of a typical biological neuron of an ANN that consists of soma, dendrites and axon. As shown in Figure 1, the activation signals are sent to the dendrite through other neurons existing in the model, and synapses in fact transfer these electrical impulses via electrochemical processes. Finally, the input signals are added by the soma and if the summation reaches a specific amount, the signals will be sent by the neuron that will spread the axon element into other neurons (Wang and Slikker, 2017). A biological neuron catches the input characteristics through other parts and then incorporates them in several ways and finally accomplishes a universally nonlinear operation on the obtained results so that the ultimate results will be given as the outputs. Nevertheless, the main aspects that are specified as prevalent functions in the artificial networks include learning and adaptation process, generalization, enormous parallelism, associative storage of corresponded information and spatiotemporal information processing (Shanmuganathan, 2016).

10

Raquel Guiné, Iman Golpour, Maria João Barroca et al.

Figure 1. Structure of biological neurons relevant to ANNs

The Perceptron Model ANNs are a computational model consisting of many nonlinear and parameterized analog signal-processing units (neurons) connected by links (synapses) of variable numerical weights between the neurons. The neuron network and its synapses establish the model that could be executed artificially via software programs and specific hardware (Guiné, 2019). An artificial neuron in the form of a simplified version is demonstrated in Figure 2. This model of an artificial neuron was proposed for the first time and extended by McCulloch and Pitts in 1943. An ANN is a set of artificial neurons as interconnected, interacting with one another in a concerted Behavior. The perceptron as the earliest artificial neuron which was suggested (Rosenblatt, 1958), is considered as the principle building block for almost all ANNs (Graupe, 2013). Therefore, an artificial neuron includes different inputs (xi) multiplied by a connection weights (wi) before reaching the principal body of the processing unit. In the simplest case, these weighted signals are then summed and fed into the transfer function for generating an important result as the output.

Application of Artificial Neural Networks … Table 1. Different activation functions Name: Equations:

Piece wise linear 1 𝑖𝑓 𝑥 ≥ 𝑥𝑚𝑎𝑥 𝑓(𝑥) = {𝑚𝑥 + 𝑏 𝑖𝑓 𝑥𝑚𝑖𝑛 < 𝑥 < 𝑥𝑚𝑎𝑥 0 𝑖𝑓 𝑥 ≤ 𝑥𝑚𝑖𝑛

Diagrams:

Name: Equations:

Gaussian 𝑓(𝑥) = 𝑒 −𝛼𝑥

2

Diagrams:

Name: Equations:

Sigmoid 𝑓(𝑥) =

1 1 + 𝑒 −𝛼𝑥

Diagrams:

Name: Equations:

Hyperbolic tangent 𝑓(𝑥) =

𝑒 𝛼𝑥 − 𝑒 −𝛼𝑥 𝑒 𝛼𝑥 + 𝑒 −𝛼𝑥

11

12

Raquel Guiné, Iman Golpour, Maria João Barroca et al. Table 1. (Continued) Diagrams:

Name: Equations:

Linear threshold (Heaviside) (𝑥) = {

1 𝑖𝑓 𝑥 ≥ 0.5 0 𝑖𝑓 𝑥 < 0.5

Diagrams:

Name: Equations: Diagrams:

Linear 𝑓(𝑥) = 𝑥

The activation function is used to transform the input into a further helpful output. It is worth noting that neurons, activation functions, and outputs are respectively analogous to synapses, soma and axon, in a biological neural model so that the weights can be positive or negative actual

Application of Artificial Neural Networks …

13

numbers (Aghbashlo et al., 2018). The activation functions commonly developed in ANNs, including nondecreasing, nonlinear, monotonic as some of the most popular functions, along with their mathematical formulations, are shown in Table 1. These activation functions also could scale the output or control its value through thresholds (Aghbashlo et al., 2018; Vega-Carrillo et al., 2007).

Figure 2. Perceptron model.

Training Algorithms In ANNs, learning algorithms can help to operate the modification process of the parameters and the values for adapting their behavior to the environment. The learning procedure in ANNs is developed by adjusting the levels of weights and bias in a network for generating a desirable response to a particular input. An ANN should be learned by means of such ways, that the application generates of a group of the inputs the desirable group of the outputs. There are different kinds of training algorithms which can be divided into three categories namely supervised, reinforced, and unsupervised algorithms as follows (Aghbashlo et al., 2018; Ladstätter and Garrosa, 2008):

14

Raquel Guiné, Iman Golpour, Maria João Barroca et al.

Supervised Learning The network can be trained by supplying it with a group of instances (the training group) of true neural network manner: {𝑝1 , 𝑡1 }, {𝑝2 , 𝑡2 }, {𝑝3 , 𝑡3 }, … , {𝑝𝑛 , 𝑡𝑛 }

(1)

where pi is as an input parameter to the network, ti is the parameter of the respective target output, and n is the specific number of the training pairs. Actually, in this method, including a teacher, a lot of pairs of input/output training patterns can supply a learning data set whereas after using the inputs, the network outputs can be compared by using the targets and then the error signal is utilized to adapt the weights so that the weights and biases of the network are modified by this training algorithm for reducing the error between the outputs and the targets of the network.

Reinforced Learning In the reinforced learning unlike the supervised training mode, there are not any targets to adjust the weights. Unsupervised Mode In the unsupervised learning, often called self-organization, the training dataset is made of input training patterns solely without outside help to cluster various input patterns into different classes. Overall, the weights and biases of the network can be singly rectified in response to the inputs of the network so that there are no available outputs of the target. Most of the algorithms accomplish this type of clustering operations.

Network Architectures The architecture of ANNs is commonly arranged in layers with individual neurons that can be conjoined to other neurons. Generally, one neuron, even with many numbers of inputs, may not be adequate for solving feasible issues. A lot of needed neurons that operate in parallel, are those

Application of Artificial Neural Networks …

15

ones which perform the same operation at the same time. So, there is a strong relationship between the network structure and learning algorithm that makes the design central. The flow and the way of the signals that neurons are conjoined between together can recognize the type of the network. With respect to the pattern of available connections among neurons, ANNs can be categorized into two kinds of architectures called feed-forward neural networks (FNNs), and Recurrent feed-back neural networks (RFNNs). One of the most important difference between the networks of FNNs and RNNs is the attendance of a mechanism as a feedback for the network amongst the available neurons in the latter on architecture. Anyway, Single Layer Perceptron (SLP), Multi-Layer Perceptron (MLP), and Radial Basis Function Nets (RBF) are three kinds of Feed-Forward Networks (Al-Jaberi, 2018; Shaker et al., 2014).

Feedforward Artificial Neural Network Feed forward ANNs refer to artificial neural networks for which there is a one-sided signal flow and the processing of information and connections among the available neurons in the ANN cannot constitute a directed cycle and is allowed only to attain information from the former neuron through the hidden nodes (if any) so that there are not any loops or cycles in the neural network.

A Single Layer Perceptron Feedforward ANN Model In the networks of feed-forward model, the signal flow is formed from the units of input to output, especially in a direction of feed-forward, from the available nodes in network input, via the network nodes in hidden layer (if any) and then to the nodes in output layer. Overall, in artificial neural network for a single layer network (the simplest network) is considered as an input layer and an output layer of neurons. The inputs are directly transmitted to the outputs by a series of weights. In addition, the output

16

Raquel Guiné, Iman Golpour, Maria João Barroca et al.

nodes utilize activation functions to produce the desirable outputs. Figure 3 illustrates a single layer of N neurons for which each element of Sdimensional input vector is conjoined to each of neurons and the weights (Aghbashlo et al., 2018; Al-Jaberi, 2018; Graupe, 2013).

Figure 3. A single layer of neurons.

Multiple Layer Feedforward ANN A schematic illustration of Multiple Layer Feedforward (MLP) feedforward ANN model is demonstrated in Figure 4. It consists of several layers as an input, one or more hidden layers, and also output layer(s). In MLP ANNs, each neuron is completely connected to all the neurons in the layers without connections between nodes in the same layer, whereas signals only dissipate via the network in the feed-forward direction. Once the individual weights of each connection have been specified, an output amount can be computed for each input of an ANN (Burghardt and Garbe, 2018; Shanmuganathan, 2016). Accordingly, in this ANN, the signals are received by input nodes and then these signals reach the first hidden layer by the corresponding connection of the first layer. Finally, the output signals are

Application of Artificial Neural Networks …

17

fed into second hidden layers and then the output signals from hidden layers are fed into output layers for generating the desirable output. The number of nodes and hidden layers are to be determined by the users. Anyway, an example of the structure of a MLP feed forward neural network created by the MATLAB software is indicated in Figure 5.

Figure 4. A schematic representation of a MLP ANN structure.

Figure 5. An example of Feedforward MLP structure of ANN with two hidden layers.

Recurrent Neural Networks (RNNs) The topology of this type of network is very universal because it allows the operation of feedback connections among neurons so that each neuron is connected to others, even to itself as shown in Figure 6. To allow the operation of feedback connections in neurons could help naturally to analyze

18

Raquel Guiné, Iman Golpour, Maria João Barroca et al.

the neural networks as the dynamic systems depend on the state at a former moment in time. Fully recurrent (Hopfield network and Boltzmann machine), simple recurrent, echo state, long short-term memory, bidirectional, hierarchical, and stochastic neural networks are different types of recurrent ANN topologies (Yi and Tan, 2004).

Figure 6. A typical recurrent network structure.

Adaptive Neural-Fuzzy Interface System (ANFIS) According to the limitations of ANNs and Fuzzy systems, the individual defects of both theses method can be eliminated by combining two softcomputing methods, ANNs and flexible knowledge representation capability of fuzzy logic systems, in a single framework (Suparta and Alhasa, 2016). ANFIS system, firstly suggested by Jang in 1993, is one of the best approaches for time-sequence computations while back-propagation method is employed during these calculations (Staub et al., 2015). It is an applied method with a hybrid intelligent process, which generates fuzzy rules from a specified dataset as input-output by implementing Takagi– Sugeno fuzzy inference system. In addition, the ANNs can be utilized to

Application of Artificial Neural Networks …

19

regulate the membership functions (MFs) and decrease the rate of errors in the determination of rules in fuzzy logic. The ANFIS is exactly a six-layer generalized network with supervised learning, composed of 1) input layer, 2) fuzzification layer, 3) rules layer, 4) normalization layer, 5) defuzzification layer, and 6) summation (output or decision) layer as demonstrated in Figure 7 (Heddam et al., 2019). This system can be trained by application of a hybrid learning algorithm via integrating back propagation with least square estimate (LSE) (Aghbashlo et al., 2018). In the architecture of ANFIS, the fuzzification and defuzzification layers play the role of adaptive layers. In the fuzzification layer, two modifiable parameters ({σi, ci}), which are identified with the input membership functions, exist, while there are three adjustable variables ({pi, qi, ri}) in the defuzzification layer (Heddam et al., 2019).

Figure 7. An ANFIS Architecture.

Hybrid Neural Network (HNN) Model A hybrid neural network (HNN) model with help of “grey-box” approaches incorporates two forms of ANNs, and possesses notable ability for modeling engineering complex problems like drying technology (see

20

Raquel Guiné, Iman Golpour, Maria João Barroca et al.

Figure 8). Specifically, this issue can be greatly overcome by using a backpropagation (BP) network (as another name for a MLP) and a radial basis function (RBF) network, whereas demonstrating the target outputs of the model as a relation mentioned the following equation:

Figure 8. Structure of the hybrid neural network.

FHNN(x) = µ1FBP(x) + µ2FRBF(x)

(2)

with µ1 and µ2 being mixing coefficients (Argatov, 2019). It is proposed to utilize a method as genetic algorithm for training the HNN (Gandomi and Roke, 2015). Accordingly, the output in the network of RBF can be indicated as following relation (Argatov, 2019): 𝑁

𝐹(𝑥) = ∑

‖𝑥−𝑐𝑗 ‖

𝑤𝑗 𝜌 (

𝑗=1

𝜎𝑗

)

(3)

where, cj denotes the center vector of the jth node of the kernel, wj and σj denote its weight and smoothing parameters, ‖𝑥 − 𝑐𝑗 ‖ denotes the norm of

Application of Artificial Neural Networks …

21

the Euclidean distance between the parameter x as input and the center of the jth node, and ρ(·) is the radial basis function, which is generally stated to be Gaussian so that is ρ(x) = exp(-x2/2).

Hybrid Mathematical-Neural Model Despite individual shortcomings of mathematical neural network models to predict the characteristics of engineering complicated problems like drying technology, combining at least two intelligent methods as a hybrid intelligent system or “grey-box” approaches like Neural networks, Genetic algorithm, Fuzzy Logic and reinforcement Learning can help to forecast, model and control the food drying process beyond the range of the used data as shown in Figure 9. Combining the various techniques in a computational model makes such systems possess a prolonged range of abilities. In general, the hybrid neural models can be categorized into series and parallel approaches (Aghbashlo et al., 2018; Rath et al., 2013).

Figure 9. Schematic representation of hybrid mathematical-neural model.

22

Raquel Guiné, Iman Golpour, Maria João Barroca et al.

Selection of Optimal ANN To obtain the predicted output by ANN according to input data, commonly about 70% of empirical data is randomly selected for training data, 15% for validation data and finally 15% for test data. Overall, to evaluate the goodness of fit for the optimal ANN models to the empirical dataset in drying process, several important statistical indexes can be utilized as the determination coefficient (R2), mean square error (MSE) and also mean absolute error (MAE) for the evaluated models as follows (Golpour et al., 2015): 1

𝑀𝑆𝐸 = 𝑛 ∑𝑛𝑘=1(𝑆𝐾 − 𝑇𝑘 )2 𝑅2 = 1 −

∑𝑛 𝑘=1(𝑆𝐾 −𝑇𝑘 ) 𝑛

∑𝑘=1 𝑆𝑘 ∑𝑛 ] 𝑘=1[𝑆𝑘 −

(4)

(5)

𝑛

𝑀𝐴𝐸 =

100 𝑛 𝑆 −𝑇 ∑𝑘=1 | 𝑘 𝑘 | 𝑛 𝑇𝑘

(6)

where Sk and Tk are the predicted and experimental values for kth pattern of the network, respectively and n is the total number of training patterns. Finally, according to the lowest error on training or cross-validation steps, the optimal model among the kinds of ANNs can be selected.

THE APPLICATION OF ANN TO THE DRYING OF FOODS With the rapid development and advancement of computer technologies along with related software, artificial intelligence technology is being used to solve many problems for systems’ modeling and processes prediction. Among the various topics of artificial intelligence, ANNs and ANFIS are important and frequently applied tools. Today, the use of these networks for

Application of Artificial Neural Networks …

23

simulation and modeling of various processes in science and food industry is developing and progressing. There are various applications for ANNs and ANFIS modeling in drying technology, which are summarized below.

Application of ANNs to Predict the Drying Properties of Agricultural Products Experimental data on moisture content, moisture ratio and hydration ratio in the process of drying green chickpea with hot air dryer were used to make predictions with artificial neural networks. In this experiment, air temperature and drying time were used as inputs for ANNs. The best topologies for predicting moisture content, moisture ratio and hydration ratio were 2-12-1, 2-8-1 and 2-10-1, respectively. The values of the regression coefficient (R2) obtained were 0.8576, 0.9974 and 0.7875, respectively (Kumar et al., 2020). Chasiotis et al. (2020) used ANNs to predict the moisture content of fruit in a hot air dryer. They used inlet air temperature, airflow rate and drying time as model inputs. The structure of the ANN in this study had two hidden layers and the number of neurons varied from 50 to 100 neurons. Their results showed that a network with a topology of 3-90-90-1 and a sigmoidsoftplus activation function with a value of R2 = 0.991 and a value of RMSE1 = 0.06 should be selected as the best network. The drying ratio of granulated grapes was predicted using ANN and the accuracy of this prediction has been compared with linear and nonlinear regression models (Çakmak and Yıldız, 2011). In the designed network, three input variables were used: moisture content, input air temperature and hot air flow ratio. Using trial and error to determine the number of neurons in the hidden layer, 10 neurons were finally selected. The training function used in this study was a hyperbolic sigmoid tangent with the LevenbergMarquardt learning algorithm. 40% of the test data were applied equally for testing and evaluation. Finally, by determining the statistical parameters, 1

RMSE = Root Mean Sqare Error.

24

Raquel Guiné, Iman Golpour, Maria João Barroca et al.

including R2, RMSE and MAE2, the artificial neural network model designed showed accurate results. Dash et al., (2020) utilized three inputs of microwave power, vacuum pressure and drying time in artificial neural networks modelling a microwave vacuum dryer to predict the moisture ratio of Bael Pulp (Aegle marmelos L.). Their results showed that using one hidden layer in ANN could have better results for predicting the output parameters. Therefore, the topology of 3-6-1, the Tansig activation function with the LM learning algorithm and the value of R2 = 0.99 showed the best performance. Taghinezhad et al. (2020b), applied ANN to predict the moisture ratio of semi-boiling paddy for three types of drying methods: hybrid infraredconvective (IRC), microwave (MIC) and infrared-convective microwave (IRCM). The variables used for IRC were three inlets of air temperature, infrared power and drying time, and for MIC were two inputs of microwave power and drying time and for IRCM were four inputs of inlet air temperature, microwave power, infrared power and drying time. Their results showed that in all three types of dryers, the structure had the best performance with two hidden layers, so that the best topology for IRC, IRCM and MIC, respectively 3-8-8-1, 2-10-10-1 and 4-10-10-1 so that the value of R2 for all dryers was above 0.99. In another study Kırbaş et al. (2019), the use of ANN in examining and estimating the drying time of pomelo fruit in different dryers (freeze drying, forced convection and microwave drying) was evaluated in which the type of dryer, sample thickness and moisture ratio were considered as the input parameters. The authors also used the feed forward network and the tanh and sigmoid activation functions. The results showed that this network can predict the drying time of pomelo with the highest R2 = 0.9996 and the lowest value of RMSE = 4.215 (Kırbaş et al., 2019).

2

MAE = Mean Absolute Error.

Application of Artificial Neural Networks …

25

Silva et al., (2015) used eight mathematical models and ANN to predict the ratio of Brazilian pepper moisture in a hot air dryer. They showed that ANN have better performance in predicting the moisture ratio of Brazilian peppers, given the higher R2 levels than the other mathematical models tested. In another study, Zare et al. (2015) accomplished the drying process of paddy in an infrared-hot air dryer. They investigated the effect of experimental levels (three levels of infrared power of 2000, 4000 and 6000 W/m2, three air temperatures of 30, 40 and 50 °C, and three air velocities of 0.1, 0.15 and 0.2 m /s) on the drying of paddy. They predicted and modeled the moisture content of paddy using an ANN method. Drying time, air velocity, inlet air temperature and infrared power were considered as the most effective input factors, and paddy moisture content was considered as the output of ANN. They found a network of ten and fourteen neurons in the first and second hidden layers, respectively, using the tansig activation function to predict paddy moisture content. Omari et al. (2018) utilized an ANN method to forecast the moisture content of mushroom in a combined infrared-hot air dryer. Independent factors included drying temperature (23, 50 and 70 °C), microwave power (1.5, 2 and 2.5 W/g), and timing of moisture content measurement as input parameters and moisture content as output. The neural network modelling was considered to predict the moisture content. The results showed that the best learning algorithm for this network was the Levenberg-Marquardt algorithm. Also, the best structure in terms of the lowest error was the 3-76-1 structure with the threshold function of sigmoid logarithm, in which the value of the MSE reached 0.2961. Kaveh et al. (2018a) used ANNs to predict drying of terebinth fruit using an infrared microwave hot air dryer. In this experiment, the kinetic drying of terebinth fruit was done in different conditions. Drying tests were performed at three temperature levels of 40, 55, 70°C, microwave power at three levels of 270, 450 and 630 W, as well as three infrared powers of 500, 1000 and 1500 W. To estimate the moisture ratio, two trained networks, namely Feed forward neural network and Cascade forward neural network,

26

Raquel Guiné, Iman Golpour, Maria João Barroca et al.

were used. The results showed that ANN with a coefficient of determination of 0.9999 predicted the moisture ratio. Kaveh et al. (2018b) examined the moisture ratio, drying rate, effective moisture diffusivity and specific energy consumption of pistachios, cantaloupes and squash using an ANN. In this study, feed forward neural network, and cascade forward neural network were used to predict the moisture ratio, drying rate, effective moisture diffusivity and specific energy consumption of pistachio, cantaloupe and squash in a fluidized bed dryer. Inlet air temperature, air velocity, drying time and product type were considered as input parameters. According to the results, the best prediction, among other networks, was the network structure with three neurons in the first hidden layer, three neurons in the second hidden layer and two neurons in the output layer, with the lowest value of MSE = 0.001414 and the highest values of R2 = 0.9677 and R2 = 0.9716, for specific energy consumption and effective moisture distribution coefficient, respectively. The application of ANN modeling in the freeze drying process in the case of strawberry has been investigated (Menlik et al., 2010), since this drying method is affected by various parameters, such as drying time, pressure, thickness and temperature of the samples, the temperature of the chamber and the relative humidity. Describing the behavior of drying and determining parameters such as moisture content and moisture ratio was very complex in this process. The authors of the study used the feed forward neural network process and the Levenberg-Marquardt algorithm in network training, and finally the 6-5-2 structure was selected as the most appropriate structure. The value of the coefficient of determination in the results of modeling with the created network was 0.999. Moisture content prediction using artificial neural networking method for three products (Hazelnut, bean and chickpea) was performed in a fluid bed dryer (Topuz, 2010). Drying time and air temperature were used as network inlets. Network predictions have a MRE of 3.92 and a MAE of 0.033.

Application of Artificial Neural Networks …

27

Fathi et al. (2011a) used an ANN to predict the kinetics of mass transfer and color change of kiwi slices that have been dried by an osmotic method. A multilayer perceptron network was created to predict the values of water loss, solids gain and color changes, and the parameters of sucrose solution concentration, temperature and drying time were considered as network input factors. At the end, it was found that the ANN with 16 neurons in the middle layer had the best performance fitting the experimental data, and the results showed that the artificial neural network can be considered as a potential method to predict the kinetic estimation of mass transfer and color changes. The value of the coefficients of explanation for water loss, solids gain and color changes were 0.92, 0.994 and 0.88, respectively. Examination of grape drying quality indicators in hot air flow dryers using neural networks showed that the feed forward neural network with 36-3 structure and learning algorithm (LM) with logarithmic sigmoid threshold function is able to predict drying time and qualitative parameters of the final product with very high determination coefficients (Khazayi, 2007). In another study, the prediction of moisture content, shrinkage and rehydration ratio of kiwifruit in a combined hot air-osmotic dryer was performed using the ANN method. In this study, four parameters (air temperature, air flow rate, osmotic temperature and drying time) were used as inputs. The results showed that a network with six neurons in the hidden layer (4-6-3 topology) was selected as the best structure. Also, the R2 values for predicting moisture content, shrinkage and rehydration ratio were 0.94, 0.93 and 0.96, respectively (Fathi et al., 2011b). In a study by Zarein and Jaliliantabar (2014), the drying of white mulberry in a microwave oven was modeled with the help of neural networks, and it was found that neural networks are able to predict the drying process of white mulberry with great accuracy and precision. In this study, the variables drying time and microwave power were considered as the network inputs, and the exponential transfer function of the BFGS algorithm was used with 8 neurons in the hidden layer in the network structure.

28

Raquel Guiné, Iman Golpour, Maria João Barroca et al.

Fazaeli et al. (2013) applied ANNs to estimate the physical and chemical properties of sprayed water drying for black and white mulberry with the help of the multi-layered perceptron neural network along with the feed forward neural network algorithm and a 6-14-5 topology. The results showed that ANNs were able to effectively predict the drying of sprayed black and white mulberry. Karakaplan et al. (2019) proposed a network with two inputs (inlet air temperature and drying time) to predict the drying moisture ratio of Mentha spicata L. in a hot air dryer. In designing this structure, they used the feed forward network, logarithmic sigmoid transfer function and a linear transfer function, and the LM algorithm. Their results showed that this network with a defined structure can predict the moisture ratio of Mentha spicata L. with R2 = 1 and RMSE = 0.00059. Mathematical modeling and neural network drying of thin layer of peach slices was carried out by Yazdani et al. (2013). First, the peach slices were dried at different temperatures (40, 50, 60, 70, 80 °C) and air velocities (1, 1.5 and 2 m/s). Multilayer perceptron network was used to model the drying kinetics. Based on the results, a network with a structure of 3-6-4-1 had the best performance in modeling peach drying with hot air, with a determination coefficient of R2 = 0.99996. Tavakolipour et al. (2014) used fuzzy logic and artificial neural networks to predict the moisture ratio of zucchini using hot air dryers. The experiments were performed at three temperature levels (60, 70 and 80 °C) and three sample thicknesses (3, 5 and 7 mm). Three input parameters were used to design the ANN, including inlet air temperature, sample thickness, and drying time. The results showed that for predicting the moisture ratio of zucchini, artificial neural networks with 3-20-20-1 topology and logarithmic sigmoid transfer function and determination coefficient of 0.998 had performed better than fuzzy logic method with determination coefficient of 0.919. In another study, it was used a network with two hidden layers to predict the drying properties of tea (drying time, total phenol content and flavonoid

Application of Artificial Neural Networks …

29

content) in a fluidized bed dryer. In this study, the inlet air temperature and air flow rate were selected as inputs. The results showed values of R2 of 0.988, 0.976 and 0.983, respectively, for predicting drying time, total phenol content and flavonoid content (Kalathingal et al., 2020). Jafari et al. (2016) used ANNs, fuzzy logic, and mathematical models to predict the moisture ratio of onion using a fluidized bed dryer. In this study, experiments were performed at three temperature levels (40, 50 and 60 °C) and two velocity levels (2 and 3 m/s) that were considered as the inputs of artificial neural networks. The results showed that ANNs predicted the onion moisture ratio with 2-5-1 topology and determination coefficient of 0.9995, fuzzy logic with determination coefficient of 0.9979 and Newton model with determination coefficient of 0.999. Kerdpiboon et al. (2006) using ANNs, predicted the rate of shrinkage and rehydration ratio of dried carrots by various methods (freezing, hot air, superheated steam, and low pressure) based on their moisture content. The researchers reported that the optimal model in all drying methods has two hidden layers of 4 to 8 neurons in each hidden layer. Drying of turnips was performed in a semi-industrial continuous dryer with three levels of inlet air (1, 1.5 and 2 m/s), three levels of inlet air temperature (45, 60 and 75°C) and three levels of belt speed (2.5, 6.5 and 10.5 mm/s). In this study, an ANN with four input parameters (distance traveled, air velocity, inlet air temperature, belt speed) and two hidden layers was designed to predict the moisture ratio and drying rate. The results showed that it was selected as the best structure for predicting the moisture ratio and network drying rate a model with LM algorithm and TanssigPurlin-Logsig threshold function with 10 neurons in the first and second layers. Also, the R2 values for moisture ratio and drying rate were 0.9990 and 0.9619, respectively (Kaveh and Chayjan, 2017). Movagharnejad and Nikzad (2007) modeled the tomato drying process in a shelf dryer by ANNs with three inputs of heating power, air flow rate and drying time, and an output of moisture ratio of dried tomato. The results showed that the lowest error was obtained by considering one hidden layer and 4 neurons. Comparison of ANN models with experimental models showed that ANNs predict tomato drying behavior more accurately than

30

Raquel Guiné, Iman Golpour, Maria João Barroca et al.

experimental models. The researchers also stated that ANN models are able to describe the characteristics of drying in a wide range of different test conditions, while experimental models are only able to describe these characteristics on limited conditions. Zenoozian et al. (2007) used ANNs to predict the color change and the percentage of shrinkage in pumpkin dried by the combined osmotic-hot air method. The researchers showed that ANN models with 1 or 2 hidden layers and 10 to 20 neurons in each hidden layer have the lowest percentage of the MSE regarding the prediction of the above factors. Golpour et al. (2015) also used an ANN method to estimate moisture content using as model inputs the air temperature, air velocity and color parameters. The results showed that the optimized ANN topology was found as 5-7-1 with Logsig transfer function in hidden layer and Tansig in output layer so that the values of the MSE and R2, of the optimized ANN were 0.001 and 0.9630, respectively. In a study of Taghinezhad et al. (2020a) the drying process of quince fruit was done in a microwave‐convective dryer. The experiments were accomplished at microwave power levels of 100, 200, and 300 W, air temperatures of 40, 55, and 70 °C, and air velocities of 0.5, 1, and 1.5 m/s. An artificial intelligence method namely ANN was used to forecast the parameters effective diffusivity (Deff) and specific energy consumption (SEC). The best result to forecast the Deff of quince fruit is corresponding to the topology of 3–8–81 and Tan-Tan-Tan transfer function with LM algorithm and with MSE = 8.12 ×10−23 and R2 = 0.9707. The results showed that the best structure for prediction of the SEC of quince fruit is relative to 3–15–15–1 topology and Tan-Log-Tan transfer function with LM algorithm, for which MSE = 0.3211, MAE = 0.1018 and R2 = 0.9794. Other researchers have used artificial neural networks to predict different properties of agricultural products, which are summarized in Table 2.

Microwavehot air

Ultrasound pre-treatment with Clevenger Fluidized-bed drier Mushroom

3–6–7–1

Moisture content

Drying temperature Drying air velocity Drying time Drying temperature Microwave power Drying time

Paddy

3–2–2–1

3-7-1

Extraction yield

Moisture ratio

2-4-1

Moisture ratio

Drying temperature Drying time Sonication time Extraction time Extraction power

Chicken breast Tarragon

3-3-1

4-25-1

Moisture content Shrinkage

Carrot

Microwave Vacuum Infraredconvection with vacuum pretreatment Convective

3-5-1

Rehydration indices

Air temperature Air velocity Rehydration temperature NMR signals Infrared power Density Pressure

Apple

Convective

Topology

Output variables

Input variables

Quince

Product

Dryer

0.9914

0.9955

0.99

0.9992

0.9963

0.9964

0.9721

R2

Tansig and Purlin Tansig and Purlin

Threshold function Tansig- Purlin

Omari et al. (2018)

Chokphoemphun and Chokphoemphun (2018)

Bahmani et al. (2018)

Kumar et al. (2019)

Kalejahi and Asefi (2019)

Sun et al. (2019)

Górnicki et al. (2019)

Reference

Table 2. ANN method used for prediction of drying properties for agricultural products

1

Rice

Kiwifruit

Celeriac

Fig

Ginkgo Biloba Seeds

BAU-STR1 dryer

InfraredConvection

Vaccum

Pulsed vacuum osmotic dehydration

Microwave

0.99, 0.998

0.9999

Energy consumption Moisture content

Moisture content

0.9999

0.9834

Moisture ratio

Color

3-6-9-1

0.98 to 0.99

Final moisture content

R2

Drying time Air temperature, Ambient air temperature Relative humidity Drying time Product weight Product top surface temperature Air temperature Relative humidity Drying time Air temperature Pressure Air temperature Wind velocity Air humidity Solar radiation Drying time Microwave power Drying time

Topology

Output variables

Input variables

Threshold function

Bai et al. (2018)

Şahin and Ozturk (2018)

Beigi and Ahmadi (2019)

Özdemir et al. (2016)

Alam et al. (2018)

Reference

The BAU-STR dryer is an economical hot air circulated dryer was developed at Bangladesh Agricultural University and consists of two cylinders (inner bin, outer bin), blower, hot air pipe, burner, bamboo, polythin sheets and rope.

Product

Dryer

Table 2. (Continued)

Product Shelled corn

Kiwi

Kiwi Aloe Vera

Kiwi

Apple

Dryer

Air-borne ultrasound on fluidized bed

Hot airinfrared

Hot air

Microwave

Osmotic dehydration

Combined heat and power

2-13-13-1 3-5-1

3-19-2

4-35-25-1

Moisture ratio

Solids gain Water loss

Moisture ratio Drying rate

Drying time Infrared lamps Air velocity Drying time Airflow temperature Microwave power Sample quantity Dehydration time Concentration of osmotic solution Osmotic solution temperature Immersion time Air temperature Sample thickness Drying time Moisture ratio Drying Rate 4-5-13-3

3-14-15-1

Drying time Moisture content Toughness Ultimate compressive strength Color Shrinkage Moisture ratio Moisture ratio

Drying air temperature Frequency Ultrasound power density

Topology

Output variables

Input variables

0.9974

0.986, 0.989

0.999

0.997

0.9998

0.97

R2

Log/ Tan/ Pure

Logarithmsigmoid

Trainlm

Logsig

Threshold function Tansig-logsig

Samadi et al. (2013)

Jabrayili et al. (2016)

Mahjoorian et al. (2017) Das et al. (2016)

Nadian et al. (2017)

Abdoli et al. (2018)

Reference

Product Sour cherry

Thyme leaves Sour cherry

Dryer

Microwave– vacuum

Microwave

InfraredConvection

0.9944, 0.9905

3-2-3-1, 3-3-3-1

0.9996

Effective moisture diffusivity Specific energy consumption

3-10-15-2

R2

0.9999

Moisture ratio Drying rate

Absolute pressure Drying time Microwave power Drying time Microwave power Sample amount Infrared power Air Temperature Air velocity

Topology

Moisture ratio

Output variables

Input variables

Table 2. (Continued)

Tansig, Tansig-logsigtansig

Threshold function Log/ Tan/ Tan

Chayjan et al. (2014)

Sarimeseli et al. (2014)

Motavali et al. (2013)

Reference

Application of Artificial Neural Networks …

35

Application of the ANFIS Model to Predict the Drying Properties of Agricultural Products Kaveh et al. (2018c) accomplished a study on the thermal properties (moisture ratio, drying rate, effective moisture diffusivity and specific energy consumption) of three products (potato, garlic and cantaloupe) in a hot air dryer. They used ANFIS and ANNs to predict these properties. The inputs to the ANFIS and ANN models were the inlet air temperature, the air velocity, the drying time, and the product type. The results showed that the ANFIS model performed better than ANNs. Also, the best membership function for each input of the ANFIS model for all 4 parameters was Gaussian (gaussmf). R2 values for moisture ratio, drying rate, effective moisture diffusivity and specific energy consumption were 0.9900, 0.9917, 0.9974 and 0.9901, respectively. The study conducted by Ojediran et al. (2020) to dry yam in a hot air dryer, was used with the ANFIS model to predict its moisture ratio. Air temperature, air flow rate, sample thickness and drying time were considered as the input parameters of the ANFIS model. They showed that the best structure for the ANFIS model with the gbell membership function and R2 = 0.9822 and RMSE = 0.0170 could predict the yam moisture ratio. In another study, Ziaforoughi et al. (2016) conducted a study to predict the moisture content of quince fruit drying in an infrared dryer using mathematical models and ANFIS. The results showed that the ANFIS model had better performance for prediction of the moisture content of quince fruit with a determination coefficient of 0.9998. Jahanbakhshi et al. (n.d.) used two models ANFIS and ANN to predict the moisture ratio of pistachio drying using a microwave dryer with ultrasonic pre-treatment. For both models, 3 parameters of microwave power, ultrasonic time and drying time were used as inputs. According to the obtained results, the ANFIS model had a higher ability to predict the moisture ratio of pistachio than ANNs due to the highest R2 value and the lowest RMSE value. Also, the structure of the ANFIS model was selected 333 according to the type of input membership function (Genfis1), the

36

Raquel Guiné, Iman Golpour, Maria João Barroca et al.

type of output membership function (Linear), and the type of learning algorithm (Hybrid) and the number of membership functions for each input. Kaveh et al., (2018d) studied the effect of drying temperature and ultrasonic time in the drying process of almond with laboratory hot air dryers and ultrasonic pretreatment. They used two models of ANN and ANFIS to predict the moisture ratio. For both models ANN and ANFIS, the drying temperature, drying time and ultrasonic time were selected as inputs. The values of R2, MSE and MAE related to the modeling of moisture ratio for drying of almond using ANN were 0.9995, 0.0008 and 0.4545, respectively, and by using ANFIS were 0.9998, 0.0003 and 1.91 respectively. The best neural network models for predicting network moisture ratio, learning algorithm type, activation function and topology were obtained: Cascade Feed Back Propagation, Levenberg- Marquardt, Tansig-Logsig-Tansig and 3-15-13-1, respectively. Also, the best ANFIS model was obtained with Gaussmf, Linear, Hybrid, and 3-3-3 according to the type of input membership function, the type of output membership function, and the type of learning algorithm and the number of membership functions for each input.

Application of Different Models to Predict Energy and Exergy Parameters of Agricultural Products in Dryers Liu et al. (2019) have studied the application of ANN to predict energy utilization, energy utilization ratio, exergy loss, and exergy efficiency of mushroom slices in a hot air dryer. The results indicated that values of R 2 for energy utilization, energy utilization ratio, exergy loss, and exergy efficiency were 0.9978, 0.985, 0.994 and 0.998, respectively. Investigation of drying index (moisture content and drying rate) and thermodynamic parameters (energy and exergy efficiency) of banana drying in a hot air flow dryer with the help of combined structure ANNs-RSM, showed that this structure is able to predict drying index and thermodynamic parameters with R2 > 0.96 and RSME < 0.060 (Taheri-Garavand et al., 2018).

Product Potato

Rough rice

Fish

Carrot

Carrot

Dryer Fluidized bed

Convective

Spray drying

Fluidized bed drying

Fluidized bed drying

Temperature Bed depth Cube size Drying time Temperature Bed depth Cube size Drying time

Drying air temperature Aspirator rate (drying air flow rate) Peristaltic pump rate (mass flow rate) Spraying air flow rate

Temperature Flow rate Relative humidity

Input variables Air temperature Air velocity Drying time

Output variables Energy efficiency Energy utilization Energy utilization ratio Exergy loss Exergy efficiency Drying time Specific energy consumption Energy efficiency Drying efficiency Thermal efficiencies Inlet exergy Outlet exergy Lost exergy Destructed exergy Entropy generation Exergy efficiency Improvement potential Energy utilization Energy utilization ratio Exergy loss Exergy efficiency Energy utilization Energy utilization ratio Exergy loss Exergy efficiency 4-30-4

4-21-4

4-20-7

3-6-1, 3-6-4-1, 3-8-7-1, 3-12-1, 3-7-6-1

Topology 3-15-5

0.9451, 0.9703, 0.9611, 0.9599

0.9722, 0.9811, 0.9827, 0.9779

0.9984, 0.9958, 0.9841, 0.9842, 0.9859, 0.9994, 0.9983

0.9946, 0.9994, 0.9964, 0.9910, 0.9912

R2 0.9926, 0.9984, 0.9870, 0.9979, 0.9839

Nazghelichi et al. (2011)

Nazghelichi et al. (2011)

Aghbashlo et al. (2012)

Beigi et al. (2017)

Reference Azadbakht et al. (2017)

Table 3. ANN and ANFIS methods used for prediction of energy and exergy characteristics of different product drying

38

Raquel Guiné, Iman Golpour, Maria João Barroca et al.

ANNs and ANFIS were used by Abbaspour-Gilandeh et al. (2020) to predict the moisture ratio, energy utilization, energy utilization ratio, exergy loss and exergy efficiency of quince slices in a hot air dryer. In this study, the inlet air temperature, air velocity and drying time were considered as input parameters. According to the results, the values of R2 in ANN with two hidden layers to predict the moisture ratio, energy utilization, energy utilization ratio, exergy loss and exergy efficiency were 0.9993, 0.9985, 0.9977, 0.9980 and 0.9970 respectively. The R2 values for the ANFIS model were 0.9997, 0.9989, 0.9988, 0.9986, and 0.9978, respectively. In this study, according to the R2 value, the ANFIS model performed better than neural networks. Azadbakht et al. (2018) examined the application of ANN methods for predicting energy and exergy in thin layer drying orange slices with osmotic pretreatment. The results of their statistical analysis showed that the osmotic time was significant for energy efficiency and exergy efficiency and specific exergy loss at the level of 1%. The highest energy and energy efficiency was observed at 900 W and osmosis time of 90 min. Microwave power was statistically significant for all parameters (energy and exergy), so that with increasing microwave power, energy efficiency and exergy also increased, while specific energy loss and exergy loss decreased. Also, a maximum of R2 was obtained in a network with 4 neurons in the hidden layer, 0.999 for energy efficiency, 0.871 for energy loss and 0.972 exergy efficiency. A number of other studies, by other researchers with ANNs and ANFIS to predict energy and exergy for drying agricultural products, are also shown in Table 3.

APPLIED RECOMMENDATIONS AND CONSIDERATIONS This book chapter has tried to review the principal introductory aspect of artificial neural networks (ANNs), its most important characteristics and its usages in the drying process technology, including advantages and shortcomings. The ANN paradigm can be used as a helpful methodology in drying technology because of its high capability to model, optimize, predict,

Application of Artificial Neural Networks …

39

recognize, monitor and control the various processes of drying. Moreover, it is not necessary a previous knowledge and also given assumptions, unlike traditional statistical and phenomenological approaches, being this intelligent technique a data-driven method to identify and simulate the nonlinear complicated relationships in those processes. In addition to study the nonlinear issues, it also consists in the investigation of nonlinear dynamic systems, self-organizing networks and chaotic neural networks, and it needs to be studied on fundamental aspects of ANNs, including stability, fault tolerance and robustness to use in the modern plants with drying technology for intelligent control of industrial dryers. Another important use of ANNs as a well-defined methodology in the sector of food processing like drying technology is the prediction of properties such as thermal conductivity, porosity and others during various processes of drying. The importance and applicability of these predictions of the properties in physics-based modelling is not well addressed and ANN studies that concentrate on comprehensive modelling of thermal drying processes are limited. Figure 10 reproduces the modelling framework. In addition, innovating and optimizing the transfer function, weights and the learning algorithm of ANNs, can be accomplished for developing the ANN technology along with further convergence, rapidity and generalization capability. Undoubtedly, the use of artificial intelligence (AI) technologies such as ANN, fuzzy logic, expert system or other developed procedures as promising technologies namely hybrid models via combining classical and mathematical models, genetic algorithm, wavelet analysis, grey coefficient, chaos theory and data mining technology can be helpful for drying food process to eliminate some available disadvantages. Accordingly, developed artificial neural network (ANN) procedures can discover their path into an extensive range of industrial drying usages because of large development in combining the neural network algorithms and computer power in the near future.

40

Raquel Guiné, Iman Golpour, Maria João Barroca et al.

Start

•Development of a physics based model for drying •Experimental validation of model •Parametric sensitivity analysis for input parameters •Identifying sensitive, i.e. characteristic input parameters •Generation of various combinations of input parameters •Simulation on physics based model for various food samples •Experimental validation of model •Extraction of temperature and moisture variation for various food samples •Neural network platform

End

•Trained neural network setup

Figure 10. Schematic representation of pathway followed in setting up the deep neural network model.

However, only a few efforts have been made so far to apply ANN method to expand the real-time monitoring procedures and implement the ANN-supported controllers on the facilities of drying process. Simultaneously, the large number of researches mostly employed the AI technology in the modes as offline. According to available challenges in automation systems with help of on AI technology, the development of these systems still faces a drastic testing. Anyway, secret and complexity of business in the world, led to few studies carried out on the usage of artificial intelligent technique in the industrial-scale dryers along with the variation

Application of Artificial Neural Networks …

41

of drying process conditions for the aims of monitoring and controlling system. Therefore, according to these reasons, the ANN approach and its relatives as adequate tools, automatically can be used to predict, monitor, and control the commercial food dryers instead of human operators working in food science in the near future. Moreover, the extrapolation capability of ANN models can be significantly improved by means of their integration with phenomenological methods, namely gray-box models. In the future, it is envisaged that great possibilities in development of hardware and software will be created by the ANN approach to discover real-time applications, but there are some hardware restrictions as the main problems in this type of the applications by ANNs that can be solved in the future. The expanded ANN models for various types of food dryers have to be incorporated into user-sociable software. This causes the designers of dryers and the managers of corresponding plants who are unfamiliar with AI techniques to easily use this software entering requested conditions via the customers and also receiving a rapid and acceptable controlling system along with a reliable confidence.

Figure 11. A comprehensive flowchart in line-monitoring and controlling of drying systems.

42

Raquel Guiné, Iman Golpour, Maria João Barroca et al.

Therefore, with the development of large data and computing systems, the combination of ANN technology with intelligent systems based on realtime control for the types of dryers by application of progressive tools for measurement like electronic tongue and nose, bioelectronics tongue, imaging, spectral, acoustical, spectroscopic procedure, and electrical techniques for pattern tracing and optimization and control of processes, can be one of the attractive subjects for online monitoring and controlling the systems of drying process in future studies. Figure 11 shows a comprehensive flowchart for online monitoring and controlling the drying systems by means of this combination. With respect to recent interest of R&D, it is anticipated that permanent and non-expensive computer software based on ANN method will be accessible in the near future for online monitoring and controlling the commercial dryers. It should be mentioned that the main problem for the successful integration with the intelligent controllers based on ANN method in drying systems is to affirm savings in the costs or time, advancing fundamental explicit ANN models and authenticating attained results to human experts.

ACKNOWLEDGMENTS This work is funded by National Funds through the FCT - Foundation for Science and Technology, I.P., within the scope of the project Refª UIDB/00681/2020. Furthermore, we would like to thank the CERNAS Research Centre and the Polytechnic Institute of Viseu for their support. The authors thank the valuable advice of the experts who revised the present chapter: Professor João Carlos Gonçalves from Agrarian School of Polytechnic Institute of Viseu, Portugal; Professor Rui Costa from Agrarian School of Polytechnic Institute of Coimbra, Portugal.

Application of Artificial Neural Networks …

43

REFERENCES Abbaspour‐Gilandeh, Y., Jahanbakhshi, A., Kaveh, M., 2020. Prediction kinetic, energy and exergy of quince under hot air dryer using ANNs and ANFIS. Food Science & Nutrition 8, 594–611. https://doi.org/ 10.1002/fsn3.1347. Abdoli, B., Zare, D., Jafari, A., Chen, G., 2018. Evaluation of the air-borne ultrasound on fluidized bed drying of shelled corn: Effectiveness, grain quality, and energy consumption. Drying Technology 36, 1749–1766. https://doi.org/10.1080/07373937.2018.1423568. Aghbashlo, M., Hosseinpour, S., Mujumdar, A., 2018. Artificial Neural Network-Based Modeling and Controlling of Drying Systems, In: Intelligent Control in Drying. CRC Press, Boca Raton, USA, pp. 155– 172. https://doi.org/10.1201/9780429443183-9. Aghbashlo, M., Mobli, H., Rafiee, S., Madadlou, A., 2012. The use of artificial neural network to predict exergetic performance of spray drying process: A preliminary study. Computers and Electronics in Agriculture 88, 32–43. https://doi.org/10.1016/j.compag.2012.06.007. Alam, Md.A., Saha, C.K., Alam, Md.M., Ashraf, Md.A., Bala, B.K., Harvey, J., 2018. Neural network modeling of drying of rice in BAUSTR dryer. Heat Mass Transfer 54, 3297–3305. https://doi.org/ 10.1007/s00231-018-2368-5. Al-Jaberi, L.A., 2018. Chapter Three - Artificial Neural Networks. pp. 35– 55. Anderson, J.-O., Westerlund, L., 2014. Improved energy efficiency in sawmill drying system. Applied Energy 113, 891–901. https://doi.org/ 10.1016/j.apenergy.2013.08.041. Argatov, I., 2019. Artificial Neural Networks (ANNs) as a Novel Modeling Technique in Tribology. Front. Mech. Eng. 5, 1–9. https://doi.org/ 10.3389/fmech.2019.00030. Azadbakht, M., Aghili, H., Ziaratban, A., Torshizi, M.V., 2017. Application of artificial neural network method to exergy and energy analyses of fluidized bed dryer for potato cubes. Energy 120, 947–958. https://doi.org/10.1016/j.energy.2016.12.006.

44

Raquel Guiné, Iman Golpour, Maria João Barroca et al.

Azadbakht, M., Torshizi, M.V., Noshad, F., Rokhbin, A., 2018. Application of artificial neural network method for prediction of osmotic pretreatment based on the energy and exergy analyses in microwave drying of orange slices. Energy 165, 836–845. https://doi.org/10. 1016/j.energy.2018.10.017. Bahmani, L., Aboonajmi, M., Arabhosseini, A., Mirsaeedghazi, H., 2018. ANN modeling of extraction kinetics of essential oil from tarragon using ultrasound pre-treatment. Engineering in Agriculture, Environment and Food 11, 25–29. https://doi.org/10.1016/j.eaef.2017.10.003. Bai, J.-W., Xiao, H.-W., Ma, H.-L., Zhou, C.-S., 2018. Artificial Neural Network Modeling of Drying Kinetics and Color Changes of Ginkgo Biloba Seeds during Microwave Drying Process. Journal of Food Quality 2018, 3278595. https://doi.org/10.1155/2018/3278595. Barroca, M.J., Guiné, R.P.F., Calado, A.R.P., Correia, P.M.R., Mendes, M., 2017. Artificial neural network modelling of the chemical composition of carrots submitted to different pre-drying treatments. Food Measure 11, 1815–1826. https://doi.org/10.1007/s11694-017-9563-9. Beigi, M., Ahmadi, I., Beigi, M., Ahmadi, I., 2019. Artificial neural networks modeling of kinetic curves of celeriac (Apium graveolens L.) in vacuum drying. Food Science and Technology 39, 35–40. https://doi.org/10.1590/fst.35717. Beigi, M., Torki-Harchegani, M., Tohidi, M., 2017. Experimental and ANN modeling investigations of energy traits for rough rice drying. Energy 141, 2196–2205. https://doi.org/10.1016/j.energy.2017.12.004. Bhatta, S., Stevanovic Janezic, T., Ratti, C., 2020. Freeze-Drying of PlantBased Foods. Foods 9, 87. https://doi.org/10.3390/foods9010087. Burghardt, F., Garbe, R., 2018. Introduction of Artificial Neural Networks in EMC, in: 2018 IEEE Symposium on Electromagnetic Compatibility, Signal Integrity and Power Integrity (EMC, SI PI). Presented at the 2018 IEEE Symposium on Electromagnetic Compatibility, Signal Integrity and Power Integrity (EMC, SI PI), pp. 165–169. https://doi.org/10. 1109/EMCSI.2018.8495246. Çakmak, G., Yıldız, C., 2011. The prediction of seedy grape drying rate using a neural network method. Computers and Electronics in

Application of Artificial Neural Networks …

45

Agriculture 75, 132–138. https://doi.org/10.1016/j.compag.2010.10. 008. Chandramohan, V.P., 2020. Convective drying of food materials: An overview with fundamental aspect, recent developments, and summary. Heat Transfer 49, 1281–1313. https://doi.org/10.1002/htj.21662. Chandramohan, V.P., Talukdar, P., 2014. Deformation of potato during convective drying. Applied Mechanics & Materials 2728. Chasiotis, V.K., Tzempelikos, D.A., Filios, A.E., Moustris, K.P., 2020. Artificial neural network modelling of moisture content evolution for convective drying of cylindrical quince slices. Computers and Electronics in Agriculture 172, 105074. https://doi.org/10.1016/j. compag.2019.105074. Chayjan, R.A., Kaveh, M., Khayati, S., 2014. Modeling some drying characteristics of sour cherry (Prunus cerasus L.) under infrared radiation using mathematical models and artificial neural networks. Agricultural Engineering International: CIGR Journal 16, 265–279. Chokphoemphun, Susama, Chokphoemphun, Suriya, 2018. Moisture content prediction of paddy drying in a fluidized-bed drier with a vortex flow generator using an artificial neural network. Applied Thermal Engineering 145, 630–636. https://doi.org/10.1016/j.applthermaleng. 2018.09.087. Ciurzyńska, A., Kowalska, H., Czajkowska, K., Lenart, A., 2016. Osmotic dehydration in production of sustainable and healthy food. Trends in Food Science & Technology 50, 186–192. https://doi.org/10.1016/j. tifs.2016.01.017. Das, C., Das, A., Kumar Golder, A., 2016. Optimality in MicrowaveAssisted Drying of Aloe Vera (Aloe barbadensis Miller) Gel using Response Surface Methodology and Artificial Neural Network Modeling. J. Inst. Eng. India Ser. E 97, 143–149. https://doi.org/ 10.1007/s40034-016-0083-7. Dash, K.K., Chakraborty, S., Singh, Y.R., 2020. Modeling of Microwave Vacuum Drying Kinetics of Bael (Aegle marmelos L.) Pulp by Using Artificial Neural Network. J. Inst. Eng. India Ser. A. https://doi.org/ 10.1007/s40030-020-00431-x.

46

Raquel Guiné, Iman Golpour, Maria João Barroca et al.

El-Sebaii, A.A., Shalaby, S.M., 2013. Experimental investigation of an indirect-mode forced convection solar dryer for drying thymus and mint. Energy Conversion and Management 74, 109–116. https://doi.org/ 10.1016/j.enconman.2013.05.006. Fathi, M., Mohebbi, M., Razavi, S.M.A., 2011a. Application of Image Analysis and Artificial Neural Network to Predict Mass Transfer Kinetics and Color Changes of Osmotically Dehydrated Kiwifruit. Food Bioprocess Technol 4, 1357–1366. https://doi.org/10.1007/s11947-0090222-y. Fathi, M., Mohebbi, M., Razavi, S.M.A., 2011b. Effect of Osmotic Dehydration and Air Drying on Physicochemical Properties of Dried Kiwifruit and Modeling of Dehydration Process Using Neural Network and Genetic Algorithm. Food Bioprocess Technol 4, 1519–1526. https://doi.org/10.1007/s11947-010-0452-z. Fazaeli, M., Emam-Djomeh, Z., Omid, M., Kalbasi-Ashtari, A., 2013. Prediction of the Physicochemical Properties of Spray-Dried Black Mulberry (Morus nigra) Juice using Artificial Neural Networks. Food Bioprocess Technol 6, 585–590. https://doi.org/10.1007/s11947-0110648-x. Gandomi, A.H., Roke, D.A., 2015. Assessment of artificial neural network and genetic programming as predictive tools. Advances in Engineering Software 88, 63–72. https://doi.org/10.1016/j.advengsoft.2015.05.007. Golpour, I., Chayjan, R., Amiri Parian, J., Khazaei, J., 2015. Prediction of Paddy Moisture Content during Thin Layer Drying Using Machine Vision and Artificial Neural Networks. Journal of Agricultural Science and Technology 17, 287–298. Górnicki, K., Kaleta, A., Trajer, J., 2019. Modelling of dried apple rehydration indices using ANN. Int. Agrophys. 33, 285–296. https://doi.org/10.31545/intagr/110796. Graupe, D., 2013. Principles of Artificial Neural Networks, 3rd Ed. ed, Advanced Series in Circuits and Systems. University of Illinois, Chicago, USA. Guiné, R., Mendes, M., Gonçalves, F., 2019. Optimization of bioactive compound’s extraction conditions from beetroot by means of artificial

Application of Artificial Neural Networks …

47

neural networks (ANN). Agricultural Engineering International : The CIGR e-journal 21, 216–223. Guiné, R.P.F., 2019. The Use of Artificial Neural Networks (ANN) in Food Process Engineering. International Journal of Food Engineering 5, 15– 21. Guiné, R.P.F., Barroca, M.J., Silva, V., 2013. Mass transfer properties of pears for different drying methods. International Journal of Food Properties 16, 251–262. https://doi.org/10.1080/10942912.2011. 551864. Guine, R.P.F., Brito, M.F.S., Ribeiro, J.R.P., 2017. Evaluation of Mass Transfer Properties in Convective Drying of Kiwi and Eggplant. International Journal of Food Engineering 13. https://doi.org/10.1515/ ijfe-2016-0257. Guiné, R.P.F., Fontes, L., Lima, M.J.R., 2019. Drying kinetics and mass transfer properties in the drying of thistle flower. Brazilian Journal of Food Technology 22, 1–13. https://doi.org/10.1590/1981-6723.05119. Haseley, P., Oetjen, G.-W., 2018. Freeze-Drying. Wiley-VCH, Weinheim, Germany. Heddam, S., Sanikhani, H., Kisi, O., 2019. Application of artificial intelligence to estimate phycocyanin pigment concentration using water quality data: a comparative study. Appl Water Sci 9, 164. https://doi.org/ 10.1007/s13201-019-1044-3. Huang, L., Zhang, M., 2012. Trends in Development of Dried Vegetable Products as Snacks. Drying Technology 30, 448–461. https://doi.org/ 10.1080/07373937.2011.644648. Jabrayili, S., Farzaneh, V., Zare, Z., Bakhshabadi, H., Babazadeh, Z., Mokhtarian, M., Carvalho, I.S., 2016. Modelling of mass transfer kinetic in osmotic dehydration of kiwifruit. Int. Agrophys. 30, 185–191. https://doi.org/10.1515/intag-2015-0091. Jafari, S.M., Ganje, M., Dehnad, D., Ghanbari, V., 2016. Mathematical, Fuzzy Logic and Artificial Neural Network Modeling Techniques to Predict Drying Kinetics of Onion. Journal of Food Processing and Preservation 40, 329–339. https://doi.org/10.1111/jfpp.12610

48

Raquel Guiné, Iman Golpour, Maria João Barroca et al.

Jahanbakhshi, A., Kaveh, M., Taghinezhad, E., Sharabiani, V.R., n.d. Assessment of kinetics, effective moisture diffusivity, specific energy consumption, shrinkage, and color in the pistachio kernel drying process in microwave drying with ultrasonic pretreatment. Journal of Food Processing and Preservation n/a, e14449. https://doi.org/10.1111/ jfpp.14449. Kalejahi, A.K., Asefi, N., 2019. Influence of vacuum impregnation pretreatment combined with IR drying on quince quality with shrinkage modeling by ANN. Chemical Engineering Communications 206, 1661– 1675. https://doi.org/10.1080/00986445.2019.1570161. Kant, K., Shukla, A., Sharma, A., Kumar, A., Jain, A., 2016. Thermal energy storage based solar drying systems: A review. Innovative Food Science & Emerging Technologies 34, 86–99. https://doi.org/10.1016/j.ifset. 2016.01.007. Karakaplan, N., Goz, E., Tosun, E., Yuceer, M., 2019. Kinetic and artificial neural network modeling techniques to predict the drying kinetics of Mentha spicata L. Journal of Food Processing and Preservation 43, e14142. https://doi.org/10.1111/jfpp.14142. Kaveh, M., Abbaspour-Gilandeh, Y., Chayjan, R.A., Taghinezhad, E., Mohammadigol, R., 2018a. Mass transfer, physical, and mechanical characteristics of terebinth fruit (Pistacia atlantica L.) under convective infrared microwave drying. Heat Mass Transfer 54, 1879–1899. https://doi.org/10.1007/s00231-018-2287-5. Kaveh, M., Chayjan, R.A., 2017. Modeling Thin-Layer Drying of Turnip Slices Under Semi-Industrial Continuous Band Dryer. Journal of Food Processing and Preservation 41, e12778. https://doi.org/10.1111/ jfpp.12778. Kaveh, M., Chayjan, R.A., Khezri, B., 2018b. Modeling Drying Properties of Pistachio Nuts, Squash and Cantaloupe Seeds under Fixed and Fluidized Bed Using Data-Driven Models and Artificial Neural Networks. International Journal of Food Engineering 14. https://doi. org/10.1515/ijfe-2017-0248. Kaveh, M., Jahanbakhshi, A., Abbaspour‐Gilandeh, Y., Taghinezhad, E., Moghimi, M.B.F., 2018c. The effect of ultrasound pre-treatment on

Application of Artificial Neural Networks …

49

quality, drying, and thermodynamic attributes of almond kernel under convective dryer using ANNs and ANFIS network. Journal of Food Process Engineering 41, e12868. https://doi.org/10.1111/jfpe.12868. Kaveh, M., Rasooli Sharabiani, V., Amiri Chayjan, R., Taghinezhad, E., Abbaspour-Gilandeh, Y., Golpour, I., 2018d. ANFIS and ANNs model for prediction of moisture diffusivity and specific energy consumption potato, garlic and cantaloupe drying under convective hot air dryer. Information Processing in Agriculture 5, 372–387. https://doi.org/ 10.1016/j.inpa.2018.05.003. Kerdpiboon, S., Kerr, W.L., Devahastin, S., 2006. Neural network prediction of physical property changes of dried carrot as a function of fractal dimension and moisture content. Food Research International, Physical Properties VI 39, 1110–1118. https://doi.org/10.1016/j. foodres.2006.07.019. Khan, M.R., 2012. Osmotic dehydration technique for fruits preservation-A review 22, 15. Khazayi, N.B., 2007. Using artificial neural network to predict the quality parameters raisins. M.Sc thesis Mechanics of Agricultural Machinery [In farsi]. Faculty of Agriculture, Tarbiat Modarres University, Tehran, Iran. Kırbaş, İ., Tuncer, A.D., Şirin, C., Usta, H., 2019. Modeling and developing a smart interface for various drying methods of pomelo fruit (Citrus maxima) peel using machine learning approaches. Computers and Electronics in Agriculture 165, 104928. https://doi.org/10.1016/j. compag.2019.104928. Kumar, D., Tarafdar, A., Kumar, Y., Badgujar, P.C., 2019. Intelligent modeling and detailed analysis of drying, hydration, thermal, and spectral characteristics for convective drying of chicken breast slices. Journal of Food Process Engineering 42, e13087. https://doi.org/ 10.1111/jfpe.13087. Kumar, M., Sansaniwal, S.K., Khatak, P., 2016. Progress in solar dryers for drying various commodities. Renewable and Sustainable Energy Reviews 55, 346–360.

50

Raquel Guiné, Iman Golpour, Maria João Barroca et al.

Kumar, Y., Singh, L., Sharanagat, V.S., Tarafdar, A., 2020. Artificial neural network (ANNs) and mathematical modelling of hydration of green chickpea. Information Processing in Agriculture. https://doi.org/ 10.1016/j.inpa.2020.04.001. Ladstätter, F., Garrosa, E., 2008. Prediction of Burnout: An Artificial Neural Network Approach. Diplomica Verlag, Hamburg. Li, Z., Raghavan, G.S.V., Orsat, V., 2010. Optimal power control strategies in microwave drying. Journal of Food Engineering 99, 263–268. https://doi.org/10.1016/j.jfoodeng.2010.02.024. Lingayat, A.B., Chandramohan, V.P., Raju, V.R.K., Meda, V., 2020. A review on indirect type solar dryers for agricultural crops – Dryer setup, its performance, energy storage and important highlights. Applied Energy 258, 114005. https://doi.org/10.1016/j.apenergy.2019.114005. Liu, Z.-L., Bai, J.-W., Wang, S.-X., Meng, J.-S., Wang, H., Yu, X.-L., Gao, Z.-J., Xiao, H.-W., 2019. Prediction of energy and exergy of mushroom slices drying in hot air impingement dryer by artificial neural network. Drying Technology 0, 1–12. https://doi.org/10.1080/07373937.2019. 1607873 Mahjoorian, A., Mokhtarian, M., Fayyaz, N., Rahmati, F., Sayyadi, S., Ariaii, P., 2017. Modeling of drying kiwi slices and its sensory evaluation. Food Science & Nutrition 5, 466–473. https://doi.org/10. 1002/fsn3.414. Menlik, T., Özdemir, M.B., Kirmaci, V., 2010. Determination of freezedrying behaviors of apples by artificial neural network. Expert Systems with Applications 37, 7669–7677. https://doi.org/10.1016/j.eswa. 2010.04.075. Michailidis, P.A., Krokida, M.K., 2013. Drying and dehydration processes in food preservation and processing, In: Conventional and Advanced Food Processing Technologies. Wiley-Blackwell, UK. Miguel Landim, A.P., Martins Jacintho Barbosa, M.I., Barbosa Júnior, J.L., 2016. Influence of osmotic dehydration on bioactive compounds, antioxidant capacity, color and texture of fruits and vegetables: a review. Ciência Rural 46, 1714–1722. https://doi.org/10.1590/01038478cr20150534.

Application of Artificial Neural Networks …

51

Mota, C.L., Luciano, C., Dias, A., Barroca, M.J., Guiné, R.P.F., 2010. Convective drying of onion: Kinetics and nutritional evaluation. Food and Bioproducts Processing 88, 115–123. https://doi.org/10.1016/j. fbp.2009.09.004. Motavali, A., Najafi, G.H., Abbasi, S., Minaei, S., Ghaderi, A., 2013. Microwave–vacuum drying of sour cherry: comparison of mathematical models and artificial neural networks. J Food Sci Technol 50, 714–722. https://doi.org/10.1007/s13197-011-0393-1. Movagharnejad, K., Nikzad, M., 2007. Modeling of tomato drying using artificial neural network. Computers and Electronics in Agriculture 59, 78–85. https://doi.org/10.1016/j.compag.2007.05.003. Mujumdar, A.S., 2006. Handbook of Industrial Drying. Taylor & Francis, New York 3rd ed., 1279. Nadian, M.H., Abbaspour-Fard, M.H., Martynenko, A., Golzarian, M.R., 2017. An intelligent integrated control of hybrid hot air-infrared dryer based on fuzzy logic and computer vision system. Computers and Electronics in Agriculture 137, 138–149. https://doi.org/10.1016/j. compag.2017.04.001. Nazghelichi, T., Aghbashlo, M., Kianmehr, M.H., 2011. Optimization of an artificial neural network topology using coupled response surface methodology and genetic algorithm for fluidized bed drying. Computers and Electronics in Agriculture 75, 84–91. https://doi.org/10.1016/ j.compag.2010.09.014. Nguyen, T.K., Mondor, M., Ratti, C., 2018. Shrinkage of cellular food during air drying. Journal of Food Engineering 230, 8–17. https://doi.org/10.1016/j.jfoodeng.2018.02.017. Ojediran, J.O., Okonkwo, C.E., Adeyi, A.J., Adeyi, O., Olaniran, A.F., George, N.E., Olayanju, A.T., 2020. Drying characteristics of yam slices (Dioscorea rotundata) in a convective hot air dryer: application of ANFIS in the prediction of drying kinetics. Heliyon 6, e03555. https://doi.org/10.1016/j.heliyon.2020.e03555. Omari, A., Behroozi‐Khazaei, N., Sharifian, F., 2018. Drying kinetic and artificial neural network modeling of mushroom drying process in

52

Raquel Guiné, Iman Golpour, Maria João Barroca et al.

microwave-hot air dryer. Journal of Food Process Engineering 41, e12849. https://doi.org/10.1111/jfpe.12849. Oyinloye, T.M., Yoon, W.B., 2020. Effect of freeze-drying on quality and grinding process of food produce: a review. Processes 8, 354-. Özdemir, M.B., Aktaş, M., Şevik, S., Khanlari, A., Özkan, M., 2016. Energy consumption and drying kinetics modeling of an infrared system with heat recovery. Presented at the 4th European Conference on Renewable Energy Systems, Istanbul, Turkey. Pandidurai, G., Vennila, P., 2018. Studies on Development of Fruit Powder from Muskmelon (Cucumis Melo L.) by Using Spray Drier. Madras Agricultural Journal 105, 215. Ramya, V., Jain, N.K., 2017. A review on osmotic dehydration of fruits and vegetables: an integrated approach. Journal of food process engineering 40. https://doi.org/10.1111/jfpe.12440. Rath, S., Sengupta, P.P., Singh, A.P., Marik, A.K., Talukdar, P., 2013. Mathematical-artificial neural network hybrid model to predict roll force during hot rolling of steel. Int. J. Comp. Mat. Sci. Eng. 2, 1350004:1– 16. https://doi.org/10.1142/S2047684113500048. Rosenblatt, F., 1958. The perceptron: A probabilistic model for information storage and organization in the brain. Psychological Review 65, 386– 408. https://doi.org/10.1037/h0042519. Şahin, U., Öztürk, H.K., 2018. Comparison between Artificial Neural Network model and mathematical models for drying kinetics of osmotically dehydrated and fresh figs under open sun drying. Journal of Food Process Engineering 41, e12804. https://doi.org/10.1111/jfpe. 12804. Samadi, S.H., Ghobadian, B., Najafi, G., Motevali, A., Faal, S., 2013. Drying of Apple Slices in Combined Heat and Power (CHP) Dryer: Comparison of Mathematical Models and Neural Networks. Chemical Product and Process Modeling 8, 41–52. https://doi.org/10.1515/ cppm-2013-0009. Sarimeseli, A., Coskun, M.A., Yuceer, M., 2014. Modeling Microwave Drying Kinetics of Thyme (Thymus Vulgaris L.) Leaves Using ANN

Application of Artificial Neural Networks …

53

Methodology and Dried Product Quality. Journal of Food Processing and Preservation 38, 558–564. https://doi.org/10.1111/jfpp.12003. Shaker, R., Aljalawi, D., Al-Jaberi, L., 2014. Fabricating a new Rheometer for Concrete. Journal of Engineering Education 20, 1–24. Shanmuganathan, S., 2016. Artificial Neural Network Modelling: An Introduction, in: Shanmuganathan, S., Samarasinghe, S. (Eds.), Artificial Neural Network Modelling, Studies in Computational Intelligence. Springer International Publishing, Switzerland, pp. 1–14. https://doi.org/10.1007/978-3-319-28495-8_1. Shofian, N.M., Hamid, A.A., Osman, A., Saari, N., Anwar, F., Dek, M.S.P., Hairuddin, M.R., 2011. Effect of Freeze-Drying on the Antioxidant Compounds and Antioxidant Activity of Selected Tropical Fruits. International Journal of Molecular Sciences 12, 4678–4692. https://doi.org/10.3390/ijms12074678. Siddiq, M., Uebersax, M.A., 2018. Handbook of vegetables and vegetable processing, 2nd ed. Wiley. Silva, B.G. da, Fileti, A.M.F., Taranto, O.P., 2015. Drying of Brazilian Pepper-Tree Fruits (Schinus terebinthifolius Raddi): Development of Classical Models and Artificial Neural Network Approach. Chemical Engineering Communications 202, 1089–1097. https://doi.org/10.1080/ 00986445.2014.901220. Staub, S., Karaman, E., Kaya, S., Karapınar, H., Güven, E., 2015. Artificial Neural Network and Agility. Procedia - Social and Behavioral Sciences 195, 1477–1485. https://doi.org/10.1016/j.sbspro.2015.06.448. Sun, Q., Zhang, M., Mujumdar, A.S., Yang, P., 2019. Combined LF-NMR and Artificial Intelligence for Continuous Real-Time Monitoring of Carrot in Microwave Vacuum Drying. Food Bioprocess Technol 12, 551–562. https://doi.org/10.1007/s11947-018-2231-1. Suparta, W., Alhasa, K.M., 2016. Modeling of Tropospheric Delays Using ANFIS, SpringerBriefs in Meteorology. Springer International Publishing, Basel, Switzerland. https://doi.org/10.1007/978-3-31928437-8. Taghinezhad, E., Kaveh, M., Jahanbakhshi, A., Golpour, I., 2020a. Use of artificial intelligence for the estimation of effective moisture diffusivity,

54

Raquel Guiné, Iman Golpour, Maria João Barroca et al.

specific energy consumption, color and shrinkage in quince drying. Journal of Food Process Engineering 43, e13358. https://doi.org/10. 1111/jfpe.13358. Taghinezhad, E., Szumny, A., Kaveh, M., Rasooli Sharabiani, V., Kumar, A., Shimizu, N., 2020b. Parboiled Paddy Drying with Different Dryers: Thermodynamic and Quality Properties, Mathematical Modeling Using ANNs Assessment. Foods 9, 86. https://doi.org/10.3390/foods9010086. Taheri-Garavand, A., Karimi, F., Karimi, M., Lotfi, V., Khoobbakht, G., 2018. Hybrid response surface methodology-artificial neural network optimization of drying process of banana slices in a forced convective dryer. Food Sci Technol Int 24, 277–291. https://doi.org/10.1177/ 1082013217747712. Tavakolipour, H., Mokhtarian, M., Kalbasi‐Ashtari, A., 2014. Intelligent Monitoring of Zucchini Drying Process Based on Fuzzy Expert Engine and ANN. Journal of Food Process Engineering 37, 474–481. https://doi.org/10.1111/jfpe.12101. Topuz, A., 2010. Predicting moisture content of agricultural products using artificial neural networks. Advances in Engineering Software, Advances in optimum engineering design 41, 464–470. https://doi.org/10.1016/ j.advengsoft.2009.10.003. Varzakas, T., Tzia, C., 2015. Food engineering handbook. CRC Press. Vega-Carrillo, H.R., Hernández-Dávila, V.M., Manzanares-Acuña, E., Gallego, E., Lorente, A., Iñiguez, M.P., 2007. Artificial neural networks technology for neutron spectrometry and dosimetry. Radiat Prot Dosimetry 126, 408–412. https://doi.org/10.1093/rpd/ncm084. Vega-Mercado, H., Marcela Góngora-Nieto, M., Barbosa-Cánovas, G.V., 2001. Advances in dehydration of foods. Journal of Food Engineering, Drying 49, 271–289. https://doi.org/10.1016/S0260-8774(00)00224-7. Vijayan, S., Arjunan, T.V., Kumar, A., 2016. Mathematical modeling and performance analysis of thin layer drying of bitter gourd in sensible storage based indirect solar dryer. Innovative Food Science & Emerging Technologies 36, 59–67. https://doi.org/10.1016/j.ifset.2016.05.014. Wang, C., Slikker, W., 2017. Neural Cell Biology, 1st Ed. ed. CRC Press, Boca Raton, USA.

Application of Artificial Neural Networks …

55

Wang, Wenjie, Wang, Wenjun, Jung, J., Yang, R., Tang, J., Zhao, Y., 2020. Investigation of hot-air assisted radio frequency (HARF) dielectric heating for improving drying efficiency and ensuring quality of dried hazelnuts (Corylus avellana L.). Food and Bioproducts Processing 120, 179–190. https://doi.org/10.1016/j.fbp.2020.01.006. Wang, Z.L., Finlay, W.H., Peppler, M.S., Sweeney, L.G., 2006. Powder formation by atmospheric spray-freeze-drying. Powder Technology 170, 45–52. https://doi.org/10.1016/j.powtec.2006.08.019. Xie, Yucen, Zhang, Y., Xie, Yongkang, Li, X., Liu, Y., Gao, Z., 2020. Radio frequency treatment accelerates drying rates and improves vigor of corn seeds. Food Chemistry 319, 126597. https://doi.org/10.1016/j. foodchem.2020.126597. Yang, R.-L., Li, Q., Hu, Q.-P., 2020. Physicochemical properties, microstructures, nutritional components, and free amino acids of Pleurotus eryngii as affected by different drying methods. Scientific Reports 10, 1–9. https://doi.org/10.1038/s41598-019-56901-1. Yazdani, M., Borghaee, A.M., Rafiee, S., Minaei, S., Beheshti, B., 2013. Mathematical and neural networks modeling of thin-layer drying of peach (Prunus persica) slices and their comparison. European Journal of Experimental Biology 3, 712–721. Yi, Z., Tan, K.K., 2004. Convergence Analysis of Recurrent Neural Networks, Network Theory and Applications. Springer, USA. https://doi.org/10.1007/978-1-4757-3819-3. Zare, D., Naderi, H., Ranjbaran, M., 2015. Energy and Quality Attributes of Combined Hot-Air/Infrared Drying of Paddy. Drying Technology 33, 570–582. https://doi.org/10.1080/07373937.2014.962143. Zarein, M., Jaliliantabar, F., 2014. ANN Modeling of White Mulberry Drying by Microwave Oven. Advances in Environmental Biology 8, 172–178. Zenoozian, M.S., Devahastin, S., Razavi, M.A., Shahidi, F., Poreza, H.R., 2007. Use of Artificial Neural Network and Image Analysis to Predict Physical Properties of Osmotically Dehydrated Pumpkin. Drying Technology 26, 132–144. https://doi.org/10.1080/07373930701781793.

56

Raquel Guiné, Iman Golpour, Maria João Barroca et al.

Ziaforoughi, A., Yousefi, A.R., Razavi, S.M.A., 2016. A Comparative Modeling Study of Quince Infrared Drying and Evaluation of Quality Parameters. International Journal of Food Engineering 12, 901–910. https://doi.org/10.1515/ijfe-2016-0074. Zielinska, M., Markowski, M., 2010. Air drying characteristics and moisture diffusivity of carrots. Chemical Engineering and Processing: Process Intensification 49, 212–218. https://doi.org/10.1016/j.cep.2009.12.005. Zielinska, M., Ropelewska, E., Xiao, H.-W., Mujumdar, A.S., Law, C.L., 2019. Review of recent applications and research progress in hybrid and combined microwave-assisted drying of food products: Quality properties. Critical Reviews in Food Science and Nutrition 0, 1–53. https://doi.org/10.1080/10408398.2019.1632788.

Reviewed by: Professor João Carlos Gonçalves from Agrarian School of Polytechnic Institute of Viseu, Portugal; Professor Rui Costa from Agrarian School of Polytechnic Institute of Coimbra, Portugal.

In: A Comprehensive Guide … Editor: Steffen Skaar

ISBN: 978-1-53618-466-2 © 2020 Nova Science Publishers, Inc.

Chapter 2

APPLICATION OF ARTIFICIAL NEURAL NETWORKS IN THE FOOD ENGINEERING Ana Jurinjak Tušek, Davor Valinger, Maja Benković, Jasenka Gajdoš Kljusurić and Tamara Jurina* University of Zagreb, Faculty of Food Technology and Biotechnology, Zagreb, Croatia

ABSTRACT Artificial neural networks (ANNs) are of great interest because of their ability to solve problems connected to interpretation of results obtained by various analytical methods. These results sometimes differ from the ordinary form in term of vast number of results for one measurement. Examples of those results include Near Infrared Spectroscopy (NIRs) spectra or results that have to be in a specific interval. ANNs are composed of group of nonlinear regression and discrimination statistical methods and are often used for their ability of visualization and prediction which is based on their learned and trained knowledge. Use of ANNs has been widely studied since they correspond to computational systems that aim to imitate some properties of biological neurons. Basically, the ANN is a * Corresponding Author’s Email: [email protected].

58

Ana Jurinjak Tušek, Davor Valinger, Maja Benković et al. system which corresponds to human brain in term of neurons that are linked by synaptic connections. The neurons are divided into i) incoming; which are stimulated by external environment, ii) internal or hidden neurons and iii) output neurons; which provide communication to the outside system. There are a lot of advantages of ANNs such as: use for nonlinear and non-parametric modeling, stability (with enough data) and high noise tolerance. Due to their characteristics, ANNs have found wide areas of application, from finance and medicine, over geology and physics to food engineering. In this chapter, the application of ANNs in food engineering will be presented. According to available novel literature, ANNs have been used in food engineering for control, monitoring and modeling of industrial food processes. Furthermore, ANNs are used for recognition, detection, classification as well as for the search of patterns, prediction of on-line parameters, image processing and optimization. Why and how ANNs are applied is explained in this chapter using examples from food/beverage matrices.

Keywords: ANNs application, food engineering, prediction, visualization

INTRODUCTION Artificial Neural Networks (ANNs) consist of a group of nonlinear regression and discrimination statistical methods with predictive capacity (Guiné 2019). The target of ANNs is to solve problems of artificial intelligence, by imitating the structure and function of the human nervous system (Figure 1) via computer programs in order to compile informationprocessing systems (Nelson, 2019; Guiné 2019; Zareef et al. 2020). The mechanism of Artificial Neural Networks includes computational model of the human brain, to achieve a machine able to mimic human thinking machine learning (Warwick 2012; Joshi 2017). The human brain and nervous system consist of the basic structural unit called neuron. The brain has about 1011 neurons with 104 connections per neuron. The rhythm of creation of new connections is very slow. In one day, 105 neurons die and zero neurons are born (Milo and Phylips 2015; Wang and Slikker 2017; Guiné 2019). Neuron is a key element due to its communication ability (Martínez-Álvarez et al. 2015). Biological neuron includes four parts: dendrites - chemical receptors that receive signals from other neurons; soma

Application of Artificial Neural Networks …

59

- the cell body of neuron that processes the input signals; axon - chemical emitter that sends out the processed signals to nearby neurons; synapses the junctions that connect neurons and regulate the signal transmittance between neurons (Huang et al. 2007; Ben-Ari and Mondada 2018). The role of synapses is to make memory and learning possible (Ben-Ari and Mondada 2018). According to Gurkaynak et al. (2016), neural networks are computational programs with implemented detection of sophisticated patterns and learning algorithms, allowing creation of predictive models from wide data basis. Since neural networks are inspired with the architecture of human brain, they constitute, together with synapses, the model that can be implemented artificially through special hardware and software programs (Guiné 2019). It can be seen in Figure 1 that the information can be transferred using different ways from a red dot to a blue one. Those ways are marked as wi. This principle is used in the ANNs and the simplified presentation of their function is given in Figure 2.

Figure 1. Human nervous system in the brain as an information-processing system (Nelson, 2019).

60

Ana Jurinjak Tušek, Davor Valinger, Maja Benković et al.

Figure 2. Function of the ANN where the w1, w2, w3 give the strength of the input signals.

ANN is characterized by architecture, activation function and training or learning. Architecture represents the topological structure of how neurons are connected. Basic parameters included in the architecture of ANN are: i) number of layers (Figure 3), ii) number of neurons per layer, iii) connection grade and iv) types of connections between neurons. Based on the number of layers, ANNs can be classified into monolayer or multilayer networks (MLP) (Martínez-Álvarez et al. 2015). Monolayer networks have only one input and one output layer while multilayer networks have hidden layers between the input and output (Rosenblatt 1958; Huang et al. 2007; MartínezÁlvarez et al. 2015). Regarding the types of connections between neurons, neurons in one layer send output information to the next layer and they may (feedback networks) or may not (feedforward networks) receive information back from the next layer. Neurons may or may not be connected with each other in the same layer. Regarding the connection grade: all neurons in a layer are totally connected with the neurons in the next layer (feedforward networks) or with the neurons in the last layer (recurrent networks). There are also partially connected networks in cases without total connection among neurons from different layers (Martínez-Álvarez et al. 2015).

Application of Artificial Neural Networks …

61

Figure 3. Neural network with three inputs and four hidden layers.

When discussing learning paradigm, learning can be defined as a process that consists of modifying the ANN weights, according to the input information or signal. The weights can be defined as connection strengths between neurons (Huang et al. 2007). Furthermore, the weights work on the principle that the node (the body of the neuron) performs two functions: it computes the sum of the weighted input signals and it applies an output function to the sum. Input signals are multiplied by weights before the sum and output functions are applied; thus modeling synapse. The output function is usually nonlinear (Ben-Ari and Mondada 2018). The networks’ learning is finished when the values assigned to the weights remain unaltered (Martínez-Álvarez et al. 2015). ANN can be also characterized regarding the representation of input/output information or signal. Input and output data are analogue which requires activation functions which are also analogue. Their role is to limit the output and introduce non-linearity in the model. The activation function is generally in a non-linear form. The most commonly used activation functions are Binary (Step), Linear (Slope), Sigmoid (Logistic) and Tanh (Hyperbolic tangent) (Guiné 2019). There are some networks that only allow discrete or even binary values as input data. Hybrid ANNs can be also found, in which input data may accept continuous values and output data would provide discrete values or vice versa (MartínezÁlvarez et al. 2015).

62

Ana Jurinjak Tušek, Davor Valinger, Maja Benković et al.

Training, test and validation are stages included in the processing of the network. Training (most commonly) comprises of about 70% of the experimental data, 15% is used for validation and the rest of the data for testing; all selected randomly from the experimental data set (Esfandian et al. 2016; Jurinjak Tušek et al. 2020). The training will start with attributed random initial weights to each of the inputs. Then, a Levenberg-Marquardt backpropagation training algorithm will induce changes in the weights and they will be adapted to the best solution. The training will finish when satisfactory results are accomplished. The next step is moving the results in the forward direction from the hidden layer towards the output layer, which has a linear activation function. The validation, as the second stage, is carried out by adjusting the combination through a number of iterations, so as to generate reasonable parameters for the learning algorithm. In the end, the network is tested to assess the prediction accuracy, by calculating the mean squared error (MSE) (Lai et al. 2016).

1. APPLICATION OF ARTIFICIAL NEURAL NETWORKS (ANNS) IN FOOD TECHNOLOGY AND ENGINEERING There is a growing interest in using artificial neural networks as problem solving algorithms due to the fact that ANNs are ideal for solving highly non-linear problems and dealing with any kind of data. ANNs can perform mapping, clustering, regression, modelling, and multivariate data analysis (Guiné 2019). They have been successfully utilized for modelling and prediction of many different processes in different fields such as agriculture, business, marketing, medicine, transports, energy (Funes et al. 2015). Artificial Neural Networks have been used in almost every aspect of food science over the past two decades (Huang et al. 2007; Dębska and Guzowska-Świder 2011; Čačić et al. 2013). In the following subchapters, the basics of ANNs application in food science will be discussed using examples from food/beverage matrices.

Application of Artificial Neural Networks …

63

Human brain consists of billions of neurons, which process information in the form of electric signals. While external information is received, processed, converted and passed to the next neuron which can choose to either accept the information or to reject it. The acceptance or rejection of the information depends on the strength of the signal. The following example will help make ANN application clearer. A food producer wants to assess whether to approve additional production of a new product with functional properties, so, he wants to predict whether a customer is likely to pay more for it based on the data shown below: Table 1. An example of data from a survey of customer satisfaction with a new (test) product Consumer

Gender Age Monthly (female=F; income (HRK) male=M) 1 F 35 5000 2 M 45 10000 3 F 43 12000 4 F 25 4000 5 M 40 3500 6 M 30 8500 # the weight (wi); $ the prediction, Xj; HRK – Croatian kuna.

Functional food consumer# (yes=1; no=0) 1 0 1 0 1 1

Probability of buying$ 0.7 0.3 0.5 0.4 0.9 0.8

Figure 4. Simplified example of the ANN for the data presented in Table 1.

64

Ana Jurinjak Tušek, Davor Valinger, Maja Benković et al.

For the data presented in Table 1, a simplified ANN architecture used for data analysis is presented in Figure 4. The presented ANN architecture is the “feed-forward network” because all input signals are streaming in one direction – from inputs to yields. The last column (Table 1) is used to train the consumers’ perception and to predict the behavior of consumers which were not interviewed in the survey. However, it is important to emphasize that ANNs are not intended for such a small amount of data as shown in Table 1.

1.1. Virtualization and Visualization Using ANNs By definition, virtualization is the process of creating a software-based (virtual) representation of something: virtual applications, servers, storage and networks. It is the single most effective way to reduce IT expenses while boosting efficiency and agility for all size businesses, and, among them, food engineering as well. Some benefits of virtualization are: (i) greater workload mobility, (ii) increased performance and availability of resources, (iii) automated operations, (iv) reduced capital and operating costs, etc. Wang et al. (2017) proposed a progressed nourishment traceability framework for nourishment quality confirmation and assessment based on the ANNs. They applied it to a system that has achieved effective traceability in the food chain and to ensure the food safety and quality. Furthermore, they also presented how to (i) evaluate the food quality with traceability information as basis in the visualization of the quality management process and (ii) meet the customer demand on food quality grading as well. Visualization based on virtualization is presented in the study of Tripathy and Kumar (2009) where neural network approach was used for potato temperature prediction during solar drying. In their work, the ANNs models were independently developed for each of potato cutting (cylinders or slices). Based on their error analysis, it arose that the neural network with 4 neurons and LOGSIG transfer function tied with TRAINRP back propagation algorithm was the most appropriate ANN configuration in terms of prediction capability of transient food temperature for both potato

Application of Artificial Neural Networks …

65

geometries. There are many similar examples and scientific search engines with the keywords “food” and ANN currently resulting in over 6200 papers, and by addition of the term “visualization,” an extremely large number of studies still remain; over 1200 papers, where just in the year 2020 more than 250 papers were published. One of them is the study which presented the analysis of phthalate plasticizer migration from PVDC packaging materials to food simulants using molecular dynamics simulations and artificial neural network (Wang et al. 2020).

1.2. Food Quality and Characteristics Food characteristics such as type, composition, nutrient contents, processing, etc., are issues that have to be taken into account regarding the healthy diet (Lule and Xia 2005; Zhou et al. 2019). Bioactive compounds or bioactives, present in different biological sources, are important for the development of food additives and are widely used in health treatment (Gökmen 2016; Xu et al. 2017). Phenolic compounds are part of medicinal or aromatic plants, with simple or complex chemical structures, essential for growth and reproduction of plants, responsible for the color, astringency and aroma in several foods (Sharma 2014). These compounds, being antioxidants fight free radicals (Rodrigo and Gil-Becerra 2014), prevent heart diseases (Jiang 2014), neurodegenerative disorders (Hamaguchi et al. 2009), circulatory problems (Medina-Remón et al. 2014), cancer (Fernández-Arroyo et al. 2012), inflammation (Wen et al. 2012), and inhibit lipid oxidation (Maqsood and Benjakul 2010). ANNs have been used in many food technology applications. Marić et al. (2020) investigated the effect of drying methods (conventional drying at T = 50°C and T = 70°C and lyophilisation) on root vegetables (celery, carrot, fennel, purple carrot, parsley and yellow carrot). All vegetable samples were subjected to 13 extraction procedure (12 with organic solvents and one water extraction) prior to and after drying. Physical characteristics such as conductivity, total dissolved solids and color, as well as chemical properties (antioxidant activity, total polyphenolic content, vitamin C, β-carotene

66

Ana Jurinjak Tušek, Davor Valinger, Maja Benković et al.

content) were parameters analysed using ANNs modeling. According to the obtained results, it could be concluded that ANNs can predict colour parameters, vitamin C and β-carotene concentrations in root vegetables after different drying methods and physical-chemical characteristics of the root vegetable extracts prepared after different drying methods, using input parameters such as type of root vegetables, drying method and the extraction procedure. dos Santos et al. (2017) developed an ANN to simultaneously determine 13 bioactive phenolic compounds in guava (Psidium guajava L.). The quantification of phenolic compounds revealed that there were no significant differences for antioxidant activity between the analyzed samples. Principal Component Analysis (PCA) was also performed in order to examine the variability between green and ripe fruits. The ANN model was applied to get more insights regarding cluster separation and the influence of each variable. ANN showed good separation in two clusters: unripe and ripe fruits. Musa et al. (2016) applied ANN for prediction of antioxidant activity of several food products (cinnamon, clove, mung bean, red bean, red rice, brown rice, black rice and tea extract). ANN was trained on a typical set of images of the DPPH radical reacting with different levels of Trolox and was able to determine the DPPH value of cinnamon, clove, mung bean, red bean, red rice, brown rice, black rice and tea extract. Results of ANN modelling were compared to experimental data obtained spectrophotometrically. ANN was in accordance with the experimental spectrophotometric procedure and it could be used to obtain semi-quantitative results of DPPH. In the research paper of Guiné et al. (2015), ANN modelling was used to predict antioxidant activity and total phenolic compounds in bananas subjected to different drying processes. According to the obtained results, they concluded that the evaluated properties can be predicted accurately from the input variables: banana variety, dryness state and type and order of extract. Antioxidant activity and phenolic compounds were greatly affected by drying processes. Yalcin et al. (2012) developed a model for fatty acid composition of vegetable oils (hazelnut, soybean, sunflower, olive, canola, corn, and cotton seed) using ANNs coupled with Adaptive Neuro Fuzzy Inference System

Application of Artificial Neural Networks …

67

(ANFIS) based on rheological measurements. The inputs for ANNs and ANFIS models were oil type, shear rate and shear stress while the outputs were C16:0 and C18:2. ANNs and ANFIS showed satisfactory prediction for the fatty acid composition of the vegetable oils studied. Cimpoiu et al. (2011) used two types of Artificial Neural Networks based applications: one to predict the antioxidant activity and a second one to establish the class of the teas used in the study (green, black or express black tea). Relationship between the total antioxidant activity and the flavonoids, catechins and methyl-xanthines content was determined by the first designed feed-forward ANN. For successful tea classification in various classes (green tea, black tea and express black tea), probabilistic ANN was also performed. Román et al. (2011) tested ANNs in order to predict problems of wine fermentation. About 20,000 data from industrial fermentations of Cabernet Sauvignon and 33 variables were used. Different sub-cases were studied by varying the predictor variables (total sugar, alcohol, glycerol, density, organic acids and nitrogen compounds) and the fermentation time (72, 96 and 256 h). It was possible to predict 100 of normal and problematic fermentations using three predictor variables: sugars, density and alcohol at 72 h. ANNs were capable of obtaining 80% of prediction using only one predictor variable at 72 h.

1.3. Application of Near Infrared Spectroscopy, Coupled with Multivariate Tools and ANNs Modelling, for the Assessment of Food Quality Fast, accurate, and automatic determination of food characteristics is a practical demand in everyday life. For detection of food properties, modern techniques such as electronic noses (Tian et al. 2014) computer vision (Brosnan and Sun 2004), spectroscopy and spectral imaging (Barbin et al. 2014) have been widely used (Zhou et al. 2019). Among them, Near Infrared Spectroscopy (NIRs) has been emerging as a rapid analysis mostly used in the detection of moisture, protein, and fat content of a wide variety of food

68

Ana Jurinjak Tušek, Davor Valinger, Maja Benković et al.

products (Pasquini 2018; Gajdoš Kljusurić et al. 2020). NIRs represents a non-invasive technique, used in agricultural applications due to its rapid data acquisition time, the capability of determining more than one parameter using the same measurement, easy usage and little or no sample preparation (Gutiérrez et al. 2015). The NIR range covers the electromagnetic spectrum between 780 and 2500 nm. Product is irradiated with NIR radiation and the reflected or transmitted radiation is measured. Radiation causes change of spectral characteristics, depending on the chemical composition of the product. Since spectrum of the scanned product will result with a large data set, many data analysis methods are used to extract significant information from the spectra (Yiqun et al. 2007; Pouladzadeh et al. 2012; Yordi et al. 2015; Cheng and Sun, 2017; Granato et al. 2018; Gajdoš Kljusurić et al. 2020). Data analysis of spectra includes multivariate tools such as factor analysis (FA), principal component analysis (PCA), partial least squares regression (PLSR), multiple linear regression (MLR), principal component regression (PCR), etc. (Cortés et al. 2019). Use of different multivariate tools implies qualitative or quantitative analysis: qualitative modelling is used in linear discriminant analysis (LDA) (Baranowski et al. 2012), partial least squares-discriminant analysis (PLS-DA) (Liu et al. 2011), soft independent modelling of class analogy (SIMCA) (Pontes et al. 2006), and support vector machine (SVM) (Chen et al. 2007). Quantitative modelling is applied in partial least square (PLS) regression, principal component regression (PCR), multiple linear regression (MLR), or broadly used artificial neural networks (ANNs) (Kumaravelu and Gopal 2015). In the research paper of Valinger et al. (2018), the aim of the work was to assess the ability of Artificial Neural Networks (ANNs) modelling in Near Infrared Spectroscopy (NIRs) calibration models for prediction of total polyphenolic content (TPC), antioxidant activity (AA) and extraction yield (EY) of the olive leaves aqueous extracts. Extracts were prepared by conventional extraction, microwave-assisted and microwave-ultrasoundassisted extraction. Partial Least Squares (PLS) models were developed from principal component analysis (PCA) factors (scores) of NIR spectra of olive leaves aqueous extracts regarding TPC, AA and EY for each extraction procedure. Based on the PLS models, the best suited PCA scores were

Application of Artificial Neural Networks …

69

chosen as input for ANN based on three output variables. According to the obtained results, ANN can be used for the prediction of TPC, AA and EY of plant extracts based on the NIR spectra. The objectives of the study by Jurina et al. (2018) were to use multiple linear regression (MLR), nonlinear regression (NR), piecewise linear regression (PLR) and artificial neural network (ANN) modelling to analyze the effect of extraction time, extraction temperature and plant species on total dissolved solids (TDS), extraction yield (EY), total polyphenolic content (TPC) and antioxidant activity (AA) of three medicinal plants aqueous extracts. The aqueous extracts of lavender, melissa and mint were prepared at defined time intervals and analyzed for physical and chemical properties. The performances of the proposed models were evaluated based on the determination coefficient (R2) and root mean square error (RMSE). According to the results, ANN has higher prediction capability, compared to MLR, NR and PLR models. Gutiérrez et al. (2015) applied support vector machine (SVM) and artificial neural networks (ANNs) for grapevine varietal classification. Modelling includes site-specific and a global scale. For building the sitespecific model training, the classifier with leaves from 20 varieties was achieved, while for building the global model, leaves from different vineyards, vintages and stages of development were used. Support vector machines and artificial neural networks were employed using the preprocessed spectra as input and the varieties as the classes of the models. SVM and ANNs showed high reliability in the creation of grapevine leaf varietal classification models from in-field NIR spectroscopy. According to the obtained results, NIR spectral range between 1600 to 2400 nm was suitable for in-field grapevine varietal discrimination. Martelo-Vidal and Vázquez (2015) applied Ultraviolet-visible (UVVIS) and Near Infrared Spectroscopy (NIRs), coupled with Artificial Neural Networks (ANNs) to quantify ethanol, glucose, glycerol, tartaric acid, malic acid, acetic acid and lactic acid in aqueous mixtures of compounds from wines. Spectral data were obtained for 152 samples. Different pre-treatments such as Standard Normal Variate (SNV), Savitzky–Golay smoothing and 2nd derived correction were applied to the spectra. ANN models were

70

Ana Jurinjak Tušek, Davor Valinger, Maja Benković et al.

developed using raw and pre-treated data in order to evaluate several spectral wavelength groupings and ANN training conditions. Feasible calibration models were obtained for ethanol, malic acid and tartaric acid. For validation of the process, 120 new samples were measured using the best ANN models. In order to obtain reliable ANN models, good parameter selection is important during training of the ANN. Gori et al. (2012) investigated the potential of Fourier Transform Infrared Spectroscopy (FTIR) coupled with Principal Component Analysis (PCA) and Artificial Neural Networks (ANNs) to differentiate butters obtained from milk creams collected in different seasons (spring, summer or winter) and produced from two feeding regimens (traditional or unified) in the Parmigiano Reggiano cheese area. ANNs predictive ability was 100% concerning the production season. From all samples, Principal Components scores/factors were extracted after PCA analysis and used as an input to develop an ANN to predict the sampling season. The ability of the ANN to predict the dietary regimens was 90.0%, 75.0% and 75.0%, respectively, for samples collected in spring, summer and winter. In the paper of Huang et al. (2012), the quality and freshness of pork meat during bacterial spoiling was determined by Fourier Transform Near Infrared Spectroscopy (FT-NIR), combined with a non-linear algorithm. The spectra of 120 pork meat samples, stored at 4°C, were measured during 11 days of spoiling process. Based on preprocessed NIR spectral data, Synergy interval partial least square (SI-PLS) was applied to select characteristic spectral variables of different components in pork meat. Prior to ANN modeling, the characteristic spectral variables of different pork meat components were determined by principal component analysis (PCA), to extract the top principal components (PCs). These PCs were used as the input of a backpropagation artificial neural network (BP-ANN) model. During development of a prediction model, BP-ANN revealed its better performance compared to the SI-PLS algorithm method. Based on the obtained results, they concluded that FT-NIR spectroscopy coupled with the BP-ANN algorithm has good potential for the determination of parameters important in evaluating quality and freshness of pork meat.

Application of Artificial Neural Networks …

71

In the study of Mutlu et al. (2011), NIR spectroscopy combined with ANN was used to predict the flour quality parameters: protein content, moisture content, Zeleny sedimentation, water absorption, dough development time, dough stability time, degree of dough softening, tenacity (P), P/G extensibility, strength, and baking test. Seventy-nine wheat flour samples were used for chemical analysis. NIR spectra of samples were also recorded and for each quality parameter, individual network was designed, trained and optimum network parameters were determined. Prediction of protein, P, P/G, moisture content, Zeleny sedimentation, and water absorption in particular gave a very good accuracy with determination coefficients (R2) higher than 0.8. Based on the obtained results they concluded that NIR in combination with ANN can successfully be used to predict the quality parameters of wheat flour. Kruzlicova et al. (2009) demonstrated the possibility of using ANNs for classification of white varietal wines. The aim was to classify wines by different variety, producer/location and the year of production. The most successful used algorithms were multilayer perceptron techniques using quick propagation and quasi-Newton propagation. For the investigated criteria, determination coefficients were higher than 93%, indicating that artificial neural networks can be a very useful tool for the classification of wines. By comparing ANNs results with three methods of discriminant analysis (LDA, QDA and KNN), it was revealed that the ANNs produced best results even though LDA, QDA and KNN analyses were very good in particular cases except classification by variety.

1.3.1. Application of ANNs on Experimental Data and Its Efficiency in Output Prediction The following examples show studies where multivariate modelling techniques are employed with the multidimensional data obtained from experiments. Marić and co-workers (2020) investigated, with the use of neural network models, effects of drying on physical and chemical properties of root vegetables. Furthermore, Jurinjak Tušek and co-workers (2020) presented different modeling techniques (multiple linear regression, nonlinear regression, piecewise linear regression, and artificial neural

72

Ana Jurinjak Tušek, Davor Valinger, Maja Benković et al.

network modelling) to predict the output(s): physical and chemical properties of the medicinal plants aqueous extracts. This relation can be formulated as: 𝑂 = 𝑓(𝑋1 , 𝑋2 , 𝑋3 )

(1)

Where: O = output (physical or chemical property) X1 = plant specie (n=9) X2 = t, minutes of plant extraction X3 = T, temperature of plant extraction Linear and non-linear models were presented with equations (e.g., for multiple linear regression model (eq. 2) and for the non-linear regression model: eq. 3). Linear model (LM): 𝑂 = 𝑎0 ∙ 𝑋1 + 𝑎1 ∙ 𝑋2 + 𝑎2 ∙ 𝑋3 + 𝑎3

(2)

Non-linear model (nLM): 𝑏

𝑏

𝑏

𝑂 = 𝑎0 ∙ 𝑋1 1 ∙ 𝑋2 2 ∙ 𝑋3 3

(3)

Five different ANN structures were simulated for the prediction of the physical nad chemical properties. The ANN models were presented with three numbers, where the first number describes number of input variables, second one number of neurons in the hidden layer and third one describes number of output variables. All models had 3 inputs and 4 outputs, having ten, eight or seven hidden layers with Logistic or Tanh hidden activation, and Tanh, Identity or Exponential output activation. The following table shows structure of the ANNs developed in the study by Jurinjak Tušek et al. (2020).

Application of Artificial Neural Networks …

73

Table 2. Example of neural network models settings Model name M1 M2 M3 M4 M5

Input-hidden-output 3-10-4 3-8-4 3-7-4 3-7-4 3-10-4

Hidden activation Logistic Tanh Logistic Tanh Logistic

Output activation Identity Tanh Exponential Identity Exponential

Four parameters were predicted for the medicinal plants aqueous extracts (total dissolved solids (TDS), extraction yield (EY), total phenolic content (TPC) and antioxidant activity (AOX)). In Table 3 the correlation coefficients for all predicted parameters and used models are presented. Table 3. Validation correlation coefficients for the prediction of the observed parameters of the aqueous extracts of medicinal plants Parameter TDS EY TPC AOX

R2 LM 0.284 0.393 0.497 0.420

nLM 0.581 0.680 0.954 0.591

ANN 0.942 0.902 0.950 0.905

This example represents the efficiency of ANNs based on the comparison with linear regression, nonlinear regression used in prediction of total dissolved solids, extraction yield, total phenolic content and antioxidant activity of the aqueous extracts of medicinal plants. The findings show that, in spite of the fact that linear and nonlinear models can be utilized for description and forecast of properties of medicinal plants aqueous extracts (Fernandez et al. 2016; Binello et al. 2017), their productivity and appropriateness is limited. On the other hand, ANN models can be used very effectively (R2>0.9) for concurrent expectation of all four observed properties of medicinal plants aqueous extracts (Jurinjak Tušek et al. 2020).

74

Ana Jurinjak Tušek, Davor Valinger, Maja Benković et al.

CONCLUSION The given overview showes that the ANNs have become an indispensable tool in the food industry, engineering and monitoring of food/feed quality. Modeling in general is not trivial and this procedure integrates knowledge and skills of mathematics, profession (in this case food chemistry and technology), statistics, etc. ANNs, due to its properties have various applications: (i) Visualization and (ii) estimation/prediction. Visualization is an ever-growing field with wide applications ranging from facial affirmation in social media, cancer detainment in pharmaceutical industry to submissive imagery planning for agrarian and food engineering processes. Prediction/Evaluating is used in commerce choices decisions which are broadly required (e.g., bargains, money related assignment between things, capacity utilization, production optimization, etc.). ANNs can provide strong decision support given their ability to prove and highlight connections in the observed set of parameters. Moreover, unlike conventional models, ANN does not impose any restrictions on inputs and residual distributions. Although, as with any form of modeling application, interdisciplinary approach is required when applying ANNs. Also, because of its virtualization, ANN requires computer programs of exceptional performance. However, the outcomes that the application of ANN provides, by simulating different conditions and/or combination of input/output parameters, makes it an indispensable tool for every biotechnical engineer.

REFERENCES Baranowski, T., Islam, N., Baranowski, J., Martin, S., Beltran, A., Dadabhoy, H., Adame, S. H., Watson, K., Thompson, D., Cullen, K. W. and Subar, A., (2012). Comparison of a web-based versus traditional dietary recall among children. Journal of the American Dietetic Association, 112: 527-532.

Application of Artificial Neural Networks …

75

Barbin, D. F., Felicio, A., Sun, D. W., Nixdorf, S. L. and Hirooka, E. Y. (2014). Application of infrared spectral techniques on quality and compositional attributes of coffee: An overview. Food Research International, 61: 23-32. Ben-Ari, M. and Mondada, F. (2018). Neural networks. In: Elements of robotics. Switzerland: Springer. Brosnan, T. and Sun, D. W. (2004). Improving quality inspection of food products by computer vision - A review. Journal of Food Engineering, 61: 3-16. Binello, A., Cravotto, G., Boffa, L., Stevanato, L., Bellumori, M., Innocenti, M. and Mulinacci, N. (2017). Efficient and selective green extraction of polyphenols from lemon balm. Comptes Rendus Chimie 20 : 921–926. Chen, Q., Jiewen Zhao, J., Fang, C. H. and Wang, D. (2007). Feasibility study on identification of green, black and Oolong teas using nearinfrared reflectance spectroscopy based on support vector machine (SVM). Spectrochimica Acta A, 66: 568-574. Cheng, J. H. and Sun, D. W. (2017). Partial least squares regression (PLSR) applied to NIR and HSI spectral data modeling to predict chemical properties of fish muscle. Food Engineering Reviews, 9: 36-49. Cimpoiu, C., Cristea, V. M., Hosu, A., Sandru, M. and Seserman, L. (2011). Antioxidant activity prediction and classification of some teas using artificial neural networks. Food Chemistry, 127: 1323-1328. Cortés, V., Blasco, J., Aleixosc, N., Cubero, S. and Talens, P. (2019). Monitoring strategies for quality control of agricultural products using visible and near-infrared spectroscopy: A review. Trends in Food Science and Technology, 85: 138-148. Čačić, J., Gajdoš Kljusurić, J. and Čačić, D. (2013). Application of regression models and artificial neural networks in agriculture – Prediction in spirit drinks sector. Journal of Food Agriculture & Environment, 11(2): 56-61. Dębska, B. and Guzowska-Świder, B. (2011). Application of artificial neural network in food classification. Analytica Chimica Acta, 705: 283-291. dos Santos, W. N. L., da Silva Sauthier, M. C., dos Santos, A. M. P., Santana, D. de A., Azevedo, R. S. A. and J. da Caldas, C. (2017). Simultaneous

76

Ana Jurinjak Tušek, Davor Valinger, Maja Benković et al.

determination of 13 phenolic bioactive compounds in guava (Psidium guajava L.) by HPLC-PAD with evaluation using PCA and Neural Network Analysis (NNA). Microchemical Journal, 133: 583-592. Esfandian, H., Samadi-Maybodi, A., Parvini, M. and Khoshandam, B. (2016). Development of a novel method for the removal of diazinon pesticide from aqueous solution and modeling by artificial neural networks (ANN). Journal of Industrial and Engineering Chemistry, 35: 295-308. Fernandes, R. P. P., Trindade, M. A., Tonin, F. G., Lima, C. G., Pugine, S. M. P., Munekata, P. E. S., Lorenzo, J. M. and de Melo, M. P. (2016). Evaluation of antioxidant capacity of 13 plant extracts by three different methods: cluster analyses applied for selection of the natural extracts with higher antioxidant capacity to replace synthetic antioxidant in lamb burgers. Journal of Food Science and Technology, 53: 451–460. Fernández-Arroyo, S., Gómez-Martínez, A., Rocamora-Reverte, L., Quirantes-Piné, R., Segura-Carretero, A., Fernández-Gutiérrez, A. and Ferragut, J. A. (2012). Application of nanoLC-ESI-TOF-MS for the metabolomic analysis of phenolic compounds from extra-virgin olive oil in treated colon-cancer cells. Journal of Pharmaceutical and Biomedical Analysis, 63: 128-134. Funes, E., Allouche, Y., Beltrán, G. and Jiménez, A. (2015). A review: Artificial Neural Networks as tool for control food industry process. Journal of Sensor Technology, 5: 28-43. Gajdoš Kljusurić, J., Jurina, T., Valinger, D., Benković, M. and Jurinjak Tušek, A. (2020). NIR spectroscopy and management of bioactive components, antioxidant activity, and macronutrients in fruits. In: Fruit crops. Diagnosis and Management of Nutrient Constraints. Amsterdam: Elsevier. Gori, A., Cevoli, C., Fabbri, A., Caboni, M. F. and Losi, G. (2012). A rapid method to discriminate season of production and feeding regimen of butters based on infrared spectroscopy and artificial neural networks. Journal of Food Engineering, 109: 525-530. Gökmen, V. (2016). Acrylamide in food: Analysis, content and potential health effects. Amsterdam: Amsterdam.

Application of Artificial Neural Networks …

77

Granato, D., Santos, J. S., Escher, G. B., Ferreira, B. L. and Maggio, R. M. (2018). Use of principal component analysis (PCA) and hierarchical cluster analysis (HCA) for multivariate association between bioactive compounds and functional properties in foods: A critical perspective. Trends in Food Science & Technology, 72: 83-90. Guiné, R. P. F., Barroca, M. J., Gonçalves, F. J., Alves, M., Oliveira, S. and Mendes, M. (2015) Artificial neural network modelling of the antioxidant activity and phenolic compounds of bananas submitted to different drying treatments. Food Chemistry, 168: 454-459. Guiné, R. P. F. (2019). The use of Artificial Neural Networks (ANN) in food process engineering. International Journal of Food Engineering, 5: 1521. Gurkaynak, G., Yilmaz, I., and Haksever, G. (2016). Stifling artificial intelligence: Human perils. Computer Law & Security Review, 32: 749758. Gutiérrez, S., Tardaguila, J., Fernández-Novales, J. and Diago, M. P. (2015). Support Vector Machine and Artificial Neural Network Models for the classification of grapevine varieties using a portable NIR spectrophotometer. PLoS ONE, 10: 1-15. Hamaguchi, T., Ono, K., Murase, A. and Yamada, M. (2009). Phenolic compounds prevent alzheimer’s pathology through different effects on the amyloid-b aggregation pathway. The American Journal of Pathology, 175: 2557-2565. Huang, L., Zhao, J., Zhang, Y. and Chen, Q. (2012). Simultaneous multicomponent analysis of pork meat during bacterial spoiling process by FT-NIR evaluated with a non-linear algorithm. Analytical Methods, 4: 3816-3823. Huang, Y., Kangas, L. J. and Rasco, B. A. (2007). Applications of Artificial Neural Networks (ANNs) in food science. Critical Reviews in Food Science and Nutrition, 47: 113-126. Jiang, F. (2014). Pharmacology of Polyphenols in Cardiac Remodeling: Potential Biological Mechanisms. In: Polyphenols in Human Health and Disease. San Diego: Academic Press.

78

Ana Jurinjak Tušek, Davor Valinger, Maja Benković et al.

Joshi, P. (2017). Artificial Intelligence with Python. Birmingham: Mumbai: Packt Publishing. Jurina, T., Cvetković, A.-M., Benković, M., Jurinjak Tušek, A., Valinger, D. and Gajdoš Kljusurić J. (2018). Application of linear, nonlinear and artificial neural network modelling for description of the medical plants extraction process. Proceedings of 3rd Natural resources, green technology and sustainable development. Jurinjak Tušek, A., Jurina, T., Benković, M., Valinger, D., BelščakCvitanović, A. and Gajdoš Kljusurić, J. (2020). Application of multivariate regression and artificial neural network modelling for prediction of physical and chemical properties of medicinal plants aqueous extracts. Journal of Applied Research on Medicinal and Aromatic Plants, 16: 1-8. Kruzlicova, D., Mocak, J., Balla, B., Petka, J., Farkova, M. and Havel, J. (2009). Classification of slovak white wines using artificial neural networks and discriminant techniques. Food Chemistry, 112: 10461052. Kumaravelu, C. and Gopal, A. (2015). A review on the applications of nearinfrared spectrometer and chemometrics for the agro-food processing industries. In: 567 Technological Innovation in ICT for Agriculture and Rural Development (TIAR), IEEE. Lai, K. C., Lim, S. K., Teh, P. C. and Yeap, K. H. (2016). Modeling electrostatic separation process using Artificial Neural Network (ANN). Procedia Computer Science, 91: 372-381. Liu, X., Yanrong Ren, Y., Peng Zhou, P. and Shang, Z. (2011). Prediction of protein 13Cα NMR chemical shifts using a combination scheme of statistical modeling and quantum-mechanical analysis. Journal of Molecular Structure, 995: 163-172. Lule, S. U. and Xia, W. S. (2005). Food phenolics, pros and cons: A review. Food Reviews International, 21: 367-388. Marić, L., Malešić, E., Jurinjak Tušek, A. Benković, M., Valinger, D., Jurina, T. and Gajdoš Kljusurić, J., (2020) Effects of drying on physical and chemical properties of root vegetables: Artificial neural network modelling. Food and Bioproducts Processing, 119: 148-160.

Application of Artificial Neural Networks …

79

Martelo-Vidal, M. J. and Vázquez, M. (2015). Application of artificial neural networks coupled to UV–VIS–NIR spectroscopy for the rapid quantification of wine compounds in aqueous mixtures. CyTA - Journal of Food, 13: 32-39. Martínez-Álvarez, F., Troncoso, A., Asencio-Cortés, G. and Riquelme, J. C. (2015). A survey on data mining techniques applied to electricity-related time series forecasting. Energies, 8: 1-32. Maqsood, S. and Benjakul, S. (2010). Comparative studies of four different phenolic compounds on in vitro antioxidative activity and the preventive effect on lipid oxidation of fish oil emulsion and fish mince. Food Chemistry, 119: 123-132. Medina-Remón, Tresserra-Rimbau, A., Valderas-Martinez, P., Estruch, R. and Lamuela-Raventos, R. M. (2014). Polyphenol Consumption and Blood Pressure. In: Polyphenols in Human Health and Disease. San Diego: Academic Press. Milo, R. and Phylips, R. (2015). Cell Biology by the Numbers, USA: Garland Science. Musa, K. H., Abdullah, A. and Al-Haiqi, A. (2016). Determination of DPPH free radical scavenging activity: Application of artificial neural networks. Food Chemistry, 194(Supplement C): 705-711. Mutlu, A. C., Boyaci, I. H., Genis, H. E., Ozturk, R., Basaran-Akgul, N., Sanal, T. and Evlice, A. K. (2011). Prediction of wheat quality parameters using near-infrared spectroscopy and artificial neural networks. European Food Research and Technology, 233: 267-274. Nelson, D. (2019). What are Neural Networks? https://www.unite.ai/ what-are-neural-networks/. Accessed 23rd June 2020. Pasquini, C. (2018). Near infrared spectroscopy: A mature analytical technique with new perspectives—a review. Analytica Chimica Acta, 1026: 8–36. Pontes, M. J. C., Santos, S. R. B., Araujo, M. C. U., Almeida, L. F., Lima, R. A. C., Gaiao, E. N. and Souto, T. C. P. (2006). Classification of distilled alcoholic beverages and verification of adulteration by near infrared spectrometry. Food Research International, 39: 182-189.

80

Ana Jurinjak Tušek, Davor Valinger, Maja Benković et al.

Pouladzadeh, P., Villalobos, G., Almaghrabi, R. and Shirmohammadi, S. (2012). A novel SVM based food recognition method for calorie measurement applications. In: 2012 IEEE International Conference on Multimedia and Expo Workshops, pp. 495-498. Rodrigo, R. and Gil-Becerra, D. (2014). Implications of Polyphenols on Endogenous Antioxidant Defense Systems in Human Diseases. In: Polyphenols in Human Health and Disease. San Diego: Academic Press. Román, R. C., Hernández, O. G. and Urtubia, U. A. (2011). Prediction of problematic wine fermentations using artificial neural networks. Bioprocess and Biosystems Engineering, 34: 1057-1065. Rosenblatt, F. (1958). The perceptron: A probabilistic model for information storage and organization in the brain. Psychological Review, 65: 386408. Sharma, R. (2014). Polyphenols in health and disease: practice and mechanisms of benefits. In: Polyphenols in Human Health and Disease. San Diego: Academic Press. Tian, H., Li, F., Qin, L., Yu, H. and Ma, X. (2014). Discrimination of chicken seasonings and beef seasonings using electronic nose and sensory evaluation. Journal of Food Science, 79: 2346-2353. Tripathy, P. P. and Kumar, S. (2009). Neural network approach for food temperature prediction during solar drying. International Journal of Thermal Sciences, 48: 1452-1459. Valinger, D., Kušen, M., Jurinjak Tušek, A., Panić, M., Jurina, T., Benković, M., Radojčić Redovniković, I. and Gajdoš Kljusuić, J. (2018). Development of Near Infrared Spectroscopy models for quantitative prediction of the content of bioactive compounds in olive leaves. Chemical and Biochemical Engineering Quarterly, 32: 535-543. Wang, C. and Slikker, W. (2017). Neural Cell Biology, USA: CRC Press: Taylor & Francis Group. Wang, J., Yue, H. and Zhou, Z. (2017). An improved traceability system for food quality assurance and evaluation based on fuzzy classification and neural network. Food Control, 79: 363-370.

Application of Artificial Neural Networks …

81

Wang, X., Song, M., Liu, S., Wu, S. and Thu, A. M. (2020). Analysis of phthalate plasticizer migration from PVDC packaging materials to food simulants using molecular dynamics simulations and artificial neural network. Food Chemistry, 317 : 126465. Warwick, K. (2012). Artificial Intelligence: The Basics. London: Routledge. Wen, C. C., Chen, H. M. and Yang, N. S. (2012). Developing Phytocompounds from Medicinal Plants as Immunomodulators. In: Advances in Botanical Research. San Diego: Academic Press. Xu, D. P., Li, Y., Meng, X., Zhou, T., Zhou, Y., Zheng, J., Zhang, J. J. and Li, H. B. (2017). Natural antioxidants in foods and medicinal plants: Extraction, assessment and resources. International Journal of Molecular Sciences, 18: 1-32. Yalcin, H., Toker, O. S., Ozturk, I., Dogan, M. and Kisi, O. (2012). Prediction of fatty acid composition of vegetable oils based on rheological measurements using nonlinear models. European Journal of Lipid Science and Technology, 114: 1217-1224. Yiqun, H., Kangas, L. J. and Rasco, B. A. (2007). Applications of artificial neural networks (ANNs) in food science. Critical Reviews in Food Science and Nutrition, 47: 113-126. Yordi, E. G., Koelig, R., Mota, Y. C., Matos, M. J., Santana, L., Uriarte, E. and Molina, E. (2015). Application of KNN algorithm in determining the total antioxidant capacity of flavonoid-containing foods. In: 19th International Electronic Conference on Synthetic Organic Chemistry. Zareef, M., Chen, Q., Hassan, M. M., Arslan, M., Hashim, M. M., Ahmad, W., Kutsanedzie, F. Y. H. and Agyekum, A. A. (2020). An overview on the applications of typical non-linear algorithms coupled with NIR spectroscopy in food analysis. Food Engineering Reviews, 1-18 (published online). Zhou, L., Zhang, C., Liu, F., Qiu, Z. and He, Y. (2019). Application of deep learning in food: A review. Comprehensive Reviews in Food Science and Food Safety, 1793-1811.

In: A Comprehensive Guide … Editor: Steffen Skaar

ISBN: 978-1-53618-466-2 © 2020 Nova Science Publishers, Inc.

Chapter 3

ARTIFICIAL NEURAL NETWORKS AS A CHEMOMETRIC TOOL IN ANALYSIS OF BIOLOGICALLY ACTIVE COMPOUNDS Strahinja Kovačević, Milica Karadžić Banjac*, Sanja Podunavac-Kuzmanović and Lidija Jevrić University of Novi Sad, Faculty of Technology Novi Sad, Department of Applied and Engineering Chemistry, Novi Sad, Serbia

ABSTRACT Prediction of biological activity and molecular properties of newly synthesized compounds and compounds with certain biological effects is a quite complex task. It requires high-quality and relevant data, as well as sophisticated chemometric methods for data analysis and extraction of the most important information from the data set. Artificial neural networks (ANNs) have become very popular in chemometric modeling of biological activity and chromatographic retention behavior of biologically active compounds in the past decades. ANNs have been widely utilized for regression purposes in Quantitative Structure–Activity Relationship *

Corresponding Author’s Email: [email protected].

84 S. Kovačević, M. Karadžić Banjac, S. Podunavac-Kuzmanović et al. (QSAR) and Quantitative Structure–Retention Relationship (QSRR) modeling as a non-linear regression method. Their efficacy and usefulness in prediction of various molecular features have been proven in numerous scientific studies whose aim was to establish high-quality models which served as chemometric guidelines for selection of the most prominent compounds. The present review chapter emphasizes the main achievements of ANN modeling in recent years in the field of QSAR and QSRR analysis of various biologically active molecules and points out its advantages and disadvantages. The applicability domain of ANNs has also been discussed, including the necessary data transformation prior to modeling. One of the main goals of this chapter is to present in a simple way contemporary developments in the application of ANNs in the design of biologically active compounds, as well as in the prediction of their physicochemical and biological properties, which are crucial for their further application in biological research.

Keywords: biological activity, chemometrics, chromatography, machine learning, QSAR analysis, QSRR analysis

INTRODUCTION Biologically active compounds, including steroids and steroidal derivatives, bezimidazoles, benzoxazoles, s-triazines, terpenes, alcaloids and many other, are in focus of numerous researches in chemical and medicinal fields due to their biological potential. Their biological activity can be expressed as anticancer, antiinflamatory, antioxidant, antifungal, antibacterial, antiviral activity, etc. For example, in particular focus are newly synthesized steroidal derivatives (Figure 1) which express antiproliferative activity towards various cancer cell lines, including, estrogen-receptor positive breast adenocarcinoma (MCF-7), estrogenreceptor negative breast adenocarcinoma (MDA-MB-231), prostate adenocarcinoma (PC-3) and colon adenocarcinoma (HT-29) [1-5]. This fact marked these compounds as worthy for further investigations and structural modifications in order to find the target compounds which would be eventually used in clinical trials.

Artificial Neural Networks as a Chemometric Tool …

85

The synthesis of novel compounds is a time-consuming and expensive task, thus in order to synthesize target compounds it is necessary to have certain guidelines. These guidelines can be formed applying chemometric methods and/or an approach based on virtual screening. The role of chemometrics has become crucial in design of novel molecules with significant biological potential. Chemometrics provides useful tools for extraction of relevant information from the huge amount of data. Considering the search for desirable compounds, chemometrics can be used for prediction of biological activity and estimation of numerous physicochemical properties which can be essential for their biological behavior [6-10]. The prediction of the biological activity is usually carried out by using various regression methods. Linear regression is one of the most used regression approaches for this purpose, including univariate linear regression (ULR), multiple linear regression (MLR), partial least squares regression (PLS), principal component regression (PCR) and so forth. However, sometimes linear regression does not result in any acceptable model since the established models may have poor statistical characteristics, so the only solution can be the application of non-linear methods. Artificial neural networks (ANNs) have become a favored non-linear regression method in prediction of not only biological activity, but also in estimation of physicochemical features of molecules [11]. The ANN modeling can result in high-quality regression models for precise prediction of desired molecular characteristics. An ANN regression model does not provide a specific mathematical function, but it generalize the relationship between one or more dependent and independent variables. Since the biological response of a compound is a consequence of many factors (chemical nature, molecular structure, reactivity, lipophilicity, bulkiness, shape, etc.), multivariate modeling is inevitable in its prediction, particularly for in vivo and in vitro correlations. In some cases, the non-linear regression approach is necessary for prediction of physicochemical properties as well, such as chromatographic behavior, lipophilicity, polarizability, distribution coefficient, etc. [12].

86 S. Kovačević, M. Karadžić Banjac, S. Podunavac-Kuzmanović et al.

Figure 1. Structures of novel steroidal derivatives which possess significant antiproliferative activity towards various types of cancer cell lines: (1) 3-oxo-16,17secoandrosta-1,4-diene-16,17A-dinitrile, (2) 17α-picolyl-androsta-3,5-diene-17β-ol, (3) 3β-hydroxy-17-oxa-17α-homoandrost-5-en-16-one (4) [1-5].

In the case of prediction of biological activity, the ANN modeling is applied for Quantitative Structure–Activity Relationship (QSAR) analysis. QSAR is well-developed and widely used field of chemometrics which is based on application of different regression approaches (linear and nonlinear) to find the correlations between molecular properties (molecular descriptors) and biological behavior (anticancer, antimicrobial, antiinflamatory activity, etc.) [13]. Besides the QSAR analysis, the Quantitative Structure–Retention Relationship (QSRR) approach was developed [14]. The QSRR modeling is based on estimation of the relationships between molecular descriptors and the retention behavior of compounds in a certain chromatographic system (i.e., Thin-Layer Chromatography (TLC) system, High Performance Liquid Chromatography (HPLC) system, Gas Chromatography (GC) system). The prediction of the chromatographic behavior in TLC and HPLC systems usually implies the prediction of so-called "chromatographic lipophilicity", as an alternative lipophilicity parameter considered to be one of the essential properties of biologically active molecules.

Artificial Neural Networks as a Chemometric Tool …

87

The ANNs have been successfully used in both QSAR and QSRR analyses. The main aim of the present chapter is to give an overview of the practical utilization of ANNs in QSAR and QSRR modeling of biologically active compounds in recent years and to present latest achievements in this field.

ARTIFICIAL NEURAL NETWORKS AS A REGRESSION TOOL Although the ANNs have been successfully used in both regression and pattern recognition analysis, the focus of this chapter will be on their performance as a regression tool. Namely, the ANNs were developed on the basis of the brain structure – they can learn, recognize patterns and manage data [15, 16]. A neuron or a nerve cell in the brain receives the information (electric signal) via filamentous structures called dendrites (Figure 2). The signal is then forwarded via a long thin strand known as axon. The neuron is connected with another neuron by connections between the axon terminals and dendrites of other neurons (Figure 3). Those connections are called synapses [17]. A huge number of neurons makes a neural network with a specific biological function. The architecture and function of an artificial neural network are based on the structure and function of a biological neuron. ANNs are consisted of several linked layers of artificial neurons (Figure 4). The input layer receives the information (independent or measured variables) and sends it out to the hidden layer. There can be one or more hidden layers which are aimed to process the input information by using defined mathematical functions. These functions are actually activation functions which run the signal propagation to the following layer so that the negative weights simulate the inhibitory stimulus, while the positive weights are responsible for simulation of excitatory stimulus [18, 19].

88 S. Kovačević, M. Karadžić Banjac, S. Podunavac-Kuzmanović et al.

Figure 2. Anatomy of a neuron.

Figure 3. The connections between neurons and information (signal) flow.

Activation function can be hyperbolic, sinusoid, radial-basis, etc. The regression function transforms the input variables (information) into a nonlinear output, so there is a non-linear correlation between the input and output data [18].

Artificial Neural Networks as a Chemometric Tool …

89

Figure 4. The structure of an artificial neural network analogous to the network of biological neurons ( – artificial neurons).

Generally, there are two main types of ANN technique based on the networks architecture [18-20]: 1) Feed-forward or back-propagation neural networks (the information flows in only one direction from the input to the output); 2) Feed-back neural networks (here, the output information of one layer routes back to a preceding layer). Feed-forward neural networks include single-layer perceptron (SLP) and multi-layer perceptron (MLP) networks, while Kohonen’s (SOM) and Bayesian regularized networks (BRANN) belong to the group of feed-back networks [18]. ANNs are used for non-parametric regression. In the first step of ANN regression modeling, the training or calibration of the networks has to be done. The training is actually a learning step in which the search for optimal network architecture and satisfactory performance is carried out. This is considered to be the crucial step of the modeling of an ANN and it is carried out by using a training data set. There are many learning algorithms used in

90 S. Kovačević, M. Karadžić Banjac, S. Podunavac-Kuzmanović et al. the training step of MLP networks. One of the most used is undoubtedly gradient descent algorithm. The Broyden-Fletcher-Goldfarb-Shanno (BFGS) optimization algorithm, as a variant of gradient descent algorithm, is also often used in networks training. This training algorithm applies the BFGS updating formula for the B matrix [21, 22]: B'BFGS = B + (Δg ΔgT / ΔgTΔx) – (B Δx ΔxT B / ΔxT B Δx)

(1)

Δx = xk+1 – xk

(2)

Δg = gk+1 – gk

(3)

In equation 1, B is the old and B'BFGS is the new second derivative approximation; Δx and Δg are vectors in ℝ𝑛 that satisfy ΔxT Δg > 0 [21, 22]. After the training (calibration) procedure, the validation of the established networks has to be performed. The validation can be carried out: 1) by using a test set, in order to determine a generalization error and to evaluate fully trained networks [19]; 2) applying a validation set so that the most suitable network’s architecture (configuration) and training parameters (validation set error and training set error are compared during training) can be found [19]; 3) applying an external test set, containing the data which did not participate in the networks training nor in the estimation of networks parameters [19, 22]; 4) by comparing experimental and predicted data and analyzing the slope and intercept of the linear relationship [22]; 5) by the analysis of the amplitude of residuals, as absolute differences between the experimental and predicted data, and the analysis of their randomness [22]; 6) by conducting cross-validation (leave-one-out or leave-more-out approach) [22, 23];

Artificial Neural Networks as a Chemometric Tool …

91

7) by the ranking of the networks based on Sum of Ranking Differences (SRD) analysis [24]. All of the validation steps take into account the calculation of certain statistical parameters which estimate statistical quality and predictive ability of a neural network used for regression purposes. Some of the most important statistical parameters used in the estimation of statistical quality of ANNs are the following [22-24]: 1) Pearson’s correlation coefficient (R) and determination coefficient (R2); 2) adjusted determination coefficient (R2ADJ); 3) determination coefficient of cross-validation (R2CV or Q2); 4) predicted residuals sum of squares (PRESS); 5) total sum of squares (TSS); 6) standard deviation of cross-validation (SPRESS); 7) root mean square error (RMSE); 8) variation coefficient (VC); 9) F-test; 10) p-value. Although the ANN regression modeling does not provide the exact mathematical equation, there are some methods whose purpose is to estimate the contribution or the influence of the variables on network’s performance. The Global Sensitivity Analysis (GSA) is widely used method for this purpose [25]. The GSA calculates the coefficients for every input variable – a high GSA coefficient of an input variable means that the network’s parameters are significantly influenced by even small changes in the input parameter [22], as illustrated in Figure 5.

92 S. Kovačević, M. Karadžić Banjac, S. Podunavac-Kuzmanović et al.

Figure 5. Higher GSA coefficient of an input variable implies its higher influence on the network’s parameters.

Considering the advantages of the application of artificial neural networks in regression modeling, it is worth mentioning that their implementation is easy and simple, they are able to model complex relationships between variables and to provide high-quality and efficient predictive models; also, ANNs possess the ability of multiple training algorithms [26, 27]. However, there are some literature sources where artificial neural networks are considered to be a “black box” since there is no possibility of obtaining the exact mathematical model which correlates the input and output variables in a clear way [18, 26-28]. The overfitting can be also a problem in ANN modeling, as well as long training time. Often, large data set is needed for reliable modeling.

PREPROCESSING OF BIOLOGICAL AND CHROMATOGRAPHIC DATA IN ANN MODELING Prior to ANN modeling, sometimes it is necessary to mathematically transform the input and/or output variables. Often, the transformation of only input or output variable gives satisfactory results, but sometimes the

Artificial Neural Networks as a Chemometric Tool …

93

transformation of both inputs and outputs is required, depending on the quality of the established networks [19]. Data scaling or data normalization in ANN modeling of chromatographic behavior (lipophilicity) and biological activity is usually needed when the input variables are on completely different scales in order to minimize bias for one variable to another in the neural network [29]. Data normalization makes the data set suitable for the training procedure and can speed up the training process [29-32]. Some normalization and variable transformation methods are presented in Table 1. Often used data scaling methods in QSRR and QSAR modeling are minmax normalization, as a minimum–maximum value based normalization method, and Z-score method, which belongs to the group of normalization methods based on mean and standard deviation [30, 31]. Min-max normalization [31] linearly transforms original variables so that their values fall into the range between 0 and 1. The calculation is carried out by using the following equation: xnorm = (x – xmin)/(xmax – xmin) + xmin

(4)

where xnorm is a normalized and x is a raw value, xmin is a minimum, and xmax is a maximum value of a variable. Z-score normalization [31] is performed so that a variable is scaled applying the equation: xnorm = (x – xmean)/σ

(5)

where xnorm is a normalized and x is a raw value of a variable, xmean is the mean value and σ is the standard deviation. Both these normalization methods are sensitive to presence of outliers in the data sets [31]. Therefore, the outliers should be detected and removed from the data set prior to the ANN modeling. Potential outlier can be detected by various methods, such as statistical, non-parametric, parametric, clustering-based and ANN-based methods [33].

94 S. Kovačević, M. Karadžić Banjac, S. Podunavac-Kuzmanović et al. Table 1. Data scaling and variable transformation methods in ANN modeling used in QSRR and QSAR studies Data scaling (normalization) methods [30] 1) Mean and standard deviation based normalization methods: - Z-score normalization - Mean centered - Power transformation - Variable stability scaling - Pareto scaling 2) Minimum–maximum value based normalization methods: - min-max normalization - max normalization 3) Median and median absolute deviation normalization 4) Decimal scaling normalization 5) Tanh based normalization 6) Sigmoidal normalization: - Logistic sigmoid - Hyperbolic tangent

Variable (x) transformation methods

1/x

log(x) log(1/x)

1/log(x) ex

1/ex

Variable selection is one of the most important steps which has to be done prior to ANN modeling. In order to select reliable variables for QSRR and QSAR models formation, several approaches can be employed (the Scheme in Figure 6) [34, 35]. Selection of variables is also important since the chosen variables should be not only mathematically related with an output variable, but also must be able to explain the predicted phenomenon. If the variables are normalized or transformed before the ANN modeling, the variable selection procedure should be performed on normalized/scaled variables, not on the raw data. Also, if linear regression analysis (ULR, MLR, PLS, PCR, etc.) is carried out before ANN modeling, the variables used in those linear models are often used as input variables in ANN modeling. Those ANN models usually have much better performance than linear models from which the variables were selected [6].

Artificial Neural Networks as a Chemometric Tool …

95

Figure 6. Some of the most used variable selection methods in QSAR and QSRR modeling [34, 35].

ARTIFICIAL NEURAL NETWORKS IN QUANTITATIVE STRUCTURE - (CHROMATOGRAPHIC) RETENTION RELATIONSHIPS QSRR method was developed as a part of Quantitative Structure– Property Relationships (QSPR) approach and found its application in predicting the retention behavior and lipophilicity of a large number of different compounds [7-10, 36-45]. QSRR modeling enables the characterization of stationary and mobile phases as well as the analysis of the efficiency of analyte separation in different chromatographic systems. The development of suitable QSRR models for chromatographic lipophilicity prediction is a very actual topic, especially the application of ANNs in this field. Many research groups around the world dedicated their time and skills in order to apply ANN modeling in definition of novel

96 S. Kovačević, M. Karadžić Banjac, S. Podunavac-Kuzmanović et al. QSRRs (Table 2). From the data in Table 2, it can be seen that the multilayer perceptron network is one of the most often used networks in QSRR studies. Table 2. The overview of the recent application of ANN in QSRR modeling The group of compounds Amiloride and methyclothiazide Atypical antipsychotic drug aripiprazole Dteroidal derivatives Essential oils of coriander and sage β-hydroxy-β-arylalkanoic acids Pharmaceuticals identified in effluent wastewater Naratriptan hydrochloride Sartans Trinucleotides 17α-picolyl and 17(E)picolinylidene androstanes Peptides originating from proteomes Pesticides, drugs of abuse, human/veterinary pharmaceuticals and mycotoxins

Candesartan cilexetil Tebipenem pivoxyl and its degradation products Doping-related compounds

Trimethylsilylated anabolic androgenic steroids

Type of ANNs used Multilayer perceptron network Multilayer perceptron network

Reference [46] [47]

Multilayer perceptron network Multilayer perceptron network Multilayer perceptron network Multilayer perceptron network

[10] [48] [49] [50]

Single-layer, double-layer, pi-sigma, sigma-pi-sigma Multilayer perceptron network Multilayer perceptron network Multilayer perceptron network

[51]

Multilayer perceptron network

[54]

Multilayer perceptron network, generalized regression neural networks, radial basis functions, linear neural networks, probabilistic neural networks Multilayer perceptron network Recursive neural network

[55]

Linear, radial basis function, probabilistic neural networks, multilayer perceptrons Multilayer perceptron network

[58]

[52] [53] [37]

[56] [57]

[59]

Artificial Neural Networks as a Chemometric Tool …

97

Different types of biologically active compounds were the subject of researches that included ANN modeling in QSRR approach, such as naratriptan hydrochloride and its impurities, β-hydroxy-β-arylalkanoic acids, candesartan cilexetil and its degradation products and doping-related compounds (Figure 7) [48, 50, 57, 59]. These compounds possess wide rage of different biological activities and pharmacological effects.

Figure 7. Molecular structures of some compounds of interest which were investigated by using ANN-based QSRR approach: (1) candesartan cilexetil degradation product, (2) aripiprazole, (3) methamphetamine (4) 3-hydroxy-2,2-dimethyl-3-(4biphenylyl)butanoic acid [47, 49, 56, 58].

Abba et al. (2020) [46] investigated amiloride and methyclothiazide that are being used for the treatment of high blood pressure (hypertension), kidney disease, treatment of fluid retention (Edema) and the liver cirrhosis. By the composition they are potassium sparing-diuretics used to prevent the body from absorbing a high amount of salt and for the potassium level regulation [46]. Authors applied two non-linear methods: a feed-forward network with back-propagation algorithms (FFNN-BP) and adaptive-neuro fuzzy inference system (ANFIS), as well as classical MLR approach. The number of the hidden neurons varied from 5 to 21 for the ANN model simulating the response surface in the HPLC optimization method development. This study concluded that, from the predictive comparison of the models, non-linear methods (ANN and ANFIS) outperformed the classical linear model (MLR) with considerable efficiency. Also, the ANN

98 S. Kovačević, M. Karadžić Banjac, S. Podunavac-Kuzmanović et al. model offered a superior alternative compared to ANFIS model. Obtained models were validated using the 4-fold cross-validation, while the data sets were divided in 75% of the data for training and 25% of the data for the test set [46]. A group of authors [47] applied the QSRR modeling in micellar liquid chromatography (MLC) and compared linear and non-linear approaches [47]. The substances of interest were group of atypical antipsychotic drug aripiprazoles. They are used to treat the symptoms of psychotic conditions such as schizophrenia and bipolar disorder (manic depression) [47]. Authors applied principal component analysis (PCA), non-negative matrix factorization (NMF), reliefF, MLR, mutual info and F-regression. The series of investigated predictive algorithms comprised linear regressions (LR), ridge regression, lasso regression, ANN, support vector regression (SVR), random forest (RF), gradient boosted trees (GBT) and k-nearest neighborhood (k-NN). The established models were evaluated using leaveone-out cross-validation, 10-fold cross-validation, y-randomization and outof-sample data. The ANN modeling was based on multilayer perceptron network and with good validation parameters [47]. Selection of lipophilicity models of series of triazole, tetrazole, toluenesulfonylhydrazide, nitrile, dinitrile and dione steroid derivatives with high biomedical importance build by LR, MLR and ANN was carried out by Karadžić Banjac et al. (2019) [10]. These steroidal derivatives express significant antiproliferative activity towards various cancer cell lines, including PC-3, MCF-7, HeLa, HT-29, MDA-MB-231, K-562 and MRC-5 [5]. The ANN models were formed by training 2 000 networks for each combination of variables. The training of MLP neural networks was carried out using the BFGS algorithm, with the data normalized by using min-max normalization method. The authors applied classical and non-parametric ranking of established linear and non-linear models, with generalized pair correlation method (GPCM) and SRD. The presented results indicate the superiority of MLR over ANN models in the case of prediction of the chromatographic lipophilicity of studied steroid derivatives [10].

Artificial Neural Networks as a Chemometric Tool …

99

QSRR modeling was employed to predict retention time of essential oil of coriander and sage compounds obtained by GC analysis [48]. Coriander and sage are commonly used in food, pharmaceutical and cosmetic industries. MLP with BFGS algorithm was used for ANN modeling, together with input and output data normalization. Good quality of conducted ANN modeling was confirmed by the relationship between the predicted and experimental retention values and statistiacal parameters as R2 and RMSE [48]. Dobričić et al. (2017) [49] applied biopartitioning micellar chromatography (BMC) and QSRR modeling with PLS, MLR and ANN for the investigation of novel β-hydroxy-β-arylalkanoic acids. These compounds possess antiinflammatory activity and are considered to be selective non-steroidal antiinflammatory drugs for processes including rheumatoid arthritis, atherosclerosis, Alzheimer's disease and colon cancer [49]. Multilayer perceptron artificial neural networks were applied in ANN modeling with back-propagation and conjugate gradient descent training algorithms. Among the PLS, MLR and ANN models, authors outlined the ANN model as optimal one based on the calculated statistical parameters. This model could be used as a guideline in design of novel β-hydroxy-βarylalkanoic acids [49]. Ultra high performance liquid chromatography (UHPLC) was employed for screening of pharmaceuticals and their metabolites in treated wastewater samples from Athens [50]. Neighborhood clustering classification (k-NN), genetic algorithm (GA) selection techniques and support vector machines (kNN-GA-SVM) were applied for modeling. The number of hidden nods ranged from minimum 3 to maximum 55 and at least 500 iterations were done for each design. The quick propagation method was applied for network training. With the use of ANN method, more than 90% of all compounds used in the optimization fell inside the desired frame [50]. Mizera et al. (2017) [51] generated a QSRR model for the determination of naratriptan hydrochloride and its impurities based on ANN coupled with GA. The authors analyzed selective serotonin 5-HT1 receptor agonists used

100 S. Kovačević, M. Karadžić Banjac, S. Podunavac-Kuzmanović et al. for the treatment of acute headache applying HPLC and GA coupled with ANN for prediction of the retention times of naratriptan hydrochloride impurities. The PLS method was also applied. Several ANN types were performed: single-layer ANN (SL-ANN), double-layer ANN (D-ANN) and higher-order architectures: pi-sigma ANN (PS-ANN) and sigma-pi-sigma ANN (SPS-ANN). Training function of ANNs was error back-propagation with Levenberg-Marquardt (LM) algorithm and Bayesian regularization. Authors concluded that PS-ANN showed to be the best model due to best score achieved. PS-ANN also outperformed GA-PLS model [51]. Six therapeutics, angiotensin receptor antagonists, called sartanans were investigated by using gradient-elution HPLC method and ANN-based QSRR modeling [52]. Trial-and-error method was used for determination of network architecture and parameters of function optimization to minimize RMSE for the training and validation sets. In the research [52], multilayer perceptron network with 8 neurons in the hidden layer was applied. In hidden and output layers logistic activation function was set. Linear postsynaptic potential function was used in hidden and output neuron and 20 000 epochs were run. Optimal results were obtained by the network trained with back-propagation algorithm. High R2 and low RMSE values, 0.985 and 0.1613, respectively, indicated a good predictive ability of the obtained ANN model. The sensitivity analysis was also performed [52]. The separation of trinucleotides by reversed-phase liquid chromatography (RPLC) was the subject of interest as well [53]. Natural trinucleotides can be primers for DNA replication or degradation products of large DNA and RNA molecules. Synthetic trinucleotides can be used in DNA sequencing, DNA amplification and synthesis of artificial genes. In the study [53], two types of regression were used: r-ANN and LR. In order to avoid overfitting in the ANN modeling, two different data sets were applied. The number of 1000 MLP networks with BFGS algorithm were trained using identity (Idt), logistic (Lgt), tangent (Tanh), sinusoidal (Sine) and exponential (Exp) activation functions [53].

Artificial Neural Networks as a Chemometric Tool …

101

Anticancer drugs based on 17α-picolyl and 17(E)-picolinylidene androstane structures were the subject of research conducted by Kovačević et al. (2016) [37]. These compounds possess antiproliferative activity towards various cancer cell lines, including MCF-7, MDA-MB-231, PC-3 and HT-29 [4]. The retention behavior of these steroidal derivatives was determined using HPLC method and it was modeled applying different chemometric approaches: LR, polynomial regression (PR), MLR, PCR, PLS and ANN. Prior to ANN modeling the input data were normalized by using min-max normalization method. For hidden and output neurons the following MLP activation functions were used: Lgt, Idt, Exp, Tanh and Sine. The total number of 28 000 networks was trained during ANN modeling with BFGS algorithm. The statistical parameters undoubtedly indicated that the ANN models are the best solution for prediction of the retention (chromatographic lipophilicity) of 17α-picolyl and 17(E)-picolinylidene androstane derivatives comparing to other linear modeling approaches [37]. Three multivariate approaches were compared in the research conducted by Žuvela et al. (2016) for non-linear relationship modeling of peptides originating from proteomes [54]. These peptides originated from bovine milk β-casein, human serum albumin, bovine serum albumin, chicken egg ovalbumin, ribonuclease B, bovine milk lactoglobulin, bovine myoglobin and insulin-like growth factor-binding protein 1. Three machine learning methods were applied: SVR, ANN and kernel PLS (kPLS). Models were thoroughly validated using an external validation set and stability was confirmed by their applicability domain defining. Authors used feedforward MLP with a back-propagation learning algorithm. SVR model had the lowest RMSE and turned out to be superior over ANN and kPLS that also had high predictive power [54]. The screening of a large number of emerging contaminants in environmental waters using ANNs for the prediction of chromatographic retention time was the aim of research conducted by Bade et al. (2015) [55]. The data set covered 544 compounds: pesticides, drugs of abuse, human/veterinary pharmaceuticals and mycotoxins. Over 100 000 network architectures for each model type were tested for their predictive ability. Five types of ANN models were used: 3- and 4-layer MLP, generalized

102 S. Kovačević, M. Karadžić Banjac, S. Podunavac-Kuzmanović et al. regression neural networks (GRNNs), radial basis functions (RBFs), linear neural networks and probabilistic neural networks (PNN). The best performance and the best correlations of predicted versus experimentally measured data were achieved by using MLP, compared to all other ANN types. The best performance had 4-layer MLPs with R2 between 0.86 and 0.90 [55]. Candesartan cilexetil and its degradation products were investigated by Golubović et al. (2015) [56] by using HPLC for retention determination and ANN for a QSRR model building. Candesartan cilexetil belongs to the class of angiotensin receptor antagonists and it is used in the treatment of hypertension of all grades [56]. Feed-forward MLP networks were used and the network architecture and parameters of function were optimized by trialand-error approach to minimize RMSE value of training and validation sets. Optimal results were obtained applying the network trained with backpropagation algorithm. The networks training was carried out in 20 000 epochs. The obtained strong agreement between theoretical and experimental output values marks the obtained model as statistically highquality one [56]. A group of authors [57] applied the ANN modeling to predict retention times of tebipenem pivoxyl and its degradation products. The carbapenem analogs are easily being degraded. This leads to the loss of their microbiological activity. The prediction error of ANNs was estimated by the cross-validation. During ANN development several learning methods were tested, but the most appropriate method was based on Levenberg-Marquardt algorithm. RMSE of the trained ANN with 4 neurons in hidden layer was higher than RMSE when recursive feature elimination (RFE) was applied. The ANN modeling was used in prediction of chromatographic retention of doping-related compounds [58]. The conducted study [58] investigated the use of ANN approach for the prediction of retention time in archived urine analysis data from the London 2012 Olympic and Paralympic Games. Several ANN types and architectures were investigated: linear, RBF, PNN and MLP. The training of ANN was stopped between 2000 and 3900 epochs, when verification and training error were the lowest. As the best networks, feed-forward, back-propagation-type MLP with 2 hidden layers of 5 and 4

Artificial Neural Networks as a Chemometric Tool …

103

nodes singled out. The authors showed that ANN models could be used to predict chromatographic retention for 93% out of all selected doping-related compounds. Comparison of MLR, PLS and ANN potential in prediction of retention times of 52 trimethylsilylated anabolic androgenic steroids was investigated as well [59]. Anabolic androgenic steroids are the most abused substances in sports worldwide and they are included in the list of prohibited substances of the World Anti-Doping Agency (WADA). Results obtained in this research were compared with the previous research [60] that presented the PLS and MLR models. In the research [59] MLP networks were trained by the back-propagation algorithm and in this way the best model was obtained (since RBF models produced very high RMSE values). The RBF, where each unit in the output layer makes a linear transformation on the data from the hidden layer, was also applied. In the MLP, the number of hidden units fluctuated from 1 to 14, while the final MLP model possessed 9 neurons in the second hidden layer. On the basis of statistical performance, the authors concluded that the MLP model is a reliable predictive model. The SRD, as a non-parametric method, was used for the ranking of MLR, PLS and ANN models and PLS method proves to be superior over MLR and ANNs methods. However, the authors concluded that the SRD ranking method underscores the ANN model compared to the MLR model [59].

ARTIFICIAL NEURAL NETWORKS IN QUANTITATIVE STRUCTURE - (BIOLOGICAL) ACTIVITY RELATIONSHIPS QSAR modeling is one of the most applied computational approaches in medicinal chemistry and, particularly, in design of novel biologically active compounds [61]. Some authors define the QSAR methodology as "an attempt to remove the element of luck from drug design by establishing a mathematical relationship in the form of an equation between biological activity and measurable physicochemical parameters" [62]. One of the

104 S. Kovačević, M. Karadžić Banjac, S. Podunavac-Kuzmanović et al. biggest challenges in QSAR modeling is how to choose the most reliable mathematical model or which modeling approach should be applied for this purpose. ANN modeling in QSAR has become quite popular in recent years, especially due to development of new training algorithms and validation approaches. Based on the SCOPUS database (www.scopus.com), in Figure 8 it can be seen that from 2010 to 2019 there is a continuous increase of the number of documents (scientific papers, books, articles, etc.) mentioning ANNs in QSAR modeling.

Figure 8. The number of documents (scientific papers, books, articles, etc.) which deal with ANNs in QSAR modeling in last ten years according to the SCOPUS database (www.scopus.com).

Non-linear models are not so rare in QSARs since the relationship between biological activity and molecular properties is often quite complex. Therefore, ANNs are certainly a convenient tool in establishing this kind of mathematical relationships. A brief overview of the recent applications of ANNs in QSAR modeling is presented in Table 3.

Artificial Neural Networks as a Chemometric Tool …

105

Table 3. The overview of the recent applications of ANNs in QSAR modeling The group of compounds 593 organic contaminants from 27 diverse chemical classes 1,2,3-triazolo[4,5d]pyrimidine hybrids (1,2,3-TPH) ExCAPE chemogenomics database project composed of 43509 compounds Nitrobenzoxadiazole derivatives Azine compounds

Traditional Chinese Medicines (TCM) ingredients Bisphosphoramide derivatives Indole-based analogs 4-anilinoquinolinetriazine derivatives Benzoxazoles and oxazolo[4,5b]pyridines

Predicted biological activity OH∙ radical rate constants

Type of ANN modeling applied in the study Deep Neural Network (DNN) with molecular fingerprints (MF)

Reference

Anti-gastric cancer activity (IC50)

Back propagation neural network (BP-NN)

[64]

Biological activity towards targeted gene

Deep Extreme Learning Machine (DELM)

[65]

glutathione-Stransferases (GSTP11) inhibition Non-nucleoside reverse transcriptase inhibition (NNRTIs) -Anti-HIV activity Nephrotoxicity

Back propagation neural network (BP-NN)

[66]

Bayesian Regularization feed-forward artificial neural network (LASSO-BR-ANN) ANN*

[67]

Urease inhibitory activity

Genetic Algorithm Artificial Neural Network (GA-ANN) Feed-forward ANN ANN*

[69]

Feed-forward multilayer perceptron (MLP) ANN function with BroydenFletcher-GoldfarbShanno (BFGS) learning algorithm

[72]

Anti-HIV activity Antimalarial activity Antifungal activity towards Candida albicans

[63]

[68]

[70] [71]

106 S. Kovačević, M. Karadžić Banjac, S. Podunavac-Kuzmanović et al. Table 3. (Continued) The group of compounds Triazine derivatives

Predicted biological activity Toxicity (inhibition of photosystem II)

Dihydroindeno and indeno thiadiazine derivatives

Antifungal activity towards Aspergillus flavus

Phenols and thiophenols

Toxicity towards Photobacterium phosphoreum as an important indicator of water quality Antiproliferative activity towards ERbreast adenocarcinoma cells (MDA-MB-231)

A- and B- modified Dhomo lactone and Dseco androstane derivatives

Phenylindole derivatives

Antiproliferative activity against hormoneindependent human MDA-MB-231 breast cancer cell line and estrogen-sensitive MCF7 breast cancer cell line *network type not specified.

Type of ANN modeling applied in the study Back propagation artificial neural network (BP-ANN) ANN model obtained by using Broyden-FletcherGoldfarb-Shanno (BFGS) learning algorithm Back propagation artificial neural network (BP-ANN)

Reference

Feed-forward multilayer perceptron (MLP) ANN function with BroydenFletcher-GoldfarbShanno (BFGS) learning algorithm Back propagation artificial neural network (BP-ANN)

[76]

[73]

[74]

[75]

[77]

Deep Neural Network with molecular fingerprints (DNN-MF) approach was applied in modeling of hydroxyl radical rate constant of a big data set which contained 593 organic contaminants from 27 diverse chemical classes [63]. The modeling was not carried out on the basis of molecular descriptors, but by using molecular fingerprints encoding the types of atoms and their

Artificial Neural Networks as a Chemometric Tool …

107

mutual connections [63]. The obtained DNN-MF model predicted hydroxyl radical rate constant of the analyzed compounds with satisfactory accuracy [63]. Back-propagation artificial neural network modeling (BP-ANN) was applied in several 2D- and 3D-QSAR analyses, including QSAR modeling of anti-gastric cancer activity of 1,2,3-triazolo[4,5-d]pyrimidine hybrids (1,2,3-TPH) [64], glutathione-S-transferases (GSTP1-1) inhibition induced by nitrobenzoxadiazole derivatives [66], toxicity of triazine derivatives by inhibition of photosystem II [73], toxicity of phenols and thiophenols towards Photobacterium phosphoreum as an important indicator of water quality [75] and antiproliferative activity of phenylindole derivatives towards hormone-independent human MDA-MB-231 breast cancer cell line and estrogen-sensitive MCF7 breast cancer cell line [77]. In these papers, 2D- or 3D-QSAR modeling was performed based on physicochemical molecular descriptors, and the comparison between the performance of ANN and the performance of multiple linear regression [64, 66, 73, 75, 77], support vector machines [77] and partial least squares regression [64] was done. It was concluded that the non-linear approach (ANNs) resulted in a model with the best statistical and predictive features. All of the established models were validated by sutable validation approaches. Recently, a QSAR study of probably the most extensive data set was published [65]. The subject of this study was the data set composed of 43 509 compounds with biological activity towards targeted genes. Here, the Deep Extreme Learning Machine (DELM) method was applied for multitarget QSAR modeling of chemo-genomic data by using a set of 40 molecular descriptors. The established ANN-based QSAR model was described by high prediction accuracy (R2 ≈ 0.971) [65]. Bayesian Regularization feed-forward artificial neural network (LASSO-BR-ANN) was recently introduced as a novel approach in QSAR analysis [67]. It was applied for prediction of non-nucleoside reverse transcriptase inhibitory activity (NNRTIs) or anti-HIV activity of a series of azine compounds [67]. Besides, a new set of descriptors (a combination of docking derived descriptors generated from ligand-receptor interactions and functional groups features [67]) for precise QSAR modeling was introduced.

108 S. Kovačević, M. Karadžić Banjac, S. Podunavac-Kuzmanović et al. The statistical and predictive quality of the presented LASSO-BR-ANN QSAR model was confirmed by internal and external validation methods. The prediction of the targeted activity applying external test set was described by satisfactory mean square error (MSE = 0.07) and determination coefficient (R2 = 0.88) [67]. Artificial neural networks approach was recently applied for QSAR analysis of nephrotoxicity of traditional Chinese medicines (TCM) ingredients [68]. Despite the fact that incidences of TCM-induced kidney injury gained considerable attention, there is an insufficient number of QSARs on TCMs-induced nephrotoxicity [68]. The analyzed data set contained 609 compounds, including natural products, drugs and their mixture. Six QSAR models were established and validated by internal and external validation. Besides the application of ANN approach, the modeling was done by using support vector machines as well. However, the ANN model showed better accuracy than the support vector machines model [68]. Urease inhibitory activity of a series of bisphosphoramide derivatives was modeled by QSAR method with Genetic Algorithm-Artificial Neural Network (GA-ANN) approach [69]. The established ANN model was described by very high determination coefficient (R2 = 0.988) and by low root mean square error (RMSE = 0.331) [69]. The model was validated by leave-one-out (LOO) and leave-multiple-out (LMO) cross-validation method. The validation parameters of the QSAR model were the following: Q2LOO = 0.827, RMSELOO = 1.355, R2L6O = 0.804 and RMSEL6O = 1.805 [69]. The modeling was carried out based on the molecular descriptors and it was determined that the most critical descriptors influencing the inhibitory activity of the newly synthesized bisphosphoramide derivatives are the rotatable bond fraction (RBF), the total energy of the molecule (Etotal), 31P chemical shift (δ), the number of aromatic bonds (nAB), the number of rings (nCIC) and total charge (Qtotal) [69]. Anti-HIV activity of a set of 128 indole-based analogs was modeled applying feed-forward ANN method on the set of molecular descriptors [70]. Besides ANN modeling, multiple linear regression and support vector machine methods were utilized. The aim of the study was the explanation of the structural requirements of HIV-1 gp120 inhibitory activity [70]. The

Artificial Neural Networks as a Chemometric Tool …

109

results of the internal and external validation indicated that the support vector machines model has slightly better statistical and predictive performance than multiple linear regression and ANN models [70]. The ANN method was recently applied in the modeling of antimalarial activity of a series of 4-anilinoquinoline-triazine derivatives based on physicochemical molecular descriptors [71]. The obtained ANN model was compared to the established multiple linear regression model and it was concluded that the ANN model has better predictive ability and statistical parameters [71]. Determination coefficient of the ANN model was R2 = 0.81 and the root mean square error was RMSE = 0.18 [71]. The models were validated by internal and external validation. The concurrence between the experimental and predicted antimalarial activity of the studied derivatives was confirmed and both QSAR models were recommended for prediction of the targeted bioactivity. Feed-forward multilayer perceptron (MLP) ANN function with Broyden-Fletcher-Goldfarb-Shanno (BFGS) learning algorithm was used in prediction of antifungal activity of benzoxazoles and oxazolo[4,5b]pyridines towards Candida albicans [72], antifungal activity of dihydroindeno and indeno thiadiazine derivatives towards Aspergillus flavus [74] and antiproliferative activity of A- and B- modified D-homo lactone and D-seco androstane derivatives towards ER- breast adenocarcinoma cells (MDA-MB-231) [76]. The modelings were carried out on the basis of molecular descriptors, which were selected by the appropriate selection procedure. The established ANN-based QSAR models were characterized by very good statistical parameters and validated by the cross-validation [72, 74, 76]. The antifungal activity of benzoxazoles and oxazolo[4,5-b]pyridines towards Candida albicans was predicted based on electrostatic and toplogical molecular descriptors and 2D-QSAR-ANN model was formed [72]. For prediction of antifungal activity of dihydroindeno and indeno thiadiazine derivatives towards Aspergillus flavus two ANN models were formed [74]. The input variables, which included lipophilicity and ADME

110 S. Kovačević, M. Karadžić Banjac, S. Podunavac-Kuzmanović et al. (absorption, distribution, metabolism and excretion) descriptors, were selected by partial least squares regression. The ANN models aimed for prediction of antiproliferative activity of Aand B- modified D-homo lactone and D-seco androstane derivatives towards ER- breast adenocarcinoma cells (MDA-MB-231) were actually designed for detection of androstane derivatives which would be non-cytotoxic towards MDA-MB-231 cells [76]. They are able to predict whether A- and B-modified D-homo lactone and D-seco androstane derivatives would have express antiproliferative activity towards MDA-MB-231 cells [76]. The authors recommended the established ANN models as a kind of guidelines for further syntheses of steroidal compounds structurally similar to those used in the modeling [76]. All the aforementioned applications of the ANN methodology in QSAR modeling pointed out the significance of its ability to model quite complex non-linear relationships between molecular properties of numerous biologically active compounds and various types of their biological activities. In many cases, the ANNs were the best solution with the highest prediction accuracy, which was confirmed by the strict internal and external validation procedures. The reliability of ANN models is higher if the training of the networks was carried out on the large data set, as it was the case in some previously mentioned QSARs based on more than 500 [63] or even more than 40 000 compounds [65].

CONCLUSION Artificial neural networks are undoubtedly a powerful regression tool with numerous advantages over classical linear regression approaches. Their application in QSRR and QSAR modeling has become an integral part of the search for high-quality QSRR and QSAR models. Despite the fact that ANNs have many advantages over classical regression methods, some disadvantages limited their use in particular fields. Network’s overtraining is the most often problem in ANN modeling, as well as the lack of the exact mathematical equation which clearly correlates the

Artificial Neural Networks as a Chemometric Tool …

111

input and output variables. This is the reason why some authors consider some ANN-based QSRR and QSAR models to be without scientific foundations and a kind of “black box.” Therefore, a critical assessment of QSRR and QSAR models based on ANNs is necessary. Also, considering the applicability domain of ANN models, it can be seen that in many studies it was not clearly defined using suitable methods such as range-based methods, geometric methods, distance-based methods etc. If applicability domain is not clearly defined, the reliable prediction is questionable. Nevertheless, the presented examples of the application of ANNs in chemometric analysis of retention behavior (QSRR analysis) and biological activity (QSAR analysis) pointed out the advantages of ANNs and their numerous possibilities in prediction of targeted feature. Also, the work of some authors on new advanced ANN approaches aimed to improve reliability of ANN-based QSRR and QSAR regression models is valuable and promising.

ACKNOWLEDGMENT This work has been supported by the research project of the Ministry of Education, Science and Technological Development of the Republic of Serbia (Project No. 451-03-68/2020-14/200134).

REFERENCES [1]

[2]

Djurendić, Evgenija A., Marina P. Zaviš, Marija N. Sakač, Janoš J. Čanadi, Vesna V. Kojić, Gordana M. Bogdanović, and Katarina M. Penov Gaši. 2009. "Synthesis and antitumor activity of new d-seco and d-homo androstane derivatives." Steroids 74:983-988. Ignjatović, Nenad, Marija Sakač, Ivana Kuzminac, Vesna Kojić, Smilja Marković, Dana Vasiljević-Radović, Victoria M. Wu, Vuk Uskoković, and Dragan P. Uskoković. 2018. "Chitosan

112 S. Kovačević, M. Karadžić Banjac, S. Podunavac-Kuzmanović et al.

[3]

[4]

[5]

[6]

[7]

[8]

Oligosaccharide Lactate Coated Hydroxyapatite Nanoparticles as a Vehicle for the Delivery of Steroid Drugs and the Targeting of Breast Cancer Cells." Journal of Materials Chemistry B 6:6957-6968. Savić, Marina P., Olivera R. Klisurić, Matarina M. Penov Gaši, Dimitar S. Jakimov, Marija N. Sakač, and Evgenija A. Djurendić. 2016. "Synthesis, structural analysis and cytotoxic activity of novel Aand B-modified D-homo lactone androstane derivative." Journal of Chemical Crystallography 46:84-92. Ajduković, Jovana J., E. A. Djurendić, E. T. Petri, O. R. Klisurić, A. S. Ćelić, M. N. Sakač, D. S. Jakimov, and K. M. Penov Gaši. 2013. "17(E)-Picolinylidene androstane derivatives as potential inhibitors of prostate cancer cell growth: antiproliferative activity and molecular docking studies." Bioorganic & Medicinal Chemistry 21:7257-7266. Penov Gaši, Katarina, Aleksandar Oklješa, Edward Petri, Anđelka Ćelić, Evgenija Djurendić, Olivera Klisurić, Janoš Csanadi, Gyla Batta, Andrea Nikolić, Dimitar Jakimov, and Marija Sakač. 2013. "Selective antitumour activity and ERa molecular docking studies of newly synthesized D-homo fused steroidal tetrazoles." Med Chem Comm 4:317-323. Kovačević, Strahinja Z., Milica Ž. Karadžić, Sanja O. PodunavacKuzmanović, and Lidija R. Jevrić. 2018. "Binding affinity toward human prion protein of some anti-prion compounds – Assessment based on QSAR modeling, molecular docking and non-parametric ranking." European Journal of Pharmaceutical Sciences 111:215-225. Karadžić, Milica Ž., Lidija R. Jevrić, Anamarija I. Mandić, Siniša L. Markov, Sanja O. Podunavac-Kuzmanović, Strahinja Z. Kovačević, Andrea R. Nikolić, Aleksandar M. Oklješa, Marija N. Sakač, and Katarina M. Penov Gaši. 2017. "Chemometrics approach based on chromatographic behavior, in silico characterization and molecular docking study of steroid analogs with biomedical importance." European Journal of Pharmaceutical Sciences 105:71-81. Karadžić, Milica Ž., Davor M. Lončar, Goran Benedeković, Ivana Kovačević, Velimir Popsavin, Strahinja Z. Kovačević, Lidija R. Jevrić, and Sanja O. Podunavac-Kuzmanović. 2017. "A comparative

Artificial Neural Networks as a Chemometric Tool …

[9]

[10]

[11]

[12]

[13]

[14]

[15]

113

study of chromatographic behavior and lipophilicity of selected natural styryl lactones, their derivatives and analogues." European Journal of Pharmaceutical Sciences 105:99-107. Kovačević, Strahinja Z., Milica Ž. Karadžić Banjac, Sanja O. Podunavac-Kuzmanović, Nataša Milošević, Jelena Ćurčić, Jelena Vulić, Vanja Šeregelj, Nebojša Banjac, and Gordana Ušćumlić. 2020. "Chromatographic and computational screening of anisotropic lipophilicity and pharmacokinetics of newly synthesized 1-aryl-3ethyl-3-methylsuccinimides." Computational Biology and Chemistry 84:107161. Karadžić Banjac, Milica Ž., Strahinja Z. Kovačević, Lidija R. Jevrić, Sanja O. Podunavac-Kuzmanović, and Anamarija I. Mandić. 2019. "On the characterization of novel biologically active steroids: Selection of lipophilicity models of newly synthesized steroidal derivatives by classical and non-parametric ranking approaches." Computational Biology and Chemistry 80:23-30. Mirshahvalad, H., R. Ghasemiasl, Nahid Raoufi, M. Malekzadeh dirin. 2018. "A Neural Network QSPR Model for Accurate Prediction of Flash Point of Pure Hydrocarbons." Molecular Informatics 38:1800094. Golubović, Jelena B., Ana D. Protić, Mira L. Zečević, and Biljana M. Otašević. 2015. "Quantitative structure retention relationship modeling in liquid chromatography method for separation of candesartan cilexetil and its degradation products." Chemometrics and Intelligent Laboratory Systems 140:91-101. Chakravarti, Suman K., and Sai Radha Mani Alla. 2019. "Descriptor Free QSAR Modeling Using Deep Learning With Long Short-Term Memory Neural Networks." Frontiers in Artificial Intelligence 2:1-17. Héberger, Károly. 2007. "Quantitative Structure-(Chromatographic) Retention Relationships." Journal of Chromatography A 1158(12):273-305. Gadzuric, Slobodan B., Sanja O. Podunavac Kuzmanovic, Aleksandar I. Jokic, Milan B. Vranes, Niksa Ajdukovic, and Strahinja Z. Kovacevic. 2014. “Chemometric estimation of post-mortem interval

114 S. Kovačević, M. Karadžić Banjac, S. Podunavac-Kuzmanović et al.

[16]

[17]

[18]

[19] [20]

[21]

[22]

based on Na+ and K+ concentrations from human vitreous humour by linear least squares and artificial neural networks modelling.” Australian Journal of Forensic Sciences 46:166-179. Jokić, Aleksandar I., Jovana A. Grahovac, Jelena M. Dodić, Siniša N. Dodić, Stevan D. Popov, Zoltan Z. Zavargo, Damjan G. Vučurović. 2012. "Artificial neural network approach to modelling of alcoholic fermentation of thick juice from sugar beet processing." Hemijska industrija 66:211-221. Miller, James N., and Jane C. Miller. 2010. Statistics and Chemometrics for Analytical Chemistry. Harlow: Pearson Education Limited. Gonçalves Maltarollo, Vinícius, Káthia Maria Honório, and Albérico Borges Ferreira da Silva. 2013. "Applications of Artificial Neural Networks in Chemical Problems." In Artificial Neural Networks Architectures and Applications, edited by Kenji Suzuki, 203-223. London: IntechOpen. Priddy, Kevin L., and Paul E. Keller. 2005. Artificial Neural Networks – An Introduction. USA: SPIE. Anderson, Dave, and George McNeill. 1992. Artificial Neural Networks Technology. A DACS State-of-the-Art Report. New York: Rome Laboratory. Anglada, Maria Josep, and Josep Maria Bofill. 1998. "How good is a Broyden-Fletcher-Goldfarb-Shanno-like update Hessian formula to locate transition structures? Specific reformulation of BroydenFletcher-Goldfarb-Shanno for optimizing saddle points." Journal of Computational Chemistry 19:349-362. Kovačević, Strahinja Z., Sanja O. Podunavac-Kuzmanović, Lidija R. Jevrić, Evgenija A. Djurendić, and Jovana J. Ajduković. 2014. "Nonlinear assessment of anticancer activity of 17-picolyl and 17picolinylidene androstane derivatives – Chemometric guidelines for further syntheses." European Journal of Pharmaceutical Sciences 62:258-266.

Artificial Neural Networks as a Chemometric Tool …

115

[23] Garkani-Nejad, Zahra, and Naser Jalili-Jahani. 2010. "Modeling the activity of 2-phenylnaphtalene inhibitors using self-training artificial neural networks." Central European Journal of Chemistry 8:877-885. [24] Kovačević, Strahinja Z., Sanja O. Podunavac-Kuzmanović, Lidija R. Jevrić, Evgenija A. Djurendić, Jovana J. Ajduković, Slobodan B. Gadžurić, and Milan B. Vraneš. 2016. "How to rank and discriminate artificial neural networks? Case study: prediction of anticancer activity of 17-picolyl and 17-picolinylidene androstane derivatives." Journal of the Iranian Chemical Society 13:499-507. [25] Pentoś, Katarzyna, Deta Łuczycka, and Tomasz Kapłon. 2015. "The identification of relationships between selected honey parameters by extracting the contribution of independent variables in a neural network model." European Food Research and Technology 241:793801. [26] Tu, Jack V. 1996. "Advantages and disadvantages of using artificial neural networks versus logistic regression for predicting medical outcomes." Journal of Clinical Epidemiology 49:1225-1231. [27] Kashyap, Patanjali. 2018. Machine Learning for Deceision Makers: Cognitive Computing Fundamentals for Better Decision Making. New York: A press. [28] Hassoun, Mohamad. 2003. Fundamentals of Artificial Neural Networks. London: A Bradford Book. [29] Jayalakshmi, T., and A. Santhakumaran. 2011. "Statistical Normalization and Back Propagation for Classification." International Journal of Computer Theory and Engineering 3:1793-8201. [30] Singh, Dalwinder, and Birmohan Singh. 2019. "Investigating the impact of data normalization on classification performance." Applied Soft Computing 105524, In Press: doi.org/10.1016/j.asoc.2019. 105524. [31] Hang Cao, Xi, Ivan Stojkovic, and Zoran Obradovic. 2016. "A robust data scaling algorithm to improve classification accuracies in biomedical data." BMC Bioinformatics 17:1-10. [32] García, Salvador, Julián Luengo, and Francisco Herrera. 2015. Data Preprocessing in Data Mining. Switzerland: Springer/

116 S. Kovačević, M. Karadžić Banjac, S. Podunavac-Kuzmanović et al. [33] Malik, Kamal, H. Sadawarti, G. S. Kalra. 2014. "Comparative Analysis of Outlier Detection Techniques." International Journal of Computer Applications 97:12-21. [34] Yasri, A., and D. Hartsough. 2001. "Toward an Optimal Procedure for Variable Selection and QSAR Model Building." The Journal for Chemical Information and Computer scientists 41:1218-1227. [35] Pérez Gonzáleza, Maykel, Carmen Terána, Liane Saíz-Urrab, and Marta Teijeira. 2008. "Variable Selection Methods in QSAR: An Overview." Current Topics in Medicinal Chemistry 8:1606-1627. [36] Jevrić, Lidija R., Milica Ž. Karadžić, Anamarija I. Mandić, Sanja O. Podunavac Kuzmanović, Strahinja Z. Kovačević, Andrea R. Nikolić, Aleksandar M. Oklješa, Marija N. Sakač, Katarina M. Penov Gaši, Srđan Z. Stojanović. 2017. "Lipophilicity estimation and characterization of selected steroid derivatives of biomedical importance applying RP HPLC." Journal of Pharmaceutical and Biomedical Analysis 134:215-225. [37] Kovačević, Strahinja Z.., Sanja O. Podunavac Kuzmanović, Lidija R. Jevrić, Pavle T. Jovanov, Evgenija A. Djurendić, Jovana J. Ajduković. 2016. "Comprehensive QSRR modeling as a starting point in characterization and further development of anticancer drugs based on 17α-picolyl and 17(E)-picolinylidene androstane structures." European Journal of Pharmaceutical Sciences 93:1-10. [38] Karadžić, Milica, Sanja Podunavac-Kuzmanović, Lidija Jevrić, Strahinja Kovačević. 2015. "QSRR analysis in characterization of some benzimidazole derivatives." Acta Chimica Slovenica 62:747753. [39] Karadžić, Milica Ž., Lidija R. Jevrić, Sanja O. Podunavac Kuzmanović, Strahinja Z. Kovačević, Eva S. Lončar. 2015. "Structure-retention analysis of some 1,6-anhydrohexose and Daldopentose derivatives by linear multivariate approach." Journal of Liquid Chromatography and Related Technologies 38:662-669. [40] Karadžić, Milica Ž., Lidija R. Jevrić, Strahinja Z. Kovačević, Sanja O. Podunavac-Kuzmanović. 2015. "Retention data from normal-phase thin-layer chromatography in characterization of some 1,6-

Artificial Neural Networks as a Chemometric Tool …

[41]

[42]

[43]

[44]

[45]

[46]

[47]

117

anhydrohexose and d-aldopentose derivatives by QSRR method." Journal of Liquid Chromatography and Related Technologies 38:1044-1051. Karadžić, Milica Ž., Lidija R. Jevrić, Sanja O. Podunavac Kuzmanović, Eva S. Lončar, Strahinja Z. Kovačević. 2015. "Structure-retention relationship study of 2,4-dioxotetrahydro-1,3thiazole derivatives." Journal of Liquid Chromatography and Related Technologies 38:1247-1253. Karadžić, Milica Ž., Lidija R. Jevrić, Sanja O. Podunavac Kuzmanović, Eva S. Lončar, Strahinja Z. Kovačević. 2015. "Lipophilicity estimation of some carbohydrate derivatives in TLC with benzene as a diluent." Journal of Liquid Chromatography and Related Technologies 38:1593-1600. Kovačević, Strahinja Z., Sanja O. Podunavac Kuzmanović, Lidija R. Jevrić, Eva S. Lončar. 2015. "Assessment of chromatographic lipophilicity of some anhydro-d-aldose derivatives on different stationary phases by QSRR approach." Journal of Liquid Chromatography and Related Technologies 38:492-500. Kovačević, Strahinja Z., Sanja O. Podunavac Kuzmanović, Lidija R. Jevrić. 2015. "Linear and nonlinear structure-retention relationship analysis of different classes of pesticides isolated from groundwater." Journal of Liquid Chromatography and Related Technologies 38:1426-1434. Kovačević, Strahinja Z.., Lidija R. Jevrić, Sanja O. Podunavac Kuzmanović, Eva S. Lončar. 2013. "Chemometric estimation of the RP TLC retention behaviour of some estrane derivatives by using multivariate regression methods." Central European Journal of Chemistry 11:2031-2039. Abba, S. I., A. G. Usman, Selin Işik. 2020. "Simulation for response surface in the HPLC optimization method development using artificial intelligence models: A data-driven approach." Chemometrics and Intelligent Laboratory Systems 201:104007. Krmar, Jovana, Milan Vukičević, Ana Kovačević, Ana Protić, Mira Zečević, Biljana Otašević. 2020. "Quantitative structure-retention

118 S. Kovačević, M. Karadžić Banjac, S. Podunavac-Kuzmanović et al.

[48]

[49]

[50]

[51]

[52]

[53]

[54]

relationship model for the determination of naratriptan hydrochloride and its impurities based on artificial neural networks coupled with genetic algorithm." Journal of Chromatography A 1623:461146. Micić, Darko, Sanja Ostojić, Lato Pezo, Stevan Blagojević, Branimir Pavlić, Zoran Zeković, Saša Đurović. 2019. "Essential oils of coriander and sage: Investigation of chemical profile, thermal properties and QSRR analysis." Industrial Crops and Products 138:111438. Dobričić, Vladimir, Jelena Savić, Katarina Nikolic, Sote Vladimirov, Zorica Vujić, Jasmina Brborić. 2017. "Application of biopartitioning micellar chromatography and QSRR modeling for prediction of gastrointestinal absorption and design of novel β-hydroxy-βarylalkanoic acids." European Journal of Pharmaceutical Sciences 10:280-284. Ibáñez, M., V. Borova, C. Boix, R. Aalizadeb, R. Badea, N. S. Thomaidib, F. Hernández. 2017. "UHPLC-QTOF MS screening of pharmaceuticals and their metabolitesin treated wastewater samples from Athens." Journal of Hazardous Materials 323:26-35. Mizera, Mikołaj, Anna Krause, Przemysław Zalewski, Robert Skibiński, Judyta Cielecka-Piontek. 2017. "Quantitative structureretention relationship model for the determination of naratriptan hydrochloride and its impurities based on artificial neural networks coupled with genetic algorithm." Talanta 164:164-174. Golubović, Jelena, Ana Protić, Biljana Otašević, Mira Zečević, 2016. "Quantitative structure-retention relationships applied to development of liquid chromatography gradient-elution method for the separation of sartans." Talanta 150:190-197. Mikulášek, Kamil, Kamil S. Jaroň, Petr Kulhánek, Miroslava Bittová, Jan Havliš. 2016. "Sequence-dependent separation of trinucleotides by ion-interactionreversed-phase liquid chromatography – A structureretention studyassisted by soft-modelling and molecular dynamics." Journal of Chromatography A 1469:88-95. Žuvela, Petar, Katarzyna Macur, J. Jay Liu, Tomasz Bączek. 2016. "Exploiting non-linear relationships between retention time

Artificial Neural Networks as a Chemometric Tool …

[55]

[56]

[57]

[58]

[59]

[60]

119

andmolecular structure of peptides originating from proteomes andcomparing three multivariate approaches." Journal of Pharmaceutical and Biomedical Analysis 127:94-100. Bade, Richard, Lubertus Bijlsma, Thomas H. Miller, Leon P. Barron, Juan Vicente Sancho, Felix Hernández. 2015. "Suspect screening of large numbers of emerging contaminants in environmental waters using artificial neural networks for chromatographic retention time prediction and high resolution mass spectrometry data analysis." Science of the Total Environment 538:934-941. Golubović, Jelena B., Ana D. Protić, Mira L. Zečević, Biljana M. Otašević. 2015. "Quantitative structure retention relationship modeling in liquid chromatography method for separation of candesartan cilexetil and its degradation products." Chemometrics and Intelligent Laboratory Systems 140:92-101. Mizera, Mikołaj, Alessandro Musenga, David A. Cowan, Leon P. Barron. 2015. "Prediction of HPLC retention times of tebipenem pivoxyl and its degradation products in solid state by applying adaptive artificial neural network with recursive features elimination." Talanta 137:174-181. Miller, Thomas H., Alessandro Musenga, David A. Cowan, Leon P. Barron. 2013. "Prediction of chromatographic retention time in highresolution anti-doping screening data using artificial neural networks." Analytical Chemistry 85:10330-10337. Fragkaki, A. G., E. Farmaki, N. Thomaidis, A. Tsantili-Kakoulidou, Y. S. Angelis, M. Koupparis, C. Georgakopoulos. 2012. "Comparison of multiple linear regression, partial least squares and artificial neural networks for prediction of gas chromatographic relative retention times of trimethylsilylated anabolic androgenic steroids." Journal of Chromatography A 1256:232-239. Fragkaki, A. G., A. Tsantili-Kakoulidou, Y. S. Angelis, M. Koupparis, C. Georgakopoulos. 2009. "Gas chromatographic quantitative structure–retention relationships of trimethylsilylated anabolic androgenic steroids by multiple linear regression and partial least squares." Journal of Chromatography A 1216:8404-8420.

120 S. Kovačević, M. Karadžić Banjac, S. Podunavac-Kuzmanović et al. [61] Cherkasov, Artem, Eugene N. Muratov, Denis Fourches, Alexandre Varnek, Igor I. Baskin, Mark Cronin, John Dearden, Paola Gramatica, Yvonne C. Martin, Roberto Todeschini, Viviana Consonni, Victor E. Kuz’min, Richard Cramer, Romualdo Benigni, Chihae Yang, James Rathman, Lothar Terfloth, Johann Gasteiger, Ann Richard, and Alexander Tropsha. 2014. "QSAR Modeling: Where Have You Been? Where Are You Going To?." Journal of Medicinal Chemistry 57:4977-5010. [62] Thomas, Gareth. 2003. Fundamentals of Medicinal Chemistry. West Sussex: John Wiley & Sons Ltd. [63] Zhong, Shifa, Jiajie Hu, Xudong Fan, Xiong Yu, and Huichun Zhang. 2020. "A deep neural network combined with molecular fingerprints (DNN-MF) to develop predictive models for hydroxyl radical rate constants of water contaminants." Journal of Hazardous Materials 383:121141. [64] Abel Kolawole, Oyebamiji, Fadare Olatomide A, and Semire Banjo. 2020. "Anti-gastric cancer activity of 1,2,3-triazolo[4,5-d]pyrimidine hybrids (1,2,3-TPH): QSAR and molecular docking approaches." Heliyon 6:e03561. [65] Anter, Ahmed M., Yasmine S. Moemen, Ashraf Darwish, and Aboul Ella Hassanien. 2020. "Multi-target QSAR modelling of chemogenomic data analysis based on Extreme Learning Machine." Knowledge-Based Systems 188:104977. [66] Almi, Imane, Salah Belaidi, Enfale Zerroug, Mebarka Alloui, Ridha Ben Said, Roberto Linguerri, and Majdi Hochlaf. 2020. "QSAR investigations and structure-based virtual screening on a series of nitrobenzoxadiazole derivatives targeting human glutathione-Stransferases." Journal of Molecular Structure 1211:128015. [67] Mozafari, Zeinab, Mansour Arab Chamjangali, and Mohammad Arashi. 2020. "Combination of least absolute shrinkage and selection operator with Bayesian Regularization artificial neural network (LASSO-BR-ANN) for QSAR studies using functional groups and molecular docking mixed descriptors." Chemometrics and Intelligent Laboratory Systems 200:103998.

Artificial Neural Networks as a Chemometric Tool …

121

[68] Sun, Yuqing, Shaoze Shi, Yaqiu Li, and Qi Wang. 2019. "Development of quantitative structure-activity relationship models to predict potential nephrotoxic ingredients in traditional Chinese medicines." Food and Chemical Toxicology 128:163-170. [69] Gholivand, Khodayar, Mahsa Pooyan, Fahimeh Mohammadpanah, Foroogh Pirastefar, Peter C. Junk, Jun Wang, Ali Asghar Ebrahimi Valmoozi, and Ahmad Mani-Varnosfaderani. 2019. "Synthesis, crystal structure and biological evaluation of new phosphoramide derivatives as urease inhibitors using docking, QSAR and kinetic studies." Bioorganic Chemistry 86:482-493. [70] Hdoufane, Ismail, Joanna Stoycheva, Alia Tadjer, Didier Villemin, Menče Najdoska-Bogdanov, Jane Bogdanov, and Driss Cherqaoui. 2019. "QSAR and molecular docking studies of indole-based analogs as HIV-1 attachment inhibitors." Journal of Molecular Structure 1193:429-443. [71] Hadni, Hanine, and Menana Elhallaoui. 2019. "Molecular docking and QSAR studies for modeling the antimalarial activity of hybrids 4anilinoquinoline-triazines derivatives with the wild-type and mutant receptor pf-DHFR." Heliyon 5:e02357. [72] Kovačević, Strahinja, Milica Karadžić, Sanja PodunavacKuzmanović, Lidija Jevrić, Evica Ivanović, and Matilda Vojnović. 2018. "Electrostatic and Topological Features as Predictors of Antifungal Potential of Oxazolo Derivatives as Promising Compounds in Treatment of Infections Caused by Candida albicans." Acta Chimica Slovenica 65:483-491. [73] Larif, Majdouline, Azeddine Adad, Rachid Hmammouchi, Abdelhafid Idrissi Taghki, Abdelmajid Soulaymani, Azzedine Elmidaoui, Mohammed Bouachrine, and Tahar Lakhlifi. 2017. "Biological activities of triazine derivatives. Combining DFT and QSAR results." Arabian Journal of Chemistry 10:946-955. [74] Karadžić, Milica Ž., Strahinja Z. Kovačević, Lidija R. Jevrić, and Sanja O. Podunavac-Kuzmanović. 2017. "Chemometric and QSAR analysis of some thiadiazines as potential antifungal agents." Acta Periodica Technologica 48:117-126.

122 S. Kovačević, M. Karadžić Banjac, S. Podunavac-Kuzmanović et al. [75] Ghamali, Mounir, Samir Chtita, Abdellah Ousaa, Bouhya Elidrissi, Mohammed Bouachrine, and Tahar Lakhlifi. 2017. "QSAR analysis of the toxicity of phenols and thiophenols using MLR and ANN." Journal of Taibah University for Science 11:1-10. [76] Kovačević, Strahinja Z., Sanja O. Podunavac-Kuzmanović, Lidija R. Jevrić, Vladimir V. Vukić, Marina P. Savić, and Evgenija A. Djurendić. 2016. "Preselection of A- and B- modified D-homo lactone and D-seco androstane derivatives as potent compounds with antiproliferative activity against breast and prostate cancer cells – QSAR approach and molecular docking analysis." European Journal of Pharmaceutical Sciences 93:107-113. [77] Adhikari, Nilanjan, Amit Kumar Halder, Achintya Saha, Krishna Das Saha, and Tarun Jha. 2015. "Structural findings of phenylindoles as cytotoxic antimitotic agents in human breast cancer cell lines through multiple validated QSAR studies." Toxicology in Vitro 29:1392-1404.

In: A Comprehensive Guide … Editor: Steffen Skaar

ISBN: 978-1-53618-466-2 © 2020 Nova Science Publishers, Inc.

Chapter 4

RIVER WATER QUALITY MODELLING USING ARTIFICIAL INTELLIGENCE TECHNIQUES Eda Göz1, Erdal Karadurmuş2 and Mehmet Yüceer3, 1

Faculty of Engineering, Department of Chemical Engineering, Ankara University, Ankara, Turkey 2 Faculty of Engineering, Department of Chemical Engineering, Hitit University, Çorum, Turkey 3 Faculty of Engineering, Department of Chemical Engineering, Inonu University, Malatya, Turkey

ABSTRACT Water pollution has become a major issue in rivers. The possibility of a pollutant to be discharged to the river as municipal and industrial waste is an important problem for those using water from rivers. Nevertheless, due to the rapid population growth in the world and the irresponsible use of water resources, the world will face a serious lack of water in the near future. Therefore, the water resources of the future must be preserved very 

Corresponding Author’s Email: [email protected].

124

Eda Göz, Erdal Karadurmuş and Mehmet Yüceer well to leave healthy and enough water for next generations. In order to prevent river water pollution, river water quality should be constantly monitored and evaluated. This way, information on the status of water quality may be obtained, and river basin management planning may be carried out. For this purpose, measurement at points can be made, or online monitoring stations can be established on river basins. According to the collected data, management actions may be created for how waterways function and how pollutants affect evaluation. In addition to this effect, seasonal changes and long-term trends must be taken into consideration. Artificial intelligence (AI) techniques have been used recently in many engineering fields. The most widely used ones among AI techniques are artificial neural networks (ANN). These are followed by support vector regression (SVR), least squares support vector regression (LS-SVR), least squares support vector machine (LS-SVM) and fuzzy logic. In the past 15 years, extreme learning machine (ELM) and its types have been used in development of many forecasting models. The statistical accuracy of classical models is commonly poor because natural systems tend to be complex and nonlinear for deterministic modelling methods. AI techniques provide a fast and flexible means of creating models for estimation of river water quality. In recent years, AI techniques have shown exceptional performance as regression tools, especially when used for pattern recognition and function estimation. In this study, AI techniques will be applied to river water quality data, and AI models will be developed. The data were collected from an on-line measurement station that was established on the Yeşilırmak River in Amasya/Turkey. In the selected region, two different measurement stations were built at a distance of about 28 km. Twelve parameters as luminescent dissolved oxygen (LDO), pH, conductivity, nitrate nitrogen (NO3-N), ammonium nitrogen (NH4-N), total organic carbon (TOC), chloride, orthophosphate, temperature, turbidity, suspended solid and flow rate were measured at five-minute intervals at these stations. Specifically, two different models as DO prediction and TOC prediction models were developed with five different approaches. These approaches were artificial neural network (ANN), support vector regression, least squares support vector regression, extreme learning machine and kernel extreme learning machine. Model performances were evaluated with some performance indices. This study is a state-of-the-art study due to the fact that parameters that are expensive to measure can be predicted from parameters that are cheaper to measure. For this reason, it will contribute significantly to reducing the cost of on-line measurement stations planned to be established in the future.

Keywords: artificial intelligence, river water quality monitoring, real-time monitoring

River Water Quality Modelling …

125

INTRODUCTION There are many types of pollution that affect people's daily life such as air pollution, water pollution, land pollution (soil pollution), noise pollution, radioactive/nuclear pollution, thermal pollution and marine pollution/ocean pollution. Environmental pollution has become an important issue on a global scale as it is the most important problem we are dealing with nowadays. In fact, environmental problems started with the industrial revolution in the 19th century. Moreover, the rapid increase in population and irresponsible use of resources by people have caused many toxins and plastic materials and disrupted the ecological balance. Water pollution examined in the environmental pollution category occurs when domestic or industrial wastes mix into lakes, rivers, seas and oceans. Pollutants may be organic, inorganic or radioactive and cause contamination either directly or indirectly. Due to the fact that some pollutants dissolve in water sources, the ecological food chain is threatened, and the water quality decreases. Water pollution directly threatens life, and for this reason, academic studies have been carried out, and governments have been taking measures within the framework of a number of laws and regulations. Within this context, a lot of laws have been enforced in Europe, there are many directives in the European Union environmental legislation, and there are numerous agencies and organization such as the International Water Association (IWA) and the World Water Forum (WWF). Developing effective management and planning of water resources and monitoring of pollution control are very difficult because of the fact that many conflicting factors and parameters are interrelated to each other in real processes. It is an effective tool to develop a mathematical model for such processes with dynamic characteristics. Mathematical models provide robust management and planning, as well as allowing us to understand the relationship and combination of various complex parameters. A number of mathematical models that have been developed related to environmental systems present some issues such as simulation of process and mitigation of pollutants. Models that are the critical element of model-based decision systems have a deterministic and stochastic structure. Models with ordinary

126

Eda Göz, Erdal Karadurmuş and Mehmet Yüceer

differential equations and a statistical structure are often unable to explain complex relationships between parameters. In order to overcome these problems, prediction models have been developed by using various machine learning methods and artificial intelligence approaches in recent years. These models provide ease of use and flexibility in defining relationships between parameters in comparison to models with a deterministic structure. Artificial intelligence models consist of artificial neural network (ANN), fuzzy-based models, support vector machine (SVM), complementary model, natural algorithms (particle swarm optimization, genetic algorithm, ant colony optimization, etc.) and others such as M5 tree, K-mean and Random forest (RF) (Tiyasha et al. 2020). When studies on river water pollution control with artificial intelligence methods are examined, applications of artificial neural networks are frequently encountered. In the study by Sarkar and Pandey (2015), the artificial neural network approach was used to estimate the dissolved oxygen (DO) concentration of River Yamuna in Mathura City. In their study, the feed forward error back propagation technique was selected. Samples were collected from different locations as Mathura (upstream), Mathura (central) and Mathura (downstream), and three different models were developed. Flow rate (Q), temperature, pH, biochemical oxygen demand (BOD) and dissolved oxygen (DO) were selected as the input variables to predict the DO level. The performance of the model was evaluated with root mean square error and coefficient of correlation, and a high correlation was observed between the predicted and measured values (Rtest: 0.928 Rtrain: 0.907). In a study carried out by Nemati (Nemati et al. 2015), multiple linear regression (MLR), ANFIS and ANN algorithms were used. In the study where dissolved oxygen concentration was predicted, chlorine, pH, conductivity, temperature, NO2-N, Ptotal, NH4N and NO3-N were chosen as the input variables. The performances of the ANN and MLR models were found to be better than that of ANFIS. In a study where ammonium nitrogen (NH3-N) was estimated, ANN, ANFIS and NARX models were proposed (Chang et al. 2015). pH, conductivity, DO, BOD, COD, total suspended solid (TSS), flow rate, Rpre, temperature, total organic carbon (TOC), total phosphate and nitrogen, NO3-N and NO2 were selected as the input variables. In Salami and Ehteshami’s study (2015), an

River Water Quality Modelling …

127

ANN structure was developed to forecast DO levels in Rivers Ireland. The data used in their study were collected from 210 stations in Ireland. Six package models as dissolved oxygen, DO percentage, biological oxygen demand, chloride, alkalinity and total hardness were proposed using temperature, pH and conductivity as the input variables, which are parameters that are relatively easier and cheaper to measure. Chang et al. (2016) developed a systematic model to define the phosphorous concentration in Dahan River in Taiwan. For this purpose, NARX, BPNN and MLR models were developed. In another study where DO prediction was carried out, the MLR, BPNN and generalized recurrent neural network (GRNN) techniques were developed for DO estimation in Danube River (Csabragi et al. 2016). According to the performance results, GRNN was found as the best model. DO level prediction in Surma River in Bangladesh was carried out by Ahmed (2017). Two different network structures as FFNN and RBFNN methods were proposed, and the FFNN approach had better results than RBFNN. Olyaie et al. (2017) used various machine learning methods as ANN, MLP, RBF, LGP and SVM implemented for prediction of DO levels in Delaware River in USA. SVM was found as the superior model in their comparative study. In order to predict water quality, Bozorg-Haddad (2017) used LS-SVR and genetic algorithm implemented with LS-SVR. According to the results, the GA implementation with LSSVR was superior to the other. In Abba et al.’s study (2018), MLR, ANN and ANFIS models were compared for DO prediction in Yamuna River. ANN was found as the most successful among the others. In a study examining the effects of artificial intelligence studies on river pollution, extreme learning machine (ELM), BPNN, linear regression and ANFIS were used to predict chl-a in Nakdong River (Yi et al. 2018). ELM was defined as the most successful model for chl-a prediction. In order to forecast water temperature of a river, three different approaches as ELM, MLP and MLR were proposed by Zhu et al. (2019). ELM and MLP had higher performance. SVM and MLR model performances were compared to estimate the water temperature in Tungabhadra River by Rehana (2019). In order to predict chla as the primary indicator for algal bloom in James River, SVM and LSSVM models were developed. In another study using the SVM and LS-SVM

128

Eda Göz, Erdal Karadurmuş and Mehmet Yüceer

methods, the water quality index was estimated by Leong et al. (2019). Three different kernel functions as linear, polynomial and RBF type kernel were tested, and the best results were obtained in the LS-SVM algorithm with a polynomial kernel. In order to create an early alarm system for DO levels, the SVM and ANN methods were used by Yahya et al. (2019). pH, TSS, DO, NH3-N, COD and BOD data were used as the input variables. SVM was able to predict the DO level and create an alarm system for river pollution monitoring. In this study, five different artificial intelligence techniques as ANN, SVR, LS-SVR, ELM and kernel extreme learning machine (KELM) were tried to develop DO and TOC prediction models. The data used in the algorithm were collected from real-time monitoring stations as a first practice in Turkey. The data were collected at 5-minute intervals at the stations’ database and sent to Ankara University Central Office. There, the data were processed. In DO modelling, conductivity, pH and temperature parameters, which are relatively easy and cheap to measure, were selected as the input variables. In TOC modelling, conductivity, pH, DO and temperature were used as the input variables. While LS-SVR was the most suitable algorithm in both developed models according to some statistical parameters, kernel ELM had no suitable result.

METHODS Modelling Approaches

Artificial Neural Network (ANN) An artificial neural network (ANN) is known as an information processing unit, and it is inspired from the biological nervous system mechanism. The first artificial neural network was proposed by McCulloh and Pitts in 1943 (McCulloh and Pitts 1943). They presented a neuron model that was used to perform any computable function by defining a finite number of artificial neurons and adjusting the synaptic weights. Modelling with the neural network approach has been defined as a very promising

River Water Quality Modelling …

129

system especially in many forecasting applications and classification problems because of its remarkable learning ability. Moreover, this approach has been used in various areas from medicine to engineering applications (Karadurmuş et al. 2018, Boztepe et al. 2020). Specifically, it is used for classification, pattern recognition, image indexing and retrieval, forecasting problems, robustness and fault tolerance problems. An artificial neural network is very successful in some complex problems such as extracting patterns and detecting trends which many computer techniques cannot solve. For this reason, a trained network structure can be defined as an “expert” in the category of information to be analyzed, and it can be used for answering “what if” questions (Zakaria et al. 2014). This method imitating the work of the human brain consists of artificial neurons known as process units. Actually, the “network” definition in Neural Network means interconnection of neurons in various layers of a system. The artificial neural network structure consists of three layers as the input, hidden and output layers. The neurons in the input layer receive the data and transfer the data to the hidden layer via synapses. After the processing of data, the hidden layer transfers them to the next layer via more synapses. The synapses store weight values. The output of the network is obtained from the neurons in the last layer. A typical ANN structure is given in Figure 1. In general, an artificial neural network can be formed based on some characteristics. The first one is the number of layers and the number of nodes in each layer. The second is the learning mechanism used in the training of the network, while the last is the activation function used in the layers. The most commonly used activation functions are sigmoid and linear functions. Linear transfer function (purelin): f(m) = m Logarithmic sigmoid function (logsig): f(m) = 1/(1 + e−m ) Hyperbolic tangent sigmoid function (tansig): f(m) = 2/(1 + e−2m )

130

Eda Göz, Erdal Karadurmuş and Mehmet Yüceer

Figure 1. Typical ANN structure.

In artificial neural network design, the signals pass through the neurons, and simultaneously, the weight and transfer functions are modified, and this is repeated until the desired output value is achieved. The flow diagram of training a neural network algorithm is given in Figure 2.

Figure 2. Flowchart of ANN algorithm

River Water Quality Modelling …

131

Artificial neural networks may have a feed-forward and feed-back network structure depending on the way the neurons connect to each other. In a feed forward neural network structure -more frequently preferred due to nonlinear behavior of the feed-back network structures, neurons are placed in regular layers from the input to the output layer as shown in Figure 3. In this structure, after entering the input layer, the data move towards the hidden layer and the output layer and reach the outer world.

Figure 3. Feed-forward neural network structure.

In a feed-back network, the output of a neuron is connected as an input to the neuron before it or in its own layers as shown in Figure 4.

Figure 4. Feed-back neural network structure

132

Eda Göz, Erdal Karadurmuş and Mehmet Yüceer

Learning algorithms, an important topic in ANN design, may be collected under three headings. These are supervised learning, unsupervised learning and reinforcement learning. Supervised learning is defined as a machine learning algorithm that specifies the network parameters by using the training data. The learning task of the artificial neural network means defining the value of its parameters based on any valid input value after having seen the output value. Training data involve the input and desired output values in a data vector form. Supervised learning is mostly used in classification studies. The second method is unsupervised learning, which is also a machine learning technique. In this approach, the parameters of a network are set by using the data, and the cost function is minimized. Unsupervised learning, which explores how to organize data, is often used in estimation problems. Its difference to supervised learning and reinforcement learning methods is that the artificial neural network is given only unlabeled examples (Zakaria et al. 2014). The reinforcement learning method is a method where data are not provided interaction with the environment. Reinforcement is mostly used as an overall learning algorithm of artificial neural networks. This approach is applied in robot control, telecommunications, etc. In order to train the network, different optimization algorithms may be used. Among these, the Levenberg-Marquardt optimization method (‘trainlm’), which is a fast-response and highly preferred method, has been chosen for training the network and adjusting the weights. The Bayesian regularization backpropagation (‘trainbr’) and BFGS quasi-Newton backpropagation (‘trainbfg’) optimization methods may be alternative methods besides the Levenberg Marquardt optimization method (Boztepe et al. 2020). In order to train an artificial neural network structure, some algorithms have been used. Although the most commonly used is the backpropagation algorithm, radial basis function (RBF), autocorrelation function, selforganizing map (SOM) neural network and Hop-field networks have also been used.

River Water Quality Modelling …

133

Support Vector Machine (SVR) The support vector machine is a kernel-based statistical learning methodology and a powerful tool for data classification and regression problems. This technique was proposed for the first time by Smola and Schölkopf (2004). A schematic representation of the SVM algorithm is given in Figure 5.

Figure 5. The basic principle of SVM algorithm (Zhang et al. 2014).

As seen in Figure 5, the input support vectors (n) and nonlinear operation of N support vectors are in the first and second layers, respectively. If one takes a sample data set defined by an n-dimensional vector for nonlinear problems, then, values in a given N domain are expressed as follows (Equation (1)): (x1 , y1 ) (x2 , y2 ) , … … . , (xN , yN ) ∈ Rn xR

(1)

In order to map samples from former space to future space, nonlinear mapping (ψ(.)) is defined as in Equation (2). ψ(x) = (ϕ(x1 ) , ϕ(x2 ) . . . . . . ϕ(xN ))

(2)

134

Eda Göz, Erdal Karadurmuş and Mehmet Yüceer The optimal decision function can be written as follows (Equation (3)): y( x) = wϕ(x) + b

(3)

In this equation, w is a weight value vector, and b is a threshold value. One way to ensure this is to minimize the norm. This problem may be written as a convex optimization problem (Equation (4.a & 4.b)): 1

Minimize 2 ‖w‖2 Subject to {

{yi − ⟨w, xi ⟩ − b ≤ ε } ⟨w, xi ⟩ + b − yi ≤ ε

(4.a)

(4.b)

If the function f actually exists that approximates all data pairs (xi , yi ) with ε precision, the convex optimization problem is possible. If this does not happen, some errors may be allowed. In this state, a “soft margin loss function” is used in the support vector machine (Cortes and Vapnik 1995). In order to overcome infeasible constraints of the optimization problem, slack variables (ξi , ξ∗i ) are used. Therefore, the final formulation (Equations (5a & 5b)) is obtained (Vapnik 1995). 1 2

Minimize ‖w‖2 + C ∑li=1(ξi + ξ∗i )

(5.a)

{yi − ⟨w, xi ⟩ − b ≤ ε + ξi Subject to {⟨w, xi ⟩ + b − yi ≤ ε + ξi∗ } ξi , ξ∗i ≥ 0

(5.b)

The constant C in this equation is positive (C > 0), and it determines the balance between the flatness of f and the amount up to which deviations larger than ε are tolerated (Smola and Schölkopf 2004). Thus, the insensitive loss function may be described as in Equation (6);

River Water Quality Modelling … 0 if|ξ| ≤ ε |ξ|ε = { |ξ| − ε otherwise

135 (6)

Least Squares Support Vector Machine (LS-SVM) Nonparametric regression is a very flexible and popular method for data analysis. Therefore, various tools have been put forward to estimate the nonlinear relationships between variables. However, these methods cause computational load, especially in large data sets. In order to decrease these difficulties, LS-SVM methods have been used to be capable of handling a large data set. LS-SVM is a reformulation of SVM. LS-SVM is defined as a ridge-type learning machine, and it has been used in many areas for the last few years because of its simplicity (Wu et al. 2006, Suykens and Vandewalle 1999, Suykens and Vandewalle 2000, Suykens 2001, Suykens et al. 2002a, Suykens et al. 2002b). In many interdisciplinary studies, LS-SVM is used for the purposes described below. LS-SVM uses a regularized least square function with equality constraints leading to a linear system which meets the Karush-Kuhn-Tucker (KKT) conditions for the optimal solution. LS-SVM is a simplified version of SVM, but the kernel parameters are very important in the regression system. For this reason, in order to select model parameters properly, a suitable method is used. The standard least squares support vector machine algorithm is given in the following equations (Tripathi et al. 2006; Güneşoğlu and Yuceer 2018) {(x1 , y1 ) … . . (xk , yk )} to be the training set The regression model may be written as follows using the nonlinear mapping function (Equation (7)) Y = w T Φ(x) + bw ϵRN bϵR ΦϵRN RM M → ∞

(7)

In this equation, w is the weight vector, and b is the bias value. When the least squares support vector is used as an approximate function, the optimization problem of LS-SVM takes the following form (Equation (8)):

Eda Göz, Erdal Karadurmuş and Mehmet Yüceer

136

1

γ

2 minw,e J (w, e) = 2 W T w + 2 ∑N K=1 ek

(8)

The constraint for this problem is given in the following Equation (9): yk = w T Φ(xK ) + b + ek k = 1,2, … , N

(9)

In this equation, the regulation parameter provides the balance between model complexity and training error. On the other hand, it is the error value based on the difference between the additional actual value and the calculated value. To solve the optimization problem with the constraint function, the Lagrange function is defined as in Equation (10). T L(w, b, e, α) = J(w, e) − ∑N k=1 αk {w Φ(xk ) + b + ek − yk }

(10)

The constant (γ) given in the equation below is the Lagrange multiplier, and it is defined as support vectors. In order to solve the equation below, the partial derivatives of the equation are taken for each of the parameters and equalized to zero. These equations are given collectively in the following Equations (11.a-c). ∂L ∂w

= 0 ⇒ w = ∑N k=1 αk Φ(xk )

(11.a)

∂L ∂b

= 0 ⇒ ∑N k=1 αk = 0

(11.b)

∂L ∂ek

= 0 ⇒ αk = γek

(11.c)

The equation in a linear form obtained from the equations above is found as in Equation (12) (Suykens, 2001) 0 [ ⃗1

⃗T 1 b ] [ ] = [0] −1 Z+γ I α

(12)

River Water Quality Modelling …

137

In this equation y = [y1 , … , yN ], ⃗1 = [1, … . ,1] α = [α1 … … … αN ] Z = {Zkj | k, j = 1, … , N}

Zk,j = F(xk )T F(xj ) = K(xk , xj ) k, j = 1, … , N

(13)

K(xk , xj ) shown in Equation (13) is defined as the kernel function. Thus, the least squares support vector machines take the form of the expression given in Equation (14). y(x) = ∑N k=1 αk K(x,xk ) +b

(14)

Extreme Learning Machine (ELM) The extreme learning machine method was first developed by Huang (Huang et al. 2006). This method is a very useful algorithm for training single hidden layer feed-forward neural networks (SLFNs). A single hidden layer feed-forward neural network has a powerful learning ability, and it can solve some problems that cannot be solved by using traditional methods. In fact, the problem with this is that a fast algorithm has not been developed. In order to fill this gap, the extreme learning machine was proposed, and it has been commonly used in machine learning, image processing and other areas (Ding et al. 2014). Conventional feed-forward network structures use the gradient descent method to train the network and can be attached to the local minimum. On the other hand, the learning speed is quite slow since all the parameters of the network are determined iteratively. However, in the extreme learning machine, the hidden layer neuron weights and bias values are determined randomly, and the output weight value is simply calculated by a matrix transformation solution. Due to these features of the extreme learning machine and less user intervention, the algorithm has a relatively fast and often better generalization ability than traditional learning algorithms. However, the extreme learning machine may require more neurons in the hidden layer for high-dimensional data. A schematic

138

Eda Göz, Erdal Karadurmuş and Mehmet Yüceer

representation of the extreme learning machine algorithm is given in Figure 6.

Figure 6. The architecture of ELM structure (Albadr and Tiun, 2017).

The algorithm of the extreme learning machine (ELM) is given below: Input: The training set, activation function and number of hidden layer neurons (N) Output: The weight value between the hidden layer and the output layer, this value is calculated by the Moore-Penrose matrix inversion method. Step 1: Hidden layer parameters are determined randomly. Step 2: The output matrix of the hidden layer is calculated. Step 3: The output layer weight value is calculated (β = H† T) According to the three steps given above, the extreme learning machine consists of two stages: At the first stage, the parameters of the hidden layer are created, which match the input data with the hidden layer activation function. At the second stage, the β weight values are solved. Considering the situation where there are N input variables in the extreme learning machine: (xi , t i )|xi ϵRn , t i ϵRn i = 1,2, … , N

River Water Quality Modelling …

139

Accordingly, the output function of the network structure with L hidden layers is defined by Equation (15). fL (x) = ∑N i=1 βi hi (x) = h(x)β

(15)

In Equation (15), β = [β1 , β2 , ⋯ , βL ] is the weight value between the hidden layer and the output layer, and h = [h1 , h2 , ⋯ , hL ] is the hidden layer output vector. In this context, the output weight value should be minimized to reduce the training error and increase the generalization performance (Equation 16). Minimum ‖Hβ‖, ‖β‖

(16)

To calculate the output weight value, the least squares method is used for the solution, and Equation (17) is obtained. 1

−1

−1

β = H T (C + HH T )

T

(17)

In Equation (17), H shows the hidden layer output matrix, C is the editing parameter, and T is the expected output matrix of the samples. According to all these calculations, the output function of the network is given by Equation (18): 1

f(x) = h(x)H T (C + HH T )

(18)

Different types of activation functions may be used in the extreme learning machine algorithm. These functions may be listed as sinusoidal, sigmoid, radial based, hyperbolic tangent and gaussian functions.

Kernel Extreme Learning Machine In the kernel-based extreme learning machine method, the kernel functions replace the activation functions in the hidden layer. In the kernelbased extreme learning machine, the hidden layer is unknown, and the kernel

140

Eda Göz, Erdal Karadurmuş and Mehmet Yüceer

matrix is expected to meet the Mercer conditions given by Equation (19) (Li et al., 2014). M = HH T = mij h(xi )h(xj ) = k(xi , xj )

(19)

Accordingly, the output function is calculated by Equation (20): 1

−1

f(x) = [k(x, x1 ), … , k(x, xN )] (C + M)

T

(20)

k(x, xN ) is defined as the kernel function in Equation (20). Different types of Kernel functions have been proposed in the literature (Li et al. 2014). The most commonly used are gaussian kernel, linear kernel, hyperbolic tangent kernel, polynomial kernel and radial basis kernel functions.

Data Collection and Study Area The data used in this study were collected from real-time on-line measurement stations near the Yeşilırmak River. The Yeşilırmak River is one of the great rivers disembogue the Black Sea, Turkey. Since the regions where the Yeşilırmak river is located are areas where agricultural activities are quite intense, nitrogen and phosphorous fertilizers are mixed into the river, and they increase the organic load. The system had two in-situ on-line monitoring stations. The study area has various pollution sources due to industrial, domestic, municipal and agricultural activities. The sampling stations were selected particularly according to the sources. The first station was built in the Aynalı Cave region after the sewage system and the Tersakan stream which has a high pollution load, while the other one was in the Administration of Hydraulic Works’ Durucasu station which had a 26.876 km distance from the first station and after the yeast factory. The study area is shown in Figure 7.

River Water Quality Modelling …

141

Figure 7. Study area and data collection.

The images of the inside/outside and center where the data were collected in Ankara are given in Figure 8.

Figure 8. Pictures of inside and outside views of the station, sample collection system, central office.

142

Eda Göz, Erdal Karadurmuş and Mehmet Yüceer

The water sample was taken from the river into the tank inside the station as seen in the picture above. Conductivity and temperature values were measured in the tank with a probe. After this, the samples were sent to other analyzers where the pollution parameters were measured. All data were collected in a database inside station and transferred to the monitoring Center in Ankara via a General Packet Radio Service (GPRS) transmission channel. The data were continuously monitored and displayed in different formats at this Central Office. Real-time data were measured at five-minute time intervals for the parameters of luminescent dissolved oxygen (LDO), pH, conductivity, nitrate nitrogen (NO3-N), ammonium nitrogen (NH4-N), TOC, o-phosphate, chloride, temperature, turbidity, suspended solid and flow rate.

RESULTS Artificial Neural Network (ANN) Model Results In this modelling, a three-layer feed-forward backpropagation network structure was used to estimate the dissolved oxygen (DO) concentrations and total organic carbon (TOC) concentrations. The number of neurons in the input layer of the network was 3, the number of neurons in the hidden layer was 15, and the number of neurons in the output layer was 1 for modelling of DO. The number of neurons in the input layer of the network was 4, the number of neurons in the hidden layer was 30, and the number of neurons in the output layer was 1 for modelling of TOC. The log sigmoid transfer function and the linear transfer function were used respectively in the hidden and output layers. The number of hidden layers and the number of neurons in the layers were determined by the trial-and-error method. The LevenbergMarquardt optimization method (‘trainlm’), which is a fast-response and highly preferred method, was chosen for training the network and adjusting the weights.

River Water Quality Modelling …

143

As a result of experimental studies, a total of 6500 units of data were obtained, and these data were normalized before being introduced to the network as it would increase the success of the model. The data were randomly divided into two parts. 70% of all data were used for network training, and the remaining data were used to test the model. The ANN model-1 utilized experimental data as an input set to identify the effects of conductivity, pH and temperature on the dissolved oxygen concentrations and total organic carbon concentrations of the Yeşilırmak river. The ANN model-2 utilized experimental data as an input set to identify the effects of conductivity, pH, dissolved oxygen concentration and temperature on the total organic carbon concentrations in the Yeşilırmak river. MATLAB and Neural Network Toolbox were used to develop the ANN model. The developed ANN model was tested with test data that the network did not see, and the success of the model was measured by statistical techniques.

Support Vector Regression (SVR) Model Results The Support Vector Regression (SVR) model proposed by Smola and Schölkopf (2004) is the regression adaptation of SVM, which is commonly used for classification problems. Two situations that can be encountered in SVMs are that the data are in a structure where they can be separated linearly or in a structure that they cannot be separated linearly. 6500 data units were randomly divided into two parts as the training and test sets. 70% of all data were used for training, and the remaining data were used to test the model. The same training and test data sets that were evaluated for ANN and LSSVM were used for the SVR model. For developing SVR, we used the sequential minimal optimization (SMO) and radial basis kernel function (RBF) for solving the optimization problem. The developed SVR model was tested with test data that the network did not see, and the success of the model was measured by statistical techniques.

144

Eda Göz, Erdal Karadurmuş and Mehmet Yüceer

Least Squares Support Vector Machine (LS–SVM) Model The least squares support vector machine (LS-SVM) is an optimized algorithm based on the standard SVM which requires solution of a linear equation set instead of the long and computationally hard quadratic programming problem involved by the original SVM. The optimization problem in LS-SVM is solved by considering the Lagrange multipliers. These multipliers should be positive in the standard SVM method, while negative values can be obtained in the LS-SVM method. In LS-SVM modelling, two parameters that need to be calibrated are needed, namely the regulation factor (γ) and the width parameter (σ) of the radial basis function (RBF). For DO modelling, the optimal values of these parameters γ and σ 2 were obtained as 800 and 0.0135, respectively. For TOC modelling, the optimal values of these parameters γ and σ2 were obtained as 154 and 0.0217, respectively. In this model, the most appropriate LS-SVM parameters were determined by the k-fold cross-validation technique to minimize the mean error squares. Further details of the LS-SVM algorithm are available in the literature The data were randomly divided into two parts as the training and test sets. 70% of all data were used for network training, and the remaining data were used to test the model. The developed LS-SVM model was tested with test data that the network did not see, and the success of the model was measured by statistical techniques.

Extreme Learning Machine and Kernel Extreme Learning Machine Results The extreme learning machine has been used since 2006. This algorithm is much faster than traditional algorithms due to less human intervention. The algorithm's only drawback is its requirement of too many neurons. In this study, two different models were used, where the first one was the TOC model, and the second one was the DO model. After preprocessing of data, 6500 data units were obtained, and these data were normalized. The data were divided into 70% -30% training and test data. The input and output variables and all data were the same in all algorithms. In both studies, 5 different activation functions and different neuron numbers were tried.

River Water Quality Modelling …

145

Within this scope the hardlim, sine, sigmoid, radbas and tribas activation functions were tested. The most successful TOC prediction model was obtained with the tribas activation function, in which there are 200 neurons in the hidden layer (Rtest/Rtrain:0.9048/0.9284 MAPEtest/MAPEtrain:6.4248/5.25859, RMSEtest/RMSEtrain:1.1693/1.0073). In the DO modelling study, a high correlation coefficient was reached in the presence of the tribas activation function. (Rtest/Rtrain:0.9403/0.9654, MAPEtest/MAPEtrain:2.0734/1.4959, RMSEtest/RMSEtrain: 0.2288/0.1684). While examining the Kernel ELM modelling for DO and TOC prediction, the obtained results were not as successful as the other algorithms. In this study, two different kernel functions as RBF kernel and linear kernel were tried. However, due to the correlation coefficient obtained around 0.60 and high MAPE values, consistent results could not be obtained.

Model Computation Using the experimental DO and TOC data, the ANNs, LS-SVM and SVR methods were implemented, and the success of the developed models were tested by statistical methods. The comparison of the experimental data and the ANNs, LS-SVM, SVR and ELM model predictions is presented in Figures 9-16. To evaluate the success of the models, four statistical parameters (i.e., correlation coefficient (R), root mean square error (RMSE) and Mean Absolute Percentage Error (MAPE) (%)) were calculated and are presented in Tables 1 and 2. The equations for the statistical evaluation functions used in this study are given below (Equations (21-23)). The experimental values were determined as follows, where xm is an observed value at the ith time step, ym is a simulated value at the same moment of time, N is the number of time steps, x̅ is the mean value of observations, and y̅ is the mean value of simulations in these equations. R =

∑N ̅ )(ym −y ̅) m=1(xm −x √∑N ̅ )2 √∑N ̅) m=1(xm −x m=1(ym −y

2

(21)

Eda Göz, Erdal Karadurmuş and Mehmet Yüceer

146

RMSE = √

2 ∑N m=1(ym −xm )

(22)

N 1

MAPE (%) = N ∑N m=1(|

ym −xm |).100 xm

(23)

A higher value of the correlation coefficient (R) and smaller values of RMSE and MAPE would indicate a better performance of the model. To model the DO and TOC concentrations in the Yeşilırmak river, ANNs, LS-SVM, SVR, ELM and KELM models were used, and the results showed good agreement between the predictions and observations. When the success of these models is evaluated from the MAPE% values, it makes LS-SVM alternative to the SVR and ANN models in prediction of the nonlinear and complex DO and TOC concentrations in the Yeşilırmak river. LSSVM could be considered as a reliable method for modelling of DO and TOC concentrations in rivers and streams. Table 1. Performance indices achieved using ANN, LS–SVM, SVR and ELM during the training and test periods for DO modelling Model ANN LS-SVM SVR ELM

Training RMSE 0.1907 0.0449 0.1888 0.1684

MAPE (%) 1.7110 0.2589 1.3852 1.4959

R 0.9536 0.9975 0.9547 0.9654

Test RMSE 0.1987 0.1506 0.1937 0.2288

MAPE (%) 1.7858 0.6475 1.4848 2.0734

R 0.9515 0.9727 0.9542 0.9403

Table 2. Performance indices achieved using ANN, LS–SVM, SVR and ELM during the training and test periods TOC modelling Model ANN LS-SVM SVR ELM

Training RMSE 0.8140 0.1944 0.9203 1.0073

MAPE (%) 4.0959 0.8625 4.5509 5.5859

R 0.9535 0.9974 0.9408 0.9284

Test RMSE 0.9478 0.6012 0.9902 1.1693

MAPE (%) 4.6953 2.0483 4.9027 6.4248

R 0.9372 0.9751 0.9312 0.9048

River Water Quality Modelling …

(a)

147

(b)

Figure 9. Comparison between the observed and predicted values for dissolved oxygen. (a) Training and (b) test data for the SVR model.

(a)

(b)

Figure 10. Comparison between the observed and predicted values for dissolved oxygen. (a) Training and (b) test data for the LS-SVM model.

(a)

(b)

Figure 11. Comparison between the observed and predicted values for dissolved oxygen. (a) Training and (b) test data for the ANN model.

148

Eda Göz, Erdal Karadurmuş and Mehmet Yüceer

(a)

(b)

Figure 12. Comparison between the observed and predicted values for dissolved oxygen. (a) Training and (b) test data for the ELM model.

(a)

(b)

Figure 13. Comparison between the observed and predicted values for total organic carbon. (a) Training and (b) test data for the SVR model.

(a)

(b)

Figure 14. Comparison between the observed and predicted values for total organic carbon. (a) Training and (b) test data for the LS-SVM model.

River Water Quality Modelling …

(a)

149

(b)

Figure 15. Comparison between the observed and predicted values for total organic carbon. (a) Training and (b) test data for the ANN model.

(a)

(b)

Figure 16. Comparison between the observed and predicted values for total organic carbon. (a) Training and (b) test data for the ELM model.

CONCLUSION Due to the increasing environmental pollution in recent years, the interest in this topic in both academic and management circles has increased. On-line monitoring stations have become popular for monitoring data specifically for water pollution control. Environmental sampling is very laborious, complicated and expensive. For this reason, it is almost impossible to monitor continuous long-term water-quality time series data with complete properties at all sampling locations in a river system, and the setup of measurement stations is very costly. Other great difficulties in river

150

Eda Göz, Erdal Karadurmuş and Mehmet Yüceer

management are pollution patterns with high complexity, dynamism and non-linearity in both spatial and temporal scales. In order to overcome this problem, various management tools have been proposed, requiring predictive methods. In this study, five different machine learning methods were used for TOC and DO prediction models. A TOC analyzer is much more expensive than other analyzers. So, a successful TOC model will be very effective in reducing costs. The DO level in river water is very important for the continuity of ecological life and must be constantly monitored. Forecasting of these two important parameters from relatively easier and cheaper parameters in this study is important for future station implementations.

REFERENCES Abba, S. I., Hadi, S. J. and Abdullahi, J. (2018). River water modelling prediction using multilinear regression, artificial neural network, and adaptive neuro-fuzzy inference system techniques. Procedia Computer Science, 120: 75–82. Ahmed, A. A. M. (2017). Prediction of dissolved oxygen in Surma River by biochemical oxygen demand and chemical oxygen demand using the artificial neural networks (Anns). Journal of King Saud University Engineering Sciences. 29 (2): 151–158. Albadr, M. A. A. and Tiun, S. (2017). Extreme learning machine: A review. International Journal of Applied Engineering Research, 12 (14): 46104623. Bozorg-Haddad, O., Soleimani S. and Loáiciga, H. A. (2017). Modeling water-quality parameters using genetic algorithm-least squares support vector regression and genetic programming. Journal of Environmental Engineering, 143(7): 04017021. Boztepe C., Künkül, A. and Yüceer, M. (2020). Application of artificial intelligence in modeling of the doxorubicin release behavior of pH and temperature responsive poly (NIPAAm-co-AAc)-PEG IPN hydrogel. Journal of Drug Delivery Science and Technology 57: 101603.

River Water Quality Modelling …

151

Chang, F. J., Chen, P., Chang, L. and Tsai, Y. (2016). Estimating spatiotemporal dynamics of stream total phosphate concentration by soft computing techniques. Science of the Total Environment 562: 228–236. Chang, F. J., Tsai, Y., Chen, P., Coynel, A. and Vachaud, G. (2015). Modeling water quality in an urban river using hydrological factors Data Driven Approaches. Journal of Environmental Management 151: 87–96. Cortes, C. and Vapnik, V. (1995). Support-vector networks. Machine Learning, 20: 273-297. Csábrági, A., Molnár, S., Tanos, P. and Kovács, J. (2016). Forecasting of dissolved oxygen in the River Danube using neural networks. Hungarian Agricultural Engineering, 7410: 38–41. Ding, S., Xu, X. and Nie, R. (2014). Extreme learning machine and applications. Neuralcomputing and Applications, 25: 549-556. Güneşoğlu, S., and Yüceer, A. M. (2018). A modeling Study of microcracking processes of polyurethane coated cotton fabrics. Textile Research Journal, 88 (24): 2766-2781. Huang, G. B., Zhu, Q., andSiew, C. (2006). Extreme learning machine: Theory and applications. Neurocomputing 70 (1-3): 489-501. Karadurmuş, E., Akyazı, H., Göz, E. and Yuceer, M. (2018). Prediction of characteristic properties of crude oil blending with ANN. Journal of Dispersion Science and Technology, 39 (9): 1236-1243. Li, B., Rong, X. and Li, Y. (2014). An improved kernel based extreme learning machine for robot execution failures. The Scientific World Journal, 2014: 1-7. Leong, W. C., Bahadori, A., Zhang, J. and Ahmad, Z. (2019). Prediction of water quality index (WQI) using support vector machine (SVM) and least square-support vector machine (LS-SVM). International Journal of River Basin Management. doi: 10.1080/15715124.2019.1628030. McCulloch, W. S., and Pitts, W. (1943). A logical calculus of the ideas immanent in nervous activity. Bulletin of Mathematical Biophysics, 5: 115–133. Nemati, S., Fazelifard, M. H., Terzi, O. and Ghorbani, M. A. (2015). Estimation of dissolved oxygen using data-driven techniques in the Tai

152

Eda Göz, Erdal Karadurmuş and Mehmet Yüceer

Po River, Hong Kong. Environmental Earth Sciences, 74 (5): 4065– 4073. Olyaie, E., Abyaneh, H. Z., Mehr, A. D. (2017). A comparative analysis among computational intelligence techniques for dissolved oxygen prediction in Delaware River. Geoscience Frontiers, 8 (3): 517–527. Rehana, S. (2019). River water temperature modelling under climate change using support vector regression.” In Hydrology in a Changing World, edited by Singh S., Dhanya C. 171–183. Cham: Springer Water, Springer. Salami, E. S., and Ehteshami, M. (2015). Simulation, evaluation and prediction modeling of river water quality properties (Case Study: Ireland Rivers). International journal of Environmental Science and Technology, 12(10): 3235-3242. Sarkar, A., and Pandey, P. (2015). River water quality modelling using artificial neural network technique. Aquatic Procedia, 4: 1070–1077. Smola A. J., and Schölkopf, B. (2004). A tutorial on support vector regression. Statistics and Computing, 14: 199-222. Suykens, J. A. K. and Vandewalle J. (1999). Least squares support vector machine classifiers. Neural Processing Letters, 9 (3):293-300. Suykens, J. A. K. and Vandewalle, J. (2000). Recurrent least squares support vector machines. IEEE Trans. Circuits Systems-I, 47 (7): 1109-1114. Suykens, J. A. K. (2001). “Nonlinear modelling and support vector machines,” in Proc. of the IEEE Int. Conf: on Instrumentation and Measurement Technology, Budapest, Hungary, May 2001, pp. 287-294. Suykens, J. A. K., Barbanter, J. De, Lukas, L. and Vandewalle, J. (2002a). Weighted least squares support vector machines: robustness and sparse approximation. Neurocomputing, 48 (1-4): 85-105. Suykens, J. A. K., Gestel, T. V., Barbanter, J. De, Moor, B. De, and Vandewalle, J. (2002b). Least squares support vector machines. Singapore: World Scientific. Tiyasha, Tran Zaher, T. M. T. and Yaseen, M. (2020). A survey on river water quality modelling using artificial intelligence models: 2000-2020, Journal of Hydrology, 585: 124670.

River Water Quality Modelling …

153

Tripathi, S., Srinivas, V. V. and Nanjundiahb, R. S. (20069. Downscaling of precipitation for climate change scenarios: A support vector machine approach, Journal of Hydrology, 330 (3-4): 621-640. Vapnik, V. (1995). The Nature of Statistical Learning Theory. Springer in USA. Wu, H. K., Chen, P., and Hsieh, J. (2006). Simple algorithms for least squares support vector machines. Paper presented in IEEE International Conference on Systems, Man and Cybernetics, Taipei, Taiwan, October 8-11. Yahya, A., Saeed, A., Ahmed, A.N., Binti Othman, F,, Ibrahim, R.K., Afan, H.A., El-Shafie, A., Fai, C.M., Hossain, M.S., Ehteram, M., Elshafie, A. (2019) Water quality prediction model based support vector machine model for ungauged river catchment under dual scenarios. Water 11(6):1231. Yi, H. S., Park, S., An, K., and Kwak, K. (2018). Algal bloom prediction using extreme learning machine models at artificial weirs in The Nakdong River, Korea.” International Journal of Environmental Research and Public Health 15 (10): 2078. Zhu, S., Heddam, S., Wu, S., Dai, J. and Jia, B. (2019). Extreme learning machine-based prediction of daily water temperature for rivers. Environmental Earth Sciences, 78 (202): 1-17. Zakaria, M., AL-Shebany, M., and Sarhan, S. (2014). Artificial neural network: A brief overview. International Journal of Engineering Research and Applications, 4 (2):7-12. Zhang, N., Williams, C. and Behera, P. (2014). Water quantity prediction using least squares support vector machines (LS-SVM) method. Systemic, Cybernetics and Informatics, 12 (4): 53-58.

INDEX A acid, 66, 69, 81, 97 activation function, 11, 12, 16, 23, 24, 25, 36, 60, 61, 62, 87, 88, 100, 101, 129, 138, 139, 144, 145 adenocarcinoma, 84, 106, 109, 110 air temperature, 5, 23, 24, 25, 26, 27, 28, 29, 30, 32, 33, 35, 37, 38 algorithm, 14, 15, 19, 20, 21, 23, 24, 25, 26, 27, 28, 29, 30, 36, 39, 51, 62, 64, 70, 77, 81, 90, 98, 99, 100, 101, 102, 103, 105, 106, 109, 115, 118, 126, 128, 130, 132, 133, 135, 137, 138, 139, 144, 150 ammonium, x, 124, 126, 142 Artificial Neural Networks (ANNs), v, viii, ix, 1, 2, 8, 9, 10, 13, 14, 15, 16, 18, 19, 22, 23, 25, 28, 29, 35, 36, 38, 39, 41, 43, 44, 46, 47, 48, 49, 50, 54, 57, 58, 59, 60, 61, 62, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 76, 77, 81, 83, 84, 85, 87, 89, 91, 92, 95, 96, 100, 101, 102, 103, 104, 105, 107, 110, 111, 114, 115, 145, 146 ANNs application, 58, 62 ANNs modelling, 2, 67

antioxidant, 50, 65, 66, 67, 68, 69, 73, 76, 77, 81, 84 artificial intelligence, vii, x, 22, 30, 39, 47, 53, 58, 77, 117, 124, 126, 128, 150, 152 assessment, 64, 81, 111, 114

B benefits, 7, 64, 80 bias, 13, 93, 135, 137 biological activity/activities, ix, 83, 84, 85, 86, 93, 97, 103, 104, 105, 106, 107, 110, 111 biologically active compounds, ix, 83, 84, 87, 97, 103, 110 brain, 9, 52, 58, 59, 63, 80, 87 breast cancer, 106, 107, 122

C calibration, 68, 70, 89, 90 cancer, 65, 74, 76, 84, 86, 98, 101, 105, 106, 107, 112, 120 carbon, x, 124, 126, 142, 143, 148, 149 cell line(s), 84, 86, 98, 101, 106, 107, 122

156

Index

chemical, vii, 2, 3, 8, 28, 44, 58, 65, 68, 69, 71, 72, 75, 78, 84, 85, 105, 106, 108, 118, 150 chemical properties, 28, 65, 69, 71, 72, 75, 78 chemometrics, 78, 84, 85, 86, 112, 113, 114, 117, 119, 120 chicken, 49, 80, 101 chromatography, 84, 86, 98, 99, 100, 113, 116, 117, 118, 119 classes, 14, 67, 69, 105, 106, 117 classification, ix, 58, 67, 69, 71, 75, 77, 80, 99, 115, 129, 132, 133, 143 climate, 4, 152, 153 clustering, 14, 62, 93, 99 colon, 76, 84, 99 color, 27, 30, 48, 50, 54, 65 color change, 27, 30 commercial, 5, 41, 42 complexity, 40, 136, 150 composition, 44, 65, 66, 68, 81, 97 compounds, ix, 8, 50, 65, 66, 69, 76, 77, 79, 80, 83, 84, 85, 86, 95, 96, 97, 99, 101, 102, 105, 106, 107, 108, 110, 112, 122 computer, 22, 39, 42, 51, 58, 67, 74, 75, 129 computing, 18, 42, 151 conductivity, x, 39, 65, 124, 126, 128, 142, 143 consumers, 2, 3, 64 consumption, 2, 26, 32, 35, 52 correlation(s), 73, 85, 86, 88, 91, 98, 102, 126, 145, 146 correlation coefficient, 73, 91, 145, 146 cost, vii, xi, 5, 6, 124, 132 cross-validation, 22, 90, 91, 98, 102, 108, 109, 144

data set, ix, 14, 62, 68, 83, 89, 92, 93, 98, 100, 101, 106, 107, 108, 110, 133, 135 database, 104, 105, 128, 142 degradation, vii, 2, 96, 97, 100, 102, 113, 119 dehydration, 4, 6, 7, 32, 33, 45, 47, 49, 50, 52, 54 dendrites, 9, 58, 87 derivatives, 84, 86, 96, 98, 101, 105, 106, 107, 108, 109, 110, 111, 112, 113, 114, 115, 116, 117, 120, 121, 122, 136 detection, ix, 58, 59, 67, 110 diffusivity, 26, 30, 34, 35, 48, 49, 53, 56 discrimination, viii, 57, 58, 69 dissolved oxygen, x, 124, 126, 142, 143, 147, 148, 150, 151, 152 distribution, 3, 26, 85, 110 doping, 97, 102, 119 drugs, 96, 99, 101, 108 drying, vii, viii, 2, 3, 4, 5, 6, 7, 8, 9, 19, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 54, 55, 56, 64, 65, 66, 71, 77, 78, 80 drying efficiency, 8, 37, 55 drying kinetics, 28, 47, 48, 51, 52 drying process, vii, viii, 2, 3, 5, 8, 21, 22, 25, 26, 27, 29, 30, 36, 38, 39, 40, 42, 43, 48, 51, 54, 66 drying process simulation, viii, 2 drying rate, 5, 7, 26, 29, 33, 34, 35, 36, 44, 55 drying ratio, 23 drying technology, 2, 4, 5, 9, 19, 21, 23, 38, 39 drying time, 4, 6, 7, 23, 24, 25, 26, 27, 28, 29, 31, 32, 33, 34, 35, 36, 37, 38

D E data analysis, ix, 64, 68, 83, 119, 120, 135 data scaling, 93, 94, 115

effective diffusivity, 30

Index energy, viii, 2, 4, 5, 6, 7, 8, 26, 30, 34, 35, 36, 37, 38, 43, 44, 48, 49, 50, 54, 62 energy consumption, 5, 7, 26, 30, 34, 35, 37, 43, 48, 49, 54 energy efficiency, 8, 38, 43 energy utilization, 36, 37, 38 energy utilization ratio, 36, 37, 38 engineering, vii, viii, x, 2, 19, 21, 52, 54, 58, 64, 74, 77, 124, 129 estrogen, 84, 106, 107 exergy efficiency, 36, 37, 38 exergy loss, 36, 37, 38 external validation, 101, 108, 109, 110 extraction, ix, 44, 46, 65, 68, 69, 72, 73, 75, 78, 83, 85 extracts, 66, 68, 69, 72, 73, 76, 78

F fish, 3, 75, 79 flavonoid content, 29 fluidized bed, 26, 29, 33, 43, 51 food, v, vii, viii, 1, 2, 3, 4, 5, 6, 7, 8, 21, 23, 39, 41, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 62, 63, 64, 65, 66, 67, 74, 75, 76, 77, 78, 79, 80, 81, 99, 115, 121, 125 food engineering, vii, viii, 54, 58, 64, 74 food industry, 3, 8, 23, 74, 76 food process engineering, viii, 2, 52, 77 food products, 2, 3, 6, 8, 56, 66, 68, 75 forecasting, x, 79, 124, 129 formation, 6, 55, 94 fruits, 2, 3, 4, 5, 49, 50, 52, 66, 76

G global sensitivity analysis, 91 glutathione, 105, 107, 120 glycerol, 7, 67, 69 growth, 65, 101, 112

157 guidelines, ix, 84, 85, 110, 114

H health, 65, 76, 80 HIV, 105, 107, 108, 121 human, viii, 9, 41, 42, 58, 96, 101, 106, 107, 112, 114, 120, 122, 129, 144 human brain, viii, 9, 58, 129 humidity, 26, 32, 37 hybrid, 8, 18, 19, 20, 21, 24, 39, 51, 52, 56 hydration ratio, 23

I image, ix, 58, 129, 137 impurities, 97, 99, 118 industry/industries, 74, 78, 99 infrared spectroscopy, 75, 76, 79 ingredients, 105, 108, 121 inhibition, 105, 106, 107 intelligence, vii, x, 22, 30, 39, 47, 53, 58, 77, 117, 124, 126, 128, 150, 152 issues, 14, 39, 65, 125

K kinetics, 27, 28, 44, 47, 48, 51, 52

L learning, x, 9, 13, 14, 15, 19, 23, 24, 25, 27, 36, 39, 49, 58, 60, 61, 62, 84, 89, 101, 102, 105, 106, 109, 124, 126, 128, 129, 132, 133, 135, 137, 138, 139, 144, 150, 151, 153 learning algorithms, 13, 59, 89, 132, 137 linear model, 72, 94, 97, 98, 101, 104 liquid chromatography, 98, 99, 100, 113, 118, 119

158

Index M

machine learning, 49, 58, 84, 101, 126, 132, 137, 150 management, x, 64, 76, 124, 125, 149 mapping, 62, 133, 135 mass, 3, 5, 27, 37, 47, 48, 119 mass transfer, 3, 27, 47, 48 materials, 3, 4, 5, 45, 65, 81, 125 matrix, 3, 90, 98, 137, 138, 139, 140 measurement, vii, viii, x, 25, 42, 57, 68, 80, 124, 140, 149 meat, 3, 70, 77 medicine, viii, 58, 62, 129 membership, 19, 35, 36 methodology, 38, 39, 51, 54, 103, 110, 133 modelling, vii, viii, x, 2, 8, 24, 25, 39, 44, 45, 50, 62, 66, 68, 69, 71, 77, 78, 114, 118, 120, 124, 128, 142, 144, 145, 146, 150, 152 models, vii, ix, x, 8, 21, 22, 25, 29, 35, 36, 39, 41, 42, 45, 51, 52, 59, 64, 67, 68, 69, 71, 72, 73, 74, 80, 81, 84, 85, 92, 94, 95, 97, 98, 99, 101, 103, 107, 108, 109, 110, 111, 113, 117, 120, 121, 124, 125, 128, 144, 145, 146, 152, 153 moisture, 5, 7, 8, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 38, 45, 48, 49, 53, 54, 56, 67, 71 moisture content, 5, 7, 23, 25, 26, 27, 29, 30, 31, 32, 33, 35, 36, 45, 49, 54, 71 moisture ratio, 23, 24, 25, 26, 28, 29, 31, 32, 33, 34, 35, 36, 38 molecular dynamics, 65, 81, 118 molecular fingerprints, 105, 106, 120 molecules, ix, 7, 84, 85, 86, 100

N nervous system, 58, 59, 128

neural network, vii, viii, ix, x, 2, 9, 14, 15, 17, 18, 19, 20, 21, 23, 24, 25, 26, 27, 28, 29, 30, 36, 38, 39, 40, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 54, 55, 57, 59, 62, 64, 68, 69, 70, 71, 73, 75, 76, 77, 78, 79, 80, 81, 83, 85, 87, 89, 91, 92, 93, 96, 98, 99, 102, 105, 106, 107, 108, 110, 114, 115, 118, 119, 120, 124, 126, 128, 129, 130, 131, 132, 137, 150, 151, 152, 153 Neural Network Model, 44, 45, 47, 53, 77 neurons, viii, 9, 10, 12, 14, 15, 16, 17, 23, 25, 26, 27, 29, 38, 57, 58, 60, 61, 63, 64, 72, 87, 88, 89, 97, 100, 101, 102, 103, 128, 129, 130, 131, 137, 138, 142, 144, 145 NIR, 68, 69, 70, 71, 75, 76, 77, 79, 81 NIR spectra, 68, 69, 70, 71 nitrogen, x, 67, 124, 126, 140, 142 NMR, 31, 53, 78 nodes, 15, 16, 103, 129

O oil, 44, 67, 99 operations, 8, 14, 64 optimization, vii, viii, ix, 2, 4, 5, 8, 42, 54, 58, 74, 90, 97, 99, 100, 117, 126, 132, 134, 135, 136, 142, 143, 144 optimization method, 97, 117, 132, 142

P partial least squares regression, 68, 85, 107, 110 pattern recognition, x, 87, 124, 129 PCA, 66, 68, 70, 76, 77, 98 PCR, 68, 85, 94, 101 pH, x, 124, 126, 128, 142, 143, 150 pharmaceutical(s), 74, 96, 99, 101, 118 phenolic compounds, 7, 66, 76, 77, 79 phosphate, 126, 142, 151

Index physicochemical properties, 3, 55, 85 physics, viii, 39, 58 plants, viii, 2, 39, 41, 65, 69, 72, 73, 78, 81 PLS, 68, 70, 85, 94, 99, 100, 101, 103 pollutant(s), ix, x, 123, 124, 125 pollution, ix, 123, 125, 140, 142, 149 population, x, 2, 123, 125 porosity, 3, 6, 7, 39 Portugal, 1, 42, 56 potato, 35, 43, 45, 49, 64 prediction, viii, ix, x, 22, 23, 26, 27, 30, 31, 35, 37, 39, 43, 44, 45, 46, 49, 50, 51, 57, 58, 62, 63, 64, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 78, 79, 80, 81, 83, 84, 85, 86, 95, 98, 100, 101, 102, 103, 107, 109, 110, 111, 113, 115, 118, 119, 124, 126, 128, 145, 146, 150, 151, 152, 153 prediction models, xi, 124, 126, 128, 150 preservation, 3, 4, 5, 7, 8, 49, 50 principal component analysis, 68, 70, 77, 98 process control and modelling, 2 product quality, 3, 7 project, 42, 105, 111 propagation, 18, 64, 71, 87, 89, 97, 99, 100, 101, 102, 103, 105, 106, 107, 126 pyrimidine, 105, 107, 120

Q QSRR analysis, ix, 84, 111, 116, 118 quantitative structure–activity relationship, vii, ix, 83, 86 quantitative structure–retention relationship, vii, ix, 84, 86, 119

R radiation, 4, 32, 45, 68 radio, 4, 7, 55 real-time monitoring, 40, 124, 128 receptor, 84, 99, 107, 121

159 recognition, ix, 58, 80 regression, viii, ix, x, 23, 57, 58, 62, 68, 69, 71, 72, 73, 75, 78, 83, 85, 86, 87, 88, 89, 91, 92, 94, 96, 98, 100, 101, 102, 107, 108, 109, 110, 111, 115, 117, 119, 124, 126, 133, 135, 143, 150, 152 regression method, ix, 84, 85, 110, 117 regression model, 23, 72, 75, 85, 89, 91, 92, 109, 111, 135 rehydration, 6, 27, 29, 46 rehydration ratio, 27, 29 reliability, 69, 110, 111 researchers, 29, 30, 38 resources, x, 64, 78, 81, 123, 125 response, 13, 14, 51, 54, 85, 97, 117, 132, 142 restrictions, viii, 2, 41, 74 river water quality monitoring, 124 root, 65, 69, 71, 78, 91, 108, 109, 126, 145

S SAR, ix, 84, 86, 111 savings, viii, 2, 8, 42 scaling, 93, 94, 115 science, 23, 41, 62, 77, 81 shrinkage, 3, 27, 29, 30, 31, 33, 48, 51, 54, 120 signals, 9, 10, 15, 16, 31, 58, 60, 61, 63, 64, 130 simulation(s), viii, 2, 23, 65, 81, 87, 125, 145 software, 10, 17, 22, 41, 59, 64 solution, 6, 7, 27, 33, 62, 76, 85, 101, 110, 135, 137, 139, 144 specific energy consumption, 26, 30, 34, 35, 37, 48, 49, 54 spectroscopy, 67, 69, 70, 71, 75, 76, 79, 81 stability, viii, 39, 58, 71, 94, 101 standard deviation, 91, 93, 94 state, xi, 4, 18, 66, 124, 134

160

Index

steroids, 84, 96, 103, 113, 119 storage, vii, 2, 3, 4, 5, 9, 48, 50, 52, 54, 64, 80 structure, vii, 15, 17, 18, 23, 24, 25, 26, 27, 28, 29, 30, 35, 36, 58, 60, 72, 87, 89, 113, 117, 118, 119, 120, 121, 125, 129, 130, 131, 132, 138, 139, 142, 143 Sun, 31, 53, 67, 75, 121 Switzerland, 53, 75, 115

T target, 14, 20, 58, 84, 85, 107, 120 techniques, vii, x, 8, 21, 41, 42, 48, 67, 71, 78, 79, 99, 124, 127, 128, 129, 143, 144, 150, 151, 152 technology/technologies, vii, 2, 4, 5, 6, 7, 8, 9, 19, 21, 22, 38, 39, 40, 42, 54, 65, 74, 78 temperature, x, 6, 24, 25, 26, 27, 28, 29, 31, 32, 33, 35, 36, 37, 64, 69, 72, 80, 124, 126, 128, 142, 143, 150, 152, 153 test data, 22, 23, 143, 144, 147, 148, 149 testing, 23, 40, 62 texture, 3, 7, 50 topology, 17, 23, 24, 27, 28, 29, 30, 36, 51 total phenol content, 28 training, 13, 14, 20, 22, 23, 26, 60, 62, 69, 70, 89, 90, 92, 93, 98, 99, 100, 102, 104, 110, 115, 129, 130, 132, 135, 136, 137, 138, 139, 142, 143, 144, 146 transformation(s), ix, 3, 84, 92, 93, 94, 103, 137 treatment, 5, 7, 31, 35, 44, 48, 55, 65, 97, 100, 102 trial, 23, 102, 142 Turkey, x, 52, 123, 124, 128, 140

U ultrasound, 7, 33, 43, 44, 48, 68 USA, 43, 46, 54, 55, 79, 80, 114, 127, 153

V vacuum, 7, 24, 31, 32, 34, 44, 48, 51 validation, 22, 62, 70, 90, 91, 98, 100, 102, 104, 107, 108 variables, 19, 23, 24, 27, 31, 32, 34, 37, 66, 67, 69, 70, 72, 87, 88, 91, 92, 93, 94, 98, 109, 111, 126, 128, 134, 135, 138, 144 vector, x, 16, 20, 68, 69, 75, 98, 99, 107, 108, 124, 126, 132, 133, 134, 135, 137, 139, 144, 150, 151, 152, 153 vegetables, 2, 3, 4, 5, 50, 52, 53, 65, 71, 78 velocity, 25, 26, 29, 30, 31, 32, 33, 34, 35, 37, 38 vision, 51, 67, 75 visualization, viii, 57, 58, 64, 74

W wastewater, 96, 99, 118 water, vii, ix, 2, 3, 5, 6, 27, 28, 47, 65, 71, 106, 107, 120, 123, 124, 125, 142, 149, 150, 151, 152, 153 water quality, vii, x, 47, 106, 107, 124, 125, 127, 151, 152 water resources, x, 123, 125 workers, viii, 2, 71

Y yield, 31, 68, 69, 73