Ground and Air Robotic Manipulation Systems in Agriculture (Intelligent Systems Reference Library, 214) 3030868257, 9783030868253

Citation preview

Intelligent Systems Reference Library 214

Andrey Ronzhin Tien Ngo Quyen Vu Vinh Nguyen

Ground and Air Robotic Manipulation Systems in Agriculture

Intelligent Systems Reference Library Volume 214

Series Editors Janusz Kacprzyk, Polish Academy of Sciences, Warsaw, Poland Lakhmi C. Jain, KES International, Shoreham-by-Sea, UK

The aim of this series is to publish a Reference Library, including novel advances and developments in all aspects of Intelligent Systems in an easily accessible and well structured form. The series includes reference works, handbooks, compendia, textbooks, well-structured monographs, dictionaries, and encyclopedias. It contains well integrated knowledge and current information in the field of Intelligent Systems. The series covers the theory, applications, and design methods of Intelligent Systems. Virtually all disciplines such as engineering, computer science, avionics, business, e-commerce, environment, healthcare, physics and life science are included. The list of topics spans all the areas of modern intelligent systems such as: Ambient intelligence, Computational intelligence, Social intelligence, Computational neuroscience, Artificial life, Virtual society, Cognitive systems, DNA and immunity-based systems, e-Learning and teaching, Human-centred computing and Machine ethics, Intelligent control, Intelligent data analysis, Knowledge-based paradigms, Knowledge management, Intelligent agents, Intelligent decision making, Intelligent network security, Interactive entertainment, Learning paradigms, Recommender systems, Robotics and Mechatronics including human-machine teaming, Self-organizing and adaptive systems, Soft computing including Neural systems, Fuzzy systems, Evolutionary computing and the Fusion of these paradigms, Perception and Vision, Web intelligence and Multimedia. Indexed by SCOPUS, DBLP, zbMATH, SCImago. All books published in the series are submitted for consideration in Web of Science.

More information about this series at http://www.springer.com/series/8578

Andrey Ronzhin · Tien Ngo · Quyen Vu · Vinh Nguyen

Ground and Air Robotic Manipulation Systems in Agriculture

Andrey Ronzhin St. Petersburg Federal Research Center of the Russian Academy of Sciences St. Petersburg, Russia Quyen Vu St. Petersburg Federal Research Center of the Russian Academy of Sciences St. Petersburg, Russia

Tien Ngo Le Quy Don Technical University Ha Noi, Vietnam Vinh Nguyen St. Petersburg Federal Research Center of the Russian Academy of Sciences St. Petersburg, Russia

ISSN 1868-4394 ISSN 1868-4408 (electronic) Intelligent Systems Reference Library ISBN 978-3-030-86825-3 ISBN 978-3-030-86826-0 (eBook) https://doi.org/10.1007/978-3-030-86826-0 © The Editor(s) (if applicable) and The Author(s), under exclusive license to Springer Nature Switzerland AG 2022 This work is subject to copyright. All rights are solely and exclusively licensed by the Publisher, whether the whole or part of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation, broadcasting, reproduction on microfilms or in any other physical way, and transmission or information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now known or hereafter developed. The use of general descriptive names, registered names, trademarks, service marks, etc. in this publication does not imply, even in the absence of a specific statement, that such names are exempt from the relevant protective laws and regulations and therefore free for general use. The publisher, the authors and the editors are safe to assume that the advice and information in this book are believed to be true and accurate at the date of publication. Neither the publisher nor the authors or the editors give a warranty, expressed or implied, with respect to the material contained herein or for any errors or omissions that may have been made. The publisher remains neutral with regard to jurisdictional claims in published maps and institutional affiliations. This Springer imprint is published by the registered company Springer Nature Switzerland AG The registered company address is: Gewerbestrasse 11, 6330 Cham, Switzerland

Preface

Agriculture robotics is an interdisciplinary scientific field that requires the involvement of a wide range of professionals engaged in artificial intelligence, robotics and agriculture. When digitalizing and robotizing agriculture in the open field crop production, it is necessary to account for the territorial distribution, structural and parametric variability of land plots that require the use of group usage of heterogeneous robotic systems, and support of their information, physical and energy interaction should be accounted for. High variability of the physical and geometric properties of agricultural products significantly complicates the process of the configuration design and selection for a robotic agricultural gripper to ensure reliable collection and movement of vegetables and fruits without damage. The book offers an introduction to intelligent control systems of heterogeneous agricultural robots. The book has resulted from activities of Laboratory of Autonomous Robotic Systems of St. Petersburg Institute for Informatics and Automation of the Russian Academy of Sciences (SPIIRAS) in the framework of agriculture robotics projects during last 5 year. In July 2020, six research institutions, including those of an agricultural profile, were included in SPIIRAS, and it was transformed into St. Petersburg Federal Research Center of the Russian Academy of Sciences (SPC RAS). Now agriculture robotics is one of the key research directions of SPC RAS. The book uncovers fundamental principles of heterogeneous robot interaction and recent developments in agriculture robot design. The purpose of the book is to present solutions to the problems of the joint application of heterogeneous ground and air robotic means when performing agricultural technological tasks that require physical interaction with agricultural products and the environment. The book considers the model-algorithmic and software-hardware control of the power supply system for unmanned aerial vehicles, a manipulator and a robotic gripper. The proposed solutions for the exchange of energy and physical resources of unmanned aerial vehicles on ground service platforms, automation of the process

v

vi

Preface

of collecting agricultural products and ensuring the stability of the air manipulation system during physical interaction with a ground object are important for the robotization of the transport and agricultural industry. St. Petersburg, Russia Ha Noi, Vietnam St. Petersburg, Russia St. Petersburg, Russia

Andrey Ronzhin Tien Ngo Quyen Vu Vinh Nguyen

About This Book

The book is aimed at solving the problems of the joint application of heterogeneous ground and air robotic means while performing the agricultural technological tasks that require physical interaction with agricultural products and the environment. The book considers the model-algorithmic and software-hardware control of the power supply system for unmanned aerial vehicles, a manipulator and a robotic gripper. The tasks’ solutions for the exchange of energy and physical resources of unmanned aerial vehicles on ground service platforms, automation of the process of collecting agricultural products and ensuring the stability of the air manipulation system at physical interaction with a ground object that are important for the transport and agricultural industry robotization are proposed. The book addresses the researchers investigating interdisciplinary issues of agricultural production robotization, problems of information, physical and energy interaction of ground and air robotic systems; it is recommended to postgraduates and students studying “Mechatronics and robotics”, “Management in technical systems” and “Technologies, mechanization and power equipment in agriculture, forestry and fisheries”. Important Advantages: • fundamental principles of heterogeneous robot interaction and their use in agriculture are offered; • interdisciplinary knowledge and experience acquired at scientific collaboration in robotics and agriculture domains are integrated; • examples of ground and air robots performing agro-technological operations are considered; • recommendations concerning the design of agriculture robots’ hardware and software are provided.

vii

Introduction

Part I of the book considers the issues of service and control automation of the interaction between heterogeneous agricultural robotic complexes. Robotic means with different levels of functioning autonomy are increasingly used in the agricultural sector, including grain sowing, fertilizing, harvesting and pesticides spraying. The joint use of heterogeneous ground and air vehicles extends the functional and sensoric capabilities of robotic processing of agricultural land. In certain cases, for instance, at servicing power supply systems and transporting air vehicles, arises a problem of physical interaction between the autonomously functioning unmanned aerial vehicle (UAV) and ground service robotic platform (GSRP). This problemsolving complexity is associated with the tasks of landing, fixation and mechanized processing of batteries and agricultural resources placed on the aerial vehicle on the service platform, as well as with the problem of control of the UAV group order of service. In the above regard, the study of models and algorithms for the interaction between heterogeneous agricultural robotic complexes is a topical research area focused on solving the problem of increasing the UAVs’ operating time at long-term autonomous works in agricultural fields that ultimately will contribute to reducing the time and cost of agricultural object processing due to automation and robotic complex application. A wide range of research and practical works of domestic and foreign scientists (Chernousko F. L., Kalyaev I. A., Ermolov I. L., Vizilter Yu. V., Pshikhopov V. Kh., Meshcheryakov R. V., Kemper P. F., Suzuki K. A. O, Morrison J. R., Kim J. W., Jung Y. D., Lee D. S., Shim D. H., Daly J. M., Ma Y., Waslander S. L. and others) deals with the problems of joint work of heterogeneous robotic means. Continuous improvement of embedded computing and sensor module hardware allows for developing more compact and energy-efficient solutions for a physical connection and exchange of energy resources between autonomous robotic complexes operating in different environments. In Part I, solutions are proposed aiming at increasing the UAVs’ operating time in long-term autonomous modes, as well as at reducing the time and cost of agricultural object processing due to developing models and algorithms for control of

ix

x

Introduction

the interaction between heterogeneous agricultural robotic complexes, particularly described as follows: 1.

2.

3.

4.

A conceptual and structurally functional model of interaction between heterogeneous agricultural robotic complexes, distinguished by the use of closed multichannel multiphase parallel queuing systems with heterogeneous nodes for dispatching and controlling the exchange of energy and physical resources of UAVs on GRSP. A logical-algorithmic model of the interaction of UAVs and GRSP, distinguished by the assessment of internal energy, physical resources, the remaining workload for groups of heterogeneous robots and providing a reduction in time and energy resources of UAVs for movement from the field, as well as takeoff and landing operations. A method for estimating the required composition and amount of equipment for agricultural land processing, distinguished by a multi-criteria assessment using a linear combination of three main criteria of the total processing time, consumed energy, cost of the equipment involved and providing numerical modeling and optimization of the volume of involved heterogeneous robotic systems. Recommendation software system AgrobotModeling, distinguished by the use of numerical and simulation modeling of UAVs and service platform amounts and providing visualization of the functioning of the selected values of input parameters, as well as the choice of the optimal composition and amount of heterogeneous robots.

Part I of the book includes Chaps. 1–4. Chapter 1 describes the problem of increasing the operating time of unmanned aerial vehicles operating in autonomous agricultural missions. The approaches to charging or replacing onboard batteries on accompanying robotic platforms are analyzed. The existing prototypes of robotic service platforms are distinguished by the complexity of the internal mechanisms, the speed of service, the algorithms for the platform and the aircraft to work together during landing and battery maintenance. Based on the analysis results, a classification of existing service systems installed on robotic platforms for servicing the batteries and built-in UAV containers has been compiled. Chapter 2 presents the formal statement of the task of interactions between heterogeneous agricultural robots. The developed model-algorithmic support for controlling the interaction between heterogeneous robots’ group at the UAV servicing in agricultural tasks is described. Chapter 3 describes the developed method and system to estimate and support the decision-making on the composition and number of heterogeneous agricultural complexes required to process the given land area, weather conditions and other aspects affecting the work cost and speed. First, a method for multi-criteria assessment of the amount and composition of heterogeneous equipment for agricultural land processing is described, then a graph model for calculating the efficiency of servicing UAV batteries is considered and a recommendation software system AgrobotModeling is presented.

Introduction

xi

Chapter 4 presents the results of numerical and simulation modeling of the amount of robotic technology for processing agricultural land. The experiments were carried out in the developed program AgrobotModeling, which provides the calculation of the number of unmanned aerial vehicles and ground service mobile platforms involved in the processing of agricultural land of a given area. The program also simulates the functioning of a selected number of robotic equipment, as well as calculates a multi-criteria assessment based on a linear combination of three main criteria: total processing time, total consumed energy and full cost of the equipment involved. Part II describes the task of multi-criteria synthesis of a robotic gripper configuration for agricultural products’ manipulations. Conventional collection and primary processing of agricultural products are the most resource-intensive tasks that require a transition from tedious manual operations to the technological processes’ automation and robotization of manipulations with physical objects. At a robotic gripper design, it is necessary to account for a variety of manipulated objects, the complexity of their identification and pointing the manipulator in a complex natural environment with obstacles. The task of synthesizing a robotic gripper mechanism is associated with meeting a number of conflicting requirements to reliability, softness, accuracy, speed and energy efficiency that form a complex space for solutions’ search. So, the study of models and algorithms for optimizing the configuration and control of a robotic gripper performing physical manipulations with agricultural products is an actual scientific direction focused on solving the problem of automation and robotization of technological processes for agricultural product processing. A wide range of scientific and practical studies made by domestic and foreign scientists considers solving the problems of robotic manipulators’ design and control (Zaborovsky V. S., Lokhin V. M., Makarov I. M., Manko S. V., Pavlovsky V. E., Poduraev Yu. V., Yushchenko A. S., Yatsun S. F., Liu J., Van Henten E. J., Feng Q., Bac C. W., Brown G. K., Lehnert C., Han K. S., Bontsema J., Hayashi S., De-An Z. and others). Continuous improvement of control systems and kinematic schemes has ensured a development of serial industrial manipulation robots. Currently, interdisciplinary studies of robotic complex control are becoming relevant, including those of agricultural grippers, subjected to increased requirements for the accuracy of manipulations as caused by high variability of agricultural products’ properties. In Part II, solutions are proposed aimed at automating the process of collecting agricultural products due to the development of models, algorithms and multi-criteria synthesis for the configuration of robotic gripper and control of its software and hardware components at physical manipulations with agricultural products objects, particularly described as follows: 1.

Conceptual and algorithmic models for selecting the parameters of a robotic manipulator and a control system for gripping agricultural products, distinguished by an automated multi-stage analysis of the geometric, mechanical and physical properties of the manipulated object, environmental parameters and potential risks of causing internal and external mechanical damage to agricultural products and ensuring the robotic gripper configuring.

xii

Introduction

2.

Algorithms for multi-criteria synthesis of a robotic gripper configuration, distinguished by a combined application of a posteriori optimization methods and determining the configuration parameter values in the kinematic scheme required at the design and control of the manipulator’s end-effector mechanism. Agro-gripper configuration and the algorithm to control operations’ cycle of its software and hardware modules for fruits’ removing, distinguished by a description of the main stages of physical manipulations of forming the high-level control commands and their execution in low-level software modules implementing interfaces to the hardware means involved in the configuration of a four-fingered robotic gripper with a vacuum bellows. AgroGripModeling software system for modeling the configuration of a robotic gripper, distinguished by the use of three multi-criteria synthesis algorithms, an ability to customize the existing kinematic schemes and parameters proposed in the original classification of agricultural grippers and agro-technological tasks.

3.

4.

Part II of the book includes Chaps. 5–8. Chapter 5 provides an overview of control tasks in regard to technological and robotic operations in agricultural production. The relevance of studying the problems of robotization in agriculture is justified by the good prospects for improving the quality of fresh vegetables and fruits, as well as reducing the cost of production, the necessary labor force and other resources through the development and implementation of agricultural robots. A classification has been compiled for agricultural grippers installed on robotic agricultural equipment and used, e.g., for weed control and harvesting. Also mentioned are the tasks of weeds’ directional spraying and/or various plant pruning, with manipulators involvement, however, no targets are reached yet. Some examples of existing research agricultural robots equipped by combined grippers matching the proposed classification and related to various types are given as follows: vacuum gripper with a video camera for grabbing tomatoes; a six-fingered pneumatic gripper with a video camera; a two-fingered gripper with pressure and collision sensors for picking apples; a threefingered gripper with a video camera for citrus fruits; eggplant grippers and others. The relevance of the joint interaction between a group of heterogeneous ground and air robots at performing agricultural tasks in an autonomous mode is also noted. Chapter 6 describes the conceptual and algorithmic models for selecting the parameters of a robotic manipulator and a control system for agricultural product gripping. The formal statement of the problem of multi-criteria synthesis of robotic gripping with the definition of target functions and imposed constraints is presented. To evaluate the performance of multi-criteria synthesis algorithms, it is proposed to use indicators responsible for the calculations’ quality and speed. In Chap. 7, a posteriori methods for solving multi-criteria synthesis problems are analyzed and modified. Here is described an example of their use at modeling and selecting values for parameters in the kinematic model of a four-fingered robotic gripper for picking tomatoes. Comparison of simulation results using NSGA-II, MOGWO and MOPSO methods is made. Versions of selecting the objective function, weight coefficients and their influence upon the set of optimal grip sizes are discussed.

Introduction

xiii

Chapter 8 describes the structure of the developed software system AgroGripModeling, providing the modeling and multi-criteria optimization of the robotic gripper configuration. The configuration of the developed robotic gripper for picking tomatoes is described that comprises a four-fingered mechatronic system, a vacuum suction nozzle and distinguishes from existing analogues by using a linear drive to move the vacuum nozzle simultaneously with the four-fingered mechanism movements. The results of modeling and optimizing the configuration of a robotic gripper implementing a posteriori algorithms MOGWO, NSGA-II and MOPSO are presented: a generalized algorithm to control the cycle of software and hardware module operation for robotic gripping at a fetus removing; the results of testing a robotic gripper with vacuum bellows for picking tomatoes are discussed. Part III considers the motion control problem for an onboard manipulator in maintaining stability of a multi-rotor UAV in hovering mode. UAV equipment with means of physical interaction with ground objects is a new scientific trend in the robotics field. Adding an onboard manipulation system to UAV significantly complicates the operation’s algorithm design and leads to an increase in overall dimensions and energy consumption. Physical interaction of the manipulator with objects complicates the process of the UAV stabilizing, which, in turn, leads to difficulties in the UAV positioning and reduces the accuracy of the end mechanism targeting, like the gripper. Besides, the physical interaction of the manipulator with ground objects requires the increased energy resources of UAV. The development and application of unmanned aerial systems in agricultural production is considered one of the most profitable markets for robotics. Along with the increase of the UAVs’ energy efficiency, there appeared a possibility to move from monitoring tasks to more complex ones, requiring physical contact with surrounding objects, manipulating agricultural products and others. Analysis of advanced research in the air robotic manipulation systems confirms the topicality of this study, as focused on solving the movement control problems for the UAV manipulator and its stabilization, specifically used in agricultural technological processes. Scientific and practical studies run by domestic and foreign scientists dwelt upon solving the problems of robotic manipulator control, UAVs and their interaction with ground objects through built-in manipulators (Bobtsov A. A., Voronova E. M., Koshkin R. P., Pavlovsky V. E., Poduraev Yu. V., Filimonov N. B., Yushchenko A. S., Banaszkiewicz M., Heredia G., Kun Xu, Suarez A., Xilun Ding, Yushu Yu, Zihao Wang and others). The studies done were aimed at upgrading the manipulator structure, increasing the stability of the air manipulation system, reducing the mass of the onboard load, minimizing the size of the UAV and increasing the permissible weight and dimensions of the payload. In Part III, solutions are proposed aimed at ensuring the stability of the air robotic manipulation system at gripping ground objects based on the development of models, algorithms for the manilulator’s movement control for an unmained aerial vehicle (UAVM) and its stabilization, particularly described as follows:

xiv

1.

2.

3.

4.

5.

Introduction

Conceptual and set-theoretical models of an air robotic manipulation system, distinguished in the description of interrelated entities: UAV, a manipulator, a ground object and environmental factors that provide the problem formulation for developing a model-algorithmic support to control the UAVM movement during physical interactions with a ground object in an environment with various disturbances and obstacles affecting the geometric patency. Algorithm to determine the UAVM acceptable configuration, distinguished by analyzing the typical trajectories of the end-effector mechanism and calculating the range sets for angles between manipulator links, ensuring their movement along the specified trajectories while maintaining the center of manipulator mass on a vertical axis of the air manipulation system. An algorithm for calculating the key points’ coordinates for all manipulator links, depending on their joints’ angles, based on solving problems of forward and inverse kinematics, distinguished by limiting the displacement of the manipulator mass center, its links and the end-effector mechanism along horizontal and vertical axes, and when the end mechanism moves along the calculated trajectory, it provides the minimum horizontal displacement of the manipulator mass center. UAVM movement control and stabilization system, characterized by the use of a fuzzy PID controller in combination with input calculated data based on polynomial trajectory equations, which ensures the acceptable positioning accuracy of the end mechanism on a given trajectory. UAVManipulatorModeling software system, distinguished by the use of modules calculating polynomial equations of manipulator link trajectories, parameters of fuzzy PID controller, ensuring the modeling and visualization for the influence of disturbing influences on the manipulator vibration occurrence and the ability of the air robotic manipulation system to maintain a stable state by minimizing the horizontal displacement of the manipulator mass center.

Part III of the book includes Chaps. 9–12. Chapter 9 provides an analytical review of existing approaches to solving the control problems for air robotic manipulation systems and the physical interaction of unmanned aerial vehicles (UAV) with ground objects, in particular, while solving problems of agricultural production. The relevance of robotic system introduction to the agricultural production is stipulated by socio-economic reasons and the reduction in the World’s freshwater resources. Among UAVs now actively used for land monitoring, land yields’ cartogram compiling and fertilization zone planning, multi-copters stand out and their obvious advantage is a vertical takeoff and high sensors’ resolution. Multicopters can be equipped with onboard manipulators and sensor means, like video cameras, thermal imager, thermometer, gas sensors, radars, wind speed sensors, pressure sensors, infrared and other sensors. Current studies of UAVs carrying manipulator aboard are professionally discussed and cover the problems of flight control, avoidance of contact with the ground, interaction with the surrounding space, as well as physical interaction with ground objects. Adding the onboard manipulation system to UAV significantly complicates the algorithms of operation, design and leads to

Introduction

xv

the increase in overall dimensions. The physical interaction between the manipulator and objects complicates the process of the UAV stabilizing, what in turn leads to difficulties in the UAV positioning and reduces the accuracy of the gripping pointing. Besides, the physical interaction between the manipulator and objects requires the UAV increased energy resources. The chapter presents an original classification of the air robotic manipulation systems’ components, describing various options for onboard facilities necessary for implementing a functional purpose of the UAV with a manipulator. Chapter 10 describes the conceptual model and general structure of the air robotic manipulation system, new tasks for manipulators’ control concerning their base instability and interaction with ground objects. An analysis of the destabilizing factors’ impact on the deviation of the end working mechanism movement relative to a specified trajectory is performed. The main task of this study is formulated, as related to the design of a control system for a manipulator installed on UAV and maintaining the center of mass on the vertical axis during movement while interacting with ground objects. Then a number of developed algorithms are described that implement the set task of motion control and stabilization of the air robotic manipulation system. Chapter 11 proposes a solution to the problem of synthesizing the kinematic and dynamic models of the UAV manipulator, describes the UAVManipulatorModeling software system structure, which provides modeling of the UAV manipulator control and motion as well as its stabilization under the influence of disturbances. The output data are the motion graphs of the manipulator mass center, graphs of the movement trajectory of each manipulator link, graphs of the link angular velocity and angular acceleration. These graphs allow for estimating the manipulator stability when working under the disturbance and without it. In design of a motion control system for the UAV manipulator, it is necessary to calculate the dynamics of the manipulator, determine the motion equations for each manipulator link and calculate the parameters of the manipulator regulator. Chapter 12 presents the results of the stability preservation modeling for an unmanned aerial vehicle in the hover mode when the manipulator is moving. Modeling was exercised in the absence of external disturbances and their impacts. The experiments were made within the developed program UAVmanipulatorModeling, supporting the calculation and visualization of the trajectories of the manipulator mass center and its end working mechanism. In the study of the horizontal shift of the manipulator mass center, its maximum value was also analyzed at the end of the manipulator movement influenced by disturbing factors that cause instability of the entire air robotic manipulation system. The described models, algorithms and software are focused on the development of robotic manipulation systems to be implemented in agricultural production and at solving logistic problems of the transportation of the ground objects to difficult accessible localities.

Contents

Part I 1

2

Automation of Service and Interaction Control of Heterogeneous Agricultural Robots

Analysis of Existing Approaches to the Service Automation and to Interaction Control of Heterogeneous Agricultural Robots . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.1 Analysis of Existing Ground-Based Robotic Service Platforms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.2 Analysis of Requirements and Restrictions of Mobile Battery Maintenance Systems for Unmanned Aerial Vehicles . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.3 Analysis of Approaches to Control the Interaction of Unmanned Aerial Vehicles and Ground-Based Robotic Service Platforms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.4 Analysis of Approaches to Physical Resources Redistribution Between Agricultural Machinery of Different Autonomy’s Degree . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.5 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Models and Algorithms of Interaction Between Heterogeneous Agricultural Robots . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.1 Conceptual Model and Formal Statement of the Problem of Interaction Between Heterogeneous Agricultural Robots . . . . . 2.2 Structural and Functional Models of Interaction Between Heterogeneous Agricultural Robots . . . . . . . . . . . . . . . . . . . . . . . . . 2.3 Algorithmic Models to Control the Interaction of Heterogeneous Agricultural Robots at Different Service Stages . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.4 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

3 3

8

10

14 20 20 25 25 27

36 42 43

xvii

xviii

3

4

Contents

Recommendation System to Select the Composition of the Heterogeneous Agricultural Robots . . . . . . . . . . . . . . . . . . . . . . . 3.1 Method of Multi-criteria Estimation of the Heterogeneous Equipment Amount and Composition for Agricultural Land Processing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.2 Graph Model to Calculate the Efficiency of the Battery Maintenance of Unmanned Aerial Vehicles . . . . . . . . . . . . . . . . . . 3.3 Description of the AgrobotModeling Recommendation Software System . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.4 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Experimental Estimation of Means Developed for Interaction Between Heterogeneous Agricultural Robots . . . . . . . . . . . . . . . . . . . . . 4.1 Methodology for Preliminary Estimation of Agricultural Robots’ Optimal Number . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.2 Numerical Modeling of the Amount of Equipment Required for Agricultural Land Processing . . . . . . . . . . . . . . . . . . . 4.3 Comparison of Numerical and Simulation Results . . . . . . . . . . . . . 4.4 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Part II 5

6

45

45 47 51 62 62 65 65 67 76 81 84

Multi-criteria Synthesis of a Robotic Gripper Configuration for Manipulating Agricultural Products

Theoretical Foundations to Control Technological and Robotic Operations with Physical Manipulations of Agricultural Products . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.1 Analysis of Control Tasks for Technological and Robotic Operations in Precision Farming . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.2 Analysis of Technical Means and Methods to Control Robotic Grippers for Manipulating Agricultural Products . . . . . . 5.3 Analysis of Methods for Design and Optimizing the Manipulators’ Structure for Working with Agricultural Products . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.4 Analysis of Robotic Grippers for Physical Manipulation of Agricultural Products . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.5 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

89 89 96

104 106 110 110

Model-Algorithmic Support of Robotic Gripper for Manipulating Agricultural Products . . . . . . . . . . . . . . . . . . . . . . . . . 115 6.1 Conceptual Model and Formal Statement of the Robotic Gripper Control Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 115 6.2 Algorithmic Model for Selecting the Control System Parameters of a Robotic Gripper for Agricultural Products . . . . . 118

Contents

Statement of the Problem of Multi-criteria Synthesis of Robotic Gripper . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.4 Criteria for Estimating the Quality of Optimization Algorithms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.5 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

xix

6.3

7

8

Algorithms for Multi-criteria Synthesis of the Robotic Gripper Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.1 Algorithms for Multi-criteria Synthesis of the Configuration of Robotic Grippers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.1.1 Non-dominated Sorting Genetic Algorithm (NSGA-II) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.1.2 Multi-objective Grey Wolf Optimizer (MOGWO) . . . . . . 7.1.3 Multi-objective Particle Swarm Optimization (MOPSO) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.2 Modeling the Configuration of a Robotic Gripper . . . . . . . . . . . . . 7.3 Optimization Algorithms’ Testing Results . . . . . . . . . . . . . . . . . . . 7.4 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Results of Modeling and Optimization of the Robotic Gripper Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.1 AgroGripModeling Software System for Configuration Synthesis of Robotic Gripper . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.2 Modeling the Configuration and Control Algorithm of a Four-Finger Robotic Gripper for Picking Tomatoes . . . . . . . . 8.3 Optimization Results of the Robotic Gripper Configuration . . . . . 8.4 Testing a Robotic Gripper with a Vacuum Bellows for Picking Tomatoes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.5 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

125 128 129 130 131 131 131 134 137 142 145 150 152 153 153 157 164 169 173 174

Part III Motion Control of Onboard Manipulator with Preservation of Stability of Multi-rotor Unmanned Aerial Vehicles in Hovering Mode 9

Analysis of Approaches to the Control of Air Manipulation Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9.1 Analysis of Control Systems for the Air Manipulation Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9.2 Analysis of Existing Air Manipulation Systems . . . . . . . . . . . . . . . 9.3 Analysis of Agricultural Tasks Being Solved by Unmanned Aerial Vehicles . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9.4 Developed Classification of Air Manipulation Systems . . . . . . . . 9.5 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

179 180 182 190 194 200 200

xx

Contents

10 Conceptual and Algorithmic Models of Air Manipulation System . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10.1 Conceptual Model and Formal Statement of the Control Problem of Air Manipulation System . . . . . . . . . . . . . . . . . . . . . . . . 10.2 New Problems Arising at Study of Control of Air Manipulation System . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10.3 Algorithmic Model for Design of Air Manipulation System . . . . 10.4 Algorithm for Calculating the Manipulator Joint Angles with the Center of Mass Preservation on a Vertical Axis . . . . . . . . 10.5 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 Mathematical Modeling of Motion Control of Air Manipulation System and Its Stabilization . . . . . . . . . . . . . . . . . . . . . . . 11.1 Synthesis of the Kinematic and Dynamic Models for Air Manipulation System . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.2 Simulation of the Links Number of Air Manipulation System . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.3 Control System of Air Manipulation Systems Based on Fuzzy PID Controller . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.4 UAVManipulatorModeling Software System for Simulating the Control and Stabilization of Air Manipulation System . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.5 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12 Experimental Results of Simulating the Motion Control of Air Manipulation Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12.1 Simulation of Manipulator Motion Control in the Absence of External Disturbances . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12.2 Simulation of Manipulator Control at Moving the Working End Mechanism Directly from the Starting Point to the Specified Point . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12.3 Simulation of Manipulator Control at Moving the Working End Mechanism Along a Given Sequence of Points . . . . . . . . . . . 12.4 Simulation of Manipulator Control at Moving the Working End Mechanism Along a Given Trajectory . . . . . . . . . . . . . . . . . . . 12.5 Simulation of Controlling the Manipulator Motion Under the Influence of Disturbances . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12.6 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

205 205 209 213 217 224 225 227 227 233 238

244 247 249 251 251

254 254 256 260 269 270

About the Authors

Prof. Dr. Eng. Andrey Ronzhin born in 1976, is the Director of St. Petersburg Federal Research Center of the Russian Academy of Sciences (SPC RAS)—a legal and consolidated (by joining five more research institutions) successor of St. Petersburg Institute for Informatics and Automation of the Russian Academy of Sciences (SPIIRAS) since 2020; he held positions of the Director of SPIIRAS (2018–2020), Deputy Director for Research of SPIIRAS (2013–2018). He headed the SPIIRAS Laboratories of Autonomous Robotics Systems (2015–2018), of Speech and Multimodal Interfaces (2008–2015) and of Speech Informatics Laboratory (2003–2007). In 2016, he was awarded the honorary academic title Professor of the Russian Academy of Sciences; he is the winner of the St. Petersburg Government Prize for outstanding results in science and technology for 2017, named after A.S. Popov, and winner of the St. Petersburg Government Award for scientific and pedagogical activity for 2016. He was awarded his MS in Engineering (M. Sc. Eng.), Ph.D. (Cand. Sc. Eng.), Associate Prof., Dr. Sc. Eng., Full Prof. in 1999, 2003, 2008, 2010, 2013, respectively. He is the founder of the scientific school for studying the multimodal interfaces in the ambient intellectual environment. The research interests of Prof. A. L. Ronzhin cover modeling of natural communication and development of interactive multimodal information-control and robotic systems; human-machine interaction and robotics. Under the leadership of A. L. Ronzhin, a number of unique technical solutions were made that implemented multimodal interfaces of the surrounding intelligent space, automation of maintenance and control of the interaction of heterogeneous robotic systems. Recently, Prof. A. L. Ronzhin has been comprehensively developing new basic and applied approaches to the introduction of agricultural robotics; his scientific contribution to this field is confirmed by numerous publications, implementations, joint research projects and is recognized internationally in Russia. Starting from 2000 to the present, Prof. Ronzhin and research team under his leadership work on a number of domestic and more than 10 international projects have been funded and subsidized by the Russian Academy of Sciences, the Russian Foundation for Basic Research, the Russian Science Foundation, the Ministry of Science xxi

xxii

About the Authors

and Higher Education of the Russian Federation, internationally oriented programs ERA.Net RUS Plus; research projects funded by international organizations include EC IST, EC INTAS, EC FP6, EOARD, and others. Professor Ronzhin is actively teaching original academic courses for the university undergraduate students and postgraduates in robotics, mechatronics, control of robots and robotic systems, developing intelligent robotic systems, agricultural robots. He is regularly invited to deliver his lectures by the universities abroad. He supervised 8 postgraduates; all the supervised applicants successfully defended their Ph.D. theses. Since 2016, he is Head of Electromechanics and Robotics Department at SUAI. Professor Ronzhin is a chairman of SPC RAS Scientific Council. He is Deputy Chairman of the Doctoral Council D 002.199.01; member of the doctoral council D 999.121.03; member of the Federal Educational and Methodological Association in the field of higher education in the enlarged group of specialties and areas of training 13.00.00 “Electricity and heat power engineering”. He has repeatedly served as the chairman, co-chairman and member of the IPCs at prestigious domestic and international conferences, such as: International Conference on Interactive Collaborative Robotics (ICR); International Conference on Engineering and Applied Linguistics “Piotrovskie Readings”; International Conference on Electromechanics and Robotics “Zavalishin’s Readings” (ER(ZR)); International Conference on Digitalization of Agriculture and Organic Production (ADOP). He is Deputy Editor-in-Chief of the Journal Informatics and Automation; Associate Editor of the Int. Journal of Intelligent Unmanned Systems; reviewer of Robotics and Autonomous Systems; a member of editorial boards of scientific journals: Speech Technologies; Analysis and data processing systems; System Engineering and Information Technologies. He is a member of International Academy of Navigation and Traffic Control; the RAS Scientific Council on Robotics and Mechatronics; Committee of the International Association for Speech Communication ISCA; Committee for Eastern Europe of the International Association for Speech Communication; Council of Directors of Research and Educational Organizations at the Department of NIT of RAS; Scientific Council for Informatization of St. Petersburg under the Government of St. Petersburg; the supervisory board of the worldclass scientific and educational center “Artificial Intelligence in Industry”; expert of the Russian Science Foundation, of the Russian Foundation for Basic Research, of the Russian Academy of Sciences, of the Federal State Scientific Institution NII RINKTSE; Skolkovo Foundation, Russian Venture Company JSC, Science Fund of the Republic of Serbia. Professor Ronzhin is the co-editor of 11 books for international conferences proceedings and the author of over 300 refereed journal papers, 7 manuals and 2 monographs. Dr. Tien Ngo born in 1987, is an Assistant Lecturer of the Aerospace Engineering Department at Le Quy Don Technical University since 2020 and part-time researcher at SPC RAS (2016–2020). He received his Ph.D. from the joint Dissertation Council incorporating the BonchBruevich St. Petersburg State University of Telecommunications (SUT), the St.

About the Authors

xxiii

Petersburg State University of Aerospace Instrumentation (SUAI) and the Baltic State Technical University “Voenmeh” of D.F. Ustinov (BGTU); he completed the postgraduate study at the SUAI Department of Electromechanics and Robotics; he received his B.Sc. in aviation engines from Moscow Aviation Institute (National Research University) (MAI) in 2020, 2019 and 2012, respectively. Dr. Tien Ngo’s research interests are agricultural robotics, including models and algorithms for service automation and control of interactions between heterogeneous agricultural robotic complexes; information, physical and energy interaction of ground and air robotic systems. Dr. Ngo is the author of over 18 refereed journal papers. Dr. Quyen Vu born in 1987, is an Assistant Lecturer of the Aerospace Engineering Department at Le Quy Don Technical University since 2021 and part-time researcher at SPC RAS (2017–2021). He received his Ph.D. from the joint Dissertation Council incorporating the BonchBruevich St. Petersburg State University of Telecommunications (SUT), the St. Petersburg State University of Aerospace Instrumentation (SUAI) and the Baltic State Technical University “Voenmeh” of D.F. Ustinov (BGTU); he completed his postgraduate study at the SUAI Department of Electromechanics and Robotics; he received his diploma of specialist in servo drive systems for aerospace vehicles from Moscow Aviation Institute (National Research University) (MAI) in 2021, 2020 and 2011, respectively. Dr. Quyen Vu’s research interests concern, along with the system analysis and information processing, agricultural robotics, including developing methods and models for multi-criteria synthesis of robotic means, like grippers and their configurations for manipulation with agricultural products. Dr. Vu is the author of over 20 refereed journal papers. Dr. Vinh Nguyen born in 1988, is a part-time researcher at SPC RAS (2017–2021). He received his Ph.D. from the joint Dissertation Council incorporating the BonchBruevich St. Petersburg State University of Telecommunications (SUT), the St. Petersburg State University of Aerospace Instrumentation (SUAI) and the Baltic State Technical University “Voenmeh” of D.F. Ustinov (BGTU); he completed his postgraduate study at the SUAI Department of Electromechanics and Robotics; he received his B.Sc. in machine engineering from Le Quy Don Technical University, Hanoi, Vietnam, in 2021, 2020 and 2012, respectively. Dr. Vinh Nguyen’s research interests lie in the field of agricultural robotics and cover design and control of manipulators and robotic grippers placed onboard the unmanned aerial vehicles; he also specializes in the issues of developing applications for manipulators’ motion control as aimed at maintaining the developed means’ stability in different modes. Dr. Nguyen is the author of over 21 refereed journal papers.

Part I

Automation of Service and Interaction Control of Heterogeneous Agricultural Robots

Chapter 1

Analysis of Existing Approaches to the Service Automation and to Interaction Control of Heterogeneous Agricultural Robots

Abstract The chapter describes model-algorithmic support of the interaction of UAV and ground-based robotic platforms that carry out the functions of their transportation and service. The problem of increasing the operating time of unmanned aerial vehicles (UAV) in autonomous agricultural missions is discussed. The approaches to charge or replace onboard batteries on an accompanying robotic platform are analyzed. The existing prototypes of service robotic platforms are distinguished by the complexity of the internal mechanisms, the speed of service, the algorithms for the platform and the aircraft to work together during landing and battery maintenance. The classification of existing service systems installed on robotic platforms for servicing batteries and built-in UAV containers has been compiled based on the results of the analysis.

1.1 Analysis of Existing Ground-Based Robotic Service Platforms An analytical review prepared on the basis of a study of recent publications available in the citation systems of the RIC, Scopus, as well as in the Elsevier, Springer libraries, showed an active study of the problems of robotization of the agro-industrial sector [1–8]. In particular, the problems of precision farming are studied [2, 9– 17], robotization in livestock farming [18, 19], application of artificial intelligence systems [20] and other aspects of the use of heterogeneous robotic tools in agriculture [21–30]. In recent years, unmanned aerial vehicles and, in particular, multi-copters (MCs) have been the subject of research by many scientific communities, military and civilian companies. Due to their versatility and the possibility of programming algorithms of their functioning, a wide range of tasks can be accomplished using multicopters, for example, searching for objects, inspecting buildings, observing, etc. One of the main unsolved problems is the need to increase the duration of autonomous work. The average flight time is usually limited to 10–25 min for lightweight multicopters using Li-Po batteries. It is possible to increase the operating time of the © The Author(s), under exclusive license to Springer Nature Switzerland AG 2022 A. Ronzhin et al., Ground and Air Robotic Manipulation Systems in Agriculture, Learning and Analytics in Intelligent Systems 27, https://doi.org/10.1007/978-3-030-86826-0_1

3

4

1 Analysis of Existing Approaches to the Service Automation …

multi-copter by searching for an improved power source or developing a system of quick battery charging. For the latter option, two types of systems have been proposed: active and passive. Active systems provide a short lead time for the multicopter, but require sophisticated electromechanical mechanisms to replace an empty battery with a new one. Passive systems are somewhat simpler, but it takes about 10 min to 1 h to charge the MC battery, which increases the delay in completing the MC’s main mission. To service UAV batteries options for using service robotic platforms, upon landing on which the UAV charges or replaces its battery to continue the implementation of an autonomous flight mission are now being investigated [31]. A multifunctional mechanism for connecting a MC with a ground-based robotic platform, which carries out the functions of their transportation and maintenance, is one of the main elements [32]. At the moment, three main tasks have been formed, which must be technically implemented on a mobile platform according to its design features, including: 1.

2.

3.

charging the MC with the possible implementation of three options for transfer of energy using an energy supply system on the platform (contact connection of the MC battery with the platform power system; replacement of the MC battery; wireless charging of the MC battery); contact interaction of the platform with a set of MC, including the MC and the platform docking mechanism, involving the safe movement of the MC on board the platform, landing and the take-off of the MC; communication of the mobile platform with the MC and the base station. Next, we will analyze the existing solutions for the above three tasks.

Autonomous UAV landing in modern research is considered not only on a fixed site, but also on a mobile platform moving in different environments. The UAV landing on the service charging station is carried out using various navigation systems and analysis of the surrounding area. The paper [33] proposes a vision system capable of detecting UAV and accompanying it before landing on the platform. Recognition of UAV templates will allow assessing its position and orientation when approaching the landing site. The proposed system operates in real time on onboard computing resources indoors and outdoors without the support of global navigation systems. A new decentralized method of controlling the joint functioning of a UAV and a mobile platform is considered in paper [34]. The presented experimental results for the small quadcopter Aeryon Scout and the Clearpath Robotics A200 Husky mobile platform confirm the possibility of landing both indoors with high-quality navigational data and outdoors in windy conditions. The work [35] proposes a system for tracking a mobile platform and monitoring the landing of a UAV on it. The system uses a detection and location algorithm for landing sites based on technical vision and an omnidirectional camera with high image quality. Analysis of the video stream makes it possible to assess the position and speed of the mobile platform relative to the UAV. The landing system was tested on a quadcopter that successfully landed on a mobile platform during outdoor flight tests.

1.1 Analysis of Existing Ground-Based Robotic Service Platforms

5

In [36], an algorithm for autonomous UAV landing on the deck of a ship is considered. A movable landing pad with six degrees of freedom used in the experiments to simulate the dynamics of various ships and sea states. The developed technical vision system uses the Kalman filter to ensure the reliability of estimates, to determine the position of the UAV relative to the platform, with special graphic marks. Analysis of energy consumption of the built-in mobile platform modules equipped with a two-axial turning area for UAV landing is considered in the paper [37]. To achieve a longer service platform operating time, it is recommended using more efficient sensors rather than increasing the size of the built-in rechargeable batteries. In addition, solar panels are installed on the platform, prolonging its operation and maintenance of the UAV. To increase the autonomy of an unmanned aerial vehicle, it is required, among other things, to recharge its energy source and replenish other consumables based on automated recharge systems. In the paper [38], two types of automatic systems for recharging the MC on a ground-based platform were developed with battery charging and replacement with a new one. Systems with a replaceable battery can significantly reduce the time for preparing an MC for a new flight and increase the total number of MCs co-located in an autonomous mission. The recharging system has a lower cost compared to the battery replacement system by minimizing mechanical components of the structure. In the work [39], the basic approaches to increasing the flight time of a UAV by reducing energy consumption are considered, in particular: 1. 2. 3. 4. 5. 6.

the use of new lightweight materials; reducing the energy consumption of on-board devices; improving the aerodynamic characteristics of the UAV; the use of hybrid constructing schemes of UAV, including the use of aerostatic unloading of aircraft and helicopter-type UAV; dynamic routing of UAV group’s flights; as well as a non-trivial solution—that is dropping of discharged power sources and thereby reducing the UAV mass.

In the work [40], it is substantiated that the search for the optimal trajectory of the UAV group, with periodic maintenance of energy supply systems on the accompanying group of ground-based mobile charging stations for some mission, is an NP-complete task, and a number of modifications to linear programming methods and heuristic approaches are proposed to solve it. In the work [41], UAV are considered to be vehicles, the cost of their mass use is estimated, including service at fixed refueling stations and the sequence of processing of UAV and dynamic delays at the arrival of UAV. In the paper [42], various approaches to controlling UAV motors are investigated in order to optimize energy consumption. Also, there are three levels of UAV control, at which energy costs can be optimized: 1.

higher—the calculation of trajectories of movement, taking into account the overflow of obstacles and minimization of time (or energy consumption);

6

2. 3.

1 Analysis of Existing Approaches to the Service Automation …

medium—calculation of kinematic and dynamic models of UAV movement along a given trajectory at the required speed; the lowest—the calculation of the parameters of voltage controllers and converters, current sources to maintain the required rotational speed of the UAV rotors. For the lowest level of control, the methods of Lyapunov, linear algebra, and PID controller are compared. At the same time, it is noted that the first methods allow reducing energy consumption, but require a long tuning process, and the PID controller is still the simplest and relatively effective approach.

Work [43] also reviewed approaches to reducing the energy consumption of UAV motor control systems. The analysis of the efficiency of various current sources, as well as the energy consumption of the UAV at various stages of operation: takeoff, climb, flight, lowering, landing. The most energy-consuming stages during the climb and landing (maneuvering while positioning to a given landing site) require special attention and optimization of control algorithms. The advantages and disadvantages of heuristic, intelligent (fuzzy logic, artificial neural networks, etc.) and optimization (dynamic programming, etc.) methods are given. In the work [44], it is noted that multi-copters are highly maneuverable UAV and are used to fly over complex trajectories in a confined space with a large number, including dynamic obstacles. Maneuverability certainly affects the high energy consumption, as well as the need to reduce the weight of the multi-copter by reducing onboard energy resources. The developed UAV wireless battery charging system features differ from the VICON camera system for accurate positioning and is fully automatic, which significantly reduces the maintenance cost. In the paper [38], three types of UAV battery charging stations are proposed: Rollin’ Mat, Concentric circles, Honeycomb, they differ in cost, capabilities, and functions. Rollin’ Mat and Concentric circles power stations are simple in design, easy to install and relatively inexpensive. However, the location and size of the terminals at the station depends on type of the aircraft, which naturally affects the size of the landing area at the station. In particular, if the UAV is rather small, then the landing accuracy provided by navigation systems may be insufficient for docking with charging pads. The Honeycomb power station has many advantages, the system is easily expandable: a large number of UAV can be charged simultaneously by adding additional cells and chargers. Another feature is that the wireless IR emitter/sensor of the communication system can be easily replaced with another wireless system. Honeycomb can be used in almost any situation where recharging is needed. But this solution is more expensive, therefore the Honeycomb platform is recommended in cases where precise landings are required on a small site in difficult weather conditions. Also in [38], a system for replacing the UAV battery was proposed. The use of the system significantly increases the coefficient of the maximum flight time and reduces the time spent and the number of UAV on the platform. On the other hand, the cost of implementing the system increases, because replacing an empty battery is more difficult than charging a UAV. For the functioning of the UAV battery replacement system, the implementation of the following functions is required: determining the

1.1 Analysis of Existing Ground-Based Robotic Service Platforms

7

position of the UAV, mechanizing the battery replacement process, charging the battery, operating of the battery store, transporting the batteries inside the station. Work [37] presents design options for functional components for replacing batteries at a service station. The developed system for replacing UAV batteries is designed to automatically replace discharged UAV batteries with new ones without human intervention. The main tasks of this system are as follows: 1. 2. 3. 4. 5. 6.

sending the UAV to the battery replacement station; UAV navigation to the station; fixing the UAV at the station; connection to the UAV: removal and placement of batteries; transportation of batteries; recharging batteries.

As a rule, ground-based service stations are located in the open air, where weather conditions cannot be predicted and the landing of the UAV is accomplished with some error. In the work [45], an approach that allows a UAV to get to the place of battery replacement, even if its landing site is not ideal due to navigation errors, weather conditions, damage to the UAV, and other factors, is considered. The mechanism by which the battery is securely fixed and physically connected to the UAV is also important, because it affects the complexity and time of battery manipulation. In addition, its additional weight will affect the size of the UAV’s payload and flight time. In order to create an interface between the UAV and the platform, mechanical and magnetic couplings, which can easily hold and release the battery, while simultaneously providing a terminal connection to the UAV, are considered. During the operation of the service station in work [45], several modules are used to replace the batteries with high accuracy: 1. 2. 3. 4.

battery fixation and orientation module; UAV locking/unlocking module; battery extraction module; battery replacement module. The station can compensate for orientation and positioning errors of the UAV during a landing. The proposed design of the ground-based station can also handle dissimilar UAV not only with different shapes and sizes, but also with different numbers of batteries.

To recharge the batteries, the MC remains on the platform until full charge, so the time that the MC spends on the platform is no less than the time it takes to charge the batteries. In a battery replacement system, this time is less, because only mechanical manipulations are carried out to change the power supply [45].

8

1 Analysis of Existing Approaches to the Service Automation …

1.2 Analysis of Requirements and Restrictions of Mobile Battery Maintenance Systems for Unmanned Aerial Vehicles Automation of the process of recovering the energy resources of the multi-copters on the ground-based robotic platform in the target solution area will allow to increase the duration of the multi-copters’ operation and the number of the tasks in the autonomous mission. Increased autonomy for the operation of UAVs and in particular multi-copters in the target area by the automatic charging system is the subject of many scientific studies. Mainly there are two types of station recharging—system of charge battery and systems with battery replacement. Systems with battery replacement can significantly shorten the preparation time of the multi-copters for a new flight and increase the total number of multi-copters that are simultaneously in the autonomous mission. The recharging system has a lower cost compared to the battery replacement system due to the minimization of the mechanical components of the structure. First, the development of charging robotic ground-based stations is relevant for solving the problems of monitoring remote territories, in which most of the energy resources are used on reaching the specified area. In this case, multi-copters have the opportunity to replenish their energy resources in the territory of operation and work in a continuous mode until the end of the platform resources. Second, the developing charging system solves the problem of automating the battery maintenance process and could be used in fixed locations. In the framework of this study, models and prototypes of the battery charging system are developed, characterized by wireless transmission of energy, mechanization of the manipulation process with the battery on the multi-copter and the service platform. Now ground-based robotic systems and unmanned aerial vehicles, including multi-copters are actively used in various industries both for solving applied specialized tasks and for the entertainment industry [8–10]. Considering the wide range of operators training and functional capabilities of the multi-copters, it is necessary to identified three most popular applications of multi-copters: 1. 2. 3.

researchers; farming; security [46].

Table 1.1 shows the basic requirements for service systems of energy supply systems for the three above categories [38]. Agricultural MCs monitor the movement of cattle, check the integrity of fences, control the quality of soil and crops, and spray agricultural fertilizers. The use of MC for entertainment, educational and research purposes are carried out during pilot training, modeling of single and group behavior, including in difficult weather conditions. Different customers also have different design constraints (budget, weight, complexity), but it is possible to identify some major common constraints; they are summarized in Table 1.2.

1.2 Analysis of Requirements and Restrictions of Mobile Battery …

9

Table 1.1 Requirements of various types of MC to the battery maintenance system in three subject areas [38] Security

Farming

Research

1. Charge UAV batteries without human assistance 2. MC safety during charging 3. Charging platform’s geometry which provides high probability of successful landing 4. Shape or color of the charging platform which are easy distinguishable from the surroundings, either by humans or navigation system 5. Erroneous landing on the charging platform inflicts no more damage than erroneous landing in normal landing pads 6. Parts of charging platform which are potentially harmful to humans, are out of reach of users 7. Simple system of charging platform set-up 8. Notification of completion for operator and/or navigation system 9. Support for regular use of the charging platform (steady landing, docking, assembly/ disassembly) 10. Easy to install and configure platform by one average human 11. The charging platform can operate without physical connection to the power grids or main base

From time to time, the platform would use the resources of a single power grid

12. Ensuring functioning of the platform under not-ideal weather conditions (light rain, light snow, wind)

Endures being wet without breaking, but mostly dry use

13. Maintenance of MC of various sizes

Maintenance of MC of one size

14. More than one UAV to charge may be served at the same time on the platform

Charge only one UAV at a given time

Table 1.2 Design constraints 1

Modifications to the helicopter, if any, should add as little weight as possible to prevent reduction in flight duration due to increase of body mass

2

Battery disabling systems must guarantee that the UAV will not be disabled unintentionally

3

UAV dimensions and physical properties

4

The battery is very sensitive to recharging voltage/current

5

UAV electronics should not be connected to battery during recharging

6

Pilot skills/auto-pilot skills

7

Human strength/skills

When developing automatic battery power maintenance system of the multi-copter should also take into account the following functional and technical requirements: – Provide identifiable landing space:

10

1 Analysis of Existing Approaches to the Service Automation …

1. 2. 3.

Provide sufficient area to land (bigger than 1.5 times the position error of navigation system and proportional to the number and size of skids/footprint/tail); Provide a visual landing mark of sufficient size and complexity for recognition by the MC navigation system; Provide means to communicate current position of platform to navigation system.

– Charge batteries: 1.

2. 3. 4.

Provide safe electrical interface between battery on multi-copter, multicopter electronics, charger on platform, and multi-copter detection system during service; Identify that multi-copter has landed in correct position; Charge battery; Identify charge needs.

– Provide power to the system: 1. 2.

Replenishment of the power supply; Adapt power to be used on the platform.

– Provide portability: 1. 2.

Provide easy setup; Provide way to transport.

Above functional and technical requirements will be taken into account when developing a ground-based robotic platform equipped with landing sites and multi-functional mechanisms for capturing unmanned aerial vehicles (UAV) and servicing their embedded power systems, taking into account previously developed technological solutions [32, 47].

1.3 Analysis of Approaches to Control the Interaction of Unmanned Aerial Vehicles and Ground-Based Robotic Service Platforms Autonomous landing of unmanned aerial vehicles in modern research is considered not only on a fixed site, but also on a mobile platform that moves in various environments. The UAV landing on the service charging station is carried out using various navigation systems and analysis of the surrounding area. Paper [33] proposes a vision system capable of detecting a UAV and accompanying it before landing on the platform. Recognition of template UAV models will allow assessing its position and orientation when approaching the landing site. The proposed system operates in real time on on-board computing resources indoors and outdoors without the support of global navigation systems.

1.3 Analysis of Approaches to Control the Interaction of Unmanned …

11

In [34], a new decentralized method of controlling the joint operation of a UAV and a mobile platform is considered. The presented experimental results for the small quadcopter Aeryon Scout and the Clearpath Robotics A200 Husky mobile platform confirm the possibility of landing both indoors with high-quality navigation data and outdoors in windy conditions. The work [35] proposes a system for tracking a mobile platform and monitoring the landing of a UAV on it. The system uses an algorithm for the detection and localization of the landing site based on technical vision and an omnidirectional camera with high image quality. Analysis of the video stream makes it possible to assess the position and speed of the mobile platform relative to the UAV. The landing system was tested on a quadcopter that successfully landed on a mobile platform during outdoor flight tests. In [36], an algorithm for autonomous UAV landing on the deck of a ship is considered. The experiments used a mobile landing pad with six degrees of freedom to simulate the dynamics of various ships and sea states. The developed system of technical vision uses the Kalman filter to ensure the reliability of the assessment of determining the position of the UAV relative to the platform, which has special graphic marks. The paper [37] analyzes the energy consumption of the built-in modules of a mobile platform equipped with a two-axial turning area for UAV landing. To achieve a longer service platform operating time, it is recommended using more efficient sensors rather than increasing the size of the built-in rechargeable batteries. In addition, solar panels are installed on the platform, which increase the duration of its operation and maintenance of the UAV. To increase the autonomy of an unmanned aerial vehicle, it is required to recharge its energy source and replenish other consumables based on automated recharge systems. In [38], two types of automatic MC recharging systems were developed—on a ground-based platform with battery charging and with replacement with a new one. A system with a replacement battery can significantly reduce the time for preparing an MC for a new flight and increase the total number of MCs simultaneously in an autonomous mission. The recharging system has a lower cost compared to the battery replacement system due to the minimization of mechanical components of the structure. In [38], three types of UAV battery charging stations are proposed: Rollin ’Mat, Concentric circles, Honeycomb. They differ in cost, capabilities, and functions. Rollin ’Mat and Concentric circles power stations are simple in design, easy to install and relatively inexpensive. However, the location and size of the terminals at the station depends on type of the aircraft, which naturally affects the size of the landing area at the station. In particular, if the UAV is rather small, then the landing accuracy provided by navigation systems may be insufficient for docking with charging pads. The Honeycomb type power station has many advantages— the system is easily expandable, because a large number of UAV can be charged at the same time by adding additional cells and chargers. Another feature is that the wireless IR emitter/sensor of the communication system can be easily replaced with another wireless system. Honeycomb can be used in almost any situation where

12

1 Analysis of Existing Approaches to the Service Automation …

recharging is needed. But this solution is more expensive, so the Honeycomb platform is recommended when precise landings are required on a small site in difficult weather conditions. Also in [38], a system for replacing the battery was proposed, the use of which significantly increases the coefficient of the maximum flight time, reduces the residence time and the number of UAV on the platform. On the other hand, the cost of implementing the system increases, because replacing an empty battery is more difficult than charging a UAV. For the functioning of the UAV battery replacement system, the following functions are required: determining the position of the UAV, mechanizing the process of replacing the battery, charging the battery, operating the battery store, transporting the batteries inside the station. Work [37] presents design options for functional components for replacing batteries at a service station. The developed system for replacing UAV batteries is designed to automatically replace discharged UAV batteries with new ones without human intervention. The main tasks of this system are presented: 1. 2. 3. 4. 5. 6.

sending the UAV to the battery replacement station; UAV navigation to the station; fixing the UAV at the station; connection to the UAV: removal and placement of batteries; transportation of batteries; recharging batteries.

As a rule, ground-based service stations are located in the open air, where weather conditions cannot be predicted, and the UAV landing is performed with some error. In [45], an approach that allows a UAV to get to the place of battery replacement is considered, even if its landing site is not ideal dues to navigation errors, weather conditions, damage to the UAV, and other factors. The mechanism by which the battery is securely fixed and physically connected to the UAV is also of great importance, as it affects the complexity and time of battery manipulation. In addition, its additional weight will be influenced by the size of the UAV’s payload and flight time. In order to create an interface between the UAV and the platform, mechanical and magnetic couplings are considered that can easily hold and release the battery, while simultaneously providing a terminal connection to the UAV. During the operation of the service station in work [59], several modules are used to replace the batteries with high accuracy: 1. 2. 3. 4.

battery fixation and orientation module; UAV locking/unlocking module; battery extraction module; battery replacement module. The station can compensate for errors in orientation and positioning of the UAV during a landing. The proposed design of the groundbased station can also handle dissimilar UAV not only with different shapes and sizes, but also with different numbers of batteries.

To restore the batteries, the MC remains on the platform until they are fully charged, so the time that the MC spends on the platform is no less than the time

1.3 Analysis of Approaches to Control the Interaction of Unmanned …

13

UAV’s battery power systems

By battery recovery method

With battery charging

With battery replacement

By type of basing

By way of navigation when landing

Abovewater

Toroidal

Terrestrial By connection method

By the way the battery is attached

Contact

Contactless

Magnetic

By store type

Plane

Local navigation systems

Global navigation systems

Linear

Mechanical

By the shape of the landing site

Disc

With mechanical guides to the center of the platform

Vision systems

Fig. 1.1 Classification of robotic systems for servicing UAV batteries

it takes to charge the batteries. In the battery replacement system, this time is less, because only mechanical manipulations are carried out to change the power source [45]. Taking into account the works analyzed in the first chapter, Fig. 1.1 shows the classification of robotic systems for servicing MC batteries. The analyzed systems can be divided according to the following criteria: 1. 2. 3. 4.

by type of basing: ground and surface; by the method of navigation during landing: local navigation systems, technical vision systems and global navigation systems; according to the shape of the landing area: flat, toroidal, system with mechanical guides to the center of the landing, flat; by the method of battery recovery: systems with battery charging, systems with battery replacement.

In a battery-charged system, there are two types: contact and non-contact. In a system with a battery replacement, depending on the method of battery fastening, there are magnetic and mechanical systems; according to the type of battery placement, there are linear and disc stores. Designs of maintenance systems for MC batteries with a battery charge are simpler and less expensive. At the same time, the service speed is not high and the number of simultaneously charging MCs is limited. Systems with a battery replacement have a more complex design, but the speed of their service is much faster and therefore the number of MCs that have been charged increases.

14

1 Analysis of Existing Approaches to the Service Automation …

The most promising battery recovery method is using wireless methods. The main problem of existing commercially available solutions for wireless power transmission systems is low transmitted power. The battery capacity of autonomous robots significantly exceeds the capacity of power supplies for smartphones and other lowpower devices, the vast market of which dictates the requirements of developers and manufacturers. Recharging robotic systems requires power that is dozens of times higher than what chargers of existing standards can offer.

1.4 Analysis of Approaches to Physical Resources Redistribution Between Agricultural Machinery of Different Autonomy’s Degree In this section, we will consider the existing approaches and systems for loading/unloading physical resources that are used by agricultural equipment, including robotic ones, when processing crops. If, with the development and active use of UAV, the problem of service charging stations began to be studied in an avalanche-like manner, then there are still very few publications on the exchange of the physical resources between robots, and in particular in the field of agriculture. Most of the works are devoted to loading liquids from stationary containers into the tank of land-based agricultural machinery [48–52]. In order to understand what useful resources we are dealing with in refueling UAV containers, first of all, we will consider the many processing operations that are performed when growing various crops. Table 1.3 describes the processing characteristics of some common crops. In particular, such features as the number of processing operations per year and the cost of their execution for each type of agricultural crop are noted. Table 1.3 Cost and number of operations for growing crops

Agricultural

Number of processing operations per year

Cost per 1.0 ha, rub

Winter cereals

25

12,807.0

Millet

21

11,441.0

Potatoes

19

62,196

Cucumber

15

67,025.0

Oats

16

7749.6

Corn

19

13,810.0

Peas

12

7161.15

Buckwheat

21

9462.5

Winter barley

16

8818.5

Sunflower

24

10,335

Soy

25

12,807.0

1.4 Analysis of Approaches to Physical Resources Redistribution …

15

Table 1.4 shows the operations of the agricultural production process, their duration and cost, as well as the possibilities of mechanization. It can be seen that the cost of non-mechanized operation is much higher. When compiling the above tables, the initial data obtained during the experimental studies described in the works [53] were used. Next, we will consider the publications of recent years, which describe methods and systems for replenishing the resources of robotic and mechanized agricultural machinery that irrigate, apply herbicides, fertilizers, seeds from their built-in container. Ground-based stations with automatic refueling of liquid resources onto ground robots are described in [54]. The station is equipped with a metal bar, at the end of which there is a manipulator with two degrees of freedom, which guides the hose into the funnel of the robot that has approached a certain position. In [55], the fuel and fertilizer consumption of a ground-based robot is estimated depending on type of the maneuver (∧ or U) when turning, and the service time when refueling. The technological process of applying mineral fertilizers requires to use various agricultural machinery: dump trucks, loaders, and fertilizers. In this case, the total field area, distance to the field, fertilizer seeding rate, fertilizer grasp width, average time for unloading a dump truck, filling a loader, unloading into a fertilizer, carrying capacity of a dump truck, loader, fertilizers, average speeds of a loader, fertilizer and other parameters were taken into account. The proposed mathematical model made it possible to determine the costs and completion time of the process of applying mineral fertilizers, the structure and composition of the optimal links “tanker—fertilizer”. The work [56] presents an unmanned aerial vehicle capable of carrying out automatic flight missions with a range of up to 5 km with a payload of up to 10 kg. The tank with the liquid used for spraying and dispensers are realized in the form of quick-detachable modules. The way to replace them is not described in the article, it is probably made by the operator. The reader’s attention is also emphasized that the use of UAV in agriculture has also an advantage—a decrease in the load on the soil cover and the complete absence of soil compaction in the field [57]. In addition to UAV, another method without physical contact with the soil is the use of cable robots, as well as crane systems [58]. Returning to the studies of the Kuban State Agrarian University, already in a subsequent work [59], the authors explicitly set the task of joint operation of “flying” dispensers with a specialized vehicle that automatically provides them with a working solution, storage, and transportation. Nevertheless, no methods of automatic replenishment of UAV resources on a ground-based refueling platform are proposed in the work. In [60], the features of UAV flight control when spraying agricultural fields with liquid agents are considered. Pesticides and chemicals must be spread at a certain distance from plants, the height of which is different even in the same field. Crops are usually planted in parallel furrows, and the flyover over them should be done evenly with an acceptable margin of error.

16

1 Analysis of Existing Approaches to the Service Automation …

Table 1.4 Characteristics of the operations of the agricultural production process Serial

Name of operations

Number of attendants, people

Cost per 1.0 ha, rub

Level of mechanization

1

Disc stubble cultivation; cm/ha

1

600.0

T-150

2

Transportation min. fertilizer for 5 km and application; kg/ha

2

635.0

MTZ-80

3

Fall plowing; cm/ha

1

1090.0

T-150

4

Chiseling (leveling) plowing; cm/ha

1

820.0

T-150

5

Seed treatment with bag-filling rizorfin; kg

2

205.0

Electric motor

6

Harrowing plowing, cm/ha 4–6 cm

1

550.0

T-150

7

Presowing cultivation, 1 cm/ha

400.0

T-150

8

Loading, 2 transportation of seeds to sowing units, 5 km

350.0

MTZ-80

9

Continuous sowing, kg/ha

1

405.0

MTZ-80

10

Harrowing of crops with rolling, cm/ha

1

360.0

T-150

11

Pre-emergence application of Prometrin, l/ha

1

455.0

MTZ-80

12

Sealing of Prometrin after application, ha

1

360.0

T-150

13

Harrowing of crops for 1 soybeans, cm

355.0

MTZ-80

14

Protection of crops from murine rodents, grain bait with Clerat, G

346.0

Manually

15

Quadruple 1 examination of crops to identify the phytosanitary state, ha

1000.0

Manually

16

Manual weeding, RUB/day, 3 times

10

15,000.0

Manually

17

Manual collection, 3 times

10

15,000.0

Manually

1

(continued)

1.4 Analysis of Approaches to Physical Resources Redistribution …

17

Table 1.4 (continued) Serial

Name of operations

Number of attendants, people

Cost per 1.0 ha, rub

Level of mechanization

18

Preparation of working solution: Forte, CE

2

250.0

Electric motor

19

Spraying of crops against pests, according to EPV

1

305.0

MTZ-80

20

Transportation of working solution of herbicides, 5 km/l

1

210.0

MTZ-80

21

Application of working solution of herbicides; l/ha

1

305.0

MTZ-80

22

Cutting of temporary irrigation ditches, ha

1

555.0

T-150

23

1st vegetation irrigation, m3/ha

1

1395.0

T-150

24

2nd vegetation irrigation, m3/ha

1

1395.0

T-150

25

Harvesting of seeds with chopping and collection of straw, ha

1

965.0

CK-5«Niva»

26

Seed transportation, 5 km/c

1

345.0

CAZ-53A

27

Chopped mass transport, 5 km/t

1

200.0

MTZ-80

The methodology for calculating the flight route of a UAV when spraying pesticides over a field of crops, taking into account weather conditions, is described in [61]. By monitoring the processing done, the route is adjusted. The advantages and features of the use of UAV in mountainous areas when solving agricultural problems are considered in [62]. Most of the private fruit farms are located in mountainous areas with fields up to 2 ha away from roads. Therefore, vertical take-off UAV with a payload of up to 15 kg are most popular for surveying and processing small fields located in the mountains. An experimental estimate of the payload mass is carried out in [63]. The influence of the rotational speed of the UAV propellers, the wind speed are taken into account when calculating the maximum mass of pesticides refueled on board the UAV. An analysis of the conditions for choosing the location of the runway is carried out in [64]. A multi-criteria assessment of the landing site quality is made using twelve input parameters based on a Bayesian network and is reduced to two main criteria: landing safety and goal attainability.

18

1 Analysis of Existing Approaches to the Service Automation …

The work [65] describes a UAV with a payload of up to 22 kg, equipped with a spraying system with a 5-L reservoir, which is sufficient for treating a field up to 0.14 km2 . The experimental results of reducing the battery charge and the mass of the sprayed liquid are shown in [66]. The volume of the tank is selected in such way that when the liquid ends, the battery charge is just enough to get to the landing site. When calculating economic costs, depreciation of equipment, insurance, taxes, fuels and lubricants, tire wear, repair costs, wages of operators of robotic and other equipment are taken into account. The prospects for using UAV not only for monitoring and spraying liquids, but also for crops and even forest restoration are also discussed. Compared to ground-based equipment, the use of UAV in agricultural tasks provides a number of basic advantages: no physical contact with the ground and no soil compaction, a wider monitoring and processing area, better processing of crops with liquid means due to the rotation of rotors without the use of additional devices. In the analytical report of J’son & Partners Consulting, UAV are classified according to the following main characteristics [67]: • • • • • • • • • • • • •

by design/configuration; by type of takeoff; for the intended purpose: by technical characteristics; by the type of power supply of the power plant; by the payload; by type of automation system; on the collision avoidance system; by type of navigation; by types of signal jamming protection; by the bandwidth of the radio frequency spectrum; onboard data processing; software specialization.

However, nothing is said about the maintenance, refueling and replenishment of the UAV’s resources, it is assumed that all these operations are performed manually immediately before the launch of the UAV. Taking into account the analysis carried out, this classification should be expanded by adding a few more criteria describing the types of service stations: – – – – –

by the degree of mobility; by the degree of autonomy of management; by the degree of autonomy of servicing the stations themselves; by type of service; by the type of resources used during UAV maintenance (Fig. 1.2).

Additional classification criteria indicate that the UAV container service systems can be divided:

1.4 Analysis of Approaches to Physical Resources Redistribution …

19

– according to the degree of mobility to stationary, trailed and mobile; – according to the degree of autonomy of control, service stations have types of remote-controlled, automatic and mechanized; – according to the degree of autonomy of service, service systems are divided into systems with refueling resources through pipes, refueling resources with special vehicles and returning to the base for refueling resources; – by the type of resources, systems are classified by the aggregate state of resources and by purpose. According to the state of aggregation, resources are divided into liquid (for example, solutions and emulsions) and solid (for example, granular and powdery). According to their purpose, resources can be divided into seeds, fertilizers, and pesticides. Fertilizers include such types as organic, mineral and bacterial. Among pesticides, the main options are: insecticides (against insects), herbicides (against weeds), fungicides (against fungi), and zoocides.

UAV’s container service system

By appointment

Fertilizers

Pesticides With the return to base for refueling resources Solid Liquid Insecticides (against insects)

Solutions

Movable

Trailed

Stationary

By form Emulsions

Granulated

Seeds

Bacterial

By state of aggregation

Organic

With refueling of resources with special vehicles

By resource type

Mineral

With refueling of resources through pipes

By the degree of autonomy of servicing the stations themselves

Zoocides

Automatic

Telecontrolled

Mechanized

By the degree of autonomy of management

By the degree of mobility

Herbicides (against weeds)

Powdered

Fig. 1.2 Classification of robotic service systems for UAV containers with payload

Fungicides (against fungi)

20

1 Analysis of Existing Approaches to the Service Automation …

1.5 Conclusion Unmanned aerial vehicles are now actively used to monitor land, map land yields, and plan fertilization zones. The first prototypes are already appearing that physically interacts with surrounding objects, which requires the consumption of even more energy resources. The joint work of robotic ground platforms and multi-copters can significantly increase the duration of an autonomous mission. The existing prototypes of service robotic platforms are distinguished by the complexity of the internal mechanisms, the speed of service, the algorithms for the joint operation of the platform and the aircraft during landing and battery maintenance. Autonomous UAV landing in modern research is considered not only on a fixed site, but also on a mobile platform those moves in various environments. The UAV landing on the service charging station is carried out using various navigation systems and analysis of the surrounding area. The requirements of unmanned aerial vehicle battery maintenance systems are analyzed for three main target areas: entertainment, farming, and security. Various classes of UAV battery charging stations have been identified, differing in the processing speed and the complexity of the servicing mechanism. Robotic systems for servicing power supply modules MC can be divided into two main categories: with charging or replacing the battery. Systems with battery replacement have more complex designs, but provide a high service rate, and therefore the number of UAV that have undergone replenishment of energy resources increases. Based on the results of the analysis of existing approaches, original classifications of robotic systems for servicing UAV batteries and UAV containers with the payload are proposed.

References 1. Khort, D.O., Filippov, R.A., Kutyrev, A.I.: Development of a universal agricultural robot. J. Adv. Res. Tech. Sci. 13, 69–73 (2019). https://doi.org/10.26160/2474-5901-2019-13-69-73 2. Fernandez, B., Herrera, P., Cerrada, J.: Robust digital control for autonomous skid-steered agricultural robots. Comput. Electron. Agric. 153, 94–101 (2018). https://doi.org/10.1016/j. compag.2018.07.038 3. Dong, F., Petzold, O., Heinemann, W., Kasper, R.: Time-optimal guidance control for an agricultural robot with orientation constraints. Comput. Electron. Agric. 99, 124–131 (2013). https:// doi.org/10.1016/j.compag.2013.09.009 4. Polo, J., Hornero, G., Duijneveld, C., García, A., Casas, O.: Design of a low-cost Wireless Sensor Network with UAV mobile node for agricultural applications. Comput. Electron. Agric. 119, 19–32 (2015) 5. Mahmuda, M., Abidina, M., Mohameda, Z., Rahmana, M., Iidab, M.: Multi-objective path planner for an agricultural mobile robot in a virtual greenhouse environment. Comput. Electron. Agric. 157, 488–499 (2019). https://doi.org/10.1016/j.compag.2019.01.016 6. Bodunde, O.P., Adie, U.C., Ikumapayi, O.M., Akinyoola, J.O., Aderoba, A.A.: Architectural design and performance evaluation of a ZigBee technology based adaptive sprinkler irrigation robot. Comput. Electron. Agric. 160, 168–178 (2019). https://doi.org/10.1016/j.compag.2019. 03.021

References

21

7. Blok, P., Boheemen, K., Evert, F., IJsselmuiden, J., Kim, G.: Robot navigation in orchards with localization based on particle filter and Kalman filter. Comput. Electron. Agric. 157, 261–269 (2019). https://doi.org/10.1016/j.compag.2018.12.046 8. Xiong, Y., Peng, C., Grimstad, L., From, P., Isler, V.: Development and field evaluation of a strawberry harvesting robot with a cable-driven gripper. Comput. Electron. Agric. 157, 392–402 (2019). https://doi.org/10.1016/j.compag.2019.01.009 9. Yakushev, V.P.: Digital technologies of precision farming in implementation of smart farming priority of Russian. Vestnik Russ. Agric. Sci. 2, 11–15 (2019) 10. Kaivosoja, J., Jackenkroll, M., Linkolehto, R., Gerhards, M.: Automatic control of farming operations based on spatial web services. Comput. Electron. Agric. 100, 110–115 (2014). https://doi.org/10.1016/j.compag.2013.11.003 11. Huuskonen, J., Oksanen, T.: Soil sampling with drones and augmented reality in precision agriculture. Comput. Electron. Agric. 154, 25–35 (2018). https://doi.org/10.1016/j.compag. 2018.08.039 12. Comba, L., Biglia, A., Aimonino, D.R., Gay, P.: Unsupervised detection of vineyards by 3D point-cloud UAV photogrammetry for precision agriculture. Comput. Electron. Agric. 155, 84–95 (2018). https://doi.org/10.1016/j.compag.2018.10.005 13. Nemenyi, M., Mesterhazi, P.A., Pecze, Zs., Stepan, Zs.: The role of GIS and GPS in precision farming. Comput. Electron. Agric. 40(1–3), 45–55 (2003) 14. Khanal, S., Fulton, J., Shearer, S.: An overview of current and potential applications of thermal remote sensing in precision agriculture. Comput. Electron. Agric. 139, 22–32 (2017). https:// doi.org/10.1016/j.compag.2017.05.001 15. Isgin, T., Bilgic, A., Lynn Forster, D., Batte, M.T.: Using count data models to determine the factors affecting farmers’ quantity decisions of precision farming technology adoption. Comput. Electron. Agric. 62(2), 231–242 (2008) 16. Chavan, T.R., Nandedkar, A.V.: AgroAVNET for crops and weeds classification: a step forward in automatic farming. Comput. Electron. Agric. 154, 361–372 (2018). https://doi.org/10.1016/ j.compag.2018.09.021 17. Liu, Y., Noguchi, N., Liang, L.: Development of a positioning system using UAV-based computer vision for an airboat navigation in paddy field. Comput. Electron. Agric. 162, 126–133 (2019). https://doi.org/10.1016/j.compag.2019.04.009 18. Belanche, A., Ignacio Martin-Garcia, A., Fernandez-Alvarez, J., Pleguezuelos, J., Yanez-Ruiz, D.R.: Optimizing management of dairy goat farms through individual animal data interpretation: a case study of smart farming in Spain. Agric. Syst. 173, 27–38 (2019). https://doi.org/ 10.1016/j.agsy.2019.02.002 19. Liu, Y., Cong, M., Zheng, H., Liu, D.: Porcine automation: robotic abdomen cutting trajectory planning using machine vision techniques based on global optimization algorithm. Comput. Electron. Agric. 143, 193–200 (2017) 20. Nagoev, Z.V., Denisenko, V.A., Lyutikova, L.A.: System of autonomous robot machine vision for agricultural application in mountain territories based on the multi-agent cognitive architectures. Sustain. Dev. Mt. Territ. 10(2)(36), 289–297 (2018) 21. Novikov, P.A., Khomonenko, A.D., Yakovlev, E.L.: Software for mobile indoor navigation using neural networks. Inf. Control Syst. 1(80), 32–39 (2016) 22. Khomonenko, A.D., Starobinets, D.Y., Lokvitskii, V.A.: A model of estimating quickness of functioning of onboard control systems of spacecraft remote sensing of the Earth. SPIIRAS Proc. 3(46), 49–64 (2016). https://doi.org/10.15622/sp.46.5 23. Drenjanac, D., Tomic, S.D.K., Klausner, L., Kuhn, E.: Harnessing coherence of area decomposition and semantic shared spaces for task allocation in a robotic fleet. Inf. Process. Agric. 1(1), 23–33 (2014). https://doi.org/10.1016/j.inpa.2014.03.001. 24. Salem, F.A.: Refined models and control solutions for mechatronics design of mobile robotic platforms. Est. J. Eng. 19(3), 212–238 (2013). https://doi.org/10.3176/eng.2013.3.04 25. Ryu, K.H., Kim, G., Han, J.S.: AE—automation and emerging technologies: development of a robotic transplanter for bedding plants. J. Agric. Eng. Res. 78(2), 141–146 (2001). https:// doi.org/10.1006/jaer.2000.0656.

22

1 Analysis of Existing Approaches to the Service Automation …

26. Jha, K., Doshi, A., Patel, P., Shah, M.: A comprehensive review on automation in agriculture using artificial intelligence. Artif. Intell. Agric. 2, 1–12 (2019). https://doi.org/10.1016/j.aiia. 2019.05.004 27. Busse, M., Schwerdtner, W., Siebert, R., Doernberg, A., Bokelmann, W.: Analysis of animal monitoring technologies in Germany from an innovation system perspective. Agric. Syst. 138, 55–65 (2015). https://doi.org/10.1016/j.agsy.2015.05.009 28. Plessen, M.G.: Coupling of crop assignment and vehicle routing for harvest planning in agriculture. Artif. Intell. Agric. 2, 99–109 (2019). https://doi.org/10.1016/j.aiia.2019.07.001 29. Erfani, S., Jafari, A., Hajiahmad, A.: Comparison of two data fusion methods for localization of wheeled mobile robot in farm conditions. Artif. Intell. Agric. 1, 48–55 (2019). https://doi. org/10.1016/j.aiia.2019.05.002 30. Zhang, T., Liao, Y.: Attitude measure system based on extended Kalman filter for multi-rotors. Comput. Electron. Agric. 134, 19–26 (2017) 31. Jeong, Y., Kweon, I.S.: Relative Pose estimation for an integrated UGV-UAV robot system. In: ICIRA 2013. Part I. LNAI, vol. 8102, pp. 625–636 (2013) 32. Nguyen, V., Vu, Q., Solenaya, O., Ronzhin, A.: Analysis of main tasks of precision farming solved with the use of robotic means. In: 12th International Scientific-Technical Conference on Electromechanics and Robotics “Zavalishin’s Readings”—2017, MATEC Web of Conferences, vol. 113, p. 02009 (2017) 33. Cocchioni, F., Frontoni, E., Ippoliti, G., Longhi, S., Mancini, A., Zingaretti, P.: Visual based landing for an unmanned quadrotor. J. Intell. Robot. Syst. 84, 511–528 (2016) 34. Daly, J.M., Ma, Y., Waslander, S.L.: Coordinated landing of a quadrotor on a skid-steered ground vehicle in the presence of time delays. Auton. Robots. 38, 179–191 (2015) 35. Kim, J.W., Jung, Y.D., Lee, D.S., Shim, D.H.: Landing control on a mobile platform for multi-copters using an omnidirectional image sensor. J. Intell. Robot. Syst. 84, 529–541 (2016) 36. Sanchez-Lopez, J.L., Pestana, J., Saripalli, S., Campoy, P.: An approach toward visual autonomous ship board landing of a VTOL UAV. J. Intell. Robot. Syst. 74, 113–127 (2014) 37. Ioannou, S., Dalamagkidis, K., Valavanis, K.P., Stefanakos, E.K.: Improving endurance and range of a UGV with gimballed landing platform for launching small unmanned helicopters. J. Intell. Robot. Syst. 53, 399–416 (2008) 38. Kemper, P.F., Suzuki, K.A.O., Morrison, J.R.: UAV consumable replenishment: design concepts for automated service stations. J. Intell. Robot. Syst. 61, 369–397 (2011) 39. Fetisov, V.S., Artem’ev, A.E., Mufazzalov, D.F.: Avtomaticheskie servisnye stancii dlja obsluzhivanija jelektricheskih bespilotnyh letatel’nyh apparatov, p. 253. Electronic Library of the USATU «Ufa State Aviation Technical University» (2017) 40. Maini, P., Sujit, P.B.: On cooperation between a fuel constrained UAV and a refueling UGV for large scale mapping applications. In: 2015 International Conference on Unmanned Aircraft Systems (ICUAS), pp. 1370–1377 (2015) 41. Zhang, K., Lu, L., Lei, C., Zhu, H., Ouyang, Y.: Dynamic operations and pricing of electric unmanned aerial vehicle systems and power networks. Transp. Res. Part C: Emerg. Technol. 92, 472–485 (2018). https://doi.org/10.1016/j.trc.2018.05.011. 42. Gandolfo, D.C., Salinas, L.R., Serrano, M.E., Toibero, J.M.: Energy evaluation of low-level control in UAVs powered by lithium polymer battery. ISA Trans. 71, Part 2, 563–572 (2017). https://doi.org/10.1016/j.isatra.2017.08.010. 43. Lei, T., Yang, Z., Lin, Z., Zhang, X.: State of art on energy management strategy for hybridpowered unmanned aerial vehicle. Chin. J. Aeronaut. 32(6), 1488–1503 (2019). https://doi.org/ 10.1016/j.cja.2019.03.013 44. Junaid, A.B., Lee, Y., Kim, Y.: Design and implementation of autonomous wireless charging station for rotary-wing UAVs. Aerosp. Sci. Technol. 54, 253–266 (2016). https://doi.org/10. 1016/j.ast.2016.04.023 45. Suzuki, K.A.O., Filho, P.K., Morrison, J.R.: Automatic battery replacement system for UAVs: analysis and design. J. Intell. Robot. Syst. 65, 563–586 (2012) 46. Vladimir, K., Boris, V., Yuri, V., Vladimir, G., Oleg, V.: Intelligent data processing technologies for unmanned aerial vehicles navigation and control. Inform. Autom. (SPIIRAS Proc.) 45, 26–44 (2016)

References

23

47. Kodyakov, A.S., Pavliuk, N.A., Budkov, V.Yu.: Study of stability of antares anthropomorphic robot under the action of an external load. Mekhatronika Avtomatizatsiya Upravlenie 18(5), 321–327 (2017) 48. Conesa-Munoz, J., Valente, J., del Cerro, J., Barrientos, A., Ribeiro, A.: Integrating Autonomous Aerial Scouting with Autonomous Ground Actuation to Reduce Chemical Pollution on Crop Soil. Springer International Publishing, Switzerland (2016). Reis, L.P., et al. (eds.): Robot 2015: Second Iberian Robotics Conference. Advances in Intelligent Systems and Computing, vol. 418, pp. 41–53 (2015). https://doi.org/10.1007/978-3-319-27149-1_4. 49. Vondricka, J., Lammers, P.S.: Real-time controlled direct injection system for precision farming. Precis. Agric. 10(5), 421–430 (2009). https://doi.org/10.1007/s11119-008-9093-x 50. Gonzalez-de-Santos, P., Ribeiro, A., Fernandez-Quintanilla, C., Lopez-Granados, F., Brandstoetter, M., Tomic, S., Pedrazzi, S., Peruzzi, A., Pajares, G., Kaplanis, G., Perez-Ruiz, M., Valero, C., del Cerro, J., Vieri, M., Rabatel, G., Debilde, B.: Fleets of robots for environmentallysafe pest control in agriculture. Precis. Agric. 18(4), 574–614 (2017). https://doi.org/10.1007/ s11119-016-9476-3 51. Esau, T., Zaman, Q., Groulx, D., Farooque, A., Schumann, A., Chang, Y.: Machine vision smart sprayer for spot-application of agrochemical in wild blueberry fields. Precis. Agric. 19, 770–788 (2018). https://doi.org/10.1007/s11119-017-9557-y 52. Chen, Y., Parish, R.L., Merhaut, D.J., Bracy, R.P.: Description of an improved hydroponic research system for screening plants for nutrient abatement in constructed wetlands. Appl. Eng. Agric. 24(5), 1–6 (2008). https://doi.org/10.13031/2013.25258 53. Nagoeva, O.V., Anchekov, M.I.: Development of a software model for a robot combine control system. News of the Kabardin-Balkar Scientific Center of RAS 3(89), 15–22 (2019) 54. Ball, D., Ross, P., English, A., Milani, P., Richards, D., Bate, A., Upcroft, B., Wyeth, G., Corke, P.: Farm workers of the future: vision-based robotics for broad-acre agriculture. IEEE Robot. Autom. Mag. 24(3), 97–107 (2017). https://doi.org/10.1109/MRA.2016.2616541 55. Spekken, M., de Bruin, S.: Optimized routing on agricultural fields by minimizing maneuvering and servicing time. Precis. Agric. 14, 224–244 (2013). https://doi.org/10.1007/s11119-0129290-5 56. Merkulov, A.A.: Konstruktivno-tehnologicheskaja shema robotizirovannogo kompleksa dlja vnesenija rabochih rastvorov. In: XI Vserossijskaya konferencya molodyh uchenyh, posvjashhennaya 95-letiju Kubanskogo GAU i 80-letiju so dnja obrazovanija Krasnodarskogo kraja, pp. 402–403. Otvetstvennyj za vypusk A. G. Koshhaev (2017) 57. Chernyshev, V.V., Briskin, E.S.: Experience in development and testing of sprinkler walking machines. Bezopasnost’ zhiznedeatel’nosti 1, 34–38 (2012) 58. Young, S.N., Kayacan, E., Peschel, J.M.: Design and field evaluation of a ground robot for high-throughput phenotyping of energy sorghum. Precis. Agric. 20, 697–722 (2019). https:// doi.org/10.1007/s11119-018-9601-6 59. Kuceev, V.V., Merkulov, A.A.: Approaches to robotization of chemical protection of plants in selection process of grain crops. In: II nauchno-prakticheskoj konferencii molodyh uchenyh Vserossijskogo foruma po selekcii i semenovodstvu, pp. 130–132 (2018) 60. Wang, Z., Song, D., Qi, J., Han, J., Miao, Y., Meng, L., Zhao, S., Li, M.: A full-functional simulation and test platform for rotorcraft unmanned aerial vehicle autonomous control. In: Kim, J.-H. et al. (eds.) Robot Intelligence Technologies and Applications, vol. 208, pp. 537–547. AISC (2012). https://doi.org/10.1007/978-3-642-37374-9_52 61. Faiçal, B.S., Pessin, G., Filho, G.P.R., Furquim, G., de Carvalho, A.C.P.L.F., Ueyama, J.: Exploiting evolution on UAV control rules for spraying pesticides on crop fields. EANN 49–58 (2014) 62. Yang, S., Yang, X., Mo, J.: The application of unmanned aircraft systems to plant protection in China. Precis. Agric. 19, 278–292 (2018). https://doi.org/10.1007/s11119-017-9516-7 63. Koo, Y.M., Hong, J.G., Haider, B.A., Sohn, C.H.: Practical payload assessment of a prototype blade for agricultural unmanned rotorcraft. J. Mech. Sci. Technol. 32(12), 5659–5669 (2018). https://doi.org/10.1007/s12206-018-1113-9

24

1 Analysis of Existing Approaches to the Service Automation …

64. Schmitt, M., Stütz, P.: Multi-UAV based helicopter landing zone reconnaissance. Information level fusion and decision. In: Harris, D. (ed.) EPCE, Part II, LNAI, vol. 10276, pp. 266–283 (2017). https://doi.org/10.1007/978-3-319-58475-1_20 65. Huang, Y., Hoffmann, W.C., Lan, Y., Wu, W., Fritz, B.K.: Development of a spray system for an unmanned aerial vehicle platform. Appl. Eng. Agric. 25(6), 803–809 (2009) 66. Martinez-Guanter, J., Agüera, P., Agüera, J., Pérez-Ruiz, M.: Spray and economics assessment of a UAV-based ultra-low-volume application in olive and citrus orchards. Precis. Agric. 1–18 (2019). https://doi.org/10.1007/s11119-019-09665-7 67. The market of Unmanned Aerial Vehicles (UAV, drones) in Russia and in the world (2017). http://json.tv/en/ict_telecom_analytics_view/the-market-of-unmanned-aerialvehicles-uav-drones-in-russia-and-in-the-world-2017

Chapter 2

Models and Algorithms of Interaction Between Heterogeneous Agricultural Robots

Abstract This chapter presents a formal statement of the problem of interaction with heterogeneous agricultural robots, describes the developed model-algorithmic support for controlling the interaction of heterogeneous robotic systems during UAV service during group agricultural tasks.

2.1 Conceptual Model and Formal Statement of the Problem of Interaction Between Heterogeneous Agricultural Robots To formalize the problem of interaction between heterogeneous agricultural robotic systems and dispatching the process of servicing unmanned aerial vehicles on ground service platforms, the following conceptual model was proposed, including the following main entities: agricultural space, unmanned aerial vehicle, groundbased service platform, ground control center. The proposed model serves as a basis for analyzing methods of control and interaction of ground service platforms and unmanned aerial vehicles in solving agricultural problems. The main elements and connections of the proposed conceptual model are shown in Fig. 2.1. The agricultural space is simplified by the basic parameters of the geometrical mobility of ground vehicles on the arable land, the parameters of the crops to be grown, the list of necessary agricultural tasks and performance schedule as well as current weather conditions. The ground controls center communicates with heterogeneous robotic systems, calculates routes for their movement and planned agricultural tasks. Also, the ground controls center includes an automated workstation for the operator, but its functions with the development and increase in the degree of autonomy of robotic means will be reduced from decontrol to supervisory control and monitoring of unforeseen situations. When describing a ground service platform, it is necessary to know the current mode of its operation, coordinates, speed, state of the power supply system, the degree of filling the container with resources, as well as the current number of UAV © The Author(s), under exclusive license to Springer Nature Switzerland AG 2022 A. Ronzhin et al., Ground and Air Robotic Manipulation Systems in Agriculture, Learning and Analytics in Intelligent Systems 27, https://doi.org/10.1007/978-3-030-86826-0_2

25

26

2 Models and Algorithms of Interaction Between Heterogeneous …

Cultivated land maps Crop characteristics Robotic farming tasks Standards for the implementation of agricultural tasks Weather

Mode of operation Coordinates Battery status UAV’s speed Container resource volume

Automated operator's workstation Communication system with robotic complexes Map of routes of movement of ground robots UAV’s flight route map Schedule for agricultural tasks

Mode of operation Coordinates Speed Power system status Container resource volume The number of serviced UAVs on the platform

Fig. 2.1 Conceptual model of the control system of heterogeneous agricultural robotic complexes

on the platform. Among the parameters of the UAV under consideration, one should also note the current mode of its operation, coordinates, speed, the degree of battery discharge, the degree of filling the container with resources. Next, we set a formal statement of the control problem of the interaction of heterogeneous agricultural robotic complexes. Let there is a working agricultural space S characterized by a set of parameters: S = H, O, G,, where H = (h1 , h2 , …, hi , …, hI )—is the set of cultivated land, O = (o1 , o2 ,…, oj , …, oJ )—set of processing agricultural objects, G = (g1 , g2 ,…, gl , …, gL )—a set of objects for basing and storing robotic equipment, C = (c1 , c2 , …, ck , …, cK )—the set of crops grown. There are many unmanned aerial vehicles U = (u1 , u2 , …, um , …, uM ) and many ground service platforms P = (p1 , p2 ,…, pn , …, pN ), used in this area. It is necessary to solve the problem of developing model-algorithmic support for the control and interaction of unmanned aerial vehicles U and ground service platforms P for servicing the working agricultural space S in the presence of a set of resource constraints L, including the operation time, the available amount of

2.1 Conceptual Model and Formal Statement of the Problem …

27

equipment, including robotic, mineral and water resources, loading of processing objects, safety of trajectories of movement of equipment and other factors. The following tuple of parameters is used to describe the current state of the unmanned aerial vehicle ui : ui = Ci , Vi , Ei , Ti , fi , where C i —the current coordinates of the UAV, V i —the current speed of the UAV, E i —current charge UAV battery, T i —current duration of the UAV flight, f i —current mode of UAV operation. The following tuple of parameters is used to describe the current state of the ground service platform pj :   p j = C j , Vj , E j , N j , f j , where C j —current platform coordinates, V j —current platform speed, E j —current energy of the platform, N j —current number of UAV on the platform, f j —current operating mode of the platform. The parameters used in calculations and modeling-algorithmic support are presented in Table 2.1.

2.2 Structural and Functional Models of Interaction Between Heterogeneous Agricultural Robots Next, we will consider the structural model of the interaction of heterogeneous agricultural robotic complexes, presented in Fig. 2.2. The technical equipment of the ground service platform can be simplified division into two main groups: 1. 2.

a platform control system containing a communication and navigation unit, an energy supply control unit, a sensor unit, and a built-in equipment control unit; UAV service system, equipped with a UAV take-off/landing unit, a UAV positioning unit, a UAV battery maintenance unit, a unit for loading/unloading resources used to work on agricultural fields, such as agricultural products, fertilizers, seeds, etc.

At the moment, a lot of functions have been formed that are implemented by a mobile platform, taking into account its design features.

28

2 Models and Algorithms of Interaction Between Heterogeneous …

Table 2.1 Description of parameters Parameter symbol

Parameter description

N

Maximum number of platforms

M

The maximum number of seats for UAV located on one platform

n

Number of platforms

m

The number of seats for UAV located on one platform

h

The volume of the target task solved by one UAV per time interval t

e

The amount of electricity consumed by one UAV per time interval t

r

The amount of resources implemented by one UAV during the time interval t for solving the target problem

ho j

Target volume for arable agricultural land o j

oj smax

Total area of cultivated land

s

The area processed by one UAV per t

U emax

Maximum UAV battery charge

U emin

The minimum amount of UAV battery charge required for a guaranteed return to the platform

etu m

The current volume of the UAV battery charge u m

U rmax

Time of consumption of the maximum volume of the UAV container

rtu m

The current volume of container UAV u m

P emax

The maximum amount of energy resources of the service platform for servicing UAV batteries

P emin

The minimum amount of energy resources of the service platform required to guarantee the return of the platform to the central base

et n

p

Current volume of energy resources of the service platform pn for servicing UAV batteries

P rmax

Time spent on the maximum amount of physical resources of the service platform for servicing UAV containers

pn

The current volume of physical resources of the service platform pn for servicing UAV containers

rt

oj

st

Current cultivated area of agricultural land o j

t Pu

UAV service time on the platform

t PS

The average flight time of a UAV from the platform to the place of the cultivated land

t CP

Average travel time of the platform between the central base and the cultivated area

u pn

Dt m

Current distance from UAV u m to platform pn

U smax t U takeo f f stu m

Maximum area processed by one UAV Launch time interval between UAV Current land area processed by UAV u m (continued)

2.2 Structural and Functional Models of Interaction Between …

29

Table 2.1 (continued) Parameter symbol

Parameter description

cP

The notional cost of one unit of the service platform, including the UAV battery maintenance system

cU

The notional cost of one UAV

t sum

Total processing time of agricultural land

esum

The total energy spent on the processing of agricultural land

csum

The total cost of the involved equipment for the processing of agricultural land

Communication and navigation block

Built-in equipment control block

Sensor block

Power supply control block

UAV u1

UAV u2

Platform control system

... UAV uM

UAV takeoff / landing block

UAV positioning block

UAV battery maintenance block

Resource loading / unloading block UAV service system

Mobile Ground Service Platform p1 Mobile Ground Service Platform p2 Working agricultural space

...

Mobile Ground Service Platform pN

Fig. 2.2 Structural model of the interaction of heterogeneous agricultural robotic complexes

F = { f 1, f 2, f 3,... f 8 }, where: f 1 —movement of the platform in two modes: (1) movement from the base to the cultivated area and back, (2) movement across the agricultural area while performing current tasks; f 2 —Charging the UAV with the possibility of implementing three options for transmit electrical energy using the power supply system equipped on the base: (1)

30

2 Models and Algorithms of Interaction Between Heterogeneous …

the contact connection of the UAV battery to the power system of the base; (2) replace the UAV battery; (3) wireless UAV battery charging; f 3 —contact interaction of the platform with a set of UAV, including the mechanism for docking the UAV and the platform, providing for the safe movement of the UAV on board the platform, landing and the takeoff of the UAV; f 4 —loading the UAV with useful resources (for example, mineral fertilizers) from the platform container for distribution over the cultivated area; f 5 —acceptance of goods (fruits collected during harvesting, stones and other debris removed during cleaning) from the UAV into the platform container; f 6 —communication of the mobile platform with a set of UAV and a base station; f 7 —navigation of the mobile platform in the global coordinate system using a system of heterogeneous sensors; f 8 —management of planning and actions of the mobile platform in cooperation with a set of UAV in solving the assigned agricultural tasks. A distinctive feature of the developed prototype of the robotic platform is the presence of built-in parking spaces for several UAV. The main structural components of the platform are: 1. 2. 3.

4.

5. 6.

a chassis that ensures the movement of the platform on agricultural land; multi-sensor system for detecting local obstacles when moving the platform; a power supply system that provides the necessary power for consumption by the platform itself and by a set of UAV when it becomes necessary to recharge them; a navigation system, consisting of two subsystems, one of which controls the movement of the platform between the base and the working area based on global navigation means, and the second local system is deployed directly on the working area before the start of an agricultural operation using radio navigation UAV; a communication system that implements communication between the mobile platform and UAV and base stations; the body part of the platform containing the payload compartment, as well as parking spaces for a set of UAV.

Figure 2.3 shows a functional model of the platform in autonomous agricultural missions and the joint operation of UAV when processing agricultural objects. The UAV transmits the following parameters to the ground-based station through the communication system: flight time, battery capacity, current location of the UAV. At the same time, the UAV receives information about the location of the ground-based station and the availability of service. In case the UAV battery has a low charge or maintenance of the built-in container of physical resources (for example, with mineral fertilizers) is required, the UAV issues a request for landing. After confirming the possibility of servicing, the UAV lands at the selected ground-based station, the UAV monitoring system controls the UAV landing process at the ground-based station.

2.2 Structural and Functional Models of Interaction Between …

31

UAV control system Battery capacity, Landing mission, Position

Control

Communication system

Control

Position, landing place

Mobile platform

UAV landing control system

Payload compartment

Has landed yes or no, UAV position control

Landed yes or not, UAV position control

Fixed system and UAV locking mechanism

Mechanism for recharging / replacing UAV batteries

Capacity battery, control

Battery capacity, takeoff mission control

Control systems are ready UAVs for takeoff

Fig. 2.3 Functional model of the mobile platform

The communication system makes it possible to transfer information between the UAV and the ground-based station according to the parameters: UAV position, platform position, UAV battery capacity, type of service. The ground-based station has a payload compartment and a docking space for UAV. The payload compartment allows storing payloads such as fertilizers, seeds, harvesting, etc. Docking area for UAV includes as follows: UAV landing control system, UAV positioning system and locking mechanism, UAV battery recharging/replacement mechanism and monitoring systems UAV readiness for the takeoff.

2 Models and Algorithms of Interaction Between Heterogeneous …

75

32

160

75

Fig. 2.4 Constructive model of a mobile platform

The seating area of 75 cm allows servicing UAV with a projection size of up to 50 cm (the landing area is more than 1.5 times bigger than the navigation system error). The fixed system and the UAV locking mechanism in the model are designed along the platform edges, which can be opened using hydraulic lifting mechanisms. Figure 2.4 shows the design model for a mobile platform. The UAV landing control system allows you to check the UAV landing status, confirming that the UAV has landed or not. Then parameters such as the current location of the UAV are passed to the platform control engine. The fixed system and the UAV blocking mechanism receive information for confirming the UAV landing, controlling the positioned and fixed blocking mechanism for the UAV at the station, unlocking the UAV upon receipt of a confirmation that the UAV is ready to take off. At the same time, it accepts the parameters of the location, battery capacity, type of UAV service. The UAV readiness control system for the take-off receives information on the battery capacity, the UAV take-off mission. Then it passes control information to other built-in mechanisms. The mechanism for recharging/replacing the UAV batteries receives confirmation of the UAV landing, performs the process of recharging/replacing the UAV batteries and transmits information about the UAV battery charge level. The UAV landing control system confirms the UAV landing status when the UAV has landed on the ground-based station. This information is transmitted to the positioning system and the UAV locking mechanism, which fixes and locks the UAV in the required place. The UAV ready control system for the takeoff checks the battery capacity, then transmits the information to the UAV battery recharging/replacement mechanism, in which the UAV battery recharging/replacement process is performed

2.2 Structural and Functional Models of Interaction Between …

33

until the UAV ready for the takeoff control system confirms that the battery is charged. The UAV ready for the take-off control system receives information about the takeoff mission. When the system confirms that the UAV is ready for the take-off, the information is transmitted to the UAV locking mechanism for unlocking. When the UAV takes off, the UAV landing control system confirms that the UAV has taken off, and there is a free landing site, the station is ready to receive the next UAV. It should be noted that among the charging systems, the most promising are non-contact ones, capable of functioning in any weather conditions [1–3]. Next, we will consider some of the design features of the UAV positioning for connecting to the power supply system of the service platform. The batteries are held on the UAV in a specific position. To be able to work with them, it is necessary to place the UAV in a certain place after it has landed. This study assumes those groundbased stations are outdoors and weather conditions can change, making it difficult to land accurately. Therefore, further on, we will focus not on landing algorithms, but on ways to position the UAV after landing with a small error. UAV flight control systems commit some permissible error during landing due to navigation errors, weather conditions, etc. The challenge is to move the UAV from an imperfect landing position to the place where the battery will be replaced. Figure 2.5 shows a model for positioning and fixing a UAV in the center using four sliding arms that can move smoothly along spiral links. The sliding levers correspond to different heights so as not to interfere with each other. When the UAV has landed, the levers translate to a central position as the shaft rotates. The UAV is centered and locked in position for battery replacement. The battery retention system in a UAV should be lightweight, securely hold the battery during flight, withstand small shocks, support the UAV battery terminals, and ensure that the battery can be easily installed and removed if necessary. Figure 2.6 shows an example of a developed mechanical connector for the UAV battery compartment.

Fig. 2.5 Constructive model of the UAV positioning system on a ground-based service platform

34

2 Models and Algorithms of Interaction Between Heterogeneous …

Fig. 2.6 An example of a mechanical connector for a UAV battery compartment

It consists of a body and elastic angled latches that securely fixes the battery. When triggered upwards, the latches bend and allow the battery to pass through, as the battery case has matching notches on its sides, this allows the latch to lock the batteries in place. Next, we will consider the algorithm for servicing an unmanned aerial vehicle on a ground-based service platform. Taking into account the increasing prospects of service stations with wireless charging methods and with the replacement of the UAV battery, an algorithmic model was developed for the interaction of a UAV set with a service ground-based robotic platform. Figure 2.7 shows an algorithmic model of the operation of the service maintenance system for powering UAV batteries when solving a target agricultural problem. The first step is to control the current charge of the UAV battery. If the battery is discharged, then the UAV flies up to the platform. The next step is to place the UAV in the service queue in accordance with the current battery charge. After receiving a signal of the demand for charging from the UAV, the system notifies the UAV about the presence of a platform with a free platform for maintenance. The UAV, having received a signal, lands on the platform with an acceptable error, which is further eliminated by an automatic positioning system. After landing, the system positions the UAV in the desired position and blocks the UAV. Then the stage of removing the battery from the UAV comes. After the discharged battery has removed and moved to the charging area, the full battery is placed in the UAV. This is followed by the stage of undocking and the takeoff of the UAV.

2.2 Structural and Functional Models of Interaction Between …

35

Start Monitoring the current charge of UAV battery No Is the battery discharged? UAV approach to the platform Queuing up for service.

There is a free space service on the platform?

No

Yes Landing a UAV on a free charging place No UAV landed on the platform Yes Moving the UAV to the right orientation and docking the UAV Removing the battery from the UAV Moving the battery to the replacement area Inserting the battery into the UAV UAV undocking End Fig. 2.7 Algorithm of UAV maintenance on a ground service platform

36

2 Models and Algorithms of Interaction Between Heterogeneous …

2.3 Algorithmic Models to Control the Interaction of Heterogeneous Agricultural Robots at Different Service Stages The result of the interaction of the UAV uj with the platform pn is the provision of the service Q(uj ) on the territory of the agricultural space S, consisting in servicing the UAV battery eu m and/or servicing the container with the resources r u m : Q(um ) = f(eu m , r u m , pj , S). Let us consider the solution of the UAV service problem from the point of view of the theory of queuing systems (QS). Each ground-based service platforms in this case is a multi-channel system with the number of channels equal to the number of equipped seats. Let us analyze the possible options for using the classical types of QS [4, 5]. In a waiting system, when all the seats are occupied, the UAV is forced to land while waiting for the end of servicing other UAV on the given platform. In our case, with the use of several service platforms, this type of QS is not effective, since the UAV can be serviced on another service platform located somewhat further. In a failed system, when all the seats are occupied and the UAV is denied service, another platform with free seat is selected. Since the service ground-based platforms is used not only for servicing UAV, but also for their transportation, the number of functioning UAV does not exceed the total number of seats on the used set of platforms. From the point of view of the QS classification, a limited number of applications, available inside the closed system, reviewed. Therefore, any UAV is guaranteed to be serviced, but when choosing a service platform, it will be necessary to assess the distance, occupancy and sufficiency of resources of each operating platform. It should also be noted that service platforms are homogeneous, therefore, QS with parallel channels will be used. In the considered task of the agricultural application of UAV, the ground-based platform exchanges energy and physical resources with the UAV. Physical resources include, for example, mineral fertilizers, pest control chemicals, agricultural products. Within this work, the physical characteristics of the listed resources and the technical difficulties of manipulating them will not be considered. Here, for calculations, we will keep in mind that there are containers on the service platform and on the UAV, between which homogeneous resources are exchanged. When servicing the UAV batteries, a charged battery is installed in its container, and the old one is returned to the service platform. In the task of fertilizing, the UAV also replenishes its container with the resources available on the service platform. In the case of harvesting, a UAV that collects agricultural products in its container, on the contrary, empties its container into a large container on the platform. Figure 2.8 shows the main types of functioning states in which a UAV can be located, as well as possible transitions between them when performing tasks on agricultural fields. For the subsequent simulation, we will use the QS apparatus

2.3 Algorithmic Models to Control the Interaction of Heterogeneous …

Transport on the platform

Loading/ unloading of the container on the platform

37

Preparation and take off from the platform

Flight to the location of the task

Completing the task

Replacing the battery on the platform

Landing on the platform

Flight back to the platform

Fig. 2.8 States of UAV functioning

with heterogeneous nodes, while we will assume that the docking mechanism is multipurpose and the seat can be occupied by a UAV to replace the battery and/or service the container with resources. The UAV maintenance process on the platform consists of several stages, therefore, the multiphase QS apparatus will be used for modeling. Figure 2.9 shows the stages of UAV maintenance during the exchange of energy resources, and Fig. 2.10—physical ones. When exchanging energy resources after landing, the system positions the UAV in the desired position. Then the stage of removing the battery from the UAV comes. After the battery is moved to the replacement area, the full battery is placed in the UAV.

Landing

Positioning

Removing the battery from the UAV

Battery to replacement area

Inserting the battery into the UAV

Preparation and takeoff

Fig. 2.9 Stages of UAV maintenance during the exchange of energy resources

38

2 Models and Algorithms of Interaction Between Heterogeneous …

Landing

Positioning

Moving resources on the platform

Unloading / loading the container onto the UAV

Loading / unloading the container from the UAV

Preparation and takeoff

Fig. 2.10 Stages of UAV servicing in the exchange of physical resources

Finally, the stage of preparation and take-off follow. Figure 2.10 shows the stages of UAV servicing in the exchange of physical resources. After the positioning stage, similarly to the maintenance of the UAV during the exchange of energy resources, the stage of unloading the container from the UAV to the platform takes place. This is followed by the stage of moving resources on the platform. After the stage of loading the container onto the UAV is completed, the stage of preparation and take-off follow. The UAV maintenance process on the service platform, in which the exchange of energy and physical resources is sequentially performed, is shown in Fig. 2.11. After the stages of servicing the UAV during the exchange of energy resources, the stage of connecting the UAV to the resource container follows. This is followed by the stages of UAV maintenance for the exchange of physical resources. In this mode, the time and energy resources of the UAV are saved for moving from the field, as well as for takeoff and landing operations. Thus, in the problem under consideration, closed multichannel multiphase parallel QS with heterogeneous nodes will be used. The laws review of the time intervals distribution between the customers entering the system and the distribution laws of the service duration of the customers, which can be applied in this problem, showed that the developed QS model has a Poisson incoming flow and a deterministic distribution of the service duration. Using Kendall’s notation, the QS model will have the form M/D/n/n. When performing a task on agricultural fields, in addition to servicing the battery, the UAV must load other resources from the platform, for example, fertilizers or, on the contrary, unload agricultural products. You should also take into account the support flotation and weather conditions when choosing the design of the ground service platform, as this will affect the total weight of the platform and its energy consumption [6–9].

2.3 Algorithmic Models to Control the Interaction of Heterogeneous …

Landing

Positioning

Removing the battery from the UAV

39

Battery to replacement area

Inserting the battery into the UAV

Connecting the UAV to the resource container

Loading / unloading a container from a UAV

Moving resources on the platform

Unloading / loading the container onto the UAV

Preparation and takeoff

Fig. 2.11 Stages of UAV maintenance during the exchange of energy and physical resources

The platform has a built-in container and, when servicing the UAV, takes a certain amount of energy and/or fertilizer resources. If it is necessary to service the UAV, it searches for a ground-based service platform, analyzing not only the availability of free space on them, but also assessing the sufficiency of its resources to charge its battery or replenish the container with resources. Platforms that do not have sufficient resources only land UAV and, when all places are filled, return to the main refueling site. Figure 2.12 shows an algorithmic model for controlling a UAV performing a task in agricultural fields [10]. When the battery is discharged or the container is full/empty, the UAV searches for the nearest platform and checks the free landing site. If there is free space on the platform, then the process of evaluating the remaining energy resources of the platform (in the case of charging/replacing the battery) or fertilizer resources (in the case of loading/unloading onto the UAV) is going on. If the resources are available, then the UAV lands and performs maintenance, if the resources are not enough, then another platform is analyzed. After the completion of the processes of charging/replacing the UAV battery or loading/unloading from the platform, the UAV is ready to take off from the platform and continue to perform the target task. In the absence of resources on the platforms, the UAV lands on the nearest platform and goes into the transport state. After the end

40

2 Models and Algorithms of Interaction Between Heterogeneous …

Start

Performing a UAV target UAV container is empty

Yes

No No

UAV battery is discharged Yes

Assessing the completion of the entire target No

There are new tasks for UAVs? Yes

Estimation of distances to platforms and their resources

No

Are there platforms with resources? Yes

Choosing the nearest free platform

Choosing the nearest free platform with sufficient resources

The transition of the UAV to the state of transportation

UAV service on the platform

End Fig. 2.12 Algorithmic model of UAV control with service on a ground-based service platform

2.3 Algorithmic Models to Control the Interaction of Heterogeneous …

41

of all resources or the completion of the target task, all UAV go into transport mode and the group of ground service platforms returns to the base ground control center. Figure 2.13 shows a logical-algorithmic model for polling the state of UAV and service platforms during the processing of agricultural land. At the beginning, the

Start

Analysis of the worked area of agricultural land o_j

Yes s_t_o_j == s_max_o_j No

End

Cycle by number of platforms n = 1, 2,… N

Analysis of p_n platform resources

e_t_p_n < e_P_min+e_u_max

Yes

HeT

r_t_p_n < r_u_max

Landing of all UAVs on platforms and return of all platforms to the central base

Yes

Waiting for the M UAV to land on the p_n platform and return the p_n platform to the central base

No Cycle by the number of UAVs m = 1, 2,… M Analysis of UAV resources u_m

e_t_u_m