Offshore Robotics: Volume I Issue 1, 2021 9811620776, 9789811620775

Citation preview

Offshore Robotics

Shun-Feng Su Ning Wang Editors

Offshore Robotics Volume I Issue 1, 2021

Offshore Robotics Editors-in-Chief Shun-Feng Su, National Taiwan University of Science and Technology, Taiwan, P. R. China Ning Wang , Harbin Engineering University, Harbin, P. R. China Associate Editors Hamid Reza Karimi, Politecnico di Milano, Italy Hongde Qin, Harbin Engineering University, P. R. China Sanjay Sharma, University of Plymouth, UK Fumin Zhang, Georgia Institute of Technology, USA Guangming Xie, Peking University, P. R. China Mergen Ghayesh, University of Adelaide, Australia Weidong Zhang, Shanghai Jiaotong University, P. R. China Yuanchang Liu, University College London, UK Daqi Zhu, Shanghai Maritime University, P. R. China Ali Zemouche, University of Lorraine, France Xianbo Xiang, Huazhong University of Science and Technology, P. R. China Choon Ahn Ki, Korea University, South Korea Mingjun Zhang, Harbin Engineering University, P. R. China Neil A. Duffie, University of Wisconsin, USA Jian Gao, Northwestern Polytechnical University, P. R. China Cunjia Liu, Loughborough University, UK Jianchuan Yin, Guangdong Ocean University, P. R. China Ying Gao, Dalian Maritime University, P. R. China

The purpose of this multidisciplinary and peer-reviewed series is to rapidly report and spread the latest technological results, new scientific discovery and valuable applied researches in the fields concerning with offshore robotics as well as promote international academic exchange. We aim to make it one of the premier comprehensive academic publications of world offshore vehicle and robotics community. Scope: Including but not limited to • • • • • • • • • • • •

Marine robotics scientific and technological discussion Autonomous marine vehicle design, test and launch Autonomous marine vehicle modeling, identification and simulation Marine guidance, navigation and control Marine robot visual servo, intelligence and autonomy Situation awareness, collision avoidance and decision making Multiple marine vehicles swarm intelligence, cooperation and optimization Autonomous power management system Safety in offshore robotics Information systems in offshore robotics AI in marine robotics Education in offshore robotics

More information about this series at http://www.springer.com/series/16379

Shun-Feng Su · Ning Wang Editors

Offshore Robotics Volume I Issue 1, 2021

Editors Shun-Feng Su Department of Electrical Engineering National Taiwan University of Science and Technology Taipei, Taiwan, China

Ning Wang School of Marine Electrical Engineering Dalian Maritime University Dalian, China

ISSN 2662-3501 ISSN 2662-351X (electronic) Offshore Robotics ISBN 978-981-16-2077-5 ISBN 978-981-16-2078-2 (eBook) https://doi.org/10.1007/978-981-16-2078-2 © Springer Nature Singapore Pte Ltd. 2022 This work is subject to copyright. All rights are reserved by the Publisher, whether the whole or part of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation, broadcasting, reproduction on microfilms or in any other physical way, and transmission or information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now known or hereafter developed. The use of general descriptive names, registered names, trademarks, service marks, etc. in this publication does not imply, even in the absence of a specific statement, that such names are exempt from the relevant protective laws and regulations and therefore free for general use. The publisher, the authors and the editors are safe to assume that the advice and information in this book are believed to be true and accurate at the date of publication. Neither the publisher nor the authors or the editors give a warranty, expressed or implied, with respect to the material contained herein or for any errors or omissions that may have been made. The publisher remains neutral with regard to jurisdictional claims in published maps and institutional affiliations. This Springer imprint is published by the registered company Springer Nature Singapore Pte Ltd. The registered company address is: 152 Beach Road, #21-01/04 Gateway East, Singapore 189721, Singapore

Contents

Survey on Collision-Avoidance Navigation of Maritime Autonomous Surface Ships . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Chengbo Wang, Ning Wang, Guangming Xie, and Shun-Feng Su

1

Ocean Explorations Using Autonomy: Technologies, Strategies and Applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35 Yuanchang Liu, Enrico Anderlini, Shuwu Wang, Song Ma, and Zitian Ding Single-Beacon Based Underwater Robot Navigation . . . . . . . . . . . . . . . . . . . 59 Zhong-ben Zhu, Hong-de Qin, and Xiang Yu Review on the Research of Ship Automatic Berthing Control . . . . . . . . . . . 87 Zhang Qiang, Nam-Kyun Im, Ding Zhongyu, and Zhang Meijuan

v

Survey on Collision-Avoidance Navigation of Maritime Autonomous Surface Ships Chengbo Wang, Ning Wang, Guangming Xie, and Shun-Feng Su

Abstract The rapid development of artificial intelligence significantly promotes collision-avoidance navigation of maritime autonomous surface ships (MASS), which in turn provide prominent services in maritime environments and enlarge the opportunity for coordinated and interconnected operations. Clearly, full autonomy of the collision-avoidance navigation for the MASS in complex environments still faces huge challenges and highly requires persistent innovations. In this chapter, major advances in state-of-the-art autonomous maritime vessels and systems, and collision avoidance technologies, are thoroughly addressed in various maritime scenarios, from academic to industrial sides. Moreover, compositions of autonomous navigation and E-navigation technologies are analyzed to efficiently systematically clarify the mechanism and principles in typical maritime environments, whereby trade-off between autonomy and navigation action are highlighted. Finally, in light of overview of promising collision avoidance and action planning technologies, it is pointed out that collision-free navigation would significantly benefits the integration of MASS autonomy in various maritime scenarios. Keywords Collision avoidance · Motion planning · Autonomous navigation systems · E-navigation · Maritime autonomous surface ships

C. Wang College of Navigation, Dalian Maritime University, Dalian 116026, China e-mail: [email protected] N. Wang (B) School of Marine Electrical Engineering, Dalian Maritime University, Dalian 116026, China e-mail: [email protected] G. Xie State Key Laboratory for Turbulence and Complex Systems, College of Engineering, Peking University, Beijing 100871, China S.-F. Su Department of Electrical Engineering, National Taiwan University of Science and Technology, Taipei City, Taiwan © Springer Nature Singapore Pte Ltd. 2022 S.-F. Su and N. Wang (eds.), Offshore Robotics, Offshore Robotics, https://doi.org/10.1007/978-981-16-2078-2_1

1

2

C. Wang et al.

1 Introduction China is known as a big shipping nation with nearly 20,000 km of coastline and 120,000 km of inland waterway, but it has not fully utilized this advantage. In recent years, with the advancement of the “Thirteenth Five-Year Plan”, the “Belt and Road strategy”, and the “South China Sea Strategy” and other shipping strategies, China has gradually moved to national shipping power. Meanwhile, the rapid development of unmanned boats, autonomous ship, and bionic robotic fish, such as aquatic unmanned surface systems and underwater unmanned systems, has not only changed the pattern of the shipping economy, but also promoted the development of safe shipping, green shipping, and intelligent shipping. In July 2017, the “Development Plan of New Generation Artificial Intelligence Technology” is promulgated and issued by the State Council of China [1], in which proposed that breakthroughs should be made in the autonomous computing architecture of unmanned system and autonomous control of drones, as well as autonomous driving of cars, ships and rail transit. Obviously, research on autonomous ship technology has been put on the agenda at the national level. In 2016, Google robot AlphaGo defeated multi-time international competition champion Li Shishi. It even caused a sensation in the field of artificial intelligence [2]. Once again, artificial intelligence represented by deep reinforcement learning has caught the attention of experts and scholars in various fields, and the same is true for the shipping industry. The history of shipping development is the continuous improvement of navigation safety performance and transportation benefits. The problem of ship navigation safety has always been a hot issue in the field of marine transportation engineering, and it is also one of the main aspects that drive the growth of MASS and their technical needs. Every year, marine accidents caused by human errors or faults, such as the negligent lookout of the on-duty driver, are common. Autonomy technology effectively replaces human pilots in ship maneuvering and cargo transportation, which greatly reduces the probability of human-induced marine accidents. At the present stage, autonomy technology is limited to applications such as unmanned boats and underwater robots, but unmanned transport cargo ships cannot yet achieve autonomous navigation and completely autonomous. In recent years, the research hotspots of many scholars and experts on autonomous navigation decision and planning are roughly divided into path planning, obstacle avoidance planning, trajectory planning and behavioral decision-making. Compared with path planning, collision avoidance and trajectory planning, behavioral decision-making considers time series and space constraints more. Behavioral decision-making systems are used to replace crew. Obstacle avoidance and approaching target ports are optimized goals. The behavioral decision-making is actually imitating the human crew’s thinking activity or process of ship maneuvering. In each collision avoidance or transportation process, the optimal navigation strategy is determined from many schemes in accordance with its own behavioral constraints.

Survey on Collision-Avoidance Navigation …

3

This work analyses current challenges and opportunities for collision avoidance and motion planning for maritime autonomous navigation systems. In particular, the following contributions are provided: 1. 2. 3. 4.

Review of state-of -the-art autonomous vessel and collision avoidance technology. Characterization of applications for maritime autonomous navigation systems. Classification of maritime autonomous surface ships and autonomous navigation systems. Overview of existing and future collision avoidance and motion planning technologies.

2 State-of-the-Art Autonomous Vessel and Collision Avoidance Technology 2.1 Advances in the MASS Autonomy technology of MASS is the integration of many intelligent ship technologies, including autonomous navigation technology (navigation situation awareness technology, navigation behavioral decision-making technology, motion control technology), intelligent engine room operation and maintenance, ship-shore communication, intelligent hull, integrated testing and other technologies. With the developing of artificial intelligence and communication technology, the level of ship automation has been gradually enhanced. Intelligent ship, marine intelligent transportation and unmanned ship technology and other related fields have also been widely researched by foreign institutions and experts represented by Europe and other countries. In recent years, the Norwegian Fraunhofer CML, as the first organization to research the demonstration of the unmanned cargo ship, completed the project maritime unmanned navigation through intelligence in network (MUNIN) from 2012 to 2015 to verify the concept of the autonomous ship, which is defined as a ship mainly guided by the autonomous decision support system and controlled by the remote control operator of the shore control center. The communication architecture solutions for the autonomous ship bridge, the autonomous machine room, the shore operation center and the operators connecting the ship to shore have developed and verified [3]. Supported by the Norwegian Research Council, the University of science and technology of Norway started the autonomous marine operations and systems (AMOS) project research in 2013. The architecture of AMOS is shown in Fig. 1. It is expected to complete the research on autonomous ships and robot systems in 2023, and develop the structure and operation of safer, smarter and more environmentally friendly ships and offshore intelligent platforms [4]. In October 2016, the Norwegian forum for autonomous ships (NFAS) was established to release information about international conferences and reports related to MASS, and in October 2017, under the organization of NFAS and SINTEF ocean, Norway, China, the United States and

4

C. Wang et al.

Fig. 1 The architecture of AMOS

other countries established the international network for autonomous ships (INAS), marking the research of MASS has been promoted to the national level, even to the international level [5, 6]. The SINTEF ocean laboratory in Trondheim, Norway, and Kongsberg, a technology company, jointly developed autonomous ship named Yara Birkeland, the first electric propulsion Unmanned Container Ship in the world. As shown in Fig. 2, the ship has a length of 70 m, a width of 15 m, and can carry 100– 150 teu. It has been tested in the water pool of SINTEF since September 29, 2017. It can use its own installed GPS, radar and camera. For example, aircraft and sensors can avoid other ships in the channel, and realize auto-docking when arriving at the Fig. 2 Yara Birkeland

Survey on Collision-Avoidance Navigation …

5

Table 1 Class of maritime autonomous surface ship in British Class of MASS

Characteristic

Notes

Ultra-light

Length overall < 7 m and maximum speed < 4kts

*

Light

Length overall ≥7– 130 nm

120 nm

—

1000 km

8h

24 h

Endurance

10

30

—

50

3.9

3.9

Maximum speed (kn)

Intelligent measurement

Offshore patrol and supervision

Sea air cooperation and independent search and rescue

—

Water sampling

Water quality sampling, online monitoring and data processing

Features

(continued)

Independent and remote-control dual mode operation

Sensor equipment includes lidar, binocular camera, laser rangefinder, optical fiber combined inertial navigation, etc.

Intelligent, unmanned and three-dimensional efficient search and rescue

With “high speed, long range, independent monitoring” and other domestic

Multi-point selection and vertical hierarchical water sample collection

Intelligent control terminal and real-time remote communication, autonomous navigation and automatic obstacle avoidance

Characteristic

12 C. Wang et al.

Ship name

Blue signal, Zhihai-1

Organization

Dalian Maritime University

Table 3 (continued)

2012

Manufacturing/application time 7.02/2

Size (m)

—

Endurance 35

Maximum speed (kn) Perform multiple tasks in a specific area

Features

It has three control modes: full autonomous navigation control mode, semi-autonomous navigation control mode and remote-control mode

Characteristic

Survey on Collision-Avoidance Navigation … 13

14

C. Wang et al.

Fig. 12 Zhihai No. 1

researches and develops the autonomous cargo ship from the aspects of the construction, autonomous navigation technology and system, control, etc., promotes the combination of research and production, and aims at the countries and research institutions with advanced unmanned technology in the field of unmanned vessel, such as Europe. In December 2017, Dalian Maritime University established “Research Institute for collaborative innovation of unmanned ship technology and system” at the annual meeting of scientific and technological innovation [25]. On July 18, 2017, the intelligent shipping seminar was held in Hangzhou, focusing on the four core topics of research status of intelligent shipping technology, application of intelligent shipping technology, maritime security in the era of intelligent shipping and prospect of intelligent shipping development, which include international conventions, revision of rules, promotion of domestic policies, R & D and application of intelligent shipping related technologies, and ships. The in-depth discussion on ship manufacturing and operation has been carried out, focusing on the close connection with relevant work of the International Maritime Organization (IMO), which will further promote the formation of domestic implementation plan and work plan for the development of intelligent shipping in the future [26]. In May 2018, the high-end forum of China’s unmanned vessel technology and innovation was successfully held in Shanghai with the theme of “new opportunities for surface unmanned vessel in the era of artificial intelligence". Famous experts and professors from the Institute of automation of the Chinese Academy of Sciences, Harbin Engineering and other important research institutions of domestic unmanned vessel were invited to participate in the exchange and discussion, which promoted the rapid development and application of the new generation of artificial intelligence in China The best opportunity and opportunity for the development of the ship industry solved the bottleneck of the development of the traditional unmanned ship industry [27].

Survey on Collision-Avoidance Navigation …

15

Autonomous navigation technology of unmanned vessel includes ship navigation situation awareness technology and collision avoidance behavioral decision technology. For the navigation situation awareness of MASS, the channel, surrounding ships and navigation status information can be obtained by the existing radar, Automatic Identification System (AIS), Electronic Chart Display and Information System (ECDIS), Global Positioning System (GPS) and other navigational instruments. The depth, water flow speed, wind speed, wind direction needs to be obtained by sensors. For non-ship obstacles, the laser scanner and radar fusion recognition should also be used [28–32]. However, MASS are still facing the scientific conundrum on intelligent recognition of small mobile targets at sea. For the behavioral decision technology of unmanned vessel collision avoidance, the application methods include improved artificial potential field, three-dimensional display, expert system, fuzzy logic, neural network, evolutionary computation, swarm intelligence and immune algorithm and other artificial intelligence and soft computing methods [33–35]. However, unmanned vessel still faces the difficult of autonomous decision-making of vessel navigation and intelligent collision avoidance behavioral decision in complex waters. With internet of things as the core technology, communication technology is mainly composed of general packet radio service (GPRS), wireless transmission technology and wireless receiving technology satellite communication [36, 37]. After the development of VHF data exchange system (VDES), navigation data (NAVDAT) and other communication technologies mature, they will be used as a part of the marine communication system to further improve the reliability of shore ship communication. Shore-based support system is composed of monitoring center and information support center. Under the “e-navigation” strategy, the unification and integration of shore-based information promotes the development of shore-based support for MASS. According to the research status of e-navigation, in the aspect of shore based support, we should focus on the shore-based information push service technology according to the navigation situation of MASS, to avoid information overload, to provide different real-time shore based support services according to the navigation stage, to provide intelligent information service and intelligent management information [38]. At present, the most prominent application is the integrated bridge system. The integrated bridge system has simple route planning, automatic collision avoidance and route tracking functions, but there is still a large distance from the real unmanned ship, especially under the conditions of high complexity, large calculation and high reliability, many integrated test theories and technologies still need further research and application.

2.2 Advances in Collision Avoidance and Action Planning Technology Collision avoidance system plays the role of copilot in the whole autonomous navigation system. The problem to be solved is to determine the obstacle avoidance and

16

C. Wang et al.

navigation strategy on the basis of knowing the environmental information of the MASS. At present, the research on the technology of collision avoidance for MASS at home and abroad is mainly divided into local path planning, multi-ship intelligent collision avoidance, behavioral planning, motion control and so on. However, there are few researches on the behavioral decision-making of collision avoidance. For the study on intelligent collision avoidance and motion planning of ships, the existing models mainly contain knowledge-based expert systems, fuzzy logic, artificial neural networks, intelligent algorithms (genetic algorithms, and ant colony algorithms etc.). In addition, a ship collision avoidance system based on the general structural model of the expert system has been established [39]. Moreover, a comprehensive and systematic study has been performed for the whole process of ship collision avoidance, and a mathematical model for the safety passing distance, pressing situation, and ship collision risk has been established. Yunsheng Fan, Xiaojie Sun, and Guofeng Wang [40] combined the velocity resolution method and backstepping tracking controller, a dynamic collision avoidance control method in the unknown ocean environment is presented. A novel dynamic programming (DP) method was proposed to generate the optimal multiple interval motion plan for MASS by Xiongfei Geng, Yongcai Wang, Ping Wang, et al. [41]. The method provided the lowest collision rate overall and better sailing efficiency than the greedy approaches. JH Ahn, KP Rhee, and YJ You [42] combined fuzzy inference systems with expert systems for collision avoidance systems. They proposed a method for calculating the collision risk using a neural network. Based on the distance to closest point of approach (DCPA) and the time to closest point of approach (TCPA), the multi-layer perceptron (MLP) neural network was applied to the collision avoidance system for compensating for the fuzzy logic. C Hua [43] optimized the shortest path and minimum heading of the local path and designed the path planning of the USV under the constraints of the close distance meeting model of the ship and 1972 International Collision Avoidance Rules. The target genetic algorithm realized the intelligent collision avoidance of USV through simulation. Marilia Abilio Ramos, Ingrid Bouwer Utne, and Ali Mosleh [44] presented a task analysis for collision avoidance through hierarchical task analysis and cognitive model for categorizing the tasks, which explored how crew can be a key factor for successful collision avoidance in future MASS navigation. The results provided valuable information for the design stage of the MASS. The above-mentioned models usually have an assume complete environmental information. However, in an unknown or uncertain environment, prior knowledge of the environment is difficult to acquire. It is difficult to form a complete and accurate knowledge base, and the rule-based algorithm is difficult to cope with various situations. Therefore, in many practices, the system needs to have strong adaptive ability. Recently, deep reinforcement learning (DRL) combined with deep neural network models and reinforcement learning have made significant progress in the field of autonomous navigation for USV, unmanned aerial vehicles (UAV), and unmanned ground vehicles (UGV). Tai L, Li S, Liu M [45] combined deep learning and decisionmaking processes into a highly compact, fully connected network, with raw depth images as the input and the generated control commands as the outputs to achieve model-free obstacle avoidance behavior. Long P, Liu W, and Pan J [46] proposed

Survey on Collision-Avoidance Navigation …

17

a novel end-to-end framework for generating effective reactive collision avoidance strategies for distributed multi-agent navigation based on deep learning. Panov A I et al. [47] proposed an approach for using a neural network to perform the path planning on the grid and initially realize it based on deep reinforcement learning. M Bojarski et al. [48] used convolutional neural networks for end-to-end training driving behavioral data, mapping the raw pixels from a single-front camera directly to the steering commands for unmanned vehicle adaptation path planning. The performance of the model and results of learning were better than the traditional model, but the only improvement was that the model was less interpretable. Cheng Y. et al. [49] proposed a simple deep reinforcement learning obstacle avoidance algorithm based on the deep Q learning network, using a convolutional neural network to train the ship sensor image information. The interaction with the environment was included by designing the incentive function in reinforcement learning. The maximum expected value of the cumulative return was obtained, and the optimal driving strategy of the underactuated was derived. However, improvement in the literature research increases the complexity of the verification environment and dynamic obstacle environment. Compared with [49] Refs, the different and better aspects of [50] Refs are: On the one hand, [50] Refs uses long short-term memory (LSTM) to do deep learning network to simplify the network structure and improve the iterative rate. On the other hand, [50] Refs learns the ship navigation state data, including relative azimuth and relative distance improve the accuracy and effectiveness of decisions. An intelligent collision avoidance decision model of MASS based on deep reinforcement learning is established. The problems encountered in intelligent avoidance decision for MASS is analyzed, and the design criteria are put forward. Based on this, a decision-making model based on Markov decision process (MDP) is established [50] Refs. Solving the optimal strategy of intelligent decision-making model makes the maximum return which is in unmanned vessel state to behavior mapping through the value function A reward function is specifically designed for target approaching, off course and safety. Finally, the simulation experiments of the intelligent collision avoidance decisionmaking method based on deep reinforcement learning are carried out respectively in static and dynamic waters, so as to demonstrate the feasibility of proposed method in actual application by Chengbo W. and Xinyu Z. et al. [51–55]. For the study on motion control and trajectory tracking control of ships, Wang Ning et al. puts forward the analytical framework model of ship domain consent. Through the parametric and hierarchical study of the quaternion ship domain model, the shortcomings of the existing ship domain model such as subjectivity and uncertainty are overcome, and the accuracy and effectiveness of ship collision avoidance are improved [56–59]. In order to effectively deal with the extremely strong unmodeled dynamics, model uncertainty and unknown external interference of the USV, an intelligent self-structured robust adaptive track tracking control strategy independent of the model is proposed by N.W. et al., realized a new method of precise track tracking control of the surface ship under the unknown time-varying complex sea conditions, and then proposed a limited time tracking control strategy of the USV, accurately suppress and cancel external interference and system uncertainty [60– 66]. To further support such a cooperative and coordinated manner for USVs, a

18

C. Wang et al.

new intelligent multi-task allocation and path planning algorithm has been proposed based upon the self-organizing map (SOM) and the fast-marching method (FMM) by Liu, Y., Zhou, X., and Tan, G. et al. [67–69].

3 Maritime Autonomous Navigation Systems Maritime autonomous navigation systems of collision avoidance can increase the safety of life at sea. The minimize the risk of collisions to assist the Master or officer of watch (OOW) in their analysis of encounter situations by simultaneous plotting of all targets in the declared range. Meanwhile, the calculation of the safe course or speed to pass clear from all targets, according to the International Regulations for Preventing Collisions at Sea (COLREG). Therefore, it is recommended to develop performance standards that will assist the shipping community in proper analysis, design, testing and approval of such system. This section provides a general perspective on navigation systems of collision avoidance for maritime autonomous surface ships and modules of autonomous navigation systems that are present in the future.

3.1 Challenges in Autonomous Navigation Systems in Uncertain Environment The marine environment is changeable. The autonomous navigation of ship in the coastal waters is more complicated than cars driving on the roads. Combined with the characteristics of the marine environment, in view of the main difficulty faced by the intelligent decision for collision avoidance of MASS, the following problems to be faced by the behavioral decision of MASS is putting forward [70, 71]: (1)

(2)

The marine environment is complex and changeable [72]. First of all, the marine natural environment is changeable, and the wind, current, surge, wave and other time-varying are strong, which greatly affects the autonomous navigation of ships. Offshore waters have strong structural navigation characteristics, with many types of divided navigation and a large amount of navigation information, such as light buoy, channel buildings, navigation signal lights, irregular small fishing boats and other external environmental factors. Therefore, the intelligent behavioral decision of MASS needs to consider the constraints of multi-source heterogeneous information and extract effective information. The sea traffic is disorderly, and there are many types of participants [73]. Due to the different tonnages and types of ships, the maneuverability of ships is different. Therefore, the information such as ship types should be considered in the decision systems. It is a major issue to avoid small obstacles for the decision system, especially for the small fishing boats that are too fast to comply

Survey on Collision-Avoidance Navigation …

(3)

19

with the rules of maritime traffic during the fishing period, which causes the phenomenon of maritime traffic chaos. The ultimate goal of autonomous navigation for MASS is to realize the thinking of brain like maneuvering [74]. When ship handling, the pilot considers people, ship and route as a whole, and carries out reactive sailing under the guidance of some maritime traffic rules, experience and intention. For intelligent collision avoidance decision of MASS, it is necessary to learn from the decision process of crew in dealing with complex traffic scenes, learn the driving experience and the fuzzy definition in the rules reasonably, and personify the navigation behavioral decision.

The complexity of the above three uncertain environments has a certain impact on the rationality and effectiveness of the autonomous navigation behavioral decision for MASS, which is mainly reflected in the closed loop of the whole voyage [75]. Three uncertain include the scene elements in the navigation situation, the space–time characteristics and status of the obstacles, the binary relationship between MASS and the obstacles effective modeling. Therefore, MASS needs effective description and modeling of behavior decision expert knowledge base (international maritime traffic rules, good seamanship) based on scene division, and intelligent collision avoidance decision and navigation decision reasoning based on self-learning of navigation situation. In the actual voyage, the navigation behavioral decision of MASS still faces more uncertainties [76–78], such as: (1)

(2)

(3)

Uncertainty of marine environment [79]. The sea is vast and infinite, and human’s understanding of the sea is very limited. In the voyage, there are not complete kinds of environmental prior knowledge. Therefore, there are many uncertainties in the sea areas lacking of environmental prior knowledge, including water depth, reef and other disturbing and obstructing factors. The uncertainty of navigation situation information perception [80]. Due to the rich information, it simply includes the information obtained by the internal sensor, the information obtained by the external sensor and the information transmitted (shared) by the third party. Internal sensors refer to the platform monitoring of MASS, generally refer to the health status of command data link, the operability and health status of sensors identified as critical, the operability and health status of onboard system (such as propeller, autopilot, collision avoidance system, etc.), watertight information, residual fuel, hull integrity, pitch, roll, heave and ship vibration dynamic. External sensors refer to GNSS, bow direction, sea condition, wind speed and direction, water depth below keel, radar target, sound signal and visual signal (other ship’s light type). Data transmitted by the third party includes AIS data, meteorological forecast data and tide calendar data. Due to the different characteristics of these sensors (principle of action, sensing mechanism, data transmission), some uncertainty of sensing information will be caused. There is uncertainty in the accuracy of the prediction of obstacle motion and collision trajectory [81–84]. The perception of all the sensors of MASS

20

C. Wang et al.

brings the space state information, and the whole decision-making process or the navigation process has distinct space–time characteristics. These sensors cannot detect or report the behavior intention and motion state of the dynamic obstacles, such as the motion direction and speed. In the uncertain environment, the navigation decision system and algorithm should have the ability of situation assessment based multi-source heterogeneous information, the ability to infer the motion state of dynamic obstacles and the ability to generate the optimal navigation strategy, to deal with the above problems and uncertainties. In view of the problems faced by marine autonomous navigation system, combined with maritime traffic rules, the system should meet the following design requirements. (1)

(2)

(3)

(4)

(5)

(6)

Autonomy planning ability. For the identified targets and obstacles, the planned route or navigation motion shall have certain safety when the optimal collision avoidance, collision avoidance time and resuming sea navigation are reasonably planned on the ECDIS. Real-time. Between the voyage, navigation environment is unpredictable, and the marine autonomous navigation and decision system must make real-time changes in the state and motion according to the changes in the navigation environment. Rationality. There are many ambiguous provisions or descriptions in the COLREGS, such as good seamanship, early avoidance, large, wide and clear description. Therefore, when designing the system, the actual navigation situation should be satisfied as much as possible, and the reasonable maneuvering mechanism for collision risks should be in line with the COLREGS. Course stability and direction keeping. In the course of navigation, in addition to crossing, overtaking, obstacle avoidance and other actions, the MASS shall not deviate from the route greatly, and shall keep driving on the route. If the ship deviates from the route due to disturbance, the route can be resumed automatically when the disturbance disappears. Speed control. In the normal navigation state, the autonomous ship should generally navigate within the maximum and minimum speed limits. In the process of intelligent decision-making, it is usually necessary to adjust the ship’s speed longitudinally to avoid obstacles. In specific sea areas, it is necessary to control the ship’s speed according to local rules, in case of emergency or accident, emergency braking can be realized. Collision avoidance. The MASS shall have the ability of collision avoidance in the course of navigation, and the actions taken include transverse turning, longitudinal acceleration and deceleration, ship stalling, etc.

3.2 Design of Maritime Autonomous Navigation Systems Maritime autonomous navigation system is a complex system [85], which integrates many advanced intelligent technologies.

Survey on Collision-Avoidance Navigation …

21

Fig. 13 The overall architecture of maritime autonomous navigation system

In this paper, the whole system is divided into four sub-systems: global route optimization, navigation situation awareness, navigation behavioral decision, and motion control and execution subsystem. As shown in Fig. 13, the overall system architecture of the navigation system for maritime autonomous surface ships is presented, which describes the collaborative relationship among the four sub-systems.

3.2.1

Global Route Optimization

The global route optimization subsystem is to set the waypoint with the help of ECDIS and GPS system in the early stage of cargo transportation for MASS, so as to realize the calculation and design of relatively better and safe routes for known obstacles and port-to-port [86]. For the whole voyage of MASS, in terms of data description, global route optimization is equivalent to optimizing the navigation strategy for global path planning. If there are obstacles or the original route is blocked in the voyage after planning, the global route optimization system will conduct quadratic programming to re-plan a reasonable and optimal global route. The commonly used algorithms are dynamic programming, a *, Dijkstra and trajectory point guidance [87–91].

22

3.2.2

C. Wang et al.

Navigation Situation Awareness

Navigation situation awareness system is to use a variety of onboard instruments and equipment to actively perceive the internal and external information of the ship or marine navigation environment, and receive the data transmitted by the third party [92, 93]. Perception system is the basis of navigation behavioral decision and motion control of MASS. The accurate perception information is also an important benchmark of MASS research and development. There are many kinds of sensors and shipborne instruments in the navigation situation awareness system of MASS, such as ECDIS, radar, lidar, HD camera, sonar, AIS, etc., which can obtain high-precision position service information, maritime safety sailing information, hydrometeorological information, ship dynamic information and port information in real time [94]. This multi-source information is fused and processed. Static and dynamic obstacles are mapped in the ECDIS and sent to the behavioral decision system.

3.2.3

Decision-Making

The navigation behavioral decision system is the core part of the whole MASS— navigation brain [74, 95, 96]. The system takes the results of the perception system as input, and collects all the information of the navigation situation, including not only the current position, speed and course of the autonomous ship, but also the information of obstacles. The decision system of maritime autonomous navigation system is to determine the route and navigation strategy of the MASS on the basis of knowing navigation safety information [97].

3.2.4

Control and Execution

After the decision instruction is given by the decision system of maritime autonomous navigation system, the control and execution system of MASS will execute the instruction, mainly including speed planning and trajectory planning, corresponding to the MASS, that is, the control of the marine telegraph and rudder [98, 99]. There is also a feedback control layer based on the integrated error of ship attitude variables in this subsystem. On the voyage of autonomous navigation for MASS, there are often some errors between the actual navigation and the plan due to the uncertainty of the ocean current, swell and other environment. Therefore, the control system will be feedback controlled again based on these errors. On the one hand, the decision instructions can be adjusted in real time for re-planning to better conform to the current navigation behavioral. On the other hand, the navigation behavioral and motion can be corrected. The ship’s navigation behavioral can avoid uncertain risks.

Survey on Collision-Avoidance Navigation …

23

4 Trade-off Between Autonomy and Navigation Action Autonomy levels, degrees of autonomy and similar concepts of MASS or maritime autonomous navigation systems have been discussed extensively all over the world after the unmanned ship is named maritime autonomous surface ships by International Maritime Organization (IMO). A general shortcoming is that most existing classification schemes define a very concrete context for the classification that may not fit other applications, such as autonomous ships. Common assumptions are that there is always a person in the vehicle as for the autonomous vessel [85] or that the controlled system always operate without any person on board, but only through teleoperation as in IMO’s classification [100]. Taxonomies for the levels of autonomy in different types of systems have been presented in the document by several countries and institutions. One of the classification societies suggesting levels of autonomy is Bureau Veritas [101]: – Level 0 Human operated—Automated or manual operations are under human control. The human makes all decisions and controls all functions. – Level 1 Human directed—Decision support, human makes decisions and actions. The system suggests actions, human makes decisions and actions. – Level 2 Human delegated—Human must confirm decisions. The system invokes functions, human can reject decisions during a certain time. – Level 3 Human supervised—System is not expecting confirmation, human is always informed of the decisions and actions. The system invokes functions without waiting for human reaction. – Level 4 Fully autonomous—System is not expecting confirmation, human is informed only in case of emergency. The system invokes functions without informing the human. Lloyd’s Register has also made a suggestion. The proposal they presented at the meeting of 13 November 2017 was at the time a draft and slightly modified from their original suggestion presented earlier in 2017. Below, their current suggestion of accessibility levels are listed, as Lloyd’s Register now calls them [101]: – Level 0 No cyber access—no assessment—no descriptive note – included for information only. – Level 1 Manual cyber access—no assessment—no descriptive note—included for information only. – Level 2 Cyber access for autonomous/remote monitoring. – Level 3 Cyber access for autonomous/remote monitoring and control (onboard permission is required, and onboard override is possible). – Level 4 Cyber access for autonomous/remote monitoring and control (onboard permission is not required, and onboard override is possible). – Level 5 Cyber access for autonomous/remote monitoring and control (onboard permission is not required, and onboard override is not possible).

24

C. Wang et al.

The UK Marine Industries Alliance has also published a document with definitions, a Code of Practice. The document gives guidelines for ships of less than 24 m in length, although some of them are to be applicable to larger ships as well. The suggested concepts of autonomy are as follows [101]: – Level 0 Manned—ship/craft is controlled by operators aboard. – Level 1 Operated—Under Operated control all cognitive functionality is within the human operator. The operator has direct contact with the unmanned ship over, for example, continuous radio (R/C) and/or cable (e.g. tethered UUVs and ROVs). The operator makes all decisions, directs and controls all vehicle and mission functions. – Level 2 Directed—Under Directed control some degree of reasoning and ability to respond is implemented into the unmanned ship. It may sense the environment, report its state and suggest one or several actions. It may also suggest possible actions to the operator, such as, for example, prompting the operator for information or decisions. However, the authority to make decisions is with the operator. The unmanned ship will act only if commanded and/or permitted to do so. – Level 3 Delegated—The unmanned ship is now authorized to execute some functions. It may sense environment, report its state and define actions, and report its intention. The operator has the option to object to (VETO) intentions declared by the unmanned ship during a certain time, after which the unmanned ship will act. The initiative emanates from the unmanned ship and decision-making is shared between the operator and the unmanned ship. – Level 4 Monitored—The unmanned ship will sense environment and report its state. The unmanned ship defines actions, decides, acts and reports its action. The operator may monitor the events. – Level 5 Autonomous—The unmanned ship will sense environment, define possible actions, decide and act. The unmanned ship is afforded a maximum degree of independence and self-determination within the context of the system’s capabilities and limitations. Autonomous functions are invoked by the onboard systems at occasions decided by the same, without notifying any external units or operators. The suggestions are made from different assumptions and the premises of the suggestions differ somewhat. There are four to six levels in the suggestions, with five levels being the most used one. However, the contents of the suggested levels are not similar, even if the number of levels is the same. There are still some similarities in most of the suggestions. The lowest level is in general a level where the human is in charge, whereas the highest level is one where the ship operates unassisted on its own. Some suggestions have left out the lowest level, whether that is because it demonstrates no automation whatsoever, or to merely decrease the amount of levels is not clear. The level of autonomy is strongly correlated with the collision avoidance decision and motion planning capabilities required by such system, as illustrated in Fig. 14. In general, high levels of autonomy means low human decision requirements, while low levels of autonomy and automation will require more human navigation decision.

Survey on Collision-Avoidance Navigation …

25

Autonomy Level Fully autonomous Autonomous functionality/ remote control Unmanned Level

Conventional Full crew

Reduced crew

Unmanned vessel

Fig. 14 Level of autonomy in a maritime navigation system

In the extreme case of local collision avoidance, the demands for real-timing, rulecomplaint and fast decision-making are very high. The lower left corner in the figure indicates an unfeasible situation, while the upper right corner is an ideal situation which enables higher performance and enhanced functionality beyond the minimum requirements.

5 Trends in Maritime Autonomous Navigation Systems In May 2006, at the 81st meeting of the Maritime Safety Committee (MSC) of the IMO (International Maritime Organization), the Seven Countries Proposal- “Development of e-Navigation Strategy” was adopted and adopted of The International Association of Marine Aids to Navigation and Lighthouse Authorities (IALA). Enavigation refers to the coordinated collection, integration, exchange, display and analysis of maritime information on board and onshore by electronic means to enhance the navigational capabilities of berths and other related services improve the level of safety and security at sea and protect the marine environment [102]. The e-navigation concept was put forward to meet the rapid development of autonomous navigation technology and navigation assistance methods. It aims to achieve the optimization of maritime transportation by integrating the existing navigation assistance technology and tools. The e-navigation technology framework mainly includes three elements, the ship environment, the shore-based support environment, and the communication system. Ship environment refers to supporting the collection, integration, exchange, display and analysis of all information provided by ship-based sensors. Shore-based support environment refers to shore-based technical services that support shore-based applications, such as search and rescue, VTS, ports, and MSI (Maritime Safety Information) services, etc. Communication systems refer to the communication equipment and communication links between ships-ships, shore-ships. To this end, the overall technical architecture of e-navigation can be simply described as the three sides of the coin, as shown in Fig. 15. The front and

26

C. Wang et al.

Unified collection, integration, exchange, display and analysis of shipboard information

Unified collection, integration, exchange, display and analysis of shore-based information

Fig. 15 E-navigation overall technical architecture

back sides of the coin represent the ship environment and the shore-based support environment, and the side of the coin represents the link ship Communication system with shore [103]. At present, with the development of shipping industry, ships have shown the characteristics of large-scale, specialized, high-speed and intelligent. E-navigation tries to integrate the existing navigation technology to maximize the safety of ship navigation and improve the efficiency of maritime cargo transportation. MASS combined with artificial intelligence technology greatly reduces the impact of human factors on maritime transportation safety and improves the level of ship navigation safety. The combination of e-navigation technology and maritime autonomous navigation technology can effectively promote the development of intelligent and information technology of maritime transportation and enhance the safety of navigation. In order to effectively improve the safety level of maritime transportation and combine the autonomous navigation with e-navigation, the overall technical framework of the autonomous navigation system of MASS based on e-navigation is shown in Fig. 16, and autonomous navigation is developed based on the intelligent environment state information perception, intelligent navigation decision and intelligent communication of e-navigation [104]. The application of e-navigation technology lays a foundation for the development of autonomous navigation of MASS, and also promotes the implementation of e-navigation strategy. E-navigation relies on four major issues of perception, data, standards and transmission, and moves from theory to practical application. The application of enavigation technology lays a foundation for the development of autonomous navigation technology of MASS, and the development of MASS also promotes the implementation of e-navigation strategy [105–108]. It integrates e-navigation technology and autonomous ship technology. In the data center, it establishes a database of the information sensed by the MASS based on the standard of S-100, and transmits information to the ship and the shore-based platform through the maritime cloud, so as to realize the autonomous navigation of MASS without collision. The four major issues that e-navigation technology system mainly solves are perception, data, standard and transmission. From the common research of enavigation and autonomous navigation of MASS, e-navigation development lays

Survey on Collision-Avoidance Navigation …

27 Monitor

Remote control

Shore and other maritime users

Maritime cloud seamless information Transmission system

Data center

Databases

S-100 Information standard

MSP

AN

CC

Perception system Control

Perception system Perception

Fig. 16 Framework of autonomous navigation system based on e-navigation

a technical foundation for the construction of autonomous navigation system in the aspects of intelligent perception, intelligent navigation decision, intelligent communication and intelligent control.

6 Conclusion The importance of maritime autonomous navigation systems is undeniable and the opportunity for coordinated and interconnected operations is clear. MASS may finish intelligent navigation through shore remote control center with long distance to the operations, so that dependence on more autonomy infrastructures such as maritime autonomous navigation system or collision avoidance decision support systems, motion planning systems must be expected. The cost, reliability, performance and availability of such systems are important issues. Moreover, there is a wide variety of scenarios with different collision avoidance decision requirements with respect to data-rates, latency and importance. These are for instance, Command and control data (telemetry), sensor data for situation awareness, payload sensor data, collision avoidance transponder broadcasts and status information. Therefore, autonomous navigation strongly depends on the system autonomy level and situation needs.

28

C. Wang et al.

This work reviews the major advancements on maritime autonomous navigation technology applied in several different scenarios, from transportation to scientific research. Moreover, it highlights how available technologies and systems can be composed in order to efficiently and effectively handling in maritime obstacle environments. Existing and prototype maritime autonomous surface ships, sensors, autonomous navigation for collision avoidance technologies are characterized, describing their requirements and capabilities. Additionally, the tradeoff between fully autonomous operations versus shore remotely operated vessels is highlighted, taking into account the availability and performance of different autonomy level. The discussed opportunities are aligned with current trends in autonomous navigation system and e-navigation technologies. Acknowledgements The authors thank the Editor-in-Chief, Technical Editor, and anonymous referees for their invaluable comments.

References 1. Notice of the State Council on Printing and Distributing the New Generation Artificial Intelligence Development Plan. http://www.gov.cn/zhengce/content/2017-07/20/content_5211996. htm 2. Artificial Intelligence Event: AlphaGo robot against Go master Li Shishi. http://www.tec hweb.com.cn/irouter/2016-03-07/2290885.shtml 3. MUNIN (Maritime unmanned navigation through intelligence in network) Homepage. http:// www.unmanned-ship.org/munin/ 4. NTNU Homepage. https://www.ntnu.edu/amos/research 5. Norwegian Forum for Autonomous Ships (NFAS) Homepage. http://nfas.autonomous-ship. org/index-en.html 6. International Network for Autonomous Ships (INAS) Homepage. http://www.autonomousship.org/events.html 7. Yara Birkeland Homepage. https://www.yara.com/news-and-media/press-kits/yara-birkel and-press-kit/ 8. AAWA—The Advanced Autonomous Waterborne Applications Homepage. http://www. rolls-royce.com/media/press-releases/yr-2016/21-06-2016-rr-publishes-vision-of-the-fut ure-of-remote-and-autonomous-shipping.aspx 9. UK Maritime and Coastguard Agency (2018) Maritime autonomous surface ships—UK code of practice. The Society of Maritime Industries on behalf of Maritime, UK 10. Richter M (2006) Operational manning considerations for Spartan scout and sea fox unmanned surface vehicles (USV). Naval postgraduate school Monterey CA graduate school of business and public policy 11. Lin X-l, Zhang B-s (2018) Technical development and operational application of unmanned surface combat system. J Unmanned Undersea Syst 26(2):107–114 12. Wan J-x (2014) Status and development trends of foreign military unmanned surface boats. Natl Def Sci Technol 35(5):91–96 13. Kumar A, Kurmi J (2018) A review on unmanned water surface vehicle. Int J Adv Res Comput Sci 9(2):95 14. Yunzhou Intelligent Technology Co., Ltd. Homepage. http://www.yunzhou-tech.com/Pro ducts/detail/id/26.html

Survey on Collision-Avoidance Navigation …

29

15. SWT Spirit Whale Technology Homepage. http://www.smartwhale.cn/a/chanpinxilie/shuizh icaiyang/2015/1221/39.html 16. School of Naval Architecture of Harbin Engineering University Homepage. http://sec.hrbeu. edu.cn/2018/0925/c443a201600/page.htm 17. Ma Y, Sheng G (2018) A sea and air autonomous cooperative search and rescue M3U platform [P], Hubei, CN108563242A, 21 Sept 2018 18. Industrial Technology Research Institute of Guangdong Huazhong University of Science and Technology Homepage. http://www.gongyanyuan.net.cn/document/201708/article1690.htm 19. Shanghai University Unmanned Boat Engineering Research Institute Homepage. http://www. jhai.shu.edu.cn/Default.aspx?Tabid=35148 20. NetEase Dalian Homepage. http://3g.163.com/local/article/E0NL5P6B04228KBM.html 21. Wang N, Gao Y, Weng Y, Zheng Z, Zhao H (2018) Implementation of an integrated navigation, guidance and control system for an unmanned surface vehicle, pp 717–722 22. Wang N, Sun Z, Jiao Y, Han G (2019) Surge-heading guidance based finite-time path-following of underactuated marine vehicles. IEEE Trans Veh Technol 68(9):8523–8532 23. He C (2016) Requirements of the six functional modules of the CCS intelligent ship code. China Ship Surv 190(3):84–85 24. Sohu Homepage. http://www.sohu.com/a/153098782_119707 25. China News Homepage. http://news.sina.com.cn/o/2017-07-06/doc-ifyhvyie0380669.shtml 26. Smart shipping seminar held in Hangzhou. http://zj.people.com.cn/n2/2017/0718/c18632730492174.html 27. The Institute successfully hosted the 2018 China Unmanned Ship Technology Innovation Forum. http://www.iim.cas.cn/xwzx/kjxw/201806/t20180605_409579.html 28. Liu C-g, Chu X-m et al (2016) Review and prospect of ship intelligence. Ship Eng 3:77–84 29. Xu H (2012) The research on target detection technology for marine rada in image processing. M.A. thesis, Wuhan University of Technology of China, Wuhan 30. Wang XS (2016) Research or USV radar image target detection system. M.A. thesis, Dalian Maritime University of China, Dalian 31. Liu R-q (2017) Application of depth learning algorithm in ship electronic chart recognition. Ship Sci Technol 39(2A):83–85 32. Li X-m, Zhang X et al (2017) 3D Lidar-based marine object detection for USV. J Shanghai Univ (Nat Sci Ed) 23(1):27–36 33. Lyu H, Yin Y (2019) COLREGS-constrained real-time path planning for autonomous ships using modified artificial potential fields. J Navig 72(3):588–608 34. Lyu H, Yin Y (2018) Fast path planning for autonomous ships in restricted waters. Appl Sci 8(12):2592 35. Hongguang L, Yong Y et al (2016) Artificial intelligence and soft computation methods in automatic collision avoidance algorithms for ships. J Navig China 39(3):35–40, 118 36. Shanshan Wu, Qingshuang L et al (2017) Design of unmanned ship for water environment monitoring based on internet of things. Digit Technol Appl 5:165–166 37. Haien X, Huihui X et al (2017) Design and implementation of unmanned ship cloud control system based on 4G IoT technology. J Softw Guid 5:56–58 38. Ruiliang G, Yuguan Ji (2016) Intelligent ship technology and unmanned navigation technology. J Ship 27(5):82–87 39. Peng L (2010) AIS-based intelligent collision avoidance expert system for inland vessels and its implementation. M.A. thesis, Wuhan University of Technology of China, Wuhan 40. Fan Y, Sun X, Wang G (2019) An autonomous dynamic collision avoidance control method for unmanned surface vehicle in unknown ocean environment. Int J Adv Robot Syst 16:1–11. https://doi.org/10.1177/1729881419831581 41. Geng X, Wang Y, Wang P, Zhang B (2019) Motion plan of maritime autonomous surface ships by dynamic programming for collision avoidance and speed optimization. Sensors 19:434. https://doi.org/10.3390/s19020434 42. Ahn JH, Rhee KP, You YJ (2012) A study on the collision avoidance of a ship using neural networks and fuzzy logic. Appl Ocean Res 37(4):162–173

30

C. Wang et al.

43. Chen H (2016) Preliminary study on local path planning of surface unmanned boats. M.A. thesis, Dalian Maritime University of China, Dalian 44. Ramos MA, Utne IB, Mosleh A (2019) Collision avoidance on maritime autonomous surface ships: operators’ tasks and human failure events. Saf Sci 116:33–44. https://doi.org/10.1016/ j.ssci.2019.02.038 45. Tai L, Li S, Liu M (2016) A deep-network solution towards model-less obstacle avoidance. In: Proceedings of IEEE/RSJ international conference on intelligent robots and systems, Daejeon, South Korea, 9–14 Oct 2016, pp 2759–2764 46. Long P, Liu W, Pan J (2017) Deep-learned collision avoidance policy for distributed multiagent navigation. IEEE Robot Autom Lett 2:656–663. https://doi.org/10.1109/LRA.2017.2651371 47. Panov AI, Yakovlev KS, Suvorov R (2018) Grid path planning with deep reinforcement learning: preliminary results. Procedia Comput Sci 123:347–353. https://doi.org/10.1016/j. procs.2018.01.054 48. Bojarski M, Del Testa D, Dworakowski D et al (2016) End to end learning for self-driving cars. arXiv preprint arXiv:1604.07316 (1), pp 1–9 49. Cheng Y, Zhang W (2018) Concise deep reinforcement learning obstacle avoidance for underactuated unmanned marine vessels. Neurocomputing 272:63–73. https://doi.org/10.1016/j. neucom.2017.06.066 50. Zhang X, Wang C, Liu Y, Chen X (2019) Decision-making for the autonomous navigation of maritime autonomous surface ships based on scene division and deep reinforcement learning. Sensors 19(18):4055 51. Wang C, Zhang X, Cong L, Li J, Zhang J (2019) Research on intelligent collision avoidance decision-making of unmanned ship in unknown environments. Evol Syst 10(4):649–658 52. Wang C, Zhang X, Li R, Dong P (2018) Path planning of maritime autonomous surface ships in unknown environment with reinforcement learning. In: International conference on cognitive systems and signal processing, Springer, Singapore, pp 127–137 53. Cheng-bo WANG, Xin-yu ZHANG, Jia-wei ZHANG, Zhi-guo DING, Lan-xuan AN (2019) Navigation behavioural decision-making of MASS based on deep reinforcement learning and artificial potential field. J Phys Conf Ser 1357(1):012026 (IOP Publishing) 54. Wang CB, Zhang XY et al (2018) On path planning of unmanned ship based on Q-learning. J Ship Ocean Eng 47(5):174–177 55. Wang C, Zhang X et al (2018) Method for intelligent obstacle avoidance decision-making of unmanned vessel in unknown waters. Chin J Ship Res 13(06):74–79 56. Wang N, Meng X, Xu Q, Wang Z (2009) A unified analytical framework for ship domains. J Navig 62(4):643–655 57. Xu Q, Meng X, Wang N, Zhang C (2009) Adaptive danger area-based danger model immune algorithm. In: International conference on intelligent computing and intelligent systems, vol 3. IEEE, pp 109–112 58. Wang N (2010) An intelligent spatial collision risk based on the quaternion ship domain. J Navig 63(4):733–749 59. Wang N, Tan Y, Liu SM (2011) Ship domain identification using fast and accurate online selforganizing parsimonious fuzzy neural networks. In: Proceedings of the 30th Chinese control conference, IEEE, pp 5271–5276 60. Wang N, Er MJ (2014) Self-constructing adaptive robust fuzzy neural tracking control of surface vehicles with uncertainties and unknown disturbances. IEEE Trans Control Syst Technol 23(3):991–1002 61. Wang N, Qian C, Sun JC, Liu YC (2015) Adaptive robust finite-time trajectory tracking control of fully actuated marine surface vehicles. IEEE Trans Control Syst Technol 24(4):1454–1462 62. Wang N, Pan X (2019) Path-following of autonomous underactuated ships: a translationrotation cascade control approach. IEEE/ASME Trans Mechatron. https://doi.org/10.1109/ TMECH.2019.2932205 63. Wang N, Deng Z (2019) Finite-time fault estimator-based fault-tolerance control for a surface vehicle with input saturations. IEEE Trans Ind Inform. https://doi.org/10.1109/TII.2019.293 0471

Survey on Collision-Avoidance Navigation …

31

64. Wang N, Sun Z, Jiao Y, Han G (2019) Surge-heading guidance based finite-time path-following of underactuated marine vehicles. IEEE Trans Veh Technol. https://doi.org/10.1109/TVT. 2019.2927893 65. Wang N, Karimi HR (2019) Successive waypoints tracking of an underactuated surface vehicle. IEEE Trans Ind Inform. https://doi.org/10.1109/TII.2019.2922823 66. Wang N, Karimi HR, Li H, Su S-F (2019) Accurate trajectory tracking of disturbed surface vehicles: a finite-time control approach. IEEE/ASME Trans Mechatron 24(3):1064–1074 67. Liu Y, Song R, Bucknall R et al (2019) Intelligent multi-task allocation and planning for multiple unmanned surface vehicles (USVs) using self-organising maps and fast marching method. Inf Sci 496:180–197 68. Zhou X, Wu P, Zhang H, Guo W, Liu Y (2019) Learn to navigate: cooperative path planning for unmanned surface vehicles using deep reinforcement learning. IEEE Access 7:165262– 165278 69. Tan G, Zou J, Zhuang J, Wan L, Sun H, Sun Z (2020) Fast marching square method based intelligent navigation of the unmanned surface vehicle swarm in restricted waters. Appl Ocean Res 95:102018 70. Lee J, Kim H, Hwang DH, Oh SH, So H (2018) Multi-radio integrated navigation system M&S software design for GNSS back-up. In: 2018 IEEE/ION position, location and navigation symposium (PLANS), IEEE, pp 1026–1033 71. Lisowski J (2019) Sensitivity of safe trajectory in a game environment on inaccuracy of radar data in autonomous navigation. Sensors 19:1816 72. Liang S, Zhi-Ming Q, Heng L (2018) A survey on route planning methods of AUV considering influence of ocean current. In: 2018 IEEE 4th international conference on control science and systems engineering (ICCSSE), IEEE, pp 288–295 73. Wang N, Jin X, Er MJ (2019) A multilayer path planner for a USV under complex marine environments. Ocean Eng 184:1–10 74. Xinping Y, Chao W et al (2017) Conceptual design of navigation brain system for intelligent cargo ship. J Navig China 40(4):95–98, 136 75. Yoo B, Kim J (2018) Ship route optimization considering on-time arrival probability under environmental uncertainty. In: 2018 OCEANS-MTS/IEEE Kobe Techno-Oceans (OTO), IEEE, pp 1–5 76. Jahnke A, Arp HPH, Escher BI, Gewert B, Gorokhova E, Kühnel D et al (2017) Reducing uncertainty and confronting ignorance about the possible impacts of weathering plastic in the marine environment. Environ Sci Technol Lett 4(3):85–90 77. Roy N, Burgard W, Fox D, Thrun S (1999) Coastal navigation-mobile robot navigation with uncertainty in dynamic environments. In: Proceedings 1999 IEEE international conference on robotics and automation (Cat. No. 99CH36288C), vol 1. IEEE, pp 35–40 78. Sormunen OVE, Goerlandt F, Häkkinen J, Posti A, Hänninen M, Montewka J et al (2015) Uncertainty in maritime risk analysis: extended case study on chemical tanker collisions. Proc Inst Mech Eng Part M J Eng Marit Environ 229(3):303–320 79. Katsanevakis S, Moustakas A (2018) Uncertainty in marine invasion science. Front Mar Sci 5:38 80. Park J, Choi J, Choi HT (2019) COLREGS-compliant path planning considering time-varying trajectory uncertainty of autonomous surface vehicle. Electron Lett 55(4):222–224 81. Park JS, Park C, Manocha D (2017) Intention-aware motion planning using learning based human motion prediction. In: Robotics: science and systems 82. Johansen TA, Perez T, Cristofaro A (2016) Ship collision avoidance and COLREGS compliance using simulation-based control behavior selection with predictive hazard assessment. IEEE Trans Intell Transp Syst 17(12):3407–3422 83. Patterson A, Lakshmanan A, Hovakimyan N (2019) Intent-aware probabilistic trajectory estimation for collision prediction with uncertainty quantification. arXiv preprint, arXiv:1904. 02765(2019)

32

C. Wang et al.

84. Soloperto R, Köhler J, Allgöwer F, Müller MA (2019) Collision avoidance for uncertain nonlinear systems with moving obstacles using robust model predictive control. In: 2019 18th European control conference (ECC), IEEE, pp 811–817 85. Liu S, Li, Jie T et al (2017) Creating autonomous vehicle systems. Synth Lect Comput Sci 6(1):1–186 86. Krata P, Szlapczynska J (2018) Ship weather routing optimization with dynamic constraints based on reliable synchronous roll prediction. Ocean Eng 150:124–137 87. Zaccone R, Martelli M (2018) A random sampling-based algorithm for ship path planning with obstacles. In: Proceedings of the international ship control systems symposium-iSCSS, Glasgow, pp 170–178 88. Lee SM, Roh MI, Kim KS, Jung H, Park JJ (2018) Method for a simultaneous determination of the path and the speed for ship route planning problems. Ocean Eng 157:301–312 89. Zaccone R, Figari M (2017) Energy efficient ship voyage planning by 3D dynamic programming. J Ocean Technol 12(4):49–71 90. Biyela P, Rawatlal R (2019) Development of an optimal state transition graph for trajectory optimisation of dynamic systems by application of Dijkstra’s algorithm. Comput Chem Eng 125:569–586 91. Liu C, Mao Q, Chu X, Xie S (2019) An improved A-star algorithm considering water current, traffic separation and berthing for vessel path planning. Appl Sci 9(6):1057 92. Sharma A, Nazir S (2017) Distributed situation awareness in pilotage operations: implications and challenges 93. Hyvönen M, Rajala M, Virtanen A, Jankkari J, Huhtala K, Ritala R (2015) Assistive situation awareness system for mobile multimachine work environments. IEEE Trans Intell Transp Syst 16(6):3403–3413 94. Raptodimos Y, Lazakis I, Theotokatos G, Varelas T, Drikos L (2016) Ship sensors data collection and analysis for condition monitoring of ship structures and machinery systems. In: Smart Ship Technology 95. Xinping Y, Feng M, Jialun L et al (2019) Applying the navigation brain system to inland ferries. In: 18th conference on computer and IT applications in the maritime industries, Tullamore, pp 156–162 96. Kienle M, Franz B, Winner H et al (2014) Concept and development of a unified ontology for generating test and use-case catalogues for assisted and automated vehicle guidance. IET Intell Transp Syst 8(3):183–189 97. Xue J, Chen Z, Papadimitriou E, Wu C, Van Gelder PHAJM (2019) Influence of environmental factors on human-like decision-making for intelligent ship. Ocean Eng 186:106060 98. Hanson BB, Hanson TE (2017) U.S. Patent No. 9,669,904. Washington, DC: U.S. Patent and Trademark Office, (2017). 99. Huang R (2018) Maritime intelligent real-time control system based on UAV. In: 2018 international conference on robots & intelligent system (ICRIS), IEEE, pp 10–12 100. Kutsuna K, Ando H, Nakashima T, Kuwahara S, Nakamura S (2019) NYK’s approach for autonomous navigation—structure of action planning system and demonstration experiments. J Phys Conf Ser 1357(1):012013 (IOP Publishing, Norway) 101. MSC 99/5/6: considerations on definitions for levels and concepts of autonomy-submitted by Finland, 12 March 2018 102. IMO’s 81st Maritime Safety Committee Document (MSC 81/23/10): Japan, Marshall Islands, Netherlands, Norway, Singapore, United Kingdom and United States propose “Development of Electronic Navigation Strategy” 103. Wang CB et al (2017) e-Navigation and its data standard analysis. J World Shipp 40(1):22–24 104. Zongjiang G, Yingjun Z et al (2017) Research summary of unmanned ship. J Dalian Marit Univ 43(2):1–7 105. Im I, Shin D, Jeong J (2018) Components for smart autonomous ship architecture based on intelligent information technology. Procedia Comput Sci 134:91–98

Survey on Collision-Avoidance Navigation …

33

106. Porathe T, Rødseth ØJ (2019) Simplifying interactions between autonomous and conventional ships with e-navigation. J Phys Conf Ser 1357(1):012041 (IOP Publishing) 107. Ahn J, Joung TH, Kang SG, Lee J (2019) Changes in container shipping industry: autonomous ship, environmental regulation, and reshoring. J Int Marit Saf Environ Aff Shipp 3(3–4):21–27 108. Jeong SH, Shim JH, Choi KS, Son YC (2018) Analysis and design of common platform core technology for maritime autonomous surface ships. J Adv Navig Technol 22(6):507–513

Ocean Explorations Using Autonomy: Technologies, Strategies and Applications Yuanchang Liu, Enrico Anderlini, Shuwu Wang, Song Ma, and Zitian Ding

Abstract Ocean exploration has become one of the most important strategies for a sustainable development for our world. To better understand the ocean and make an efficient use of its resources, autonomous marine vehicles (AMVs) including both surface and underwater vehicles play an essential role to extend and accelerate the exploration capabilities. This chapter provides an in-depth review of the key technologies in the development of autonomous surface vehicles (ASVs) and autonomous underwater vehicles (AUVs), which are two main types of AMVs. With the illustration of some typical vehicle prototypes, the control methods and deployment strategies of ASVs and AUVs, especially the collaborative operation of these two types of vehicles, have been discussed to inspire a wide application of marine autonomy in future ocean explorations. Keywords Autonomous marine systems · Marine robotics · Artificial intelligence · Robust control · Unmanned surface vehicles · Autonomous underwater vehicles

1 Introduction The ocean is the origin of life on Earth. Its total area is about 360 million square kilometres, which is 71% of total Earth’s surface area. Simultaneously, the ocean is abundant in resources and an important part of the human living environment. Nowadays, with the rapid development of society, people are facing an acute issue of land resources shortage. Therefore, the exploitation and utilization of marine resources is closely related to the survival and development of human beings. Y. Liu (B) · E. Anderlini · S. Wang · S. Ma · Z. Ding Department of Mechanical Engineering, University College London, Torrington Place, London WC17JE, UK e-mail: [email protected] S. Wang School of Energy and Power Engineering, Wuhan University of Technology, Wuhan, PR China Intelligent Transport System Research Centre, Wuhan University of Technology, Wuhan, PR China © Springer Nature Singapore Pte Ltd. 2022 S.-F. Su and N. Wang (eds.), Offshore Robotics, Offshore Robotics, https://doi.org/10.1007/978-981-16-2078-2_2

35

36

Y. Liu et al.

Since the twenty-first century, with the continuous changes in global situation, the ocean exploration has become a vital priority in many countries. Competition between countries on marine resources has become intense. In today’s increasingly strong international competition, safeguarding the security of the maritime territories is a priority for each country. Hence, accelerating the construction of marine field is of great significance for safeguarding national security and stimulating national economy. In the construction of marine field, the environment safety is particularly important. In marine environments, where many dangers such as high-water pressure and fluctuations in ocean currents exist, it becomes extremely difficult to perform missions by manpower making the autonomous operation become a promising solution. At present, autonomous marine vehicles (AMVs) and their development and deployment have begun to receive increasing widespread attention. AMVs mainly include two categories of autonomous surface vehicles (ASVs) and autonomous underwater vehicles (AUVs). An ASV is a vessel that is equipped with control and communication equipment to perform special missions on surface. An AUV is a vessel that travels underwater without the need of operator input. Compared with traditional vehicles, AMVs have the advantages of modularity, high intelligence, good manoeuvrability, strong autonomy, safety and etc. [1]. It can safely and efficiently complete complex missions in dynamic, unknow and complex marine environments [2]. Due to these advantages, AMVs are broadly employed in both military and civilian applications [3]. With the development of navigation technology, the role of AMVs in military and civilian missions continues expanded. In military field, AMVs can be utilized for scout, maritime patrolling, weapons delivery, and antisubmarine missions and etc. [4]. With regard to civilian missions, the applications of AMVs are more extensive, mainly including data collection, search and rescue, ocean sampling, disaster warning and etc. [5–8]. In most situations, an AMV needs to perform missions successfully supported by high-level of intelligence and key aspects that will contribute to the improvement of the autonomy level of AMVs include: • developing a reliable guidance, navigation and control system for AMVs; • promoting the implementation of artificial intelligence (AI) technologies for effective decision making; • developing multiple vehicles to strengthen the system robustness and mission efficiency. This chapter therefore provides a thorough review on the development of AMVs by specifically discussing the advances in above-mentioned three aspects. In order to provide readers a coherent and comprehensive understanding, review will be carried out by targeting ASVs and AUVs, respectively. The rest of the is organised as follows. In Sect. 2, introduction to ASVs will be provided with a specific emphasis on how the deployment and coordination of multiple ASVs. In Sect. 3, technologies developed in underwater vehicles will be reviewed. Sections 4 and 5 mainly discuss the application of ASVs and AUVs. Section 6 concludes the paper.

Ocean Explorations Using Autonomy …

37

2 Autonomous Surface Vehicles (ASVs) and the Deployment Strategies In this section, a review to autonomous surface vehicles (ASVs) is introduced. Note that many existing literatures have provided an enriched insight into this area mainly covering the technology development in motion planning [9], sensing and perception [10] and control [11]. The core to achieve the autonomy in ASVs is the construction of autonomous navigation system, which consists of sensing, planning and control modules. Within the planning module, apart from the motion planning which is to generate an optimised trajectory for vehicles to follow, another important aspect is task management/allocation. Such a problem mainly considers allocating multiple tasks for either single or multiple vehicles to efficiently complete a mission. Therefore, this section mainly focuses on task allocation with a special interest in multiASV systems. Inspired by other types of autonomous vehicles, especially mobile robots, key developments within this area will be discussed. Also, with the advance in machine learning algorithms in recent decades, the implementation of various learning algorithms in multi-task allocation problem will be reviewed.

2.1 Tasks Management in ASVs Under different circumstances, multi-vehicle autonomous system can be used to solve various problems, but most importantly the multi-task problem where an ASV fleet is required to execute a number of tasks in a collaborative and cooperative way. One of the critical points to deploy a multi-vehicle system for multi-task is to develop an efficient task assignment strategy which can ensure every single task to be assigned to a reasonable vehicle in a fleet. In terms of the development of multi-task allocation, the early stage research concentrated on the study of multi-robot systems with an ambition that the developed techniques can be applied on autonomous spacecraft to support complex space mission [12]. Among these early works, Simmons and Apfelbaum [13] proposed a pioneering task description language based on a three-layer control architecture which consists of planning layer, executive layer and behaviour layer for robot control. Similarly, a task-level mission management was proposed for the intelligent autonomous system in [14]. Since the millennium, the research interests gradually started moving towards the area in multi-vehicle autonomous systems. Javirs et al. [15] implemented a multiagent cooperative system for autonomous manufacturing. Meanwhile, McLurkin and Yamins [16] tested and carried out a trade-off among a sequence of algorithms to achieve the dynamic task assignment. In this work, each task can only be completed by collaborations among several mini-robots. There were also researches about a more general-purpose task allocation for autonomous systems. For instance, Schwertfeger and Jenkins [17] designed a multi-robot belief propagation to process the

38

Y. Liu et al.

distributed task allocation problem; Botelho and Alami [18] proposed a cooperation model for multi-robot cooperation based on negotiated task allocation named M+. These researches mostly focused on the achievement of collaboration and were only implemented on indoor prototypes. As the topic of this chapter reveals, the interests of this research are drawn by the higher-level task management strategy for multi-ASV. Liu and Bucknall [3] proposed a hierarchical structure for formation control of ASV fleet, which consists of three layers, the Task Management Layer, the Path Planning Layer and the Task Execution Layer (shown in Fig. 1). Raboin et al. [19] proposed a heuristic strategy to help with defending an asset vessel from the intruder in the context of the rampant piracy operations in some sea area. Ulam et al. [20] tested both the case-based reasoning (CBR) mission planner and contract net protocol (CNP) base task allocation for a naval mine removal operation. Most of the studies in this area are highly practical.

2.2 Machine Learning Used in ASVs Machine learning algorithms are widely employed in solving problems laid in Task Management Layer and Path Planning Layer in the context of the hierarchical architecture proposed by [3]. For instance, the self-organising map (SOM), which is an unsupervised learning algorithm with a 2-layer-neuralnetwork structure (network structure is shown in Fig. 2), is used to solve the travelling salesman problem (TSP) [21, 22]. Some task allocation and path planning problems that indicate to form a closed loop cruising path (also, some non-robust open-loop trails [23]) have employed this SOM-TSP concept with a circle topology [24]. The competitive mechanism of the SOM algorithm is also used in the relevant field, for example the task allocation of multi-ASV path planning [25]. In the context of a fleet dealing with multiple tasks (the task number is larger than the vehicle number), the clustering concept in machine learning can help with the task assignment. Standard clustering algorithms include k-means clustering, quality threshold clustering and hierarchical clustering based on the greedy algorithm. The unsupervised clustering algorithm, k-means, has properties such as robustness and computational efficiency, which satisfy the requirement of the task-assignment problem mentioned in [26]. Nevertheless, the conventional k-means clustering also has some drawbacks including tending to form clusters with similar size, difficulties in dealing with centrosymmetric inputs and local optimisation, which cannot guarantee an expected result in all circumstances. Consequently, [27, 28] employ k-means algorithm directly for task allocation of multi-robot system, but the objectives of the optimisation are limited to aspects such as the minimum travel distance. Moreover, some research also focuses on undertaking motion planning and lower level control by machine learning algorithms. [29] proposed a ASV control framework based on the extreme learning theory, which is firstly proposed by [30]. Differing from the typical neural network which is based on the back-propagation,

Ocean Explorations Using Autonomy …

39

Fig. 1 The hierarchical structure for formation control of ASV fleet [3]

an extreme-learning neural network is a much more convenient method to obtain the weight matrix. Furthermore, some government-dominated research also aims to combine the ASV and machine learning to achieve military objectives, like anti-submarine [31].

40

Y. Liu et al.

Fig. 2 The two-layer network structure for self-organising map (SOM)

2.3 Multi-ASV Systems and Control Architectures A multi-ASV system (MAS) is a computerized system composed of multiple interacting intelligent ASVs. The modes of interacting behaviours can be divided into a fully cooperative mode, a fully competitive mode, and a hybrid mode. ASVs within a system share a common environment, which can be perceived using multiple sensors and data-fusion algorithms. The control architecture of MAS (shown in Fig. 1) is usually divided into three categories: centralized topology, distributed topology and hybrid topology (Fig. 3). A centralized topology MAS is mostly used in the formation control task, in which multiple ASVs are consistently controlled based on a control centre. The goal of a centralized topology MAS is to complete the task with the lowest total cost with several multi-agent tasks fall into this category. For example, the ASV formation task [6] and the drone display task [7], etc. A distributed topology MAS is mostly used in the distributed control task, in which multiple ASVs are separately controlled based on intercommunication. The goal of a distributed topology MAS is to complete the task with the lowest cost for each agent. Agents can be fully cooperative or fully competitive in the task. For example, the collision avoidance task for cooperative ASV [8] and the competitive task. Game theory [9] is usually used in these tasks. A hybrid topology MAS is the combination of the above two systems. In some real tasks, different sets of ASVs are consistently controlled based on the different control centres. At the same time, different sets of agents are full competitive [10]. In this case, the hybrid topology MAS is more suitable for modelling such a task.

Ocean Explorations Using Autonomy …

41

Agent

Agent

Communication

Agent

Control center

Communication

Agent

Agent

Agent

Agent

Agent

(a) Centralized topology

(b) Distributed topology Agent Communication

Agent

Control center

Agent

Agent

(c) Hybrid topology Fig. 3 The control architectures of multi-ASV systems

3 Unmanned Underwater Vehicles 3.1 Unmanned Underwater Vehicles Technologies Unmanned underwater vehicles (UUVs) were first developed in the 1960s by the US with the Navy Cable-Controlled Underwater Recovery Vehicle (CURV and CURV II), followed in the 1970s by the CURV III [32]. The main function of these large vehicles was the recovery of damaged equipment and rescue of personnel. The systems became famous as a consequence of two high-profile operations: the retrieval of a lost hydrogen-bomb off the coast of Palomares, Spain, in 1966 and the rescue of two pilots of the PISCES III submersible off the coast of Cork, Ireland, in 1973. Since then, UUVs have evolved into two separate technology types: remotely operated vehicles (ROVs) and autonomous underwater vehicles (AUVs). A thorough review of ROVs was compiled by Capocci et al. [33]. Since the 1980s, these vehicles have been extensively developed and used by the defence, energy and scientific sectors. Recently, there have been renewed interests in mining of the ocean floor for rare minerals [34, 35]. Example applications include survey of ships and underwater structures, exploration of the ocean floor for marine life, energy resources and minerals, monitoring of pipelines and underwater foundations of offshore structures, maintenance of pipelines, cables and offshore installations and construction of underwater structures. Hence, ROVs are separated into two categories: inspection-

42

Y. Liu et al.

and intervention-class vehicles [33]. The latter are much larger in size due to their functional requirements. To perform their tasks successfully, ROVs need high levels of positioning and execution accuracy and high power. These requirements result in designs that present a large number of thrusters in addition to a trim and compensation system (ROVs can typically be over-actuated). Additionally, since high speed is not necessary, the vehicles are boxy in shape to maximise the use of the space available on the platform at the expense of the drag force. Furthermore, ROVs are characterised by an umbilical tether, through which they may be powered (depending on the cable length and powering requirements, some ROVs present energy storage on-board), can send sensory information to and receive commands from the operator. As a result, ROVs are mostly operated from a mothership support vessel or platform. Although control systems have been developed to enable path tracking, trajectory control and object manipulation [36–39], expert human pilots are tasked with the operation of the ROVs. Reviews of AUVs for scientific exploration and defence applications can be found in [40–42], respectively. AUVs are routinely used by the energy industry for pipeline or cable inspection, underwater structure maintenance, payload delivery, communication/navigation network nodes and seafloor mapping. The oceanography industry employs AUVs for the study of the oceans properties and currents, seafloor mapping, benthic studies and the study of marine biology, e.g. through acoustic localisation of marine mammals or fish species. The defence industry relies on AUVs for intelligence, surveillance and reconnaissance missions, mine countermeasures, anti-submarine warfare and communication/navigation network nodes. Here, conventional AUVs propelled by a rudder and actuated by sternplanes, rudders and a variable buoyancy device are classified separately from underwater gliders. An example of a commercial AUV is described in [43], while reference [44] covers a deep-diving, oceanographic AUV. These vehicles work by first controlling buoyancy either through a piston or an oil bladder (for the deeper diving vehicles). Then, the devices fly at a specific depth thanks to the thrust provided by a propeller unit, using a rudder to control the heading and a sternplane to control the depth. AUVs can also present vertical thrusters and forward plans to provide controllability at low speeds or to enable station keeping, e.g. as described in [45, 46]. Batteries (typically lead–acid, but recently also lithium-ion and rechargeable despite the gas build-up risks) usually power the vehicles. AUVs can present cylindrical or multiple spherical pressure hulls depending on the depth rating. New trends include the development of ever-larger AUVs, which are defined as extra-large UUVs (or XLUUVs) for enhanced capabilities [47]. Underwater gliders were first conceptualised by Stommel in 1989 [48] as an evolution of ARGO floats that would be able to travel both horizontally and vertically. However, the first actual glider technologies were realised only in the late 1990s [49– 51]. More recent vehicle technologies are described in Claustre et al. [52–54]. Good reviews of underwater gliders for oceanography applications can be found in the following references [55, 56]. The function of underwater gliders is to monitor the oceans along their depth up to 6,000 m deep [56]. Thanks to an increasing range of sensors, these vehicles can

Ocean Explorations Using Autonomy …

43

measure salinity, temperature, pressure, density, chlorophyll levels, acoustic signals from animals and other vehicles, etc. Hence, underwater gliders enable the study of the oceans on both large- and small-scales in both space and time. As a result, the devices have represented a step change for the oceanography industry, enable an increased number of observations for a lower cost than traditional technologies, e.g. ship-based studies. Additionally, underwater gliders are also being used for naval applications, especially for the monitoring of regions of interest. The main propulsion system for underwater gliders is represented by the variable buoyancy device, which changes the buoyancy of the vehicle either through an oil bladder and a hydraulic pump for deep-water applications or a piston for shallow-water applications ( ε do ¯ x(l) = ∂h k (xk ,ve,k ) | 6: H (l) (l) . ∂xk x =ˆx ,v =vˆ k

7: 8: 9: 10:

k|k

e,k

e,k|k

(l+1) (l+1) Calculate Kk(l+1) , xˆ k|k and Pk|k through Eqs. 77, 78, and 79, respectively. (l) (l) (l+1) (l+1) Calculate Aˆ , Bˆ , λˆ and νˆ through Eqs. 81, 82, 83, and 84, respectively. k

k

k|k

k|k

(l+1) through Eq. 85. Calculate Pe,k|k−1 (l+1) (l+1) Calculate τˆ and βˆ through Eqs. 87 and 88, respectively. k|k

k|k

11:

(l+1) Calculate μ(l+1) k−1 and vˆ e,k|k−1 through Eqs. 89 and 90, respectively.

12:

Hve,k

(l+1)

=

∂h k (xk ,ve,k ) |x =ˆx(l+1) ,v =vˆ (l+1) . ∂ve,k k e,k k|k e,k|k−1 (l+1) (l+1) (l+1) through K e,k , vˆe,k|k and Pe,k|k

Eqs. 93, 94, and 95, respectively. 13: Calculate 14: l = l + 1. 15: end while (l) (l) (l) (l) 16: xˆ k|k = xˆ k|k , Pk|k = Pk|k , vˆe,k|k = vˆe,k|k , Pe,k|k = Pe,k|k . Output:ˆxk|k , Pk|k , vˆe,k|k , Pe,k|k .

Single-Beacon Based Underwater Robot Navigation

3.4.4

77

Parameter Selection

The tuning parameters ρα and ρλ and the nominal ESV uncertainty parameters μ¯ k and σ¯ e,k must be selected in the implementation of the VB-based adaptive single-beacon navigation method. • Selection of ρλ By combining Eqs. 61, 68, 69, 83, 84 and 85, we obtain the recursive form of (l+1) as follows: Pe,k|k−1 (l+1) Pe,k|k−1 =

2 ˆ (l) ρλ (Pe,k−1|k−1 + σ¯ e,k−1 ) + Aˆ (l) k + Bk

ρλ + 1

.

(96)

(l+1) , is the According to Eq. 96, the modified a priori variance of the ESV, i.e., Pe,k|k−1 2 weighted sum of the prior information Pe,k−1|k−1 + σ¯ e,k−1 and the new information Aˆ k(l) + Bˆ k(l) with weights of ρλ and 1, respectively. The nominal parameter σ¯ e,k−1 dominates the accuracy of the nominal a priori variance of the ESV uncertainty, and the parameter ρλ dominates the confidence level of σ¯ e,k−1 . • Selection of ρα From Eqs. 66, 67, 87 and 89, we obtain

μ(l+1) k−1 =

(l) − vˆe,k−1|k−1 ) μ¯ k−1 + ρα (vˆe,k|k

ρα + 1

.

(97)

Eq. 97 indicates that μ(l+1) ¯ k−1 and k−1 equals the weighted average between μ (l) (vˆe,k|k − vˆe,k−1|k−1 ). Similar to σ¯ e,k−1 and ρλ , the nominal mean μ¯ k−1 dominates the accuracy of the nominal a priori mean of the ESV uncertainty, and the parameter ρα dominates the confidence level of μ¯ k−1 . • Selection of μ¯ k−1 and σ¯ e,k−1 Because the VB approach only guarantees the local convergence of variational iterations, accurate prior information is necessary. Therefore, the nominal parameters μ¯ k−1 and σ¯ e,k−1 must be close to the true values μk−1 and σe,k−1 , respectively. Commonly, there is not a strong prior belief in μ¯ k−1 ; thus, it is reasonable to set μ¯ k−1 = 0m/s.

4 Numerical Studies We conducted a comprehensive numerical study to compare the performance of different navigation methods according to field data. The field data were collected from a surface boat that cruised at a speed of approximately 3m/s, and the groundtruth trajectory was provided by a global positioning system. The boat was equipped with a hydrophone to receive the acoustic signal transmitted by the beacon mounted

78

Z.-B. Zhu et al.

on the seafloor, at a surveyed location 123.745m underwater. Because the slant range between the vehicle and beacon could be computed using their locations (which were known), the ground-truth ESV at every instant could be computed by dividing the slant range by the corresponding transit time obtained from the TOA measurement subtracted by the known TOE. Four navigation methods described in this chapter will be compared: (1) the basic single-beacon navigation method (presented in Sect. 2, referred to as “SM”); (2) the state-augmented method (presented in Sect. 3.2, referred to as “SAM”); (3) the EM-based single-beacon navigation method (presented in Sect. 3.3, referred to as “EMSM”); and (4) the VB-based single-beacon navigation method (presented in Sect. 3.4, referred to as “VBSM”). In implementing the aforementioned four navigation methods, the following initial settings were selected: (1) 0.5 m/s in both the x and y directions for the initial oceancurrent offset, (2) 1540 m/s for the nominal ve , (3) two initial position cases will be considered: (1) a precisely known initial position (referred to as the “Case 1”) and (2) a relative large initial position offset (10 m in both the x and y directions, referred to as the “Case 2”). The tuning parameters for these four methods were as follows: (1) σc = 0.01 m/s, (2) σw = 0.1 m/s, (3) σe = 1 m/s for the SAM, (4) σ¯ e = 1 m/s and μ¯ = 0m/s for the VBSM, (5) Rr m = 52 m2 for the SM, (6) Rtm = 0.0012 s2 for the SAM, EMSM and VBSM, (7) Rcm = 0.012 I2 m2 /s2 , (8) ρα = 9, and ρλ = 1 for the VBSM, and (9) threshold value (ε) of 0.0001m/s for the EMSM and VBSM. The average root-mean-square (ARMS) errors of the horizontal distance

ARMS H

" # T #1 %

 (xk − xˆk )2 + (yk − yˆk )2 =$ T k=1

and the ESV ARMSve

" # K #1 % =$ (ve,k − vˆe,k )2 K k=1

were utilized as the evaluation indices. T and K represent the total number of fixed sampling intervals and transit-time measurements, respectively.

4.1 Case 1 In Case 1, we assume that the initial position of the vehicle is precisely known. Figures 1 and 2 show a comparison of the estimated trajectory and the corresponding horizontal distance error  H = x 2 + y 2 for the four aforementioned navigation methods. The ARMS H values for these four navigation methods are presented in Table 1. As indicated by Figs. 1 and 2, along with the ARMS H , the accuracy of the EMSM is the highest if the initial position of the vehicle is precisely known.

Single-Beacon Based Underwater Robot Navigation

79

Table 1 The comparison of ARMS H and ARMSve among SM, SAM, EMSM and VBSM in Case 1 SM SAM EMSM VBSM ARMS H (m) ARMSve (m/s)

10.31722 28.72394

2.46011 3.95667

0.24838 0.40800

1.79023 3.44684

Fig. 1 Planar position estimates comparison among SM, SAM, EMSM and VBSM in Case 1. The square markers in the subgraph represent the estimated and real positions of the vehicle at the same epoch

The SAM, EMSM, and VBSM have better performance than the SM because they consider the impact of an inaccurate ESV. The VBSM is slightly better than the SAM, which is explained as follows: the VBSM can adaptively adjust the statistical parameter related to the ESV uncertainty, whereas the SAM treats these parameters as constants, and their values may be inaccurate. Figure 3 presents the ESVs estimated via the SM, SAM, EMSM, and VBSM. In the SM, the ESV is a predefined constant with the value of the nominal ve , i.e., 1540m/s. The ARMSve values for the four methods are presented in Table 1. Figure 3 indicates that the SAM, EMSM, and VBSM followed the variation trend of the ESV. Similar to the localization accuracy, the ESV estimation result of the VBSM was slightly better than that of the SAM, owing to the adaptiveness of the VBSM to the ESV uncertainty parameters. Among the methods tested, the EMSM had the best ESV estimation performance. We also examined the number of iterations in each correction step. As shown in Fig. 4, the algorithm needed approximately 10 iterations for the EMSM to converge, and the iteration times has a large fluctuation, but all the acoustic transit time correc-

80

Z.-B. Zhu et al.

Fig. 2 Horizontal distance error comparison among SM, SAM, EMSM and VBSM in Case 1

Fig. 3 The ESV estimated by SM, SAM, EMSM and VBSM in Case 1

tions can converge to a steady state within 50 iterations for the VBSM. In practical applications, the acoustic communication rate is commonly low, and the measurement interval of the transit time is commonly long enough to process the iterations (30s for the field data of this study).

4.2 Case 2 In Case 2, we consider that the initial position offset of the vehicle is 10m in both the x and y directions. Comparisons of the estimated trajectories and horizontal distance error among the SM, SAM, EMSM, and VBSM are shown in Figs. 5 and 6, respectively. The ARMS H values for the four navigation methods are pre-

Single-Beacon Based Underwater Robot Navigation

81

Fig. 4 Iteration times of EMSM and VBSM during each correction step in Case 1

Fig. 5 Planar position estimates comparison among SM, SAM, EMSM and VBSM in Case 2. The square markers in the subgraph represent the estimated and real positions of the vehicle at the same epoch

82

Z.-B. Zhu et al.

Fig. 6 Horizontal distance error comparison among SM, SAM, EMSM and VBSM in Case 2

Table 2 The comparison of ARMS H and ARMSve among SM, SAM, EMSM and VBSM in Case 2 SM SAM EMSM VBSM ARMS H (m) ARMSve (m/s)

13.08824 28.72394

7.16246 6.92449

14.24036 23.04378

5.90028 4.95326

sented in Table. 2. In contrast to Case 1, the EMSM had the worst localization accuracy for Case 2. The positioning error of the EMSM remained nearly constant with its initial position offset. The reason why the EMSM had a large positioning error is described in Remark 1. The Bayesian treatment of the ESV is helpful for the SAM and VBSM to achieve good results in the case of an initial position offset. Owing to the adaptiveness of the VBSM, it had the best localization accuracy among the methods tested (slightly better than the SAM). The trend of the results for the ESV estimation performance of the four navigation methods was similar to that for the localization accuracy. As indicated by the estimated ESV shown in Fig. 7 and the ARMSve values presented in Table 2, the VBSM had the best ESV estimation results, whereas the EMSM was unsatisfactory. The ESV estimation of the VBSM was slightly more accurate than that of the SAM owing to the adaptiveness of the VBSM to the ESV uncertainty parameters.

5 Conclusions and Future Directions The performance of single-beacon navigation is affected by the accuracy of the ESV, which is difficult to precisely determine in practical applications. In this chapter, we described the basic single-beacon navigation methods, which need a precisely

Single-Beacon Based Underwater Robot Navigation

83

Fig. 7 Estimated ESV by SM, SAM, EMSM and VBSM in the Case 2

known ESV. Then, to eliminate the impact of the ESV setting error, several methods for online estimation of the ESV were presented, including the state-augmented method, EM method, and VB approximation method. The performance of these navigation methods was compared through a comprehensive numerical study based on field data. In addition to the ESV setting error, other factors also affect the performance of the single-beacon navigation system, such as the beacon position setting error and the unknown clock drift between the clock of the beacon and the core clock of the AUV. It will be a valuable future direction to design a single beacon navigation method considering the influence of ESV setting error, clock drift and beacon position setting error. Additionally, outliers frequently arise in acoustic transit-time measurements in practical applications, and dealing with them is an interesting topic.

References 1. Paull L, Saeedi S, Seto M, Li H (2014) AUV navigation and localization: a review. IEEE J Oceanic Eng 39(1):131–149. https://doi.org/10.1109/JOE.2013.2278891 2. Chen Y, Zheng D, Miller PA, Farrell JA (2016) Underwater inertial navigation with long baseline transceivers: a near-real-time approach. IEEE Trans Control Syst Technol 24(1):240– 251. https://doi.org/10.1109/TCST.2015.2429613 3. Miller PA, Farrell JA, Zhao Y, Djapic V (2010) Autonomous underwater vehicle navigation. IEEE J Oceanic Eng 35(3):663–678. https://doi.org/10.1109/JOE.2010.2052691 4. Larsen MB (2000) Synthetic long baseline navigation of underwater vehicles. In: IEEE conference and exhibition of the OCEANS 2000 MTS. (Cat. No.00CH37158), vol 3, pp 2043–2050. https://doi.org/10.1109/OCEANS.2000.882240 5. Zhu Z, Hu SJ (2018) Model and algorithm improvement on single beacon underwater tracking. IEEE J Oceanic Eng 43(4):1143–1160. https://doi.org/10.1109/JOE.2017.2754018

84

Z.-B. Zhu et al.

6. Claus B, Kepper IVJH, Suman S, Kinsey J (2018) Closed-loop one-way-travel-time navigation using low-grade odometry for autonomous underwater vehicles. J Field Robot 35(4):421–434. https://doi.org/10.1002/rob.21746 7. Kepper J, Claus B, Kinsey J (2018) A navigation solution using a MEMS IMU, model-based dead-reckoning, and one-way-travel-time acoustic range measurements for autonomous underwater vehicles. IEEE J Oceanic Eng. https://doi.org/10.1109/JOE.2018.2832878 8. Indiveri G, De Palma D, Parlangeli G (2016) Single range localization in 3-D: observability and robustness issues. IEEE Trans Control Syst Technol 24(5):1853–1860. https://doi.org/10. 1109/TCST.2015.2512879 9. De Palma D, Arrichiello F, Parlangeli G, Indiveri G (2017) Underwater localization using single beacon measurements: observability analysis for a double integrator system. Ocean Eng 142:650–665. https://doi.org/10.1016/j.oceaneng.2017.07.025 10. Wang S, Chen L, Hu H, Gu D (2013) Single beacon based localization of AUVs using moving horizon estimation. In: 2013 IEEE/RSJ international conference on intelligent robots and systems, pp 885–890. https://doi.org/10.1109/IROS.2013.6696455 11. Harris ZJ, Whitcomb LL (2016) Preliminary study of cooperative navigation of underwater vehicles without a DVL utilizing range and range-rate observations. In: 2016 IEEE international conference on robotics and automation (ICRA), pp 2618–2624. https://doi.org/10.1109/ICRA. 2016.7487420 12. Bayat M, Crasta N, Aguiar AP, Pascoal AM (2016) Range-based underwater vehicle localization in the presence of unknown ocean currents: theory and experiments. IEEE Trans Control Syst Technol 24(1):122–139. https://doi.org/10.1109/TCST.2015.2420636 13. Crasta N, Moreno-Salinas D, Bayat B, Pascoal AM, Aranda J (2018) Range-based underwater target localization using an autonomous surface vehicle: observability analysis. In: 2018 IEEE/ION position, location and navigation symposium (PLANS), pp 487–496. https://doi. org/10.1109/PLANS.2018.8373417 14. Zhu Z, Hu SJ, Li H (2016) Kalman-based underwater tracking with unknown effective sound velocity. In: OCEANS 2016 MTS/IEEE monterey, pp 1–9. https://doi.org/10.1109/OCEANS. 2016.7761086 15. Zhu Z, Hu SJ, Li H (2016) Effect on kalman based underwater tracking due to ocean current uncertainty. In: 2016 IEEE/OES autonomous underwater vehicles (AUV), pp 131–137. https:// doi.org/10.1109/AUV.2016.7778660 16. Qin HD, Yu X, Zhu ZB, Deng ZC (2020) An expectation-maximization based singlebeacon underwater navigation method with unknown ESV. Neurocomputing 378:295–303. https://doi.org/10.1016/j.neucom.2019.10.066, http://www.sciencedirect.com/science/article/ pii/S0925231219315048 17. Qin HD, Yu X, Zhu ZB, Deng ZC (2020) A variational Bayesian approximation based adaptive single beacon navigation method with unknown ESV. Ocean Eng 209:107484. https://doi.org/ 10.1016/j.oceaneng.2020.107484 18. Gadre AS, Stilwell DJ (2004) Toward underwater navigation based on range measurements from a single location. In: IEEE international conference on robotics and automation (ICRA ’04 2004), vol 5, pp 4472–4477. https://doi.org/10.1109/ROBOT.2004.1302422 19. Gadre AS, Stilwell DJ (2005) A complete solution to underwater navigation in the presence of unknown currents based on range measurements from a single location. In: 2005 IEEE/RSJ international conference on intelligent robots and systems, pp 1420–1425. https://doi.org/10. 1109/IROS.2005.1545230 20. Casey T, Guimond B, Hu J (2007) Underwater vehicle positioning based on time of arrival measurements from a single beacon. OCEANS 2007:1–8. https://doi.org/10.1109/OCEANS. 2007.4449186 21. Simon D (2006) Optimal state estimation: kalman, H infinity, and nonlinear approaches, pp 123–129 22. Julier SJ, Uhlmann JK (2004) Unscented filtering and nonlinear estimation. Proc IEEE 92(3):401–422. https://doi.org/10.1109/JPROC.2003.823141

Single-Beacon Based Underwater Robot Navigation

85

23. Qin H, Yu X, Zhu Z, Deng Z, Tian R (2018) Unscented kalman filter based single beacon underwater localization with unknown effective sound velocity. In: 2018 OCEANS-MTS/IEEE kobe techno-oceans (OTO), pp 1–6. https://doi.org/10.1109/OCEANSKOBE.2018.8559342 24. Arasaratnam I, Haykin S (2009) Cubature kalman filters. IEEE Trans Autom Control 54(6):1254–1269. https://doi.org/10.1109/TAC.2009.2019800 25. Doucet A, Johansen A (2009) A tutorial on particle filtering and smoothing: fifteen years later. In: Handbook of nonlinear filtering, vol 12 26. Doucet A et al (2000) On sequential Monte Carlo methods for Bayesian filtering. Stat Comput 27. Sarkka S (2013). Bayesian filtering and smoothing. https://doi.org/10.1017/ CBO9781139344203 28. Valluru J, Patwardhan SC, Biegler LT (2018) Development of robust extended Kalman filter and moving window estimator for simultaneous state and parameter/disturbance estimation. J Process Control 69:158–178. https://doi.org/10.1016/j.jprocont.2018.05.008, http://www. sciencedirect.com/science/article/pii/S0959152418301094 29. Sabet MT, Daniali HM, Fathi A, Alizadeh E (2018) Identification of an autonomous underwater vehicle hydrodynamic model using the extended, cubature, and transformed unscented kalman filter. IEEE J Oceanic Eng 43(2):457–467. https://doi.org/10.1109/JOE.2017.2694470 30. Sabet MT, Sarhadi P, Zarini M (2014) Extended and unscented kalman filters for parameter estimation of an autonomous underwater vehicle. Ocean Eng 91:329– 339. https://doi.org/10.1016/j.oceaneng.2014.09.013, http://www.sciencedirect.com/science/ article/pii/S0029801814003370 31. Garcia-Velo J, Walker BK (1997) Aerodynamic parameter estimation for high-performance aircraft using extended kalman filtering. J Guidance Control Dyn 20(6):1257–1260. https:// doi.org/10.2514/2.7597 32. Deng ZC, Yu X, Qin HD, Zhu ZB (2018) Adaptive Kalman filter-based single-beacon underwater tracking with unknown effective sound velocity. Sensors (Switzerland) 18(12). https:// doi.org/10.3390/s18124339 33. Dempster AP, Laird NM, Rubin DB (1977) Maximum likelihood from incomplete data via the EM algorithm. J R Stat Soc 39(1):1–38 34. Schon T, Wills A, Ninness B (2011) System identification of nonlinear state-space models. Automatica 47(1):39–49. https://doi.org/10.1016/j.automatica.2010.10.013 35. Ge M, Kerrigan E (2017) Noise covariance identification for nonlinear systems using expectation maximization and moving horizon estimation. Automatica 77:336–343. https://doi.org/ 10.1016/j.automatica.2016.11.011 36. Jiang K, Zhang H, Karimi HR, Lin J, Song L (2018) Simultaneous input and state estimation for integrated motor-transmission systems in a controller area network environment via an adaptive Unscented Kalman filter. IEEE Trans Syst Man Cybern Syst:1–10. https://doi.org/10. 1109/TSMC.2018.2795340 37. Wu Z, Jiang B, Kao Y (2019) Finite-time H∞ filtering for ito stochastic markovian jump systems with distributed time-varying delays based on optimisation algorithm. IET Control Theory Appl 13(5):702–710. https://doi.org/10.1049/iet-cta.2018.6119 38. Huang Y, Zhang Y, Xu B, Wu Z, Chambers JA (2018) A new adaptive extended kalman filter for cooperative localization. IEEE Trans Aerosp Electron Syst 54(1):353–368. https://doi.org/ 10.1109/TAES.2017.2756763 39. Sarkka S, Nummenmaa A (2009) Recursive noise adaptive kalman filtering by variational bayesian approximations. IEEE Trans Autom Control 54(3):596–600. https://doi.org/10.1109/ TAC.2008.2008348 40. Tzikas DG, Likas AC, Galatsanos NP (2008) The variational approximation for Bayesian inference. IEEE Signal Process Mag 25(6):131–146. https://doi.org/10.1109/MSP.2008.929620

Review on the Research of Ship Automatic Berthing Control Zhang Qiang, Nam-Kyun Im, Ding Zhongyu, and Zhang Meijuan

Abstract To fully understand the key problems that the design of under-actuated ship automatic berthing controller under the restricted water area, the uncertain mathematical model and the relatively strong environmental disturbances, considering the requirements of the navigation practice and conduct systematic theoretical exploration and research. Research and engineering practice on ship automatic berthing control in the domestic and abroad are introduced, three control tasks and five research difficulties are summarized in this paper. Combining the ship automatic berthing research with the “shipbuilding industry 4.0 and e-Navigation” project of IMO, the development of ship automatic berthing trends to be unified model, intelligent control, whole process of berthing, accurate measurement and engineering realization. Keywords Ship automatic berthing · Shipbuilding industry 4.0 · Ship control · Closed-loop gain shaping · Review

1 Introduction Berthing means bringing a vessel to her berth until the ship is made fast. As is shown in Fig. 1, the berthing ship (indicated by a red color) comes alongside from seaside and follows the position A to B and C by means of engines, helm movements and tugboats assistance. Since a ship may berth port or starboard side to, with no wind or tide, with the tide ahead, with the wind onshore or offshore, the master, the pilot and the tug skippers must have good local knowledge of the wind conditions, tides, depths and aids to navigation, but they also have to take into consideration the propeller transverse thrust, rudder effect and the astern power of the main engine. With the development of large-scale, rapid and automatic ships, as well as the reduction of the number of crew or the lack of experience, ship berthing becomes Z. Qiang (B) · D. Zhongyu · Z. Meijuan Shool of Navigation and Shipping, Shandong Jiaotong University, Weihai, China e-mail: [email protected] N.-K. Im Division of Navigation Science, Mokpo National Maritime University, Mokpo-si, South Korea © Springer Nature Singapore Pte Ltd. 2022 S.-F. Su and N. Wang (eds.), Offshore Robotics, Offshore Robotics, https://doi.org/10.1007/978-981-16-2078-2_4

87

88

Z. Qiang et al.

m F ro

Tugboats

e sid sea

A B

Ship in berthing

C Berthing ship 1

Berthing position

Berthing ship 2

Fig. 1 Ship berthing plan

one of the most difficult and complex maneuvers. According to statistics, 70% of the accidents are related to the bad ship skills of the drivers in the port [1]. The control actuators such as paddle and rudder are usually designed for constant speed, but the actual ship berthing is affected by shallow water, low speed, bank effect, the operation is becoming more complex. In practice, large ships often rely on tugs to assist in berthing, and can also conduct independent berthing when conditions permit. Therefore, it is of great practical significance to study the automatic berthing control of ships [2]. The research on automatic berthing control began in the early 1990s. So far, this research mainly depends on human experience and precise control algorithm [3]. Because the problem of automatic berthing involves the low-speed movement of the ship in shallow water, the interference of wind, wave and current is relatively large, the amount of system information is increased, and the operation and control are more difficult [4]. Therefore, automatic berthing of ships has become an urgent problem in ship motion control. Ship automatic berthing control is a multi input and multi output problem. With the development of ship automation in recent years, automatic berthing control has attracted a lot of attention abroad, but the domestic research is relatively less, and only limited to theoretical research. The hidden scientific problems in the study of automatic berthing control are the study of ship maneuvering and positioning control in restricted water area and strong environmental disturbances. In this paper, the control tasks and control methods of ship automatic berthing control are summarized, and the solutions to the main problems are summarized. Based on the strategic background of “shipbuilding industry 4.0” and “E-navigation” of IMO, the development trend of ship automatic berthing control research is analyzed.

Review on the Research of Ship …

89

2 Automatic Berthing Types Ship berthing control involves track planning, path tracking and stabilization control. It includes deceleration control, parking control, reversing control, parallel berthing control, inbound track keeping control, steering control, turning control, etc. At present, the main research results come from Japan, China, South Korea, Vietnam and other countries. Different researchers have different emphases on the research of control tasks. Combing the main research results, three types of automatic berthing are summarized: stabilization control outside the berth, direct berthing, stabilization control outside the berth and then parallel berthing, as shown in Table 1.

2.1 Stabilization Control Outside the Berth Ahmed, Hasegawa, et al. [5] believe that the ship sails along the virtual route to 1.5 times the length of the ship from the wharf to stabilization, that is, to complete the automatic berthing control task. The virtual route is the route that guides most of the captains control the ships to the berth, and the angle between the virtual route and the quay shoreline is generally 30°. The use of 1.5 times the length of the ship is influenced by Kose et al. By analyzing the operation procedure of large ships, Kose believes that there should be enough room for operation to deal with any adverse Table 1 The classification of ship automatic berthing control Control tasks

Main researchers

Stabilization control outside the berth

Japan: Ahmed, Hasegawa, et al. Sail to the virtual berthing [5] route, then decelerate continuously through the telegraph to stop, and finally reverse to stabilize the ship to 1.5 times the strength of the ship from the berth

Direct berthing

Korea: Park et al. [6]

Two oars and two rudders cooperate with bow thruster to complete parallel approach

China: Bu Renxiang, Guo Chen, Liu Yang, et al. [7, 8]

Stabilization control using rudder and propeller of underactuated ship

Japan: Mizuno et al. [9–11]

Using the head and tail thruster to approach directly berthing

Vietnam: Tran, Korea: Im, bui, Kim, France: djouani, China: Lee, Japan: ohtsu, etc. [12–17]

Using rudder, oar, pusher (or pusher) to complete stabilization control outside the berth and parallel berthing

Stabilization control outside the berth and then parallel berthing

Implementation method

90

Z. Qiang et al.

situation. Therefore, berthing operation is not to make the ship close to the wharf, but to stabilize the ship at a certain distance outside the berth.

2.2 Direct Berthing Park et al. [6] use the oars and bow thrusters of the twin-screw twin-rudder ship to complete parallel berthing and make the ship close to the wharf. References [7, 8] researched the automatic berthing control of Underactuated Ships. The problem of berthing is transformed into the problem of stabilizing control to achieve berthing. In references [9–11], the optimal maneuvering planning algorithm was used to control the head and tail thrusters directly berthing.

2.3 Stabilization Control Outside the Berth and then Parallel Berthing Some references [12–15] used rudder, oar, head and tail thruster to complete the stabilization from the starting point to the outside of the berth, and then use vehicle, rudder and thruster to propel the berth horizontally to make the ship close to the wharf. Tamaru et al. [16] of Japan designed a controller for the wharf by using the head and tail thrusters, and designed automatic mooring systems such as suction cups and bumpers to make the ship approach and moor in parallel.

3 Difficulties of Research Automatic berthing is usually considered as one of the most complex problems in the field of ship control, and the main difficulties involved are as follows: (1)

(2)

(3)

Uncertainties of models. Due to the reduction of ship speed, the frequent use of rudders, oars and external disturbances, which cause the change of various hydrodynamic derivatives, the equation of motion is more complex and has high-order nonlinearity, so it is difficult to accurately predict ship motion. Finally, the parameters and even the structure of the mathematical model of ship motion are perturbed [18]. Uncertainties of disturbances. In shallow water, when the ship moves at low speed, the external interference such as wind and current increases relative to the steady ship speed, and the uncertainty increases. Stabilization problems of underactuated systems. Ship maneuvering involves two tasks. The first is to make the ship position converge to the expected path; the second is to meet the attitude to the expected path [19]. Because

Review on the Research of Ship …

(4)

(5)

91

conventional surface ships are not equipped with driving devices in the transverse direction (or when the speed is greater than 3–4 kn, it is not suitable to use side thrusters), there is a large drift angle in low-speed movement, and the steering efficiency and maneuverability become poor [20]. The traditional control strategy can’t satisfy the stabilization of freedom of configuration or attitude. Optimal path planning. Route planning is the most important work for the captain before berthing and unberthing. Under actuated ships can not track any trajectory, and need to consider the nonlinear state constraints and actuator saturation, so the automatic berthing controller needs to meet the feasibility requirements in the optimal path planning. Control device accuracy. Different navigation practices have different requirements for accuracy. The ship does not need to be accurate to centimeter level in ocean navigation, and the deviation of a few centimeters in berthing operation may cause extreme damage. In practice, the approach steering often depends on the bow mast and the reference object to determine the course and the angular acceleration, and its sensitivity is higher than compass. Therefore, the accuracy of the control device should be considered when the ship berths automatically.

4 Current Research 4.1 Uncertainties of Models To solve the problems of model uncertainty, it is necessary to establish a mathematical model to approach the real ship motion, which needs to fully consider the influence of low speed, shallow water, shore wall, etc. on the ship motion. At the same time, some literatures also use control algorithm to solve the problem of model uncertainty. The mathematical models of ship automatic berthing include MMG (mathematical model group) separated model and Abkowitz integrated model. A large number of documents have absorbed the MMG separation modeling idea of Japan to establish mathematical model. References [1, 5, 21] only considered the influence of wind, and adds wind disturbances to MMG modeling, neglecting the influence of other factors. In references [9, 10, 22], MMG model was added to the thruster model. In reference [23], a comprehensive disturbances term were added to the MMG modeling, and using controller design to solve the problems of model uncertainty. In 1984, Kose et al. [24] proposed a mathematical model for low speed shallow water in harbor based on MMG. Considering that the ship speed u may be zero or negative at low speed, the dimensionless factor was improved by replacing u with g (g is the acceleration coefficient of gravity). The model has been further applied in references [25, 26]. In 1989, Japan’s MSS began to report the low speed model. In 1993, Hasegawa made a comprehensive review of the low-speed model. To solve the relatively small values of the lateral velocity v and the yaw rate r in the second and third order expansions of the MMG model, and the variable problems of the relatively

92

Z. Qiang et al.

large values in the low-speed model, as well as the zero value problems of the U, he summarized and reviewed the multiple model, the Fourier expansion model, the lateral flow model and the response model. In 1994, Hasegawa gave the mathematical model of parking, and comprehensively considered the transverse force and torque of the propeller in front and back. In 1996, Yang [27] put forward a mathematical model of berthing and unberthing operation according to the characteristics of berthing and unberthing at low speed and large drift angle, considering the factors of ship, oar, rudder, wind, cable, anchor, bank wall and tugboat. The practical calculation of the forces acting on the upper flow of the hull and the forces of the oars, rudders, tugs, cables and anchor chains was put forward. In 1997, Zhang [3] considered the influence of wind and shallow water and adopted the marine administration model. In 1999, Jia et al. [28] ignored the transverse force of the front vehicle. Based on the regression of six ship model test parameters, the transverse force and moment model of the reversing propeller was given. The model is simple and practical. In 2000, Le et al. [29] obtained low-speed hydrodynamic coefficient through VLCC free model test. The hydrodynamic coefficients and the added mass estimated by the least square and Kalman filter. Endo et al. [30] completed the stop procedure in strict accordance with the navigation practice steps of “full speed forward → half speed forward → micro speed forward → stop”, and obtained the corresponding relationship between ship speed U and propeller speed n through the form of expert questionnaire. This achievement has been applied to the design of berthing controller for many times. However, this model took 498t small ship as the research object, whether it was suitable for large ships needed further discussion. In 2005, Fossen et al. [31] solved the problems of ship motion control under different ship speeds and sea conditions by using the mathematical model of feedback control and unified state space form. In 2007, Lataire et al. [32] proved that the effects of the bank wall depends on the geometry of the bank wall and the ships through the hull through extensive measurement and calculation. The measurement results were compared with the NORRBIN formula in 1974, and a new calculation formula was given. Eloot et al. [33] compared the control force (steering force) and disturbing force (Shore force), and gave a new calculation method. This method can quickly judge the feasibility of manipulation. In 2009, Lee et al. [34] used Abkowitz integral ship mathematical model and Shengziyin shallow water model to research the influence of entering shallow water on ship handling. As MMG and Abkowitz models were not ideal for low-speed ships, in 2012, Oh et al. [35] reviewed several main low-speed mathematical models, including transverse flow model, polynomial model, Fourier expansion model, CFD (computer fluid dynamics) mathematical model based on RANS (Reynolds averages Navier Stokes), and according to the experimental results of relevant literature results, the characteristics of each model were compared and the application suggestions were given. In 2012, Shih et al. [36] proposed to use a unified nonlinear state space model to solve the problems of model and disturbances uncertainty in the near-shore restricted waters. In 2014, Sian et al. [37] used CFD to simulate the shore effect of large LNG ship in shallow water, and obtained the coefficient of ship hydrodynamic force and ship-shore effect force. In recent years, using CFD to calculate and predict shallow

Review on the Research of Ship …

93

water and low-speed models was becoming more and more popular, and the validation of data needed further attention. In 2014, Zhang established a responsive low-speed shallow water mathematical model. The model integrated the influence of ship speed and water depth, and was transformed into an equivalent state space form through matrix transformation, which was used for turning control in port [38]. In 2016, Ferrari [39] applied the State-Dependent Algebraic Riccati Equation nonlinear control technique to the automatic berthing manoeuvre of a catamaran with waterjet propulsion, and researched the mathematical models of ship maneuvering and water jet propulsion, then the control algorithm was developed and applied to the case of the ship moving to the berth automatically. In 2019, Kamolov [40] proposed a system for automatic berthing of ships based on the Internet of things and last model technologies. Its main goal was to automatically find the space through the sensors fixed in the port, and transmitted the data to the ships, so as to improve the berthing efficiency of ships. In 2019, Maki [41] solved the off-line automatic berthing problem, modeled the optimal control problem as the minimum time problem, and used the covariance matrix adaptive evolution strategy (CMA-ES) to optimize the real value variables. This method can also provide new ideas for online (real-time) control. In addition, the artificial neural network (ANN) [42, 43], sliding mode iterative method [7], adaptive neural network [8] and other algorithms solve the problem of disturbance uncertainty as well as the problem of model uncertainty.

4.2 Uncertainties of Disturbances In order to solve the problems of wind and flow disturbances, the researchers used various algorithms such as artificial neural network, sliding mode control, expert system [17, 44], feedback control [45] to design the automatic berthing controller. Experienced drivers can handle external interference flexibly, and can use disturbances to assist berthing. Therefore, experienced experts are the main consulting objects for intelligent controller designers [18]. If artificial intelligence technology can be used to copy the experience of human brain in berthing and unberthing operation, automatic berthing is possible. ANN is widely regarded as a tool for dealing with complex nonlinear systems. In 1990, Yamato et al. first used ANN algorithm to design berthing controller [46]. In 1993, Hasegawa et al. [47] used ANN algorithm and expert system to research automatic berthing. The expert system assisted ANN to output the best rudder angle at a certain speed and angular speed. By using 3layer neural network and BP algorithm to train the weight factor and offset. In 1993, Yamato et al. [17] used the expert decision system to research the port entry and berthing in the restricted waters. The expert system is responsible to make decisions of courses for ship maneuvering and restricted navigation environment. However, the knowledge base is obtained from simulation experiments not from real ship data. In 1997, Zhang et al. [3] designed a multivariable neural network controller, using BP algorithm to train ANN online, which can adjust parameters online, accurately

94

Z. Qiang et al.

evaluate performance, and has certain generalization ability. The simulation results show that the controller is suitable for the disturbances of wind and shallow water. In 2002, Im et al. [48] used two different neural networks to output rudder angle and rotation speed. He used motion identification technology to estimate the environmental disturbances, and obtained the ship’s lateral velocity v and yaw rate r. Using two algorithms to improve the wind resistance of the controller. The two algorithms are respectively used to cancel the horizontal and vertical disturbances, and were successfully applied to the berthing control under limited wind speed, but they are not ideal for parallel wind disturbances. Later, some scholars improved their research results by increasing training samples, added the ship model test and ship operating experience data to the training samples. However, the test data is lack of generality and data consistency is difficult to guarantee. The problem of how to create appropriate training samples and deal with the disturbances of higher wind speed and arbitrary wind direction has not been solved. In 2007, Mizuno et al. [10] proposed the minimum time automatic berthing method by using the nonlinear programming method. According to the performance index of predictive control error caused by external interference, the input parameters of minimum time berthing control are adjusted. In order to solve the problem that it is difficult to adjust the optimal parameters, an advanced real-time simulator which can replace the on-line training and compensate the control system error was introduced. The nonlinear mathematical model of ship and off-line neural network interpolation were used to calculate the optimal solutions of various minimum time. At the same time, the position and course of the ship were predicted. In 2007, Bu et al. [7] combined with incremental feedback technology, did not need to estimate the wind and current disturbances to complete the automatic control of forward berthing. In 2012, Mizuno et al. [11] proposed a parallel simulation system based on predictive control algorithm. Based on the minimum time berthing method of nonlinear programming, the nonlinear model predictive controller was improved, and the parallel simulation method became more accurately track the signal under the wind disturbances by using the Graphical Processing Unit. The sea trial of Shioji Maru proved that the method is feasible. To solve the problem of low threshold of maximum interference wind speed, Ahmed et al. [5] used experiments to verify the control effect of ANN based on virtual window technology for the first time in 2012, and divided automatic berthing into three basic control problems: steering, deceleration and parking. The virtual window technology is used to ensure the consistency of training samples, and the nonlinear programming method is used to calculate the output rudder angle in the virtual window. However, the inconsistency of training samples can easily lead to the jitter of rudder angle. Xu et al. [49] established consistent training samples on the basis of reference [10]. In steering control, nonlinear programming method and optimal steering constraints were used to create training samples. However, problems such as too many steering constraints and rudder angle jitter still need to be verified. In 2013 and 2014, Ahmed et al. [1, 50] used the nonlinear programming method and the virtual window theory to increase the consistency training samples without rudder angle jitter. These samples contains navigation methods for most ports, Using BP Levenberg Marquardt algorithm and using two

Review on the Research of Ship …

95

independent feedforward neural networks instead of centralized network controller, the output rudder angle and rotation speed parameters were trained respectively. In the case of no wind, the effectiveness of ANN training is verified, and considering gust disturbance, ANN parameters are trained separately. At the same time, a PD controller was proposed, which not only controls the course, but also reduced the distance from the ship center of gravity to the virtual route. Using the 3 m “Esso Osaka” ship model to test and simulation, it was verified that the controller can effectively interfere with the 1.5 m/s wind speed (equivalent to 15 m/s of the real ship) without training samples. In 2017, Zhang [51] solved the calculation load problem of multi parameters ship automatic berthing neural network controller, according to the practice of berthing operation, by reasonably canceling the parameters of virtual navigation line and position auxiliary line, optimizing the network input parameters and reducing the calculation dimension. In 2018, Nguyen et al. [52] proposed a new neural network controller to control the ship to the berths of different ports without retraining the neural network structure by using the head up coordinate system including relative orientation and ship to berth distance. The effectiveness of the controller were verified by numerical simulation. In 2018, Nguyen and Im [53] designed a ship automatic berthing system based on neural network by using the ship parameters from the distance measurement system. The system can realize the automatic berthing of ships in different ports without retraining the neural network. In addition, the system ensured the accurate and continuous measurement of neural network input parameters. To verify the effectiveness of the algorithm, two virtual ports and one real port were simulated. The results showed that the system has good performance of automatic berthing. In 2018, Zou [54] proposed an iterative sliding mode control method based on chaos particle swarm optimization. To improve the performance of the controller, chaos particle swarm optimization algorithm with shrinkage factor was introduced, and the parameters of the controller were optimized according to the performance index of berth path. The mathematical model of 5446 TEU Container ship was used for control and simulation. The simulation results showed that the designed controller could park quickly under the disturbances of wind and waves. 2019, Shuai et al. [55] proposed a method of automatic berthing of ships based on artificial neural network for environmental disturbances, and established a mathematical model. In the simulation, sufficient and reliable data can be obtained through manual operation. To keep consistent with the manual control, a neural network with two parallel structures is proposed to control the ship thrust and rudder angle respectively. Feature selection technology and genetic algorithm are used to optimize the structure and reduce the training cost. The numerical simulation of different environmental disturbances, including no wind, constant wind and dynamic wind, is carried out. The results show that the ship can reach the wharf safely, which proves the validity of the method. In 2019, Piao [56] adopted active disturbances rejection control (ADRC) method to keep the ship on the predetermined track in order to resist the wind interference. The results showed that ADRC controller had good robustness, could solve the problems of insufficient autonomous route determination and strong anti-interference ability in the process of automatic berthing of underactuated

96

Z. Qiang et al.

unmanned vessel, and improved the safety of navigation. It layed a theoretical and experimental foundation for the further development of unmanned ship control.

4.3 Stabilization Problems of Underactuated Ship The degrees of freedom of configuration and attitude must be in stabilization in the process of berthing. Kim et al. [57] proposed a method based on logic control to drive the ship forward and backward repeatedly along the route, and gradually closed to the termination state. Theoretically, it overcomed the limitation of continuous excitation condition and allowed the ship speed to be negative. However, for large ships, it was difficult to control the vehicle and rudder repeatedly in a short time, so its practicability needed to be verified. In 2007, Bu et al. [7] combined with incremental feedback technology, designed a dynamic output feedback controller with acceleration nonholonomic constraints, the nonlinear sliding mode was defined in the extended state space. By using the recursive decomposition iterative design method, the track planning and tracking problem of the control system was transformed into the stabilization control problem of the scalar zero order system, which realized the automatic control of the forward berthing operation without the interference of wind and flow estimation. In 2012, Liu et al. [8] used the nonlinear adaptive control algorithm to design the path tracking controller and berthing controller for the restricted water area respectively, turning the nonholonomic system into a cascade nonlinear system, and then tracking the path using the nonlinear control law. Adaptive neural network algorithm was used to estimate the uncertainty of disturbances. The stability of the closed-loop system was analyzed by Lyapunov theory. In 2019, Nguyen [58] proposed a support system for ship navigation in waterway, the system performed three tasks. First, the ship could be controlled automatically according to the planned track by adjusting the rudder. Secondly, reducing the speed gradually to enter the berth area at low speed. Finally, at low speed, when the effectiveness of the rudder is not enough to control the bow at the required angle, the bow thruster would adjust the bow appropriately, and then changed the control mode to automatic berthing. Through the proposed system, the automatic system could be combined to obtain a fully automatic system. In order to verify the effectiveness of the proposed system, the training ship model was used for numerical simulation. In 2019, Zhang [59] solved the auto-berthing control problem for underactuated marine ships subject to unknown dynamics and external disturbances under the circumstance of the restricted waters in port, firstly, an additional control method is adopted to solve the underacurated problem. Secondly, a robust neural network (NN) adaptive approach based on the navigation dynamic deep-rooted information (DRI) is proposed to reconstruct the lumped uncertainties caused by unknown ship dynamics and external disturbances. Meanwhile, the dynamic surface control (DSC) and the minimum learning parameter (MLP) techniques are used to reduce the computational load of the adaptive NN control scheme. Considering the input saturation effects of control actuator (rudder,

Review on the Research of Ship …

97

propeller, etc.) and the coupling characteristics of uncertainties, this approach integrates neural network weights, approximation errors and external disturbance term as the composite uncertain parameters, which is estimated online by parameter adaptive technique. Finally, simulations are carried out on an underactuated model ship to verify the effectiveness of the proposed auto-berthing control scheme.

4.4 Path Planning In 1994, Djouani et al. [60] designed an automatic berthing controller with optimal control strategy mapping by using optimal path planning and ANN algorithm. Considering the nonlinear state constraints and actuator saturation, the problems of optimal path planning for ship obstacle avoidance were transformed into nonlinear and non convex set mathematical programming. Penalty function was used to simulate obstacle avoidance constraints. The discrete augmented Lagrangian method and Uzawa algorithm were used to solve the optimal planning problem. The optimal path planning could be used for off-line ship handling and collision avoidance control. In 1995, Djouani et al. [13] designed the optimal path planning according to the nonlinear mathematical model of the ship. Considering the nonlinear dynamic characteristics and internal and external constraints of the ship, the optimal constraint controller was designed by energy optimization to solve the problem of ship optimal path planning. The berthing was divided into two stages. The first stage used optimal path planning and feedback control. The second part used adaptive or neural network control. Due to the highly nonlinear mathematical model of the ship in the case of low speed and large maneuvering, the minimum time control could provide a reference path for automatic berthing. The minimum time manipulation problem could be solved by solving the nonlinear two-point boundary value problem. To fully consider the nonlinearity, Shoji et al. [61] put these problems as a nonlinear two-point boundary value variation problem, and finally solved the minimum time steering, minimum time parallel deviation, minimum time parking, minimum time berthing, and minimum time handling under wind disturbances by using the continuous conjugate gradient method [9]. However, the solution took a long time to calculate, did not obey the online control law, and because of the safety concerns in real ship handling, it was impossible to complete the real minimum time scheme. To solve the above problems, Mizuno et al. [9] proposed the minimum time manipulation method based on online and offline neural networks. Through ANN offline training, the minimum time solution was precalculated. Using AIC (Akaike’s information criterion) and MDL (minimum description length principle) to select the best neural network architecture. The sea trial of Shioji Maru practice ship was combined with simulation to verify that the method was ideal. In 2007, Im et al. [21] proposed to use the selection controller to solve the applicability problem of port geographical conditions. He divided the sea area into several areas and chose different weights of neural network according to different environments, so the ship can maintain the stability of berthing control. Different Ann algorithms are used to guide the ship

98

Z. Qiang et al.

from one area to another. The basis and practicability of each region division deserve further study. In 2019, Nguyen and Im [62] proposed a support system for automatic berthing of ships in the last stage of berthing. According to different tasks of berthing algorithms, three kinds of fuzzy controllers were established. The first controller was designed to control the ship’s longitudinal normal motion towards the wharf with the ship’s propeller. The second controller was designed to stabilize the relative azimuth error with the tugboat. At the same time the final controller would send the ship to the wharf according to the bow propeller and tugboat. Finally, the performance of the system was verified by numerical simulation. The results showed that the berthing support system has good performance for ships. In 2019, Li [63] proposed a layered trajectory planning method based on the multi-constraint analysis of environmental obstacles and berth and small, unmanned surface vehicle (USV) dynamics. The autoberthing open field experiment performed with the small Dolphin-I USV proved that the proposed method is effective and feasible.

4.5 Control Device Accuracy The measurement accuracy of ship speed and position is one of the most concerned problems in berthing. The accuracy of control device mainly depends on modern navigation technology. In view of the problem that ship path planning relies on manual judgment, especially the low input accuracy in low-speed mode, Maritin et al. [64] designed an embedded simulation test platform using dynamic planning algorithm. The route plan was stored in ECDIS in the form of waypoints. The operation decision module judges the parameter values according to the front and rear waypoints and outputs them to the execution module. The module calculates parameters for feedforward and feedback control with different priorities. Takai et al. [65] designed a fully automatic berthing system. The system consists of photoelectric position measurement system and ship control software. The error of photoelectric displacement measurement system was within 4 cm. Kawai et al. [66] designed a ranging system based on image sensor, which could be used for automatic berthing of ships with roll, pitch and head roll motion. The measurement accuracy was less than 3.1 m within the distance of 5–100 m. Ogura et al. [67] designed a remote stereo camera measurement system which could be used for automatic berthing. The experimental results of the system show that the average error is less than 1% in the range of 20– 100 m. Peng et al. [68, 69] designed the auxiliary berthing instrument and software display system for large ships. The two-point position differential global positioning system (DGPS) technology, ship automatic identification system (AIS) technology and wireless network communication technology are used to complete the design of the berther. It was pointed out that the positioning accuracy of general berthing should be controlled within 1 m. The test results showed that the positioning accuracy of the berther could reach 60 cm, and the speed accuracy could reach 5 cm/s. The distance sensor was not used in this equipment, and the influence of information analysis errors such as DGPS and AIS on the accuracy needed further research. Lee et al. [70,

Review on the Research of Ship …

99

71] proposed a fuzzy control scheme based on image processing. Setting two objects up on the shore, and the navigation is carried out by using the technology of overlapping and serial vision. The target detection camera is mounted on the bow, and the autopilot system analyzed the deviation between the string and the ship position, and used the fuzzy logic control system to give the rudder angle. The experimental results showed that the deviation of berthing distance is 10–20%. The accuracy, stability and anti-interference ability needed further study. Gao [72] introduced the application of real time kinematic (RTK) technology in berthing, which could provide continuous and efficient berthing parameters in real time and achieved centimeter level accuracy. In 2017, Ilyas [73] analyzed the influence of microwave propagation conditions on the operation of automatic berthing and positioning system, and considered the influence of microwave phase change on positioning accuracy. The system consisted of two card readers installed on the ship and two active balise installed on the wharf. The placement diagram of reader and transponder was shown in Fig. 2. Each reader measured its distance from each transponder. So we could get four distances d ij , the distance b1 between the readers and the distance b2 between the transponders were known, a basic geometric expression can be used to determine the exact ship position relative to the piers. Then, in 2017, Ilyas [74] proposed a development principle and method of local navigation system based on zero difference signal transformation, which enabled us to determine the distances with the help of the simplest system design. In 2017, Yang [75] had designed a parallel automatic mooring system with mooring lines, which used mooring devices on the upper deck of the ship to berth, and used bond graph method to model the system. Figure 3 shows the structure of the experimental apparatus. The basic experiments were conducted by setting the same system as the mooring device of the ship on the outside of the tank. Using PID control method to water tank test, the test shown in Fig. 4 was carried out to verify the effectiveness. Fig. 2 System components placement

100

Z. Qiang et al.

Fig. 3 Structure of the experimental apparatus

Fig. 4 Experimental devices

In 2018, Mizuno [76] designed a new method to solve the nonlinear optimal control problem by using genetic algorithm and simulation method, with the development of high-performance GPU, it was possible to execute parallel genetic algorithm and simulation in a short time.

5 Engineering Practice In 2018, Folgefonn, a ferry owned by Norled, a Norwegian operator, had become the first ferry in the world to install an automatic docking system. The 83 m long ferry “Folgefonn” was equipped with Wärtsilä’s innovative automatic berthing system, hybrid propulsion device and Wärtsilä wireless charging

Review on the Research of Ship …

101

system. The ship had completed a three-month port berthing test, mainly testing the actual berthing of the ship in the port. During the test, the captain did not carry out any manual control operation. According to the program design, the automatic berthing system could be activated when the ship traveled at normal speed to the position 2000 m away from the berth. Then the system started to perform deceleration operation, and started the opposite row and berthing operation through the full-automatic way until the ship stopped safely in the berth. When the ship was ready to sail again, the system could perform the opposite steps to complete the ship departure operation. All ship operations, including turning and propulsion, were automatically controlled by software. However, the crew could end the automatic program and intervene and control the ship manually at any time. The automatic function could liberate the crew from the cab and put their energy into the perception of external situation, so as to improve the safety and reliability of operation. Wärtsilä’s automatic berthing technology had brought great benefits to ship operators. These advantages include: improving safety, reducing the human errors greatly; reducing energy loss, improving the utilization rate of propeller effectively; improving the efficiency of berthing, so that the ship had more time to berthing and rest. Norled company has given the “Folgefonn” to Wärtsilä for modification and upgrading, adding some other intelligent marine products and systems developed by Wärtsilä. Wärtsilä was released on “Complete autonomous navigation and autodocking ferry”, shown in Fig. 5. It’s auto-berthing with wireless inductive charging and auto-mooring system shown in Fig. 6. Auto-berthing used mooring system without cable. 2018.6.16, Volvo penta officially demonstrated a new automatic berthing technology for yachts in Gothenburg, Sweden.

Fig. 5 Complete autonomous navigation and autodocking ferry

102

Z. Qiang et al.

Fig. 6 Auto-mooring system

Volvo penta, a new technology for automatic berthing of yachts, is based on the auto parking technology of the same group. On the same day, Volvo penta automatically parked a 68 foot yacht between two 65 foot racing yachts participating in Volvo Ocean Race, shown in Fig. 7. 2018.12.03, Rolls-Royce’s “completely auto-navigation and auto-berthing ferry” was released, shown in Fig. 8. And officials claim that they were “the world’s first fully autonomous sailing ferry”. 2018.12.04, ABB successfully demonstrated the active “remote control” mode of passenger transit. The idea was slightly different from Rolls-Royce and Wärtsilä. It mainly rely on “power positioning system” and “remote control system” (Fig. 9). The world’s first ship intelligence situational awareness system made by RollsRoyce laster year, shown in Fig. 10. In this year, NYK joint JMS had successfully developed the ship berthing assistance system, which could given control orders to steer. Japanese merchant ship Mitsui announced that its joint test project “auto-berthing and auto-unberthing” had

Fig. 7 Autoberthing of 68 foot yacht

Review on the Research of Ship …

Fig. 8 Rolls-Royce’s auto-berthing ferry

Fig. 9 ABB’s “remote control” mode

Fig. 10 Intelligence situational awareness system

103

104

Z. Qiang et al.

been completed. The project had been tested 54 times. The world’s first autonomous container ship “Yara Berkeland” would be launched next year. The auto-berthing system was simple and easy to apply.

6 Prospect Based on Germany’s “industry 4.0” strategy, the main characteristics of ships are gradually forming the ability of perception, evaluation, prediction, optimization, control, management, remote support, etc., and even designing and building unmanned, self-organized, self reconfigurable “thinking” intelligent ships [77]. At the same time, the International Maritime Organization (IMO) is carrying out the strategic implementation plan of “e-Navigation". In terms of ship automatic control, the core of the two strategic plans is to make the operation more intelligent, more evolved and more accurate. Therefore, it is urgent to solve the most difficult and complex control problem in the field of automatic berthing. Therefore, combined with the research background and current situation, the research of automatic berthing control is developing towards intelligent control, whole process of berthing, accurate measurement and engineering realization. (1)

(2)

(3)

Unified model. According to the actual berthing environment, researchers often focus on building mathematical models in some aspects, but lack of unified and modular research. The main direction of the mathematical model of automatic berthing is MMG modeling, which includes MMG ship basic model (including thruster and propeller model), environment (wind, wave and current) interference model, low-speed model, shallow water model, bank wall effect model, ship to ship effect model, tugboat model, cable model, anchor chain model, etc. At the same time, the MMG model needs to be modified for the large-scale and fast ships. Intelligent control. Ship berthing control is a practical engineering task. Using artificial intelligence algorithm to copy human berthing experience is still a research direction of automatic berthing. The fuzzy logic algorithm can simulate the thinking and decision-making of the captain when berthing, and form the rule base of fuzzy control. Due to the complexity of berthing operation, the corresponding fuzzy rules are also complex. How to abstract the simple and applicable fuzzy rules is the focus of research. Whole process of berthing. The voyage plan stipulates that the route design needs to be from the departure port to the destination port. Berthing operation is an important part of the ship’s voyage plan. Combined with three kinds of control tasks in the study of automatic berthing (stabilization outside the berth, direct berthing, stabilization outside the berth and then parallel berthing), the whole process of berthing is put forward, which is the integrated whole process control of inbound track planning, track keeping control and berth stabilization

Review on the Research of Ship …

(4)

(5)

105

control. Under the premise of ensuring safety, the intermediate stabilizing link is omitted. Accurate measurement. With the advent of the era of digital navigation, advanced electronic navigation equipment for the development of navigation technology has brought a major change. DGPS, AIS, RTK, ECDIS, anemometer, bathymeter, log, attitude analyzer and image sensing ranging system have been applied to the ship industry, providing real-time and accurate berthing data for the automatic berthing controller. With the help of these advanced measuring equipment, it is possible to realize automatic berthing in an all-round way. Engineering realization. There are some achievements in the research of ship automatic berthing controller in theory, but most of them have some assumptions, which is not consistent with the actual engineering environment. With the development of research, nonlinear problems such as actuator characteristics, system saturation and time delay have become the research direction of automatic berthing control.

7 Conclusion In this paper, the research literatures of automatic berthing control at home and abroad are summarized, and the main research methods and current situation are proposed. Combined with the background of “shipbuilding industry 4.0” and “ENavigation” strategic plan of the world maritime organization, the research trend of intelligent control, whole process of berthing, accurate measurement and engineering realization of automatic berthing control are prospected. It is believed that with the deepening of the theory, simulation and sea trial of automatic berthing control, the research goal will be realized.

References 1. Ahmed YA, Hasegawa K (2013) Implementation of automatic ship berthing using artificial neural network for free running experiment. In: 9th IFAC conference on control applications in marine systems, Osaka, pp 25–30 2. Jia XY, Zhang XK (2002) Ship motion intelligent control and H∞ robust control. Dalian Maritime University Press, Dalian 3. Zhang Y, Grant EH, Pratyush S (1997) A multivariable neural controller for automatic ship berthing. IEEE Control Syst 17(4):31–44 4. Zhang XK (2012) Concise and robust control of ship motion. Science Press, Beijing 5. Ahmed YA, Hasegawa K (2012) Automatic ship berthing using artificial neural network based on virtual window concept in wind condition. In: 13th IFAC symposium on control in transportation systems, Sofia, pp 12–14

106

Z. Qiang et al.

6. Park JY, Nakwan K (2014) Design of an adaptive backstepping controller for auto-berthing a cruise ship under wind loads. Int J Nav Arch Ocean Eng 6(2):347–360 7. Bu RX, Liu ZJ, Hu JQ (2007) Berthing controller of underactuated ship with nonlinear sliding mode. J Traffic Transp Eng 7(4):24–29 (in Chinese) 8. Liu Y, Guo C (2012) Automatic berthing control of underactuated surface ships in restricted waters based on nonlinear adaptive control method. In: Proceeding s of the 31st Chinese control conference, Hefei, pp 939–944 9. Mizuno N, Tamai Y et al (2002) A ship’s minimum-time maneuvering system using neural networks. In: IECON 02.Sevilla, Spain, pp 1848–1853 10. Mizuno N (2007) Minimum time ship maneuvering method using neural network and nonlinear model predictive compensator. J Control Eng Pract 15(6):757–765 11. Mizuno N, Kakami H, Okazaki T (2012) Parallel simulation based predictive control scheme with application to approaching control for automatic berthing. In: 9th IFAC conference on manoeuvring and control of marine craft, 2012. The International Federation of Automatic Control, Arenzano, Italy, pp 19–21 12. Tran VL, Im NK (2012) A study on ship automatic berthing with assistance of auxiliary devices. Int J Nav Arch Ocean Eng 4(3):199–210 13. Djouani K, Hamam Y (1995) Minimum time-energy trajectory planning for automatic ship berthing. IEEE J Ocean Eng 20(1):4–12 14. Ohtsu K (1991) A fully automatic berthing test using the training ship Shioji Maru. J Navig 44(2):213–223 15. Bui VP, Kim BY, Choi YW, Kawai H (2009) A study on automatic ship berthing system design. In: Proceedings of the 2009 IEEE international conference on networking, sensing and control, Okayama 16. Tamaru H, Hagiwara H et al (2005) Development of automatic berthing system for Kaisho Maru and its performance evaluation. Jpn Inst Navig 113:157–164 17. Yamato HT, Koyama T et al (1993) Automatic berthing system using expert system (in Japanese). J Soc Nav Arch Jpn 174:327–337 18. Zhang XK (2010) Ship control system. Dalian Maritime University Press, Dalian 19. Skjetne R, Fossen TI, Kokotovic PV (2005) Adaptive output maneuvering with experiment for a model ship in a marine control laboratory. Automatica 41(2):289–298 20. Guo C, Wang Y, Sun FC et al (2009) Survey for motion control of underactuated surface vessels. Control Decis 24(3):321–329 (in Chinese) 21. Im NK, Keon LS, Do BH (2007) An application of ANN to automatic ship berthing using selective controller. Int J Mar Navig Saf Sea Transp 1(1):101–105 22. Okazaki T, Ohtsu K (2008) A study on ship berthing support system-minimum time berthing control. In: Conference proceedings—IEEE international conference on systems, man and cybernetics, Singapore, pp 1522–1527 23. Le MD (2001) An automatic control system for ship harbour manoeuvres using decoupling control. In: Proceedings of the 2001 IEEE international conference on robotics & automation, Seoul, Korea, pp 2108–2113 24. Kose K, Hinata H et al (1984) On a new mathematical model for manoeuvring motions of ships in low speed. Soc Nav Arch Jpn 155:15–24 25. Hasegawa K, Fukutomi T (1994) On harbor manoeuvring and neural control system for berthing with tug operation. In: Proceedings of the 3rd international conference on manoeuvring and control of marine craft, Southampton, UK, pp 197–210 26. Hasegawa K, Kitera K (1993) Mathematical model of manoeuverability at low advance speed and its application to berthing control. In: The 2nd Japan-Korea joint workshop of ship and marine hydrodynamics, Osaka 27. Yansheng Y (1996) Study on mathematical model for simulating ship berthing or unberthing. J Dalian Marit Univ 22(4):11–15 (in Chinese) 28. Jia XY, Yang YS (1999) Mathematical model of ship motion—mechanism modeling and identification modeling. Dalian Maritime University Press, Dalian

Review on the Research of Ship …

107

29. Le MD, Kose K (2000) Estimation of ship hydrodynamic coefficients at low speed range and application to control ships. Jpn Inst Navig 103:33–39 30. Endo M, Hsegawa K (2003) Passage planning system for small inland vessels based on standard paradigms and manoeuvres of experts. In: MARSIM’03, Kanazawa, Japan 31. Fossen TI (2005) A nonlinear unified state-space model for ship maneuvering and control in a seaway. Int J Bifurc Chaos 15(9):2717–2746 32. Lataire E, Vantorre M et al (2007) Navigation in confined waters: influence of bank characteristics on ship-bank interactions. In: ICMRT 2007, Ischia, Italy 33. Eloot K, Verwilligen J, Vantorre M (2007) A methodology for evaluating the controllability of a ship navigating in a restricted channel. Arch Civ Mech Eng 7(3):91–104 34. Lee G, Surendran S et al (2009) Algorithms to control the moving ship during harbour entry. Appl Math Model 33(5):2474–2490 35. Oh KG, Hasegawa K (2012) Ship manoeuvring hydrodynamic forces and moment in low speed. In: Proceedings of 5th PAAMES and AMEC2012, Taiwan 36. Shih CH, Huang PH et al (2012) Design optimal control of ship maneuver patterns for collision avoidance: a review. J Mar Sci Technol 20(2):111–121 37. Sian A, Maimun A et al (2014) Assessment of ship-bank interactions on LNG tanker in shallow water. J Teknol 66(2):141–144 38. Zhang XK, Zhang GQ (2014) Concise robust control for ship’s turing manoeuvring in harbor. Navig China 37(2):31–34 (in Chinese) 39. Ferrari V, Sutulo S, Guedes Soares C (2016) Non-linear control for the automatic berthing of waterjet catamaran. In: Proceedings of 3rd international conference on maritime technology and engineering, MARTECH 2016, vol 1, pp 201–210 40. Kamolov A, Park SH (2019) An IoT based smart berthing (parking) system for vessels and ports (Lecture notes in electrical engineering), vol 513, pp 129–139 41. Maki A (2019) Application of optimal control theory based on the evolution strategy (CMA-ES) to automatic berthing. J Mar Sci Technol (Japan) (article in press) 42. Im NK, Hasegawa K et al (2001) A study on automatic ship berthing using parallel neural controller. J Kansai Soc Nav Arch Jpn 236:65–70 43. Im NK, Hasegawa K (2001) A study on automatic ship berthing using parallel neural controller (2nd report)-motion identification considering lateral speed and angular velocity to cope with disturbances. Soc Nav Arch Jpn 237(3):127–132 44. Koyama T, Yan J et al (1993) A systematic study on automatic berthing control (1st report) (in Japanese). J Soc Nav Arch Jpn 162:201–210 45. Kose K, Fukudo J, Sugano K (1986) On a computer aided maneuvering system in harbors. J Soc Nav Arch Jpn 160:103–110 46. Yamato H et al (1990) Automatic berthing by neural controller. In: Proceedings of the ninth ship control systems symposium, vol 3, Bethesda, USA, pp 183–201 47. Hasegawa K, Kitera K (1993) Automatic berthing control system using neural network and knowledge-base (in Japanese). J Kansai Soc Nav Arch 220:135–143 48. Im NK, Hasegawa K (2002) Motion identification using neural networks and its application to automatic ship berthing under wind. J Technol 6(1):16–26 49. Xu G, Hasegawa K (2012) Automatic berthing using artificial neural network on teaching data generated by optimal steering. Jpn Soc Nav Arch Ocean Eng 14(2):295–298 50. Ahmed YA, Hasegawa K (2014) Experiment results for automatic ship berthing using artificial neural network based controller. In: Preprints of the 19th world congress, The International Federation of Automatic Control, Cape Town, South Africa 51. Zhang Q, Zhang XK, Im NK (2017) Concise nonlinear neural network controller for ship auto-berthing. Navig China 40(04):11–15, 29 52. Nguyen MC (2018) Artificial neural network controller for automatic ship berthing using head-up coordinate system. Int J Nav Arch Ocean Eng 235–249 53. Nguyen VS, Do1 VC, Im NK (2018) Development of automatic ship berthing system using artificial neural network and distance measurement system. Int J Fuzzy Log Intell Syst 18(1):41–49

108

Z. Qiang et al.

54. Zou TY, Shen ZP, Dai CS (2018) Adaptive iterative sliding mode berthing control for underactuated ship based on chaotic particle swarm. In: Chinese control conference, CCC, July 2018, pp 2881–2886 55. Shuai YH, Li GY, Cheng X, Skulstad R, Xu JS, Liu HH, Zhang HX (2019) An efficient neural-network based approach to automatic ship docking. Ocean Eng 191:106514 56. Piao ZJ, Guo C, Sun S (2019) Research into the automatic berthing of underactuated unmanned ships under wind loads based on experiment and numerical analysis. J Mar Sci Eng 7(9):300. https://doi.org/10.3390/jmse7090300 57. Kim TH, Basar T et al (2002) Asymptotic stabilization of an underactuated surface vessel via logic-based control. In: Proceedings of the American control conference, Anchorage, pp 4678–4683 58. Nguyen, VS (2019) Research on a support system for automatic ship navigation in fairway. Futur Internet 11:2 59. Zhang Q, Zhu GB, Hu X, Yang RM (2019) Adaptive neural network auto-berthing control of marine ships. Ocean Eng 177:40–80 60. Djouani K, Hamam Y (1994) Ship optimal path planning and artificial neural nets for berthing. In: Oceans engineering for today’s technology and tomorrow’s preservation proceedings, vol 1, pp 785–790 61. Ohstu K, Yamai Y et al (1996) Minimum-time maneuvering of a ship with wind disturbances. Control Eng Pract 4(3):385–392 62. Nguyen VS, Im NK (2019) Automatic ship berthing based on fuzzy logic. Int J Fuzzy Log Intell Syst 19(3):163–171 63. Li Y, Jia ZH, Zhang WB, Liao YL (2019) Layered trajectory planning and experiment for the autoberthing of unmanned surface vehicles. J Harbin Eng Univ 40(06):1043–1050 64. Martin K, Oliver K et al (2013) Full-state manoeuvre planning system for marine vehicles. In: 9th IFAC conference on control applications in marine systems, Osaka, Japan 65. Takai T, Ohtsu K (1990) Automatic berthing experiments using “Shioji Maru”. Jpn Inst Navig 83(5):267–276 66. Kawai H, Kakamoto J et al (2010) Distance measurement system based on image sensors for automatic berthing of ships. In: International conference on control, automation and systems, Gyeonggi-do, Korea, pp 201–204 67. Ogura T, Mizuchi Y et al (2013) Distance measurement system using stereo camera for automatic ship control. In: 13th international conference on control, automation and systems, Gwangju, Korea, pp 1020–1024 68. Peng GJ, Liu Y, Zhang XG et al (2012) Design of auxiliary berthing instrument of large ship. J Traffic Transp Eng 12(6):48–54 (in Chinese) 69. Zhou XL, Zhang SG, Peng GJ (2014) Reliability analysis of ship-based auxiliary berthing system. J Jimei Univ (Nat Sci) 19(3):196–200 (in Chinese) 70. Lee SD, Tzeng CY et al (2010) Design and application of an image-processing-based fuzzy autopilot for small boat approaching maneuvers. J Mar Sci Technol 18(4):558–567 71. Lee SD, Tzeng CY, Shu KY (2012) Design and experiment of a small boat auto-berthing control system. In: 12th international conference on ITS telecommunications, Taipei, Taiwan, pp 397–401 72. Gao QS (2013) Introduction to application of RTK technology in ship berthing. China Water Transp 13(11):7–8 (in Chinese) 73. Ablyakimov IS, Shirokov IB (2017) Analyzis of influence of microwave propagation conditions above sea surface on system operation of automatic ship berthing. In: Conference proceedings— 2017 radiation and scattering of electromagnetic waves, RSEMW 2017, pp 222–225 74. Ablyakimov IS, Shirokov IB (2017) Operation of local positioning system for automatic ship berthing. In: Proceedings of 2017 IEEE east-west design and test symposium, EWDTS 2017, 14 Nov 2017 75. Yang KU, Hurb JG, Choic MS, Yeoa DJ, Byuna JH (2017) Study on ship automatic berthing system with mooring lines. China Ocean Eng 31(1):19–29

Review on the Research of Ship …

109

76. Mizuno N, Kita T, Ishikawa T (2018) A new solving method for non-linear optimal control problem and its application to automatic berthing problem. In: IECON 2018—44th annual conference of the IEEE industrial electronics society, pp 2183–2188 77. Zhang HJ (2014) China ship news 2014-10-22(002) 78. Xu GP, Zhang XK (2013) An overview of ship autopilot research. Shipbuild China 54(2):191– 200 (in Chinese)