Human-Automation Interaction: Transportation (Automation, Collaboration, & E-Services, 11) 3031107837, 9783031107832

Citation preview

Automation, Collaboration, & E-Services

Vincent G. Duffy Steven J. Landry John D. Lee Neville Stanton   Editors

Human-Automation Interaction Transportation

Automation, Collaboration, & E-Services Volume 11

Series Editor Shimon Y. Nof, PRISM Center, Grissom Hall, Purdue University, West Lafayette, IN, USA

The Automation, Collaboration, & E-Services series (ACES) publishes new developments and advances in the fields of Automation, collaboration and e-services; rapidly and informally but with a high quality. It captures the scientific and engineering theories and techniques addressing challenges of the megatrends of automation, and collaboration. These trends, defining the scope of the ACES Series, are evident with wireless communication, Internetworking, multi-agent systems, sensor networks, cyber-physical collaborative systems, interactive-collaborative devices, and social robotics – all enabled by collaborative e-Services. Within the scope of the series are monographs, lecture notes, selected contributions from specialized conferences and workshops.

Vincent G. Duffy · Steven J. Landry · John D. Lee · Neville Stanton Editors

Human-Automation Interaction Transportation

Editors Vincent G. Duffy School of Industrial Engineering Department of Agricultural and Biological Engineering Purdue University West Lafayette, IN, USA John D. Lee Department of Industrial and Systems Engineering University of Wisconsin-Madison Madison, WI, USA

Steven J. Landry Industrial and Manufacturing Engineering The Pennsylvania State University University Park, PA, USA Neville Stanton Human Factors in Transport within Engineering and Physical Sciences University of Southampton Southampton, UK

ISSN 2193-472X ISSN 2193-4738 (electronic) Automation, Collaboration, & E-Services ISBN 978-3-031-10783-2 ISBN 978-3-031-10784-9 (eBook) https://doi.org/10.1007/978-3-031-10784-9 © The Editor(s) (if applicable) and The Author(s), under exclusive license to Springer Nature Switzerland AG 2023 This work is subject to copyright. All rights are solely and exclusively licensed by the Publisher, whether the whole or part of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation, broadcasting, reproduction on microfilms or in any other physical way, and transmission or information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now known or hereafter developed. The use of general descriptive names, registered names, trademarks, service marks, etc. in this publication does not imply, even in the absence of a specific statement, that such names are exempt from the relevant protective laws and regulations and therefore free for general use. The publisher, the authors, and the editors are safe to assume that the advice and information in this book are believed to be true and accurate at the date of publication. Neither the publisher nor the authors or the editors give a warranty, expressed or implied, with respect to the material contained herein or for any errors or omissions that may have been made. The publisher remains neutral with regard to jurisdictional claims in published maps and institutional affiliations. This Springer imprint is published by the registered company Springer Nature Switzerland AG The registered company address is: Gewerbestrasse 11, 6330 Cham, Switzerland

ACES Series Editor Foreword

Our Springer ACES Series is delighted to welcome the unique three-book excellent collection of editors, chapter co-authors and contributors on Human-Automation Interaction. This collection includes: • Human-Automation Interaction: Mobile Computing • Human-Automation Interaction: Transportation • Human-Automation Interaction: Manufacturing, Services, and UX. When we consider collaboration today, during the age of cyber-collaborative world and society, we cannot limit it any longer to human–human collaboration, the foundation and future of any human civilization. At the same time, we cannot ignore the fact that automation, while invented and implemented by humans, is made solely for the sake of humans. Hence, we have an essential need to understand and explore the science, engineering and management of HAI, Human-Automation Interaction. After all, the purpose of interaction is collaboration. That is the theme defined by the committee for the Gavriel Salvendy International Symposium for Emerging Frontiers in Industrial Engineering. (The committee includes Robert Proctor, Chair; Vincent Duffy, Shimon Nof and Yeuhwern Yih.) While during the pandemic years it could not be held in person, it was possible to engage many colleagues worldwide, who are the participants in this three-book important, collaborative endeavor. Thanks again to all the participants and contributors, all of us who for many years have been inspired and learned from the leadership of Prof. Gavriel Salvendy. Thanks also to the Springer team, who supported the publication of these books. We would like to welcome and invite many readers of various academic backgrounds to enjoy these exciting articles as part of their exploration of HAI. West Lafayette, IN, USA June 2022

Shimon Y. Nof Editor, Springer ACES Series

v

Preface (HAI: Mobile Computing)

Human-Automation Interaction (HAI) has become present and design considerations are now important in so many aspects of our lives. The themes of the three books are Transportation, Mobile Computing and Manufacturing and Services and User Experience (UX). This initiative is intended as a look toward the future and a tribute to our esteemed colleague, Gavriel Salvendy, who contributed to research literature and the infrastructure development in engineering, human factors and ergonomics over the past six decades. We celebrate Prof. Salvendy’s birthday this year with a compilation of articles in three main themes of Human-Automation Interaction. He reviewed and expressed interest in very many of the articles contributed this year. Over the past 40 years, he has been an editor of handbooks and journals in areas of overlapping research interest with most of our contributing authors. Dr. Salvendy is a founding chair of Human–Computer Interaction International (HCII) and Applied Human Factors and Ergonomics International (AHFE) conferences. As co-editors, we invited and appreciated the opportunity to interact with the authors that contributed chapters within the HAI theme of their interest. We look forward to sharing these articles with a general audience that has an interest in human factors and ergonomics. We greatly appreciated the opportunity to celebrate international collaborations and contributors through this initiative. We are grateful to those who contributed to this special compilation of articles. Papers from these volumes were included for publication after a minimum of one single-blind review from among the co-editors within the thematic areas. I would again like to thank the co-editors for their contributions, cooperation, support and efforts throughout. Eighty-six contributing authors from 11 countries contributed 30 articles to the book. Authors and editors in this book are representing China, Germany, India, Japan, Korea, Malaysia, Nigeria, Portugal, Switzerland, the UK and the USA.

vii

viii

Preface (HAI: Mobile Computing)

The co-editors are Martina Ziefle, Patrick P. P. L. Rau and Mitchell M. Tseng. The main parts for the HAI Mobile Computing book are shown below: Section A: Health, Care and Assistive Service Section B: Usability, User Experience and Design Section C: Virtual Learning, Training and Collaboration Section D: Ergonomics in Work, Automation and Production Section E: Interaction with Data and User Modeling in Special Applications.

West Lafayette, IN, USA

On behalf of the co-editors Vincent G. Duffy

Contents

Interaction with Vehicle Automation The Shorter Takeover Request Time the Better? Car-Driver Handover Control in Highly Automated Vehicles . . . . . . . . . . . . . . . . . . . . . Hsiang-Chun Wang, Zhi Guo, and Pei-Luen Patrick Rau

3

Personalized Risk Calculations with a Generative Bayesian Model: Am I Fast Enough to React in Time? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Claus Moebus

19

Etiquette Equality or Inequality? Drivers’ Intention to be Polite to Automated Vehicles in Mixed Traffic . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Tingting Li and Peng Liu

57

Human Collaboration with Advanced Vehicle Technologies: Challenges for Older Adults . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Joseph Sharit, Dustin J. Souders, and Neil Charness

75

Design for Inclusion and Aged Population in Transportation and Human-Automation Interaction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Jimmy Onyedikachi Uba, Jessica Adanma Onwuzurike, Chidubem Nuela Enebechi, and Vincent G. Duffy

91

Utilizing Bibliometric Analysis Tools to Investigate Automation Surprises in Flight Automation Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111 Evan Barnell HCI in an Automated Vehicle Human-Computer Interaction in Mobility Systems . . . . . . . . . . . . . . . . . . . 131 Heidi Krömker, Cindy Mayas, and Tobias Wienken

ix

x

Contents

Cognitive Analysis of Multiscreen Passenger Vehicles . . . . . . . . . . . . . . . . . 147 Alex Krochman and Thorsten Kuebler Systematic Review on the Emergence of Kan-sei Engineering as a Human Factors Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 157 Daniel J. Tillinghast and Suhas G. Aekanth A Practitioner’s Guide to Evaluating Distraction Potential of In-Vehicle Voice Assistants . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 171 Kerstan Cole, Johanna Josten, Philipp Seewald, and Christian Roth Automating the Driving Task—How to Get More Human-Centered . . . . 195 Klaus Bengler, Burak Karakaya, and Elisabeth Shi A Systematic Literature Review of the Effect of Increased Automation on the Air Traffic Control Industry . . . . . . . . . . . . . . . . . . . . . . 207 Benjamin Mardiks Trust in Vehicle Automation Trust in Automated Vehicle: A Meta-Analysis . . . . . . . . . . . . . . . . . . . . . . . . 221 Zhengming Zhang, Renran Tian, and Vincent G. Duffy Crazy Little Thing Called Trust—User-Specific Attitudes and Conditions to Trust an On-Demand Autonomous Shuttle Service . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 235 Hannah Biermann, Ralf Philipsen, and Martina Ziefle From Trust to Trust Dynamics: Combining Empirical and Computational Approaches to Model and Predict Trust Dynamics In Human-Autonomy Interaction . . . . . . . . . . . . . . . . . . . . . . . . . . 253 X. Jessie Yang, Yaohui Guo, and Christoper Schemanske Calibration of Trust in Autonomous Vehicle . . . . . . . . . . . . . . . . . . . . . . . . . . 267 Seul Chan Lee and Yong Gu Ji Human-Automation Interaction for Semi-Autonomous Driving: Risk Communication and Trust . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 281 Jing Chen, Scott Mishler, Shelby Long, Sarah Yahoodik, Katherine Garcia, and Yusuke Yamani A Systematic Literature Review of Human Error and Machine Error in Accident Investigation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 293 Nathan B. Rowland Miller

Contents

xi

Safety, Maintenance and Physical Modeling of Vehicle Packaging and Assembly Systematic Literature Review of Safety Management Systems in Aviation Maintenance Operations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 311 Natalie Zimmermann and Vincent G. Duffy Multimodal Interactions Within Augmented Reality Operational Support Tools for Shipboard Maintenance . . . . . . . . . . . . . . . . . . . . . . . . . . . 329 Victoria L. Claypoole, Clay D. Killingsworth, Catherine A. Hodges, Jennifer M. Riley, and Kay M. Stanney Task Simulation Automation via Digital Human Models: A Case Study on Cockpit Fire and Smoke Emergencies . . . . . . . . . . . . . . . . . . . . . . 345 Mihir Sunil Gawand and H. Onan Demirel Facility Layout Design Optimization of Wing Assembly of Unmanned Aerial Vehicle Based on Particle Swarm Optimization . . . 363 Hai-Zhe Jin, Zi-Jian Cao, Xin-Yi Chi, and Xue-Xin Fan Forklift Operator Discomfort and Vision Assessment Through Computer-Aided Ergonomics Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 379 Suhas G. Aekanth, Thorsten Kuebler, and Vincent G. Duffy Promoting Safety and Injury Prevention of Electric Transportation . . . . 399 WooJune Jung Smart Cities and Connected Vehicles Designing for Me! What Older Dwellers’ Want to Improve Mobility in an Age-Friendly City . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 415 Pei-Lee Teh, Ver Nice Low, Deepa Alex, Qasim Ayub, and Shaun Wen Huey Lee A Literature Review of Technological Trends in Urban Logistics: Concepts and Challenges . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 433 Bruno Machado, Carina Pimentel, Amaro Sousa, Ana Luísa Ramos, José Vasconcelos Ferreira, and Leonor Teixeira A Cost-Effective and Quality-Ensured Framework for Crowdsourced Indoor Localization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 451 Lulu Gao and Shin’ichi Konomi Intelligent Connected Vehicle Information System (CVIS) for Safer and Pleasant Driving . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 469 Xin Zhou, Jingyue Zheng, and Wei Zhang

xii

Contents

Travel Behaviour and Mobility in Smart Cities: An Interdisciplinary Review of Mass Transit in a Smart City in Malaysia . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 481 Santha Vaithilingam, Pei-Lee Teh, Pervaiz K. Ahmed, Chee Pin Tan, and Sui-Jon Ho A Systematic Literature Review of Improvements to Transportation Safety Through Crowdsourced Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 503 Brent Homcha Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 517

Interaction with Vehicle Automation

The Shorter Takeover Request Time the Better? Car-Driver Handover Control in Highly Automated Vehicles Hsiang-Chun Wang, Zhi Guo, and Pei-Luen Patrick Rau

Abstract The future development of fully automated vehicle cannot be predicted yet. Car-driver handover control need be existed in a certain period, but it is not well understood. The present study is to gain deeper insight into car-driver handover control in the highly automated vehicle by investigating the effects of mode transition, the time of transition to takeover request (TTR-time) and take over request time (TOR-time). Two experiments were conducted via driving simulator. Experiment 1 invited twenty participants to analyze the influence of TOR-time when resuming control. Experiment 2 recruited forty participants to further explore the effects of mode transition, TTR-time and TOR-time with a 2 * 2 * 2 mixed design. The shorter TOR-time resulted in less takeover reaction time but worse quality. Six seconds was the balance point between less takeover reaction time and higher handover control quality. In addition, the shorter TTR-time aggravated the significant difference of takeover reaction time caused by TOR-time, but the longer TTR-time abridged the significant difference and improved the attitude towards TOR-time during transition from level 3 to level 2 of automation. If the TOR-time is needed to be reduced in the design for acute threats, no less than six seconds would be better. If the future autonomous vehicle involves the mode transition of automation levels, designer and engineer also should take the time of mode transition to takeover request into account. These findings provide practical implications for the takeover control design of highly automated vehicles. Keywords Autonomous driving · Levels of automation · Driver behavior · Human-automation interaction · Vehicle design

H.-C. Wang · Z. Guo · P.-L. P. Rau (B) Department of Industrial Engineering, Tsinghua University, Beijing, China e-mail: [email protected] Z. Guo e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Switzerland AG 2023 V. G. Duffy et al. (eds.), Human-Automation Interaction, Automation, Collaboration, & E-Services 11, https://doi.org/10.1007/978-3-031-10784-9_1

3

4

H.-C. Wang et al.

1 Introduction and Background Autonomous driving is widely assumed to be beneficial for road safety due to the operations of safety driving, like no speeding, no emotional driving, et cetera. But automated vehicles may also have negative effects on road safety [1–5]. For instance, crashes may happen when drivers have to resume the control while they are out of the loop and when the system fails to deal with the unpredictable behaviors of other road users, bad weather and complex urban traffic situations. The future development of fully automated vehicle cannot be predicted yet due to technology limitation. A mixture of vehicles with different automation levels will gradually change into a mixture of vehicles with higher automation levels to deal with these issues in the near future [4]. National Highway Traffic Safety Administration (NHTSA) in USA and internationally the Society of Automotive Engineers (SAE) have defined various levels of vehicle automation in order to differentiate the responsibilities between the driver and an automated driving system [6, 7]. The definitions indicate drivers also need to resume the driving control even in the highly automated vehicles. But the driver’s responsibilities during autonomous driving between partial automation and high automation is hard to understand and will be an unfamiliar aspect of autonomous driving for novice drivers faced such systems. For example, the studies on highly automated systems showed that the operator’s manual skills could be inhibited or even deteriorated in the presence of long periods of automation [8]. Given the problem, researcher and developer proposed adaptive automation in highly automated systems, like several complex automation modes (e.g. level 2 and 3 of automation) may be used in one vehicle [8, 9]. Therefore, it is a critical research topic to understand car-driver handover control. In the past, studies on handover control in highly automated vehicles have focused on the effects of trust, traffic density, takeover scenarios, the secondary task and take-over request time (TOR-time) on take-over performance [10–20]. But studies on TOR-time are very limited to establish a well-understood foundation about how long it will take to get the driver back into the loop. Damböck et al. [21] found drivers were capable of taking over control within a time budget of 4–8 s, depending on the complexity of the situation. Gold et al. [14] found driver had the less take over reaction time with the shorter TOR-time by the comparison of 5 s and 7 s in a highly automated driving scenario with a secondary task and acute threats. But Walch et al. [10] found there was no difference of takeover reaction time between 4 and 6 s in a highly automated driving scenario with fog and hazards. Merat et al. [11] stated that it took drivers around 10–15 s to take over the car and 35–40 s to stabilize the lateral control of the vehicle in a highly automated driving scenario without hazards. The previous works of TOR-time mainly focused on inattentive drivers in an autonomous driving with level 3 of automation defined by NHTSA, which did not consider the human-machine coordination with different automation levels. The transition among different automation levels would be better for driver to take over when confront the acute threats because the transition provides gradually change of monitoring levels. So what will happen for driver in the condition of mode transition

The Shorter Takeover Request Time …

5

Fig. 1 The definitions of TTR-time and TOR-time

when taking over the automated vehicle? In addition, these studies concerned about different TOR-times and different scenarios, resulting in lack of comparison among the findings. Most studies concerned about tactical level [10, 14], like gaze behavior before the resuming control. Only one study was about the resumption of control on the operational level like lateral and vertical control after drivers resumed the driving [11]. In fact, both tactical and operational levels play important roles during the transition of control. In order to well understand the effects of TOR-time during take-over control, these issues should be taken into account. Therefore, the present study focus on the operation level of driving task of attentive drivers within the situation of acute threats to (1) examine the specific influence of TOR-time in the same driving scenario via four levels occurred in previous studies [10, 14], and (2) investigate the effects of mode transition between level 2 and level 3 of automation, the time of transition to takeover request (TTR-time) and take over request time (TOR-time).

2 The Effects of Different Levels of TOR-Time 2.1 Objectives Literature review shows that 4–8 s are the time budgets for most takeover driving studies. The time budget for take over request increase 1 s, the driver’s takeover time will increase 0.33 s, and the takeover quality decrease [22]. But most comparative study of time budget apply more than 1 s as the interval. There is no study investigating which ranging from 4 to 8 s would be a balance point for TOR-time design with a 1 s interval. Therefore, the present experiment would examine the difference of time budget of 4, 5, 6, and 7 s in takeover driving.

2.2 Methods Participants. Twenty participants (11 males, 9 females) took part in this experiment. Their average age was 27.16 years (SD = 2.80). All of them had a driving license

6

H.-C. Wang et al.

for 4.76 years on average (SD = 2.36), and drove totally at least 5000 km before participating in this experiment. All participants had a normal vision and hearing. Experimental Design. The experiment used a single-factor (TOR-time: 4, 5, 6 and 7 s) within-subject design. The order of the TOR-time conditions was random. TOR-time is the time budget of takeover request, namely the time between the onset of takeover request and colliding or reaching the system limits if driver would not intervene. The dependent variables included takeover performance [10, 14], driving performance within one minute after takeover [11] involving operational level of driving, and subjective experience ratings. The takeover performance was assessed using take-over reaction time and the number of collision within 100 m after driver’s take over. Take-over time was defined as the time between the onset of take-over request and the moment when the participants starts to take over manual control of the automated car. Driving performance after take over was mainly assessed by lateral control indexes, namely the maximum steering angle, the maximum lateral acceleration and the standard deviation of steering angle which evaluates the driving stability, as the present study focus on the driver’s steering behavior after resuming the car control. Subjective experience ratings were assessed by the workload, satisfaction and attitudes towards TOR-time. The workload was measured by NASA-TLX [23]. The satisfaction was measured on a 7-point Likert scale by After-Scenario Questionnaire with three items, respectively the satisfaction with the ease, the time and the support information of finishing task [24]. The attitudes towards TOR-time were measure on a 7-point Likert scale by Semantic Differential Scale, including “urgent-non-urgent”, “able to react-unable to react”, and “safe-dangerous” for TOR-time, in which 1 represents the closest to the meaning of the left side and 7 represents the closest to the meaning of the right side [25]. Scenarios and Apparatus. The automated vehicle drive at a speed of 100 km/h on the right lane of a two-lane highway with a hard shoulder in a driving simulator. There are no other vehicles in the front of automated vehicle. On the left lane, other vehicles are driving at 90 km/h and driving distances were random (see Fig. 2a). Three acute threatening take-over scenarios occurred on the right lane were designed to explore the effect of time budget of takeover request (TOR-time) on driver’s takeover performance and feelings during hand-over control. They were respectively the scenario of blocked road by sudden road construction without cues, the scenario of the sudden braking of the leading vehicle, and the scenario of illegally parking of a stranded vehicle ahead between the right lane and the road shoulder without cues (see Fig. 2). For all scenarios, when the emergency occurs, there are no vehicles on the corresponding and front part of the left lane and other vehicles driving at 90 km/h behind the left part of the automated driving vehicle. It provide an opportunity for the driver to take over the control and change lane. The experiment was conducted in a six DOF (degree of freedom) driving simulator with a 270-degree field of view in Department of Automotive Engineering, Tsinghua University (see Fig. 3). Participants drove in the simulated car and were able to steer,

The Shorter Takeover Request Time …

7

a

b

c

d

Fig. 2 The basic driving scene (a) and the scenarios of blocked road (b), sudden braking of leading vehicle (c), and illegally parking of a leading vehicle (d)

brake, accelerate and observe the real-time information about the driving on the dashboard, such as speed, tachometer and so on. The simulator could also automatically control longitudinal and lateral motion via the automatic system using programming. The sound effects of the engine, passing vehicles, audio signals and road environment were provided via audio simulating system of the driving cabin. The vehicle data were recorded in detail by the simulator. The road environment of the driving scene was a two-lane highway with a hard shoulder, and the speed was limited at 120 km/h in accordance with the speed traffic regulation in China. Procedure. Participants drove freely 5 min to be familiar with the manipulation and presentation of TOR-time of driving simulator after the brief introduction of the experiment and the signature of informed consent. And then participants were required to finish three scenarios at each level of TOR-time. The three scenarios were displayed in a random order. All participants need to fill questionnaires of subjective ratings after each level of time budget. The study was approved by ethics committee. All participants were provided with informed consent and obtained monetary compensation for their participation.

Fig. 3 The six DOF driving simulator used in the experiment

8

H.-C. Wang et al.

2.3 Results For each dependent variable, the one-way repeated measures analysis of variance (ANOVA) was used to examine the effect of TOR-time. Multiple comparison test with bonferroni corrections were conducted to identify specific significant differences among means. Take-over performance. The ANOVA result revealed a significant effect of TORtime (F(1, 19) = 11.92, p < 0.001). According to the findings in the previous studies [10, 14], the comparisons between 4 and 5 s, 5 and 6 s, 6 and 7 s, 4 and 6 s, and 5 and 7 s were planned to be analyzed. Paired sample test showed there was a significant difference between 5 and 6 s (M TOR-5 s : 1.58 ± 0.051, M TOR-6 s : 1.87 ± 0.073, p < 0.01), between 4 and 6 s (M TOR-4 s : 1.46 ± 0.061, M TOR-6 s : 1.87 ± 0.073, p < 0.01), and between 5 and 7 s (M TOR-5 s : 1.58 ± 0.051, M TOR-7 s : 1.97 ± 0.121, p < 0.01); but the differences between 4 and 5 s (M TOR-4 s : 1.46 ± 0.061, M TOR-5 s : 1.58 ± 0.051, p = 0.706) and between 6 and 7 s (M TOR-6 s : 1.87 ± 0.073, M TOR-7 s : 1.97 ± 0.121, p = 1.000) were not significant, which was depicted in Fig. 4. In addition, the collisions of 4 s, 5 s, 6 s and 7 s were respectively 5, 2, 2, 1, but it was not examined because the expected values of over 20% cells were less than 5. Driving performance. The driving performance was evaluated by lateral control indexes, including the maximum steering angle, the maximum lateral acceleration and the standard deviation of steering angle. The ANOVA results indicated the effect of TOR-time was significant for both the maximum steering angle (F(1, 19) = 12.00, p < 0.001) and the standard deviation of steering angle (F(1, 19) = 13.54, p < 0.001), and the effect of TOR-time on the maximum lateral acceleration was marginally significant (F(1, 19) = 2.77, p = 0.078). The descriptive statistic showed that the shorter TOR-time accompanied by larger maximum steering angle, larger standard deviation of steering angle and larger maximum lateral acceleration. But the significant difference occurred only in the pair of 5 s and 7 s, and the pair of 4 s and 7 s for both maximum steering angle (M TOR-5 s = 28.18 ± 2.890 vs. M TOR-7 s = 17.24 ± 1.347, marginally significant p = 0.012 > 0.008; M TOR-4 s = 35.17 ± 3.576 vs. M TOR-7 s = 17.24 ± 1.347, p < 0.001) and standard deviation of steering 2.1 2

Take-over RT (s)

Fig. 4 The influence of TOR-time on take-over reaction time

1.9 1.8 1.7 1.6 1.5 1.4 1.3 4s

5s

6s

TOR-time

7s

The Shorter Takeover Request Time …

9

angle (M TOR-5 s = 14.17 ± 1.359 vs. M TOR-7 s = 8.59 ± 0.606, p < 0.008; M TOR-4 s = 17.22 ± 1.604 vs. M TOR-7 s = 8.59 ± 0.606, p < 0.001). Although TOR-time had a marginally significant effect on the maximum lateral acceleration, the differences of each two levels of TOR-time were not significant according to the paired sample test with bonferroni corrections (all p > 0.008). Workload and satisfaction. The ANOVA analysis showed the effect of TOR-time was significant for both subjective workload (F(1, 19) = 4.25, p < 0.05) and satisfaction related to the task (F(1, 19) = 5.23, p < 0.05). According to the descriptive statistic, there was a slight decrease of workload and an increase of satisfaction with the increase of TOR-time. Further analysis of multiple comparison indicated the difference between 4 and 7 s was marginally significant for workload (M TOR-4 s = 2.75 ± 0.123 vs. M TOR-7 s = 2.36 ± 0.108, p = 0.01 > 0.008) and significant for satisfaction (M TOR-4 s = 4.62 ± 0.214 vs. M TOR-7 s = 5.72 ± 0.213, p = 0.004 < 0.008). Attitude towards TOR-time. “urgent-non-urgent”, “able to react-unable to react”, and “safe-dangerous” were used to assess the attitude towards TOR-time. A significant effect of TOR-time was observed for “urgent-non-urgent” (F(1, 19) = 5.82, p < 0.01), “able to react-unable to react” (F(1, 19) = 4.04, p < 0.05), and “safedangerous” (F(1, 19) = 5.85, p < 0.01). The descriptive statistic indicated that participants reported TOR-time they perceived was less likely to be “urgent” and more likely to be “able to react” and “safe” with the increase of TOR-time. Further analysis of multiple comparison indicated the differences between 4 and 7 s and between 4 and 6 s were significant for “urgent” (M TOR-4 s = 2.65 ± 0.254 vs. M TOR-7 s = 4.05 ± 0.359, p < 0.008; M TOR-4 s = 2.65 ± 0.254 vs. M TOR-6 s = 3.70 ± 0.363, p < 0.008) and “safe” (M TOR-4 s = 4.24 ± 0.315 vs. M TOR-7 s = 2.65 ± 0.296, p < 0.008; M TOR-4 s = 4.24 ± 0.315 vs. M TOR-6 s = 2.71 ± 0.268, p < 0.008). The differences between 4 and 7 s and between 5 and 7 s were significant for “able to react” (M TOR-4 s = 3.18 ± 0.300 vs. M TOR-7 s = 2.18 ± 0.231, p < 0.008; M TOR-5 s = 2.82 ± 0.324 vs. M TOR-7 s = 2.18 ± 0.231, p < 0.008).

3 The Effects of Mode Transition, TTR-Time, and TOR-Time 3.1 Objectives As fully automated vehicles are not predictable in the future due to its technology restrictions, a mixture of vehicles with different levels of automation will be existed [4]. A human factors research on transitions in automated driving [26] shows that driving states are different between various levels of automation. Take level 2 and level 3 of automation as an example to illustrate the driving state. Level 2 of automation, partial automation driving (PAD), is that the automation system are responsible

10

H.-C. Wang et al.

for the longitudinal and lateral controls, driver’s control are both off, and the driver need to monitor permanently to be able to resume the control anytime needed. Level 3 of automation, conditional automation driving (CAD) or high automation driving (HAD), is that both longitudinal and lateral controls are done by automated system and drivers are out of control, but drivers are not monitoring permanently, ranging from monitoring permanently and not monitoring at all. But what drivers and system should do is not equal to what these driver and the system actually do. In real situation, drivers and automation system need to jointly conduct the driving task and adjust dynamically their weight according to the momentary situation. It is shared control [27]. It means both level 2 and 3 of automation would be appeared in one vehicle. The previous studies about taking-over control mainly focus on partial automation driving and high automation driving [22]. In shared control situation, it might be existed switch from level 2 to level 3 or from level 3 to level 2. Therefore, it is necessary to explore the effect of mode transition and the time between mode transition and take over request on driver’s resuming control when facing system failures. As for the time of transition to takeover request (TTR-time) and take over request time (TOR-time), most previous studies on hand-over control examine the difference of 5 and 7 s [22], and the first experiment in the present study show 6 s was a balance point, so we also choose the two time levels of 5 and 7 s for the TTR-time and the two time levels of 4 and 6 s for TOR-time.

3.2 Methods Participants. Forty subjects (22 males, 18 females) took part in the experiment. Their average age was 26.62 years (SD = 3.75). All of them had a driving license for 4.81 years on average (SD = 2.32), and drove more than 5000 miles per year. The participants had a normal vision and hearing, and were randomly divided into two groups. One group went through the transition of automation level from level 3 to level 2 (11 males and 9 females, average age = 26.91 ± 3.68, average driving experience = 4.66 ± 2.31 years), and the other group went through the transition from level 2 to level 3 (11 males and 9 females, average age = 26.33 ± 3.81, average driving experience = 4.95 ± 2.34 years). None of those participants attended the first experiment. Experimental Design. The experiment used a 2 (Mode transition: L3-L2, L2L3) * 2(Transition-takeover request time: 5, 7 s) * 2(Take-over request time: 4, 6 s) mixed within/between factorial design. Mode was a between-subject variable, and transition-takeover request time (TTR-time) and take-over request time (TOR-time) were within-subject variables. Four combinations of time were formed based on two levels of each of the two within-subject variable, and the corresponding four driving conditions were conducted in a counterbalanced order. Figure 1 demonstrates the definitions of TTR-time and TOR-time. Mode transition described the switch among different levels of automation at two levels: the switch from level 3 to level 2 and its

The Shorter Takeover Request Time …

11

reverse transition. According to the definition of automation levels by NHTSA and SAE [4, 28], level 2 in the experiment was defined as the situation in which participants were asked to keep their hands on the wheel and system could control lateral and longitudinal movements; level 3 in the experiment referred to as the situation that participants were required to just pay their attention to the road center ahead of the vehicle without putting their hands on the wheel and system automatically control the driving. When the transition from one automation level to another one occurs, the cue information about what the participant need to do in the current automation level would be presented visually and audibly in the front of drivers. The dependent variables included takeover performance [10, 14], driving performance within one minute after takeover [11], and subjective experience ratings. The takeover performance was assessed using take-over reaction time and the number of collision within 100 m after driver’s take over. Take-over time was defined as the time between the onset of take-over request and the moment when the participants starts to take over manual control of the automated car. Driving performance after take over was mainly assessed by lateral control indexes, namely the maximum steering angle, the maximum lateral acceleration and the standard deviation of steering angle which evaluates the driving stability. Subjective experience ratings were assessed by the workload, satisfaction and attitudes towards TTR-time and TOR-time. The workload was measured by NASATLX [23]. The satisfaction was measured on a 7-point Likert scale by After-Scenario Questionnaire with three items, respectively the satisfaction with the ease, the time and the support information of finishing task [24]. The attitudes towards TTR-time and TOR-time were measure on a 7-point Likert scale by Semantic Differential Scale, including “sufficient-insufficient”, “urgent-non-urgent”, and “able to react-unable to react” for TTR-time, and “urgent-non-urgent”, “able to react-unable to react”, and “safe-dangerous” for TOR-time, in which 1 represents the closest to the meaning of the left side and 7 represents the closest to the meaning of the right side [25]. Scenarios and Apparatus. The scenario and apparatus are the same as those in the first experiment. Procedure. Participants drove freely 5 min to be familiar with the manipulation and presentation of mode transition, TTR-time and TOR-time of driving simulator after the brief introduction of the experiment and the signature of informed consent. In the formal driving task, one half of the participants went through the transition from level 3 to level 2, and the other half went through the transition from level 2 to level 3. After a certain period of time (TTR-time), the system suffered from the acute threatening driving scenarios and reached its boundary, take over request (TOR) was prompted to remind driver of resuming manual control to avoid the potential accident. All participants had to drive 3 trials for each combination of two time within-subject variables and filled questionnaires of subjective ratings after each combination. The whole experiment took approximately an hour. The study was approved by ethics committee. All participants were provided with informed consent and obtained monetary compensation for their participation.

12

H.-C. Wang et al.

3.3 Results For each dependent variable, a 2 * 2 * 2 mixed-model repeated measures analysis of variance (ANOVA) was used to examine the main and interaction effects. In addition, multiple comparison test with bonferroni corrections was conducted to identify specific significant differences among means. Take-over performance. The control shift from automated system to the driver was executed as soon as participants firstly manipulated the steering wheel two degrees or hit ten percent braking pedal position after the takeover request was prompted [14]. Therefore, take-over reaction time was the time between the onset of take-over request and the moment of the participants’ first conscious input of control shift. The ANOVA results revealed a significant main effect of TOR-time (F(1, 38) = 48.32, p < 0.001), and a significant interaction effect of TTR-time and TOR-time on take-over reaction time (F(1, 38) = 4.83, p < 0.05), but there were no significant main effects of mode transition and TTR-time and any other significant interaction effects. The results apparently showed a decrease reaction time with a shorter TOR-time in general (M TOR-4 s : 1.46 ± 0.053, M TOR-6 s : 1.94 ± 0.099). The results indicated the shorter TTR-time aggravated the significant difference caused by TOR-time (M diff in 5 s = 0.61 ± 0.102, p < 0.001), but the longer TTR-time abridged the difference (M diff in 7 s = 0.36 ± 0.076, p < 0.001), see Fig. 5. Furthermore, a TOR-time of 4 s resulted in significantly more collisions (11.7%) within 100 m after driver’s takeover than TOR-time of 6 s (5.8%), χ 2 = 5.11, p < 0.05. But there were no significant differences of collision for mode transition (10.4% for L2-L3, 7.1% for L3-L2, χ 2 = 1.67, p > 0.05) and TTR-time (7.9% for 5 s, 9.6% for 7 s, χ 2 = 0.42, p > 0.05). The interaction effects of TTR-time and TOR-time or mode transition and TOR-time were not examined because the expected values of over 20% cells were less than 5. Driving performance. Participants were more likely to resume manual control to avoid accidents by changing lane. Therefore, the driving performance was evaluated by lateral control indexes, including the maximum steering angle, the maximum lateral acceleration and the standard deviation of steering angle. For the 2 * 2 * 2 2.6

TTR-time 5s 7s

2.4

Take-over RT (s)

Fig. 5 The influence of TTR-time on the effect of TOR-time on take-over reaction time

2.2 2 1.8 1.6 1.4 1.2 1

4 s

TOR-time

6s

The Shorter Takeover Request Time …

13

ANOVA analysis of each of the three dependent variables, only the main effect of TOR-time was significant (the maximum steering angle, F(1, 38) = 52.37, p < 0.001; the maximum lateral acceleration, F(1, 38) = 62.78, p < 0.001; the standard deviation of steering angle, F(1, 38) = 6.39, p < 0.05). A shorter TOR-time resulted in larger maximum steering angle (M TOR-4 s = 30.40 ± 1.678 vs. M TOR-6 s = 20.89 ± 1.159), larger standard deviation of steering angle (M TOR-4 s = 15.38 ± 0.803 vs. M TOR-6 s = 10.07 ± 0.534), and larger maximum lateral acceleration (M TOR-4 s = 5.59 ± 0.363 vs. M TOR-6 s = 4.31 ± 0.375), which indicates higher risk of lane change. Workload and satisfaction. As is known to all, highly automated driving was developed to reduce the workload of driver. Therefore, workload should be taken into account to evaluate the effects. The ANOVA analysis showed that only the main effect of TOR-time was significant (F(1, 38) = 8.78, p < 0.01). The results revealed participants with the TOR-time of 4 s (2.88 ± 0.111) had a higher workload during fully resuming control compared to those with 6 s (2.61 ± 0.104). In addition, only the significant main effect of TOR-time was also observed for satisfaction measured by items involved the ease, the time and support information of finishing task (F(1, 38) = 20.60, p < 0.001). The shorter TOR-time was, the less satisfaction participants had (M TOR-4 s = 5.02 ± 0.189 vs. M TOR-6 s = 5.64 ± 0.184). Attitude towards TTR-time. “sufficient-insufficient”, “urgent-non-urgent”, and “able to react-unable to react” were used to assess the attitude towards TTR-time. A significant main effect of TOR-time and a significant interaction effect of mode transition and TTR-time were observed for “urgent-non urgent” (F(1, 38) = 4.38, p < 0.05; F(1, 38) = 5.01, p < 0.05) and “able to react-unable to react” (F(1, 38) = 7.48, p < 0.01; F(1, 38) = 4.59, p < 0.05). Only a significant main effect to TOR-time was observed for “sufficient-insufficient” (F(1, 38) = 14.77, p < 0.001). Participants within a TOR-time of 4 s reported TTR-time they perceived was less sufficient (M TOR-4 s = 3.73 ± 0.263 vs. M TOR-6 s = 2.71 ± 0.239), more urgent (M TOR-4 s = 3.24 ± 0.227 vs. M TOR-6 s = 3.75 ± 0.208) and less likely to able to react (M TOR-4 s = 2.93 ± 0.196 vs. M TOR-6 s = 2.43 ± 0.181) compared to those with 6 s. Participants with TTR-7 s reported less urgent compared to TTR-5 s (M TTR-5 s = 3.40 ± 0.261 vs. M TTR-7 s = 3.90 ± 0.309, marginally significant p = 0.06) when they went through the mode transition from level 3 to level 2, but there was no significant effect of TTR-time on subjective attitudes towards TTR-time (M TTR-5 s = 3.50 ± 0.261 vs. M TTR-7 s = 3.18 ± 0.309, p = 0.22 > 0.05) when they went through the mode transition from level 2 to level 3. However, the longer TTR-time was, the higher the possibility of the reaction participants perceived was, which occurred only in the condition of mode transition from level 2 to level 3 (M TTR-5 s = 2.93 ± 0.230 vs. M TTR-7 s = 2.50 ± 0.276, p < 0.05). Attitude towards TOR-time. “urgent-non-urgent”, “able to react-unable to react”, and “safe-dangerous” were used to assess the attitude towards TOR-time. A significant main effect of TOR-time and a significant interaction effect of mode transition and TTR-time were observed for “urgent-non urgent” (F(1, 38) = 29.31, p < 0.001; F(1, 38) = 5.14, p < 0.05). Only a significant main effect to TOR-time was observed

14

H.-C. Wang et al.

for “able to react-unable to react” (F(1, 38) = 16.02, p < 0.001) and “safe-dangerous” (F(1, 38) = 29.15, p < 0.001). Participants with TOR-4 s reported TOR-time they perceived was more urgent (M TOR-4 s = 2.25 ± 0.189 vs. M TOR-6 s = 3.38 ± 0.211), less likely to able to react (M TOR-4 s = 3.36 ± 0.226 vs. M TOR-6 s = 2.61 ± 0.190) and more likely to be dangerous (M TOR-4 s = 4.41 ± 0.257 vs. M TOR-6 s = 3.19 ± 0.214) compared to those with 6 s. Participants with TTR-7 s reported TOR-time they perceived was less urgent compared to TTR-5 s (M TTR-5 s = 2.58 ± 0.253 vs. M TTR-7 s = 3.13 ± 0.294, p < 0.05) when they went through the mode transition from level 3 to level 2, but there was no significant effect of TTR-time on subjective attitudes towards TOR-time (M TTR-5 s = 3.00 ± 0.253 vs. M TTR-7 s = 2.90 ± 0.294, p = 0.63) when they went through the mode transition from level 2 to level 3.

4 General Discussion The goal of the study is to explore the effects of mode transition, TTR-time and TORtime on the takeover control of driver in autonomous driving. The results showed TOR-time was a critical factor influencing takeover time and quality and subjective feelings. The first experiment was further conducted to examine the specific influence of TOR-time via four levels based on previous study findings. Both experiments showed the trend: the shorter TOR-time resulted in less takeover reaction time, more collision and more unstable lateral control, and made drivers perceive more workload, less satisfaction and more urgent and less safe perceived TOR-time which was also felt to be less likely to be able to react. If the driver has not enough time to maneuver control or enough situation awareness, the driving behaviors generally belong to operational level involving immediate control of vehicle and reactionary driving based on events that have happened [29, 30]. When the driver has no experience of the situation, it is hard for he/she to deal with the event [30]. The results of the first experiment provided more specific findings. Starting off at 7 s of TOR-time, 5 s started to reduce significantly in the takeover reaction time and the stability of lateral control, but significant decrease of satisfaction and attitudes towards TOR-time and significant increase of workload began almost at 4 s. The finding indicated the effect of TOR-time was more sensitive to objective performance than to subjective feelings or attitudes. Overall, a preliminary recommendation for designer or engineer of autonomous vehicles is that no less than six seconds would be better when the TOR-time need to be reduced in design for acute threats. The results about the difference between 5 and 7 s of TOR-time is the same as the findings of Gold et al. [14]. But the results about the difference between 4 and 6 s of TOR-time is not the same as the findings of Walch et al. [10]. One possible explanation for this finding is that the fog scenario in the Walch’s study was easy to provide the anticipation for driver ahead compared to the sudden driving scenarios in the present study. The anticipation could undermine the effect of TOR-time on takeover control. Another important distinction between the present study and previous work is that the current study asked participants to monitor the environment but not to play a

The Shorter Takeover Request Time …

15

secondary driving task. The attention driver paid to the driving task in previous work was less than the present study, thus the difference of 4 and 6 s did not be revealed. Future research should be conducted in this aspect via more driving scenarios to figure out the general conclusion. When a highly automated vehicle involves mode transition among different levels of automation, the second experiment found TTR-time modulated the effects of TOR-time on takeover control. The shorter time of mode transition to takeover request (TTR-time) aggravated the significant difference of takeover reaction time caused by TOR-time, but the longer TTR-time abridged the significant difference. It confirmed the speculation according to SOA theories about dual-task interference [31, 32]. Therefore, whatever the task before takeover request is the secondary task or mode transition of automation levels, the time from the previous task to takeover request should be taken into account when designing for car-driver handover control interface, or the practice about car-driver take over should be considered when autonomous driving is promoted. In addition, mode transition modulated the effect of TTR-time on driver’s attitudes towards TOR-time and TTR-time. Participants perceived the TOR-time and TTR-time to be less urgent with the longer TTR time when they went through the mode transition from level 3 to level 2, but they perceived the possibility of the reaction about TTR-time to be higher with the longer TTR-time when they were in the mode transition from level 2 to level 3. It implies different level of monitoring would influence the perception of TOR-time and TTR-time. In order to provide best user experience, designer and engineer also should take the interaction of mode transition and the time of mode transition to takeover request into account in the future.

5 Conclusion To gain deeper insight into car-driver handover control in highly automated vehicle, the specific influences of take over request time were examined via four levels, and the effects of mode transition, the time of transition to takeover request and take over request time were investigated. Overall, both experiments confirmed that the shorter takeover request time resulted in less takeover reaction time but worse quality. If the take-over request time is needed to be reduced in the design for acute threats, a preliminary recommendation for designer or engineer is that no less than six seconds would be better. Also, if the future autonomous vehicle involves the mode transition of automation levels, designer and engineer also should take the time of mode transition to takeover request into account due to its effects on takeover reaction time and attitudes towards TOR-time when designing the interaction of car-driver handover control. Therefore, these findings provide practical implications for the takeover control design of vehicles with different levels of autonomy. Additional research is also needed before adopting these results, given the limited driving scenario and traffic situation in the study.

16

H.-C. Wang et al.

Acknowledgements This study was funded by the National Key Research and Development Plan 2016YFB1001200-2.

References 1. Gurney JK (2013) Sue my car not me: products liability and accidents involving autonomous vehicles 2. Goodall NJ (2016) Can you program ethics into a self-driving car? IEEE Spectr 53(6):28–58 3. Brown B, Laurier E (2017) The trouble with autopilots: assisted and autonomous driving on the social road. In: Proceedings of the 2017 CHI conference on human factors in computing systems. ACM, Denver, Colorado, USA, pp 416–429 4. Vlakveld WP (2016) Transition of control in highly automated vehicles: a literature review 5. Sivak M, Schoettle B (2015) Road safety with self-driving vehicles: general limitations and road sharing with conventional vehicles 6. SAE I (2014) Taxonomy and definitions for terms related to on-road motor vehicle automated driving systems. Standard J3016 7. Wood SP et al (2012) The potential regulatory challenges of increasingly autonomous motor vehicles. Santa Clara L Rev 52:1423 8. Parasuraman R, Rizzo M (2008) Neuroergonomics: the brain at work. Oxford University Press 9. Feldhütter A, Segler C, Bengler K (2017) Does shifting between conditionally and partially automated driving lead to a loss of mode awareness? In: International conference on applied human factors and ergonomics. Springer 10. Walch M et al (2015) Autonomous driving: investigating the feasibility of car-driver handover assistance. In: Proceedings of the 7th international conference on automotive user interfaces and interactive vehicular applications. ACM 11. Merat N et al (2014) Transition to manual: driver behaviour when resuming control from a highly automated vehicle. Transport Res F: Traffic Psychol Behav 27:274–282 12. Jamson AH et al (2013) Behavioural changes in drivers experiencing highly-automated vehicle control in varying traffic conditions. Transp Res Part C: Emerg Technol 30:116–125 13. Hester M, Lee K, Dyre BP (2017) “Driver take over”: a preliminary exploration of driver trust and performance in autonomous vehicles. In: Proceedings of the human factors and ergonomics society annual meeting. 2017. SAGE Publications, Los Angeles, CA 14. Gold C et al (2013) “Take over!” How long does it take to get the driver back into the loop? In: Proceedings of the human factors and ergonomics society annual meeting. SAGE Publications, Los Angeles, CA 15. Bahram M, Aeberhard M, Wollherr D (2015) Please take over! An analysis and strategy for a driver take over request during autonomous driving. In: Intelligent vehicles symposium (IV). IEEE 16. Koo J et al (2015) Why did my car just do that? Explaining semi-autonomous driving actions to improve driver understanding, trust, and performance. Int J Interactive Des Manuf (IJIDeM) 9(4):269–275 17. Kim HJ, Yang JH (2017) Takeover requests in simulated partially autonomous vehicles considering human factors. IEEE Trans Human-Mach Syst 47(5):735–740 18. Fagnant DJ, Kockelman K (2015) Preparing a nation for autonomous vehicles: opportunities, barriers and policy recommendations. Transp Res Part A: Policy Pract 77:167–181 19. Alessandro C, Claudio AP, Sergio D (1998) Automatic vehicle control in emergency situations: technical and human factor aspects. In: Ollero A (ed) Intelligent components for vehicles, pp 153–159 20. Gold C et al (2016) Taking over control from highly automated vehicles in complex traffic situations: the role of traffic density. Hum Factors 58(4):642–652

The Shorter Takeover Request Time …

17

21. Damböck D et al (2012) Übernahmezeiten beim hochautomatisierten Fahren. Tagung Fahrerassistenz. München 15:16 22. McDonald AD et al (2019) Toward computational simulations of behavior during automated driving takeovers: a review of the empirical and modeling literatures. Hum Factors 61(4):642– 688 23. Hart SG, Staveland LE (1988) Development of NASA-TLX (Task Load Index): results of empirical and theoretical research. Advances in psychology. Elsevier, pp 139–183 24. Lewis JR (1991) Psychometric evaluation of an after-scenario questionnaire for computer usability studies: the ASQ. ACM Sigchi Bulletin 23(1):78–81 25. Osgood CE, Suci GJ, Tannenbaum PH (1957) The measurement of meaning 26. Lu Z et al (2016) Human factors of transitions in automated driving: a general framework and literature survey. Transp Res Part F-Traffic Psychol Behav 43:183–198 27. Abbink DA, Mulder M, Boer ER (2012) Haptic shared control: smoothly shifting control authority? Cogn Technol Work 14(1):19–28 28. Banks VA, Stanton NA (2016) Keep the driver in control: automating automobiles of the future. Appl Ergon 53:389–395 29. Lindstrom-Forneri W et al (2010) Driving as an everyday competence: a model of driving competence and behavior. Clin Gerontol 33(4):283–297 30. Stahl P, Donmez B, Jamieson GA (2013) Anticipatory driving competence: motivation, definition & modeling. In: Proceedings of the 5th international conference on automotive user interfaces and interactive vehicular applications. ACM 31. Pashler H (1994) Dual-task interference in simple tasks: data and theory. Psychol Bull 116(2):220 32. Ruthruff E et al (2003) Vanishing dual-task interference after practice: has the bottleneck been eliminated or is it merely latent? J Exp Psychol Hum Percept Perform 29(2):280

Personalized Risk Calculations with a Generative Bayesian Model: Am I Fast Enough to React in Time? Claus Moebus

Abstract We present a Bayesian modeling and decision procedure to answer the question of whether the reaction speed of a single individual is slower and thus more risky than the speed of a randomly selected individual in a reference population. The behavioral domain under investigation is simple reaction times (SRTs). To do this, we need to consider aspects of Bayesian cognitive modeling, psychometric measurement, person-centered risk calculation, and coding with the experimental, Turing-complete, functional, probabilistic programming language WebPPL. We pursue several goals: First, we lean on the new and paradoxical metaphor of a cautious gunslinger. We think that a whole range of risky situations can be embedded into this metaphor. Second, the above described gunslinger metaphor can be mapped to the framework of Bayesian decision strategies. We want to show by way of example that within this framework the research question ‘transfer the locus of longitudinal control’ in Partial Autonomous Driver Assistant Systems (PADAS) can be tackled. Third, evidence-based priors for our generative Bayesian models are obtained by reuse of meta-analytical results. For demonstration purposes we reuse reaction-time interval estimates of Card, Moran, and Newell’s (CMN’s) meta-analysis, the Model Human Processor (MHP). Fourth, the modification of priors to posterior probability distributions is weighted by a likelihood function, which is used to consider the SRT data from a single subject as evidence and to measure how plausibly alternative prior hypotheses generate these data. Fifth, we want to demonstrate the expressiveness and usefulness of WebPPL in computing posterior distributions and personal probabilities of risk. Keywords Personal Bayesian risk calculation · Context-dependent risk potential of an individual subject · Single-case diagnostics · Cognitive engineering model · Reuse of meta-analyses as Bayesian priors · Generative Bayesian model · Model human processor · Single subject response time · Probabilistic programming language WebPPL · Bayesian decision strategy · Transfer the locus of longitudinal control · Partial autonomous driver assistant system · PADAS C. Moebus (B) Learning and Cognitive Systems, Department of Computing Science, C.v.O University, Oldenburg, Germany e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Switzerland AG 2023 V. G. Duffy et al. (eds.), Human-Automation Interaction, Automation, Collaboration, & E-Services 11, https://doi.org/10.1007/978-3-031-10784-9_2

19

20

C. Moebus

1 Introduction 1.1 Motivation This is a study in the development of a Bayesian cognitive engineering model and Bayesian psychometric decision procedure. It is accompanied by the reuse and integration of psychological meta-analysis data. All computations are supported by code written in the Turing-complete functional probabilistic programming language WebPPL. We feel being in the tradition of Westmeyer [34], Bessiere et al. [2], Pearl [26], Lee and Wagenmakers [15], Goodman et al. [10], and Levy and Mislevy [17]. We pursue several goals: First, we lean on the new and paradoxical metaphor of a cautious gunslinger. We think that a whole range of risky situations (e.g. [16]) can be embedded into this metaphor. The agent has to answer himself three increasing complex counterfactual and metaphoric questions: (1) Can I draw my revolver fast enough, if my opponent needs only τc milliseconds to do so?, (2) Can I draw my revolver as fast as a randomly selected person of a (younger) reference population, if my opponent needs only τc milliseconds to do so?, (3) Is the probability that I can draw my revolver as fast as a randomly selected person of a (younger) reference population less than p = 0.05, if my opponent needs only τc milliseconds to do so?. Second, the above described gunslinger metaphor can be mapped to the framework of Bayesian decision strategies. We want to show by way of example that within this framework the research question ’transfer the locus of longitudinal control’ in Partial Autonomous Driver Assistant Systems (PADAS) can be tackled. Third, evidence-based priors for our generative Bayesian models are obtained by reuse of meta-analytical results. For demonstration purposes we reuse reaction-time interval estimates of Card, Moran, and Newell’s (CMN’s) meta-analysis, the Model Human Processor (MHP). According to the MHP total simple reaction times (SRTs) of an arbitrary computer user are composed from three latent time components related to perception, cognition, and motor processes. Fourth, the modification of priors to posterior probability distributions is weighted by a likelihood function, which is used to consider the SRT data from a single subject as evidence and to measure how plausibly alternative prior hypotheses generate these data. Posteriors are obtained by runs of the Metropolis-Hastings MarkovChain-Monte-Carlo (MH-MCMC) algorithm provided in Turing-complete, functional WebPPL. Fifth, we want to demonstrate the expressiveness and usefulness of the experimental WebPPL in computing posterior distributions and personal probabilities of risk. When SRT-specific values-at-risk ([7, 18], p. 114ff; [29], p. 178) are externally provided prior risk probabilities can be compared to posterior risk probabilities. It can be checked whether there is a substantial or even striking increase, which we call risk-excess. This way it is possible to answer the above mentioned questions. So, hazardous scenarios (e.g. traffic scenarios) with only a few behavioral data of a

Personalized Risk Calculations with a Generative …

21

single subject (e.g. a driver) can be mapped to the paradoxical and counterfactual cautious gunslinger scenario and to Bayesian psychometric decision procedures.

1.2 Generation of Simple Reaction Times (SRTs) Under MHP Guidance Card, Moran, and Newell’s Model Human Processor (MHP) In their seminal book The Psychology of Human-Computer Interaction Card et al. [4, 5] present the solution of several Human-Computer Interaction (HCI) design problems. The proposed solutions are based on some basic and abstract human information-processing mechanisms and are summarized in the Model Human Processor (MHP). MHP is a (simplified) engineering model and a static calculation guide of the human perceptual-cognitive-motor system. “It can be divided into three interacting subsystems: (1) the perceptual system, (2) the motor system, and (3) the cognitive system, each with its own memories and processors” ([5], p. 24). This way, MHP can be used as a simple static calculation guideline, e.g. to predict human response times. According to CMN MHP-guided design activities belong to applied information-processing psychology supporting engineering activities like task analysis, calculation, and approximation ([5], p. 9f; [4]). A more recent metaanalysis can be found in [14]. In MHP the meta-analytic knowledge is quantified by time intervals with interval boundaries and ‘typical’ values. “We can define three versions of the model: one in which all the parameters listed are set to give the worst performance (Slowman), one in which they are set to give the best performance (Fastman), and one set for a nominal performance (Middleman).” ([5], p. 44) Interval data from even more recent meta-analyses can easily be integrated into our Bayesian SRT model by substituting the CMN intervals by more recent ones. As an example we refer to Gratzer and Becke ([12], p. 126). They present interval data similar to CMN [5] but for reaction phases in braking events. They report intervals for basic reaction time (‘Reaktionsgrundzeit’), conversion time (‘Umsetzzeit’), and response time (‘Ansprechzeit’) which add up to total reaction time. MHP-Composition of Processor Cycle Times The uncertainties in the parameters of the MHP are captured by three subversions of the MHP ([5], p. 44): • Middleman is the version ... in which all the parameters ... are set to give the normal performance. • Fastman is the version ... in which all the parameters ... are set to give the best performance. • Slowman is the version ... in which all the parameters ... are set to give the worst performance.

22

C. Moebus

Cycle times of hypothetical perceptual (P), cognitive (C), and motor (M) processor are reported according to the interval-template τ X := τ X Middleman [τ X Fastman ∼ τ X Slowman ] ; X = P, C, M

(1)

and similarly for the total reaction time T τT := τTMiddleman [τTFastman ∼ τTSlowman ]

(2)

According CMN τT should be the sum of the specific component cycle times τ X Z man ; X = P, C, M ; Z = Middle, Fast, Slow : Σ τTZ man = τ X Z man (3) X ∈{P,C,M}

The meaning of (1) is that τ X is ranging from τ X Fastman to τ X Slowman with a ’typical’ value τ X Middleman and (2) has a similar meaning for the total reaction time τT . CMN not only provide quantitative intervals for (1) but also for (2) which obey the constraints (3). But a given left side of (2) can be fulfilled by many more 3-tuples (τ P , τC , τ M ) not considered on the right-hand side of (3). This is why we introduce for modelling purposes a new less constrained variable τΣ replacing τT . Σ τΣ := τ P + τC + τ M = τX . (4) X ∈{P,C,M}

The semantics of a ‘typical value’ is not formally specified by CMN ([5], p. 44f). We attempt various formal interpretations of the ambiguous term ‘typical value’ through statistics such as mode, median, and mean. These interpretations lead to different weakly informed prior belief distributions. MHP-τ X -intervals Cycle-times τ X and related τ X ; X ∈ {P, C, M}-intervals are displayed in Table 1 (CMN, Fig. 2.1, p. 26). Generation of SRTs in MHP’s Example 9 One of the standard problems in The Psychology of Human-Computer Interaction [5] to motivate the use of MHP is example

Table 1 MHP-cycle-times of C,P,M-processors Interval τX τP τC τM τT

100 [50–200] ms 70 [25–170] ms 70 [30–100] ms 240 [105–470] ms

Reference Perceptual processor ([5], p. 32f) Cognitive processor ([5], p. 42f) Motor processor ([5], p. 34f) Total SRT ([5], Fig. 2.1, p. 26, p. 66, p. 433f)

Personalized Risk Calculations with a Generative …

23

9: A user sits before a computer display terminal. Whenever any symbol appears, he is to press the space bar. What is the time between signal and response? ([5], p. 66). Solution. “Let us follow the course of processing through the Model Human Processor in Figure ... The user is in some state of attention to the display ...When some physical depiction of the letter A (we denote it α) appears, it is processed by the Perceptual Processor, giving rise to a physically-coded representation of the symbol (we write it α' ) in the Visual Image Store and very shortly thereafter to a visually coded symbol (we write it α'' ) in Working Memory... This process requires one Perceptual Process cycle τ P . The occurrence of the stimulus is connected with a response..., requiring one Cognitive Processor cycle, τC . The motor system then carries out the actual physical movement to push the key..., requiring one Motor Processor cycle, τ M . Total time required is τ P + τC + τ M . Using Middleman values, the total time required is 100 + 70 + 70 = 240 ms. Using Fastman and Slowman values gives a range 105–470 ms.” (CMN, Fig. 2.1, p. 26, p. 66, p. 433f) These are the cycle times in ms of the hypothetical perceptual processor, the cognitive processor, and the motor processor, respectively.

2 Priors Cycle times of hypothetical perceptual (P), cognitive (C), and motor processor (M) are reported according to the definition template (1). The meaning of (1) is that τ X is ranging from τ X Fastman to τ X Slowman with a ‘typical’ value τ X Middleman . The formal semantics of a ‘typical’ value are left unspecified by CMN ([5], p. 44f). We try various interpretations like mode (Pt 2.1, Pt 3.1), median (Pt 2.2, Pt 3.2) or mean (Pt 2.3, Pt 3.3) in our generative model’s prior belief pdfs. We expect that the corresponding prior triangular pdfs are more left-skewed in that same order because there is a well-known heuristic called ‘mode-median-meaninequality’ mode < median < mean ([1], p. 1372). So we expect that the prior with interpretation ‘typical value’ is mean is more left-skewed than the others. If the interpretation of ‘typical value’ is the mode of the prior pdf, we get along with a simple triangular prior (Pt 2.1, 3.1) T rianglemode (a = τ Fastman , b = τ Slowman , c = τ Middleman ).

(5)

If the interpretation is the median of the prior pdf, then we can stick to the triangular prior but with a slightly complicated parameter c c = modeT riangle = h median T riangle (τ X Fastman , τ X Slowman , τ X Middleman ) The function h median T riangle is a mapping from the median of a triangular pdf to its mode. In this case the prior pdf is (Pt 2.2, 3.2)

24

C. Moebus

T rianglemedian (τ Fastman , τ Slowman , h median T riangle (a, b, median T riangle )).

(6)

If the interpretation of ‘typical value’ is the mean of the prior, we cannot use a symmetric pdf like the Gaussian, because the MHP-intervals asymmetric around the ‘typical value’. For this interpretation ‘typical value’ is mean we can either reuse the triangular distribution similar to the median interpretation (Pt 2.3, 3.3) or use the Gamma distribution [19] as prior belief pdfs ([22], Pt 2.4, 3.4). In the case of reusing the triangular distribution the pdf is T rianglemean (τ Fastman , τ Slowman , h mean T riangle (a, b, mean T riangle )).

(7)

where the function h mean T riangle (a, b, mean T riangle ) is a mapping from the mean of a triangular pdf to its mode.

2.1 Triangular Priors for Interval Modes Extracted From CMN’s MHP Because the MHP-intervals provide only scarce information in form of the ’typical’ value, the lower, and upper bounds we chose as weakly informative priors the triangular (Pt 2.1–2.3) and the Γ -distribution ([22], Pt 2.4). When using triangular priors the ’inspired guesses’ are the mode, the median, and the mean, and again the mean in the case of Γ priors. Here in Pt 2.1 the “inspired guess” is the mode of the triangular prior pdf. In other words we force the MHP-value τ X Middleman to be the mode of the prior pdf. The PDF is ⎧ 0 f or x < a, ⎪ ⎪ ⎪ 2(x−a) ⎪ ⎪ ⎨ (b−a)(c−a) f or a ≤ x < c, 2 f or x = c, f T riangle (x|a, b, c) := (8) (b−a) ⎪ 2(b−x) ⎪ ⎪ (b−a)(b−c) f or c < x ≤ b, ⎪ ⎪ ⎩ 0 f or b < x. The generation of triangular-distributed random variables is done by the following mapping (https://en.wikipedia.org/wiki/Triangular_distribution). Given a random variate U drawn from the uniform distribution in the interval (0, 1), then the variate X T riangle |a, b, c is defined as { X T riangle |a, b, c :=

√ a√ + U (b − a)(c − a) f or 0 < U < F(c) b − (1 − U )(b − a)(b − c) f or F(c) ≤ U < 1

where F(c|a, b) := (c − a)/(b − a).

(9)

Personalized Risk Calculations with a Generative …

25

Fig. 1 T rianglemode prior of τ P MHP-Intval: 100.0 [50.0–200.0] τ P ∼ T rianglemode (50, 200, 100) 3σ-interval: 116.7 [22.6–210.8]

Fig. 2 T rianglemode prior of τC MHP-Intval: 70.0 [25.0–170.0] τC ∼ T rianglemode (25, 170, 70) 3σ-interval: 88.3 [−2.5 to 179.0]

The prior pdfs are displayed in Figs. 1, 2, 3, 4 and 5. We see that in Figs. 1, 2 and 3 the MHP-values τ X Middleman were defined to be the mode of the corresponding triangular priors. The τΣ -distribution in Fig. 4 is the convolution of the priors τ P , τC , τ M in Figs. 1, 2 and 3. This was obtained by summing the samples of the latent components (4). Figure 5 displays the prior pdf στΣ ∼ Γ (k = 4, θ = 20) for the standard deviation στΣ of the Gaussian likelihood function (10) for i.i.d. SRT-data L N (τΣ , στΣ | S RT s) :=

m 

N (S RTi | τΣ , στΣ ) ; m = #S RT s.

(10)

i=1

We chose Γ (4, 20) because we thought that a mean of μ = 80 and a σ = 40 for the Γ -pdf of the στΣ would provide sufficient unexplained variation in the Gaussian likelihood independent of the true variation due to τΣ . The independence assumption

26 Fig. 3 T rianglemode prior of τ M MHP-Interval: 70.0 [30.0–100.0] τ M ∼ T rianglemode (30, 100, 70) 3σ-interval: 66.7 [23.7–109.8]

Fig. 4 T rianglemode prior of τΣ MHP-Intval: 240.0 [105.0–470.0] 3σ-interval: 271.7 [134.0–409.4]

Fig. 5 Prior στΣ prior ∼ Γ (k := 4, θ := 20) in Gaussian likelihood (10)

C. Moebus

Personalized Risk Calculations with a Generative …

27

by introducing the product in (10) could be criticized because all SRT-data (Fig. 16) stem from a single subject. But for the moment we stick to this assumption. Three parametrizations are known for the Γ distribution. We chose that one with shape k and scale θ parameter (with k > 0, θ > 0). Under this selection expectation, variance, and standard deviation are functions of these hyperparameters k = 4 and θ = 20: E(στΣ |k, θ) = μ(στΣ |k, θ) = kθ = 4 · 20 = 80, V ar (στΣ |k, θ) = σ 2 (στΣ |k, θ) = kθ2 = 4 · θ2 = 4 · 202 = 1600 and

√

V ar (στΣ |k, θ) = σ(στΣ |k, θ) =

√ √ kθ = 4 · 20 = 40.

2.2 Triangular Priors for Interval Medians Extracted From CMN’s MHP As a second alternative for a weakly informative prior we chose the triangular distribution with the median as an “inspired guess” for MHP’s ‘typical value’. In other words we force the MHP-value τ X Middleman to be the median of the prior pdf. The median of a triangular distribution is defined as ⎧ √ ⎨ a + (b−a)(c−a) , f or c ≥ a+b 2 2 √ (11) median T riangle = ⎩ b − (b−a)(b−c) , f or c < a+b 2

2

If we interpret the ‘typical’ value τ X Middleman as the median mdT riangle of a T riangle(a, b, c) pdf with bounds a = τ X Fastman and b = τ X Slowman then we need to determine the unknown third parameter c = modetriangle in T riangle(a, b, c) from the information given by mdT riangle , τ X Fastman , and τ X Slowman . In short, we apply the function h median T riangle to instantiated median T riangle , τ X Fastman , and τ X Slowman statistics. The result of the function application provides the desired modeT riangle = c. c := modeT riangle = h median T riangle (τ X Fastman , τ X Slowman , τ X Middleman )

(12)

c := modeT riangle = h median T riangle (a, b, median T riangle )

(13)

which is

First, we derive c when c ≥

and md ≡ median : √ (b − a)(c − a) (md − a) = 2 a+b 2

28

C. Moebus

(md − a)2 =

(b − a)(c − a) 2

2(md − a)2 = (b − a)(c − a) 2(md − a)2 = (c − a) (b − a) c := h median T riangle (a, b, mdT riangle ) = Second, we derive c when c τΣc ) = αc prior (τΣc ).

(25)

After testing the subject we have a set of personal SRTs and we can compute the personal risk probability by marginalizing and filtering the posterior f posterior (τ P , τC , τ M , τΣ , στΣ |S RT s) P(τΣ > τΣc |S RT s) = αc posterior (τΣc )

(26)

Now with (26) it is possible to answer question 1 of our gunslinger scenario. The only problem left is to define the vague concept ‘fast enough’ by a tolerable probability αc posterior (τΣc ). αc posterior (τΣc ) ∫ ∫ ∫ = ... τΣ >τΣc τ P

f posterior (τ P , τC , τ M , τΣ , στΣ |S RT s) dστΣ ...dτ P dτΣ .

σ τΣ

In WebPPL (26) will be determined numerically by Monte-Carlo methods. First we marginalize the posterior pdf with the Infer-function. Then the marginal posterior

Personalized Risk Calculations with a Generative …

47

will be filtered to obtain the particles satisfying τΣ > τΣc |S RT s. In the last step the probability αc posterior (τΣc ) can be estimated by the ratio of the length of this filtered array to the length of the unfiltered marginal posterior array.

4.3 Answer to Question 2 Next the psychometric personal risk-excess calculation is done by comparing the posterior personal risk probability αc posterior with that from the prior risk αc prior P(τΣ > τΣc |S RT s) − P(τΣ > τΣc ) = αc posterior (τΣc ) − αc prior (τΣc ). αcdi f f (τΣc ) = αc posterior (τΣc ) − αc prior (τΣc )

(27) (28)

Risk-excess (27) and (28) is a monotonic increasing function of the standard deviation of priors and the τΣc −thresholds. The posterior 3στΣ -intervals from pt 3.1–3.3 and the Γ −priors left-out from this paper are displayed in Table 2. The widest posterior interval is obtained from T rianglemean -priors of the τΣ components. This prior was most strongly influenced by the subjects’ data. Thus the use of this posterior for risk excess calculations is most unfavorable for the person under study, when the subject is suspected to be slower than a randomly chosen person from the reference population. Answering question 2 we compute (28) for the range {τΣc |τ Middleman ≤ τΣc ≤ τ Slowman } αcdi f f (τΣc ) ; τ Middleman ≤ τΣc ≤ τ Slowman . (29) Results of (29) are displayed in Figs. 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43 and 44. The most important results can be seen for τΣc in Figs. 36, 40, and 44. The maximal risk-excess is nearby the threshold τΣc = 290 ms. This is the most unfavorable threshold for the subject. Depending on the prior the person-specific risk-excess ranges from p = 0.25 to p = 0.65. Risk-excess reduces with growing threshold τΣc more and more till it reduces surprisingly to zero. This happens for thresholds τΣc > 340 ms.

Table 2 Posterior 3στΣ -intervals τ P,C,M -priors

Posterior στΣ

Posterior 3στΣ -interval

Interval range (ms)

T rianglemode T rianglemedian T rianglemean Γ

24.2 25.1 25.6 25.5

310.0 [237.5–382.5] 307.2 [232.1–382.8] 306.6 [229.7–383.4] 294.9 [218.5–371.3]

145.0 150.7 153.7 152.8

48

C. Moebus

Fig. 33 Risk excess probabilities αcdi f f (T rianglemode ) for thresholds τ Pc ∈ [100−200) ms

Now we have provided a formal answer to question 2 of the counterfactual and metaphorical gunslinger scenario. This is even true for varying thresholds (τΣc ).

4.4 Answer to Question 3 Only if the risk-excess (27) and (28) is substantial greater than e.g. p = .05 then the subject’s SRT should be considered as more risky than that of a randomly chosen subject from the reference population. We think that p = 0.05 can be accepted by convention. To answer question 3 we formalize it as τ X crit =

min

τΣc ∈{τMiddleman ≤τΣc ≤τSlowman }

αcdi f f (τΣc ) ≤ 0.05

(30)

The critical thresholds τ X crit ms ; X ∈ {P, C, M, Σ} are collected in Table 3. They partition single subject’s risk-excessive from non-risk-excessive SRT-regions: The entries of Table 3 can be identified quite easily in Figs. 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43 and 44. They have the following meaning. If the opponent’s SRT τ X c in process measure X ∈ {P, C, M, Σ} is τ X c > τ X crit then the probability that the subject is slower than a randomly selected subject from the reference population is below p=0.05. In other words we can be quite certain that the single subject’s SRT is no more risky than that of any subject from the reference population when the opponent’s SRT τ X c is greater than τ X crit . Following the entries from Table 3 we can answer question 3 when we know the opponent’s SRT-value τc . Furthermore we can see that the choice of prior is *not* important for certain risk-avoiding decisions. Let’s concentrate on the T rianglemode prior. E.g. is the SRT-value-at-risk τΣ < 336.6 ms the increase in single subject’s prior-to-posterior risk probability [risk-excess (27)] is greater than 0.05 (Fig. 36). Otherway round we can say that if the SRT-value-at-risk τΣ ≥ 336.6 ms the increase

Personalized Risk Calculations with a Generative … Fig. 34 Risk excess probabilities αcdi f f (T rianglemode ) for thresholds τCc ∈ [70−170) ms

Fig. 35 Risk excess probabilities αcdi f f (T rianglemode ) for thresholds τ Mc ∈ [70−100) ms

Fig. 36 Risk excess probabilities αcdi f f (T rianglemode ) for thresholds τΣc ∈ [240−470) ms

49

50

C. Moebus

Fig. 37 Risk excess probabilities αcdi f f (T rianglemedian ) for thresholds τ Pc ∈ [100−200) ms

Fig. 38 Risk excess probabilities αcdi f f (T rianglemedian ) for thresholds τCc ∈ [70−170) ms

in single subject’s prior-to-posterior risk probability [risk-excess (27)] is smaller than 0.05. This means that for all SRT-values-at-risk greater than 336.6 ms the reaction times of our single subject in the age of 72 years do not include a significant higher risk than a typical (younger) person of the MHP-reference population. This result is also true for the two other priors and a slightly greater τΣcrit = 341.2 (Figs. 40, and 44). The results of this kind of individual Bayesian risk calculation are much more precise than general statements such as “... that the chronological age of a driver cannot be a clear indicator of his sensory-motor performance.” Instead we think that our Bayesian model combines in a near ideal way results of meta-analyses with evidence-based single-case diagnostics.

Personalized Risk Calculations with a Generative … Fig. 39 Risk excess probabilities αcdi f f (T rianglemedian ) for thresholds τ Mc ∈ [70−100) ms

Fig. 40 Risk excess probabilities αcdi f f (T rianglemedian ) for thresholds τΣc ∈ [240−470) ms

Fig. 41 Risk excess probabilities αcdi f f (T rianglemean ) for thresholds τ Pc ∈ [100−200) ms

51

52

C. Moebus

Fig. 42 Risk excess probabilities αcdi f f (T rianglemean ) for thresholds τCc ∈ [70−170) ms

Fig. 43 Risk excess probabilities αcdi f f (T rianglemean ) for thresholds τ Mc ∈ [70−100) ms

Fig. 44 Risk excess probabilities αcdi f f (T rianglemean ) for thresholds τΣc ∈ [240−470) ms

Personalized Risk Calculations with a Generative …

53

Table 3 Critical SRT-at-risk values τ X crit partitioning single subject’s risk-excessive from nonrisk-excessive SRT τ Pcrit (ms) τCcrit (ms) τ Mcrit (ms) τΣcrit (ms) Prior T rianglemode T rianglemedian T rianglemean

172 178 178

144 148 150

83.8 86.8 88.0

336.6 341.2 341.2

5 Transfer the Locus of Longitudinal Control by a Bayesian Decision Strategy We try to map the answers developed in the scenario of the cautious gunslinger into a Bayesian decision strategy. The strategy should provide a solution sketch to the applied engineering problem ‘transfer the locus of longitudinal control’ within a partial autonomous driver assistant system (PADAS). This problem seems to have been solved satisfactorily in the case of airbags. The airbag takes control and supports the driver only in those situations that are out of his control and in which he can no longer protect himself. We have something similar in mind for longitudinal control. The PADAS agent should only become active if it knows almost for certain from previous driver behavior that the driver cannot avoid a collision. Thus the research question of in time take-over control from the driver to avoid collision has to be solved. The description of Bayesian decision agents can be found here [25, 31]. Our earlier proposals to use Bayesian models in PADAS design [23, 24] were not cast in the framework of Bayesian decision strategies. This will be changed now. With the answers (25)–(30) to questions 1–3 as building blocks we are able to propose two strategies δ1 and δ2 for a Bayesian agent transferring the locus of longitudinal control. Of course priors and likelihood of the generative model have to be modified to fit into the new domain longitudinal control. Priors could be obtained from e.g. [12]. Data to be plugged into the likelihood have to be assessed from the driver of interest. Using (26) strategy δ1 can be formulated with τT T X (TTX = time-to-the-lastpossible-damage-avoiding-PADAS-intervention) as { δ1 (τT T X , S RT s) :=

αc posterior (τT T X ) > α(loss) |→ contr ol(P AD AS) αc posterior (τT T X ) ≤ α(loss) |→ contr ol(dri ver ).

(31)

Equivalently, using (27) and (28) strategy δ2 is { δ2 (τT T X , S RT s) :=

αcdi f f (τT T X ) > α(loss) |→ contr ol(P AD AS) αcdi f f (τT T X ) ≤ α(loss) |→ contr ol(dri ver ).

(32)

54

C. Moebus

τT T X is the last possible time intervention point of a PADAS for preventing collision or damage. α(loss) is the critical threshold probability for transferring the locus of control from the PADAS to the driver. It should have a very small value and it depends on costs in continuous operation. That is e.g. the cost of PADAS operation and the feeling of discomfort for the driver being monitored by a PADAS.

6 WebPPL-Code and Simulation Runs All computations and graphics were done within WebPPL. WebPPL-code and simulation runs for all parts of this paper can be found on my personal university website [22] https://uol.de/en/lcs/probabilistic-programming.

7 Summary We developed a Bayesian methodology for studying the risks implied by the timed reactive behavior of single subjects when data of a reference population are at hand. Many hazardous traffic situations can be mapped to this scenario. To communicate the goal of this research we invented the metaphorical and counterfactual scenario of a cautious gunslinger deliberating in a Bayesian way whether s/he can draw his revolver in time, if the opponent needs τΣc ms. The metaphorical gunslinger asks himself three questions which could be answered by the inference capabilities of our generative Bayesian model. These questions are: (1) “Can I draw my revolver fast enough, if my opponent needs only τc milliseconds to do so?”, (2) “Can I draw my revolver as fast as a randomly selected person of a (younger) reference population, if my opponent needs only τc milliseconds to do so?”, (3) “Is the probability of drawing my revolver slower than a randomly selected person of a (younger) reference population at most p = 0.05 if my opponent needs only τc milliseconds?”. In our study the data of a meta-analysis are compiled into weakly informed evidence-based triangle prior distributions. The likelihood of the single subject’s simple reaction time (SRT) data dependent on the prior hypothesis are formalized by a Gaussian distribution. Then the risk-excess of the single-subject’s SRT-behavior is calculated for various thresholds by comparing prior against posterior probability distributions (pdfs). It could be demonstrated that for all opponent’s challenges τc longer than a critical threshold τΣcrit = 340 ms the SRT-behavior of a 72-year old BMW-car driver is no more risky than the SRT-behavior of a randomly chosen driver of the (younger) reference population. The answers to the three questions of our cautious gunslinger agent can be used as building blocks of a Bayesian decision strategy. The strategy controls the transfer of the locus of longitudinal control from the driver to a PADAS and back again. This transfer depends on a τT T X -random variable which can be defined

Personalized Risk Calculations with a Generative …

55

e.g. as the situation-dependent time-to-the- last-possible-damage-avoiding-PADASintervention. Furthermore the transfer depends on a risk probability-threshold which is a function of the loss, when the human-PADAS-system is in operation. This can depend on, among other things, a feeling of discomfort with being monitored by a PADAS. Though the results are dependent on the data of a single subject the process model of our Bayesian psychometric risk diagnosis is not. We think that our Bayesian model combines in a near ideal way results of meta-analyses with evidence-based single-case diagnostics.

References 1. Arens T, Hettlich F, Karpfinger Ch, Kockelkorn U, Lichtenegger K, Stachel H (2018) Mathematik. Springer, Heidelberg 2. Bessière P, Laugier Ch, Siegwart R (eds) (2008) Probabilistic reasoning and decision making in sensory-motor systems. Springer, Heidelberg 3. Bishop Ch (2006) Pattern recognition and machine learning. Springer, Heidelberg 4. Card SK, Moran TP, Newell A (1986) The model human processor: an engineering model of human performance. In: Boff KR, Kaufman L, Thomas JP (eds). Handbook of perception and human performance, vol 2: Cognitive processes and performance. Wiley, New York, pp 1–35 5. Card SK, Moran TP, Newell A (1983) The psychology of human-computer interaction. Lawrence Erlbaum, Hillsdale, NJ 6. CogniFit: reaction time, cognitive ability, neuropsychology. https://www.cognifit.com/science/ cognitive-skills/response-time. Accessed 16 Aug 2022 7. Cottlin C, Döhler S (2009) Risikoanalyse, 1st edn. Vieweg+Teubner, Wiesbaden 8. Gelman A, Carlin JB, Stern HS, Dunson DB, Vehtari A, Rubin DB (2014) Bayesian data analysis, 3rd edn. CRC Press, Boca Raton, FL 9. Ge H, Xu K, Ghahramani Z (2018) Turing: a language for flexible probabilistic inference. In: Proceedings of the twenty-first international conference on artificial intelligence and statistics, proceedings of machine learning research. PMLR, vol 84. MLResearchPress, pp 1682–1690. http://proceedings.mlr.press/v84/ge18b.html 10. Goodman ND, Tenenbaum JB (2016) The ProbMods contributors: Probabilistic models of cognition, 2nd edn. CocoLab. https://probmods.org/. Accessed 15 Nov 2020 11. Goodman, N.D., Stuhlmüller, A.: The Design and Implementation of Probabilistic Programming Languages. CocoLab. Stanford (2020). http://dippl.org/. Accessed 15 Nov 2020 12. Gratzer W, Becke M (2009) Kinematik. In: Burg H, Moser A (eds) Handbuch Verkehrsunfallrekonstruktion, 2nd edn. Vieweg+Teubner, Wiesbaden, pp 89-169 13. Human benchmark. https://humanbenchmark.com/tests/reactiontime. Accessed 21 Oct 2020 14. Jastrzembski TS, Charness N (2007) The model human processor and the older adult: parameter estimation and validation within a mobile phone task. J Exp Psychol: Appl 13(4):224–248. https://doi.org/10.1037/1076-898X.13.4.224 15. Lee MD, Wagenmakers EJ (2013) Bayesian cognitive modeling. Cambridge University Press, Cambridge, UK 16. Lefèvre St, Vasquez D, Laugier Ch (2014) A survey on motion prediction and risk assessment for intelligent vehicles. ROBOMECH J. 1(1):1–14. https://robomechjournal.springeropen.com/ articles/10.1186/s40648-014-0001-z. Access 02 Jan 2021 17. Levy R, Mislevy RL (2016) Bayesian psychometric modeling. CRC Press, Boca Raton, FL 18. Linsmeier ThJ, Pearson ND (1996) Risk measurement—an introduction to value at risk, No. 1629-2016-134959. https://ageconsearch.umn.edu/record/14796/. Accessed 31 Dec 2020

56

C. Moebus

19. Luce RD (1986) Response times. Oxford University Press, Oxford, UK 20. Lunn D, Jackson Ch, Best N, Thomas A, Spiegelhalter D (2013) The BUGS book—a practical introduction to Bayesian analysis. CRC Press, Boca Raton, FL 21. Mackay DJC (2003) Information theory, inference, and learning algorithms. Cambridge University Press. Cambridge, UK 22. Moebus C (2022) Personal risk calculations with a generative Bayesian model. https:// uol.de/en/lcs/probabilistic-programming/webppl-a-probabilistic-functional-programminglanguage/simple-reaction-times-srts. Accessed 16 Aug 2022 23. Moebus C, Eilers M (2011) Integrating anticipatory competence into a Bayesian driver model. In: Cacciabue PC, Hjälmdahl M, Luedtke A, Riccioli C (eds) Human modelling in assisted transportation: models, tools and risk methods. Springer, Heidelberg, pp 225–232. https://doi. org/10.1007/978-88-470-1821-1_24 24. Moebus C, Eilers M (2011) Prototyping smart assistance with Bayesian autonomous driver models. In: Mastrogiovanni F, Chong NY (eds) Handbook of research on ambient intelligence and smart environments: trends and perspectives. IGI Global Publications, pp 460–512. https:// doi.org/10.4018/978-1-61692-857-5 25. Murphy KP (2012) Machine learning—a probabilistic perspective. The MIT Press, Cambridge, MA 26. Pearl J (2009) Causality—models, reasoning, and inference, 2nd edn. Cambridge University Press. Cambridge, UK 27. Pearl J, Glymour M, Jewell NP (2016) Causal inference in statistics—a primer. Wiley, Hoboken, NJ 28. Pearl J, MacKenzie D (2018) The book of why—the new science of cause and effect. Basic Books, New York 29. Petters AO, Dong X (2016) An introduction to mathematical finance with applications. Springer, Heidelberg 30. Robert ChP, Casella G (2004) Monte Carlo statistical methods. Springer, Heidelberg 31. Robert ChP: The Bayesian choice—from decision-theoretic foundations to computational implementation. Springer, Heidelberg (2007) 32. Staton S (2021) Probabilistic programs as measures. In: Barthe G, Katoen JP, Silva A (eds) Foundations of probabilistic programming. Cambridge University Press, Cambridge, UK, pp 43–74. https://doi.org/10.1017/9781108770750. https://www.cambridge.org/core. Accessed 17 Jan 2021 33. UbiCar. The effect of dangerous driving behaviours on reaction time. https://ubicar.com.au/ blog/the-effect-of-dangerous-driving-behaviours-on-reaction/. Accessed 16 Aug 2022 34. Westmeyer H (1975) The diagnostic process as a statistical-causal analysis. Theory Decis 6(1):57–86

Etiquette Equality or Inequality? Drivers’ Intention to be Polite to Automated Vehicles in Mixed Traffic Tingting Li and Peng Liu

Abstract Human drivers and automated vehicles (AVs) might share public roads in mixed traffic in the near future. We focused on a specific question: Will human drivers interact with AVs in the same ways that they interact with other human drivers or traditional vehicles (TVs)? Theories and evidence in the literature on AVs and humancomputer interaction (HCI) imply two opposing hypotheses: etiquette equality and etiquette inequality. To examine them, we investigated drivers’ intention to be polite or impolite to AVs in mixed traffic and TVs. Data from two vignette-based experiments (total N = 825) supported the etiquette equality hypothesis: driver participants expressed similar intentions to demonstrate polite driving behaviors in Experiment 1 and impolite driving behaviors in Experiment 2 when they were assumed to interact with AVs and TVs. This finding seems to echo the “Computers-Are-Social-Actors” paradigm in HCI, which suggests that people will interact with computers in much the same ways that they interact with other people. Theoretical and practical implications of our results are discussed. Keywords Mixed traffic · Automated vehicles · Computers-are-social-actors · Etiquette equality · Polite driving behaviors

1 Introduction Automated vehicles (AVs) promise great gains in social welfare through improved mobility, safety, and environmental impacts [1, 2]. Deploying dedicated AV lanes throughout the road network is impractical and may reduce the total traffic efficiency when the AV flow rate is low [3]. AVs cannot reach 100% market penetration rate in the near future [4, 5]. Foreseeably, human drivers will share roads with machine T. Li Tianjin University, Tianjin 300072, China e-mail: [email protected] P. Liu (B) Zhejiang University, Hangzhou 310058, China e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Switzerland AG 2023 V. G. Duffy et al. (eds.), Human-Automation Interaction, Automation, Collaboration, & E-Services 11, https://doi.org/10.1007/978-3-031-10784-9_3

57

58

T. Li and P. Liu

drivers [6–8]. Certain AVs blend into roads populated with human drivers. They are being tested on public roads. Robotaxi services are being piloted on a small scale worldwide [9, 10]. Human can usually recognize an AV on public roads by the visible sensory equipment attached to its roof such as stereo cameras and light detection and ranging (LiDAR) and sometimes its exterior signage indicating that it is operating in the automated mode. Human drivers will know that they are driving alongside an AV. Driving is a social activity and road is a social place. Human drivers and AVs need to negotiate and coordinate with each other over the shared road space. As the era of automated driving approaches, we must to consider in advance how human drivers will interact with AVs and their “driving etiquette” in their interactions in mixed traffic.

1.1 Human-AV Interaction in Mixed Traffic Studies on human-AV interactions in mixed traffic are increasing. An important line of research is to enable to AVs to learn the “driving etiquette” in current driving culture, mimic, and seamlessly mesh with human behaviors in driving interactions [11, 12]. Researchers suggested focusing on capturing the typical human driving behaviors from naturalistic driving database [12]. However, the interacting behaviors of human drivers towards other human drivers (other traditional vehicles) might not substitute those towards AVs [13, 14]. They might change their driving strategies and behaviors, for instance, taking advantage of AVs’ defensive driving style [14– 16]. Pedestrians and cyclists might have different ways of interacting with AVs and human drivers [17, 18]. Millard-Ball [17] predicted that pedestrian behaviors will change when pedestrians interact with matured AVs as their perceived risk of crossing is almost non-existent and they know that AVs will stop and will not be drunk or distracted. Pedestrians will take advantage of this in an encounter with AVs [17]. In summary, human-AV interactions may be not the same as human-human interactions on public roads. Their mix might be similar to oil and water [16]. Difficulties or complexities in human-AV interactions can be indicated by AVrelated crashes in mixed traffic. Dozens of crashes have occurred because human road users bullied and attacked AVs being tested on roads as reported in news articles [19–21], for instance, through “rude gestures and utterances, challenging the cars to brake, driving up close behind them, and tending not to give the cars right of way at junctions” [20]. Traffic crashes of AVs have a different pattern than those of conventional cars. For instance, 7 out of 10 incidents involving Google cars in the automated mode were due to a human driver rear-ending the automated car [22]. In contrast, only 14% of conventional cars were rear-ended by another car [22]. These overrepresented rear-end collisions were probably because AVs behaved in ways that human drivers did not anticipate or expect them to behave [23]. Certain opinion surveys had indicated human drivers’ concerns on mixed traffic and their interactions with AVs. Bansal et al. [24] reported that almost one-half

Etiquette Equality or Inequality? Drivers’ Intention …

59

of participants (48%) were concerned about AVs’ interactions with conventional vehicles. Tennant et al. [15, 25] directly asked participants the question “How would you feel about driving alongside autonomous cars?” and 41% felt uncomfortable about driving alongside (either totally, very, or quite), 29% felt comfortable (either totally, very, or quite), and the remaining 21% said they were neither comfortable nor uncomfortable. In addition, 34% did not like the idea of mixing human drivers and AVs, whereas only 20% were not troubled by the idea [15]. Liu and Xu [26] reported that more than one-half of participants were ambivalent about whether self-driving vehicles should be allowed on public roads before they experienced a real AV. Despite growing interest in mixed traffic, few studies surveyed drivers’ opinion about how they will interact with AVs in mixed traffic. Participants in China and South Korea expressed a greater intention to drive aggressively toward AVs than towards other human drivers [14]. Wong [27] studied participants’ willingness and unwillingness to negotiate with automated and traditional cars in four road scenarios (e.g., in cases where the automated car/traditional car was blocked by a stopped truck, the participants were asked to rate whether they would keep driving or wait) and found that there were no differences between reactions to automated and traditional cars except for one behavior: participants were more likely to keep driving when facing an AV than facing a traditional car. Thus, Liu et al. [14] and Wong [27] contributed somewhat different findings. Certain simulator-based and field studies have examined human-AV interactions and specific driving behaviors such as cut-in manoeuvers [28], overtake manoeuvers [8], and car following behaviors [13]. In these studies, differences in driving behaviors towards AVs and traditional cars were rarely examined. Zhao et al. [13] conducted the first field study on behaviors of human drivers following an AV or traditional vehicle and found that participant drivers’ following behaviors on the lead vehicle depended on their trust in AVs rather than the actual driving behavior. They did not offer a detailed comparison between participants’ behaviors when they followed the AV or traditional vehicle. As aforementioned, AV-related crashes have demonstrated the challenges in human-AV interactions on public roads, public opinion surveys have expressed human drivers’ concerns about human-AV interactions in mixed traffic, but few studies have examined human drivers’ intention and willingness about how they will interact with AVs and their actual interactive behaviors. Empirical knowledge on human-AV interactions is lacking.

1.2 Hypothesis Development Sufficient knowledge about human-AV interactions is needed to ensure the benefits that AV deployment brings. We were interested in whether human drivers will interact with AVs in the same ways that they interact with other human drivers. Specifically, we examined human drivers’ intentions to be polite and impolite to AVs and traditional cars and proposed two opposing hypotheses: an etiquette equality

60

T. Li and P. Liu

hypothesis (see Sect. 1.2.1) and an etiquette inequality hypothesis (see Sect. 1.2.2). We referred to theories and evidence for or against the two hypotheses not only in the AV literature but also in the broader human-computer interaction (HCI) literature.

1.2.1

Etiquette equality hypothesis

The etiquette equality hypothesis is: human drivers express the same level of intention to perform polite behaviors (or impolite behaviors) to AVs and traditional vehicles (TVs; i.e., other human drivers). The HCI literature provides theories or evidence about how humans interact with computers and other humans, offering insights for examining the two opposing hypotheses related to human-AV interactions. Reeves and Nass [29] proposed the Media Equation paradigm, suggesting that humans hardly distinguish between computers and other humans in the way they behave towards them. In the same vein, Nass and Moon [30] proposed the “Computers-Are-Social-Actors” paradigm, suggesting that people will interact with computers in much the same ways that they interact with other people despite their knowledge that computers are neither human nor deserving of human-like treatment. The social scripts to human-human interaction are applied mindlessly as a heuristic shortcut in human-computer interaction. People essentially ignore “the cues that reveal the essential asocial nature of a computer” [30]. The “Computers-Are-Social-Actors” paradigm has also been applied more recently to studies of human interaction with autonomous agents and empirical supports from certain studies. Edwards et al. [31] found non-significant differences in participants’ perceptions in terms of source credibility, communication competence, and interactional intentions between human Twitter agents and Twitterbots. Hong and Williams [32] reported that participants blamed artificial intelligence (AI) crimepredicting agents at similar rates as they blamed human crime-predicting agents for these agents’ racist decisions. In summary, the etiquette equality hypothesis is indirectly supported by the Media Equation and “Computers-Are-Social-Actors” paradigms in HCI that indicate a similarity between human-human and human-computer interactions. According to these two paradigms, human drivers will interact with robot drivers in a manner similar to other human drivers.

1.2.2

Etiquette inequality hypothesis

The etiquette inequality hypothesis is: human drivers express different levels of intention to perform polite behaviors (or impolite behaviors) to AVs and TVs. Certain HCI studies [33–35] reported inconsistencies between human-human and human-machine interactions, indirectly supporting the etiquette inequality hypothesis. For instance, participants tended to be more open, agreeable, extroverted, and conscientious and self-disclosing when chatting with humans than with AI and

Etiquette Equality or Inequality? Drivers’ Intention …

61

demonstrated more socially desirable traits while communicating with humans [34]. Participants responded more slowly to robots’ greetings than human greetings [35]. Thus, as opposition to the Media Equation and “Computers-Are-Social-Actors” paradigms, the Media Inequality paradigm was proposed in human-computer conversation [34, 36]. Socially interacting with machines is not mindless, but mindful [33]. People would mindfully realise that they are not real humans and have different cognitive, affective, and social responses to them [34, 36]. The etiquette inequality hypothesis might be indirectly supported by recent AV studies demonstrating obvious human-AV differences in various participants’ responses to them, including safety requirements [37, 38], perceived severity of crashes involving them [39], blame attribution for crashes involving them [40–43], and intention to drive aggressively towards them [14]. These studies found that AVs are treated harshly than human drivers. For example, participants had a greater intention to drive aggressively toward AVs than towards other human drivers in hypothetical mixed traffic scenarios [14]. AVs were blamed more than human drivers when both were involved in traffic accidents [40–42]. In summary, the Media Inequality paradigm in HCI [34, 36] and the findings of human-AV differences in the AV literature [14, 39–43] indirectly support the etiquette inequality hypothesis.

1.3 Present Study Smooth interactions between human drivers and AVs are central to road safety, efficiency, and satisfaction in mixed traffic [11, 44, 45]. We focused on a core question that has rarely been examined in the AV literature: will human drivers interact with AVs in the same ways that they interact with other human drivers? Existing theories and evidence in HCI and AV research imply two opposing hypotheses: etiquette equality and etiquette inequality (see Sect. 1.2). If the former hypothesis holds true, then the current driving etiquette or driving culture in the human world can be used to programme and regulate AV driving behaviors in mixed traffic. Otherwise, a different type of driving etiquette will emerge or should be developed. We examined the etiquette equality and etiquette inequality hypotheses. Two vignette-based experiments were conducted to survey participants’ intention to perform nine polite behaviors towards an AV and TV in Experiment 1 and perform the opposite of these behaviors (i.e., impolite behaviors) towards these two cars in Experiment 2. To investigate participants’ intentions, we adopted a vignette-based design that has been widely used to explore participants’ responses to AVs and interactions with AVs [14, 46]. In this design, participants read these hypothetical vignettes about AVs (e.g., specific decisions and behaviors of AVs) and respond as required. A comparison between participants’ intention to be polite and impolite to AVs and TVs will offer support for the etiquette equality hypothesis or etiquette inequality hypothesis.

62

T. Li and P. Liu

2 Methodology 2.1 Participants In the two between-subjects designs, a total of 895 respondents (495 in Experiment 1 and 400 in Experiment 2) were recruited through online social media and received a compensation of 5 Chinese Yuan ($0.75) for their participation. Among them, 48 participants in Experiment 1 and 20 in Experiment 2 were excluded because they did not have a driving license (although prior to the survey they were informed that only those with a driving license were qualified for the survey), and two participants under the age of 18 years in Experiment 1 were excluded (because people below 18 years are not allowed to apply for a driving license in China). In Experiment 1, for the remaining 445 licensed drivers (215 females and 230 males), their average age was 31.2 (standard deviation, SD = 8.5, Min = 18, Max = 60) and their average years of driving experience were 4.7 (SD = 4.7, Min = 0, Max = 40). They were randomly assigned to the TV condition (n = 225) or AV condition (n = 220). These two groups did not differ in age (MTV = 31.1, MAV = 31.3, t = − 0.31, p = 0.756), years of driving experience (MTV = 4.4, MAV = 5.0, t = −1.40, p = 0.163), and gender (Percentage of FemaleTV = 45.8%, Percentage of FemaleAV = 50.9%, χ 2 = 1.17, p = 0.279). In Experiment 2, among the 380 remaining participants (202 females and 178 males), the average age was 28.9 (SD = 8.5, Min = 18, Max = 65), the average years of driving experience was 4.2 (SD = 4.0, Min = 0, Max = 24), and they were randomly assigned to the TV condition (n = 191) or AV condition (n = 189). These two groups did not differ in age (MTV = 28.5, MAV = 29.3, t = −0.88, p = 0.378), years of driving experience (MTV = 4.1, MAV = 4.3, t = −0.38, p = 0.706), and gender (Percentage of FemaleTV = 51.3%, Percentage of FemaleAV = 55.0%, χ 2 = 0.53, p = 0.468).

2.2 Questionnaire Design Our polite driving behavior questionnaire (PDBQ) was designed based on two scales, the positive driver behaviors scale [47] and the multidimensional driving style inventory [48]. Özkan and Lajunen’s positive driver behaviors scale has 14 behaviors (e.g., “no sounding horn to disturb the driver in front waiting even after green light”), asking participants to indicate how often they committed each of the 14 behaviors. Among them, those behaviors related to the interactions with other drivers were selected. The multidimensional driving style inventory [48] measures drivers’ eight driving styles including dissociative, anxious, risky, angry, high-velocity, distress-reduction, patient, and careful driving styles. Patient driving style items that are relevant to patience and politeness while driving were selected. PDBQ was designed with nine polite behaviors (i.e., scenarios or items).

Etiquette Equality or Inequality? Drivers’ Intention …

63

In our study, we designed PDBQ in Experiment 1 and its opposite in Experiment 2. Specifically, participants rated their intention to perform the polite behaviors in Experiment 1 and perform the opposite of these behaviors (i.e., impolite behaviors) in Experiment 2. Because the framing of the questions could influence the participants’ responses [49], we used this design to assess whether Experiments 1 and 2 contributed convergent and robust results. The shortened description of the scenarios, diagrams, and phrasing of the questions in two experiments are shown in Fig. 1. Among the nine scenarios, all scenarios except Scenario 6 were adapted from the positive driver behaviors scale [49] and Scenario 5, 6, and 9 were adapted from the multidimensional driving style inventory [48]; thus, Scenario 5 and 9 were adapted from both scales.

2.3 Procedure Design The two experiments had the identical procedures. Experiment 1’s procedure is explained. The participants were first informed that only those with a driving license were qualified for the survey. They were then instructed to choose a condition according to the parity of the last digit of their cell number to ensure they were randomly allocated to one of the two conditions (TV vs. AV). They then read the nine traffic scenarios one by one in a random order, which were described both in words and using diagrams, and then indicated their intention to perform the nine polite behaviors. Take Scenario 9 (S9) for example. The participants read the description in the TV questionnaire: “Imagine the following scenario. On an urban road, you are driving a car (the yellow car). At the moment, a car (the blue car) in front of you is stopping abruptly” and saw the associated diagram behind (see Fig. 1). They then responded to the question: “In this case, how likely are you to not honk to express your displeasure?” The participants reported their intention on a 10-point scale (1 = very low; 10 = very high). In the AV questionnaire, “a car” was replaced with the “an automated car” in the scenario description. Considering that AVs are equipped with radars, stereo cameras, and LiDAR to detect their environment, the AV in the diagram was designed with a radar symbol on the roof to distinguish it from a conventional car. There are other differences between the TV and AV questionnaires. In the AV questionnaire, the participants first read a section of text on AVs that said “As technologies progress, we will enter into an era of automated driving. Imagine that technology of automated driving cars has matured and they become part of our daily life” [37]. After responding to the nine driving scenarios, the participants then responded to 11 items measuring their attitudes towards AVs and mind perception of AVs in the AV questionnaire, which were beyond the current study and available at the Open Science Framework (OSF) website (https://osf.io/h6zsq/). As previously stated, in Experiment 2, the participants were asked to rate their intention to perform impolite behaviors. Thus, for example, the question in S9 was changed to “In this case, how likely are you to honk to express your displeasure?”.

64

T. Li and P. Liu Scenario 1

Scenario 2 There is a car ahead of you in the lane.

On your left-rear side, there is a car trying to overtake.

E1: How likely are you to avoid close following and not to disturb the car ahead?

E1: How likely are you to slow down and allow the car to overtake successfully?

E2: How likely are you to keep close following the car ahead?

E2: How likely are you to maintain current velocity or even accelerate to not allow the car to overtake?

Scenario 3 There is a car (the silver-white car) driving normally ahead in your lane and you intend to change the lane and overtake it. E 1 : Afte r overtaking, how likely is it that you quickly return to your previous lane when safety is guaranteed to avoid occupying the fast track lane for a long time and blocking the blue car behind? E2: Af ter overtaking, how likely is it that you do not return to your previous lane, even if it could influence the blue car behind? Scenario 5

Scenario 4 There is a car driving slowly ahead in your lane.

You are ready to turn right and an oncoming car is turning left.

E1: how likely are you not to honk your horn to urge the car ahead?

E1: How likely are you to wait patiently for the car to pass first?

E2: how likely are you to honk your horn to urge the car ahead?

E2: How likely are you to pass first and not yield to the car? Scenario 7

Scenario 6 When the light turns green, the blue car ahead does not start in time.

The re a car in your right-hand lights up and intends to enter the left lane.

E1: How likely are you to wait patiently until it gets going?

E1: How likely are you to slow down and let the car enter the left-turn lane first?

E2: How likely are you to honk your horn and urge the car ahead to get going quickly?

E2: H ow likely are you to maintain cu r r e nt velocity or even accelerate, to not allow the car to drive ahead of you? Scenario 9

Scenario 8 There is a car on your right intends to turn left and bypass the obstacle ahead. E1: H ow likely are you to slow down and let the right car pass first? E2: How likely are you to maintain current velocity or even accelerate, to not allow the car to drive ahead of you?

There is a car in front of you is stopping abruptly. E 1 : H ow likely are you not to honk to express your displeasure? E2: How likely are you to honk to express your displeasure?

Fig. 1 Scenario design: its simplified description, diagram, and question phrasing. Note: in the scenario description in the AV scenario, “a car” was replaced by “an automated car”. “E1” and “E2” represent “Experiment 1” and “Experiment 2”, respectively. The only difference between the two experiments was the question. In each scenario, its description starts with “Imagine the following scenario. On an urban road, you are driving a car (the yellow car)” and in its questions, it starts with “In this case” (see Sect. 2.3)

Etiquette Equality or Inequality? Drivers’ Intention …

65

The participants submitted their demographics information including gender (female/male), exact age, possession of a driving license, and driving experience (years). The participants’ responses to the question on possessing a driving license were used to exclude invalid participants from the survey. In our survey, only driver participants were needed.

3 Results Our data were analyzed by JASP (https://jasp-stats.org/) [50]. Our data and analysis are available at the OSF (https://osf.io/h6zsq/).

3.1 Exploratory Factor Analysis (EFA) We pooled data of Experiments 1 and 2 for the EFA. The EFA was conducted with the principal component as the fitting method and Varimax as the rotation method to explore PDBQ’s underlying factor structure. It revealed two factors (eigenvalue > 1) that explained 66.64% of the total variance. As shown in Table 1, these items’ minimum factor loading was 0.62, meeting the acceptance criterion for factor loadings (≥ 0.50) [51]. The corrected item-total correlation (ITC) representing the correlation of an item with the total score of all other items was examined. Based on the 0.30 criterion for the ITC [52], all the items were identified as acceptable (their minimum was 0.49). Factor 1 explained 42.38% of the variance (Cronbach’s α = 0.89) consisting of six scenarios (S1, S2, S3, S5, S7, and S8). In these scenarios, the driver participants rated their intention to show their courtesy to others in Experiment 1. Take S7 for Table 1 EFA results for the polite driving behavior questionnaire Item

Factor loading

Item-total correlation (ITC)

I

II

S1

0.72

0.21

0.65

S2

0.82

0.23

0.76

S3

0.74

0.16

0.65

S4

0.31

0.73

0.55

S5

0.62

0.30

0.58

S6

0.21

0.75

0.49

S7

0.87

0.18

0.82

S8

0.89

0.12

0.82

S9

0.09

0.80

0.50

66

T. Li and P. Liu

example: “On an urban road, you are driving a car (the yellow car) in the left-turn lane. At the moment, a car (the blue car) in your right-hand lights up and intends to enter the left lane. In this case, how likely are you to slow down and let the car enter the left-turn lane first?” Note that the participants rated their intention to perform the opposite of these behaviors in Experiment 2. Thus, this factor was called “courtesy” in Experiment 1 and “discourtesy” in Experiment 2. Factor 2 explained 22.26% of the variance (Cronbach’s α = 0.70) and consisted of three scenarios (S4, S6, and S9). In these scenarios, the driver participants rated their intention to be patient with others. Take S6 for example: “On an urban road, you are driving a car (the yellow car) and queuing up at a traffic light. When the light turns green, the blue car ahead does not start in time. In this case, how likely are you to wait patiently until it gets going?” This factor was labeled as “patience” in Experiment 1 and “impatience’ in Experiment 2. As we measured the participants’ intention to perform polite and impolite behaviors in two separate experiments (see Fig. 2), we considered PDBQ’s construct validity by examining two theoretical relationships. The nine scenarios are taken as a sample (n = 9). First, for the nine scenarios, the ratings of the participants’ intention to perform their associated polite behaviors in Experiment 1 and impolite behaviors in Experiment 2 were negatively correlated in both situations (TV: r = − 0.70, p = 0.035; AV: r = − 0.78, p = 0.013). Second, bearing in mind that the minimum and maximum values in PDBQ were 1 and 10, the sum of participants’ intention to perform the polite and impolite behaviors was supposed to be similar from 11 across the nine scenarios, which was confirmed by t-tests (TV: t df =8 = 1.46, p = 0.183; AV: t df =8 = 1.19, p = 0.268).

3.2 Analysis of Covariance (ANCOVA) The items for (dis)courtesy and (im)patience were averaged respectively. Figure 3 shows the mean values of courtesy (to TV: M = 7.53, SD = 1.57; to AV: M = 7.50, SD = 1.60), patience (to TV: M = 6.10, SD = 2.03; to AV: M = 6.29, SD = 1.87), discourtesy (to TV: M = 3.85, SD = 1.76; to AV: M = 3.66, SD = 1.81), and impatience (to TV: M = 5.12, SD = 2.27; to AV: M = 5.14, SD = 2.21). Courtesy and patience were significantly correlated in Experiment 1 (r = 0.447, p < 0.001), and so were discourtesy and impatience in Experiment 2 (r = 0.472, p < 0.001). In Experiment 1, two ANCOVA tests were conducted, with courtesy and patient as the dependent variables, respectively, interactive condition (TV = 0, AV = 1) as the independent variable, and gender (male = 0, female = 1), age, and driving experience as covariates. The participants’ courtesy (F = 0.10, p = 0.748, η2 p < 0.001) and patience (F = 0.70, p = 0.402, η2 p = 0.002) did not differ across the TV and AV conditions (see Table 2). In Experiment 2, similar ANCOVA tests revealed that participants’ discourtesy (F = 0.92, p = 0.339, η2 p = 0.002) and impatience (F = 0.08, p = 0.772, η2 p < 0.001) were similar across the TV and AV conditions. Thus,

Etiquette Equality or Inequality? Drivers’ Intention …

67

Fig. 2 Mean values of intention to perform nine polite driving behaviors in Experiment 1 and nine impolite driving behaviors in Experiment 2 across the TV and AV situations. Error bars = 95% Confidence Interval (CI)

68

T. Li and P. Liu

Table 2 Results of ANCOVA Predictor

Experiment 1 Courtesy B

Patience F

p

η2 p

B

F

p

η2 p

Scenario

−0.05

0.10

0.748

< 0.001

0.16

0.70

0.402

0.002

Gender

0.18

1.46

0.228

0.003

0.13

0.49

0.485

0.001

Age

0.01

1.02

0.312

0.002

0.02

3.56

0.060

0.008

Driving experience

0.01

0.27

0.602

< 0.001

0.03

1.56

0.212

0.004

Predictor

Experiment 2 Discourtesy

Scenario

Impatience

B

F

p

η2 p

B

F

p

η2 p

−0.18

0.92

0.339

0.002

0.07

0.08

0.772

< 0.001

Gender

−0.30

2.57

0.110

0.007

−0.27

1.30

0.254

0.003

Age

−0.01

0.25

0.619

< 0.001

−0.06

9.92

0.002

0.026

Driving experience

0.03

1.23

0.267

0.003

0.05

1.46

0.227

0.004

B = unstandardized coefficients; Scenario: TV = 0, AV = 1; Gender: male = 0, female = 1.

Experiments 1 and 2 supported the etiquette equality hypothesis over the etiquette inequality hypothesis. Gender and driving experience did not influence courtesy (p = 0.228 and p = 0.602, respectively), discourtesy (p = 0.110 and p = 0.267, respectively), patience (p = 0.485 and p = 0.212, respectively), and impatience (p = 0.254 and p = 0.227, respectively). Age did not influence courtesy (p = 0.312) and discourtesy (p = 0.619), whereas it marginally influenced patience and significantly influenced impatience. Compared with younger participants, older participants reported marginally higher intention to be polite to others (B = 0.02, p = 0.060, η2 p = 0.008) and lower intention to be impolite (B = −0.06, p = 0.002, η2 p = 0.026). The results of the demographic factors’ influence were not our focus and thus were omitted from the following discussion.

4 Discussion We surveyed the driver participants’ intention to be polite to an automated car and a traditional car in Experiment 1 and to be impolite to these two cars in Experiment 2 and found that their intentions to perform polite behaviors (courtesy and patience) or impolite behaviors (discourtesy and impatience) were similar when interacting with the two different cars. Thus, our finding supported the etiquette equality hypothesis. If this the hypothesis is confirmed by more evidence, its practical implication is that the driving etiquette will not change in an era of mixed traffic and that AV developers

Etiquette Equality or Inequality? Drivers’ Intention …

69

Fig. 3 Mean values of a courtesy and patience in Experiment 1 and b discourtesy and impatience in Experiment 2 across the TV and AV conditions. Error bars = 95% CI

should focus on enabling AVs to learn the current driving etiquette and mimic current human driving behaviors. This finding seemingly echoes the Media Equation [29] and “Computers-AreSocial-Actors” [30, 53] paradigms in HCI. For example, Nass [54] demonstrated etiquette equality that people apply the same politeness rules to computers that they do to other people. In our two experiments, the driver participants expressed similar intention to perform polite or impolite behaviors to automated cars and traditional human-driven cars. This finding is also in line with certain observations in the AV literature. Wong [27] found that overall their participants did not convey differences in their agreements with several negotiation behaviors towards an AV and traditional car, except that their participants more supported one behavior that “I would keep driving, because it is my right of way" when they negotiated with the AV. Ritchie et al. [8] provided indirect support for our finding. They examined participants’ acceptability of overtaking manoeuvers as a function of pull-in distance in two scenarios (participants were overtaken by a car when they drove a car vs an automated car drove participants and overtook a leading car) and found that the two scenarios (overtaking and being overtaken) did not influence participants’ acceptability of overtaking manoeuvers. Contrary to our finding on the non-significant human-AV difference in terms of participants’ intention to be polite or impolite towards AVs and traditional cars, certain prior studies revealed significant human-AV differences or TV-AV differences. The participants had a greater intention to “bully” AVs in hypothetical mixed traffic scenarios [14], perceived traffic crashes involving AVs more severe [39], and perceived AVs more culpable when they were struck by another car [45] and when they caused traffic crashes in amoral conditions [40, 41] or moral dilemmas [43]. These discussions might imply a conflict in human-AV differences in people’s various responses to AVs and traditional human-driven cars. Our non-significant human-AV difference could be due to many factors. Three possible accounts are

70

T. Li and P. Liu

offered. First, as suggested by the Media Equation [29] and “Computers-Are-SocialActors” [30, 53], people regard an AV as a normal car and apply the same driving etiquette and interactive behaviors to them when AVs perform common driving behaviors. Second, the vehicle type (automated vs traditional car) was not a salient cue for influencing participants’ intention to how to behave. Third, prior studies on human-AV differences [14, 39–43] manipulated AVs to cause unwanted outcomes or be involved with unwanted events, for which people would have lower tolerance; on the contrary, the AVs in this study were assumed to perform common driving activities on public roads. This difference in study design might explain the mixed findings on the human-AV differences between these previous studies and our research. This encourages further investigation of human-AV differences in transportation. This is important for tracking how human drivers will share the road with AVs. Certain research limitations and future directions should be noted. First, our vignette-based design could only examine participants’ intention to be polite or impolite to others rather than their actual behaviors. Considering the intention-behavior gap [55], future studies should examine drivers’ interactive behaviors using simulators, virtual reality, or naturalistic observations if possible. Second, our sample was limited to one country. Cross-national differences in the public’s attitude and acceptance of AVs have been noted [56–58], implying that the generalizability of our finding supporting the etiquette equality hypothesis in mixed traffic should be examined in more countries. Third, participants might not have encountered AVs as AVs have not yet been commercialized and are being tested and piloted on a small scale, which might have affected our results.

5 Conclusions Human drivers will negotiate with AVs to share public roads. Road are a social space and driving is a social activity. In an era of mixed traffic, it is important to assess how human drivers will drive alongside AVs. In two vignette-based, between-subjects experiments in which driver participants imagined interacting with automated and traditional cars, they expressed similar intention to be polite (Experiment 1) or impolite (Experiment 2) to both cars. This finding lends support for the etiquette equality hypothesis. If it is confirmed by more evidence, this implies that driving etiquette will not change in mixed traffic and that AV developers should focus on enabling AVs to be capable of learning the current driving etiquette to mimic human driving behaviors. Acknowledgements This work was supported by the Natural Science Foundation of China (No. 72071143).

Etiquette Equality or Inequality? Drivers’ Intention …

71

References 1. National Highway Traffic Safety Administration (2016) Federal automated vehicles policy: accelerating the next revolution in roadway safety. National Highway Traffic Safety Administration (NHTSA), U.S. Department of Transportation, Washington DC 2. Fagnant DJ, Kockelman K (2015) Preparing a nation for autonomous vehicles: Opportunities, barriers and policy recommendations. Transp Res Part A Policy Pract 77:167–181 3. Liu Z, Song Z (2019) Strategic planning of dedicated autonomous vehicle lanes and autonomous vehicle/toll lanes in transportation networks. Transp Res Part C Emerg Technol 106:381–403 4. Litman T (2018) Autonomous vehicle implementation predictions: implications for transport planning. Victoria Transport Policy Institute, British Columbia, Canada 5. Milakis D, Snelder M, van Arem B, van Wee B, de Almeida Correia GH (2017) Development and transport implications of automated vehicles in the Netherlands: Scenarios for 2030 and 2050. Eur J Transp Infrastruct Res 17(1):63–85 6. Lee S, Jeong E, Oh M, Oh C (2019) Driving aggressiveness management policy to enhance the performance of mixed traffic conditions in automated driving environments. Transp Res Part A Policy Pract 121:136–146 7. Sharma A, Ali Y, Saifuzzaman M, Zheng Z, Haque MM (2017) Human factors in modelling mixed traffic of traditional, connected, and automated vehicles. In: Proceedings of the AHFE 2017 international conference on human factors in simulation and modeling. Los Angeles, CA 8. Ritchie OT, Watson DG, Griffiths N, Misyak J, Chater N, Xu Z, Mouzakitis A (2019) How should autonomous vehicles overtake other drivers? Transp Res F Traffic Psychol Behav 66:406–418 9. Can robotaxis ease public transport fears in China? https://www.bbc.com/news/business-523 92366. Last accessed 4 June 2020 10. Autonomous vehicle passenger service pilot programs, https://www.cpuc.ca.gov/avcpilotinfo/. Last accessed 7 Aug 2020 11. Chater N, Misyak J, Watson D, Griffiths N, Mouzakitis A (2018) Negotiating the traffic: can cognitive science help make autonomous vehicles a reality? Trends Cogn Sci 22(2):93–95 12. Huang X, Zhang S, Peng H (2020) Developing robot driver etiquette based on naturalistic human driving behavior. IEEE Trans Intell Transp Syst 21(4):1393–1403 13. Zhao X, Wang Z, Xu Z, Wang Y, Li X, Qu X (2020) Field experiments on longitudinal characteristics of human driver behavior following an autonomous vehicle. Transp Res Part C Emerg Technol 114:205–224 14. Liu P, Du Y, Wang L, Ju DY (2020) Ready to bully automated vehicles on public roads? Accid Anal Prev 137:105457 15. Tennant C, Howard S, Franks B, Bauer MW, Stares S (2016) Autonomous vehicles-negotiating a place on the road. A Study on How Drivers Feel About Interacting with Autonomous Vehicles on the Road. London School of Economics and Political Science, London, UK 16. An inconvenient truth: Human drivers and autonomous cars mix like oil and water. https://www.forbes.com/sites/lanceeliot/2019/05/07/an-inconvenient-truth-human-dri vers-and-autonomous-cars-mix-like-oil-and-water/#370f92773b84. Last accessed 11 Aug 2020 17. Millard-Ball A (2018) Pedestrians, autonomous vehicles, and cities. J Plan Educ Res 38(1):6–12 18. Vlakveld W, van der Kint S, Hagenzieker MP (2020) Cyclists’ intentions to yield for automated cars at intersections when they have right of way: Results of an experiment using high-quality video animations. Transp Res F Traffic Psychol Behav 71:288–307 19. A slashed tire, a pointed gun, bullies on the road: Why do Waymo self-driving vans get so much hate? https://www.azcentral.com/story/money/business/tech/2018/12/11/waymo-self-drivingvehicles-face-harassment-road-rage-phoenix-area/2198220002/. Last accessed 31 July 2019 20. Uber says people are bullying its self-driving cars with rude gestures and road rage. https:// www.businessinsider.com/uber-people-bullying-self-driving-cars-2019-6. Last accessed 31 July 2019

72

T. Li and P. Liu

21. Humans slapped and shouted at robot cars in two of six DMV crash reports this year, https://www.latimes.com/business/autos/la-fi-hy-human-attacks-robot-cars-20180305story.html. Last accessed 12 Apr 2020 22. Teoh ER, Kidd DG (2017) Rage against the machine? Google’s self-driving cars versus human drivers. J Safety Res 63:57–60 23. Why people keep rear-ending self-driving cars. https://www.wired.com/story/self-driving-carcrashes-rear-endings-why-charts-statistics/. Last accessed 1 Dec 2019 24. Bansal P, Kockelman KM, Singh A (2016) Assessing public opinions of and interest in new vehicle technologies: an Austin perspective. Transp Res Part C Emerg Technol 67:1–14 25. Tennant C, Stares S, Howard S (2019) Public discomfort at the prospect of autonomous vehicles: building on previous surveys to measure attitudes in 11 countries. Transp Res F Traffic Psychol Behav 64:98–118 26. Liu P, Xu Z (2020) Public attitude toward self-driving vehicles on public roads: Direct experience changed ambivalent people to be more positive. Technol Forecast Soc Chang 151:119827 27. Wong PNY (2019) Who has the right of way, autonomous vehicles or drivers? Multiple perspectives in safety, negotiation and trust. In: Automotive user interfaces and interactive vehicular applications. Utrecht, Netherlands, pp 22–25 28. Zhao C, Wang W, Li S, Gong J (2020) Influence of cut-in maneuvers for an autonomous car on surrounding drivers: experiment and analysis. IEEE Trans Intell Transp Syst 21(6):22665– 32276 29. Reeves B, Nass C (1996) The media equation: how people treat computers, television, and new media like real people and places. CSLI Publications, Stanford, CA 30. Nass C, Moon Y (2000) Machines and mindlessness: social responses to computers. J Soc Issues 56(1):81–103 31. Edwards C, Edwards A, Spence PR, Shelton AK (2014) Is that a bot running the social media feed? Testing the differences in perceptions of communication quality for a human agent and a bot agent on Twitter. Comput Hum Behav 33:372–376 32. Hong J-W, Williams D (2019) Racism, responsibility and autonomy in HCI: testing perceptions of an AI agent. Comput Hum Behav 100:79–84 33. Fischer K, Foth K, Rohlfing KJ, Wrede B (2011) Mindful tutors: linguistic choice and action demonstration in speech to infants and a simulated robot. Interact Stud 12(1):134–161 34. Mou Y, Xu K (2017) The media inequality: Comparing the initial human-human and human-AI social interactions. Comput Hum Behav 72:432–440 35. Kanda T, Miyashita T, Osada T, Haikawa Y, Ishiguro H (2008) Analysis of humanoid appearances in human–robot interaction. IEEE Trans Rob 24(3):725–735 36. Shechtman N, Horowitz LM (2003) Media inequality in conversation: how people behave differently when interacting with computers and people. In: Proceedings of the SIGCHI conference on human factors in computing systems. Fort Lauderdale, FL 37. Liu P, Wang L, Vincent C (2020) Self-driving vehicles against human drivers: equal safety is far from enough. J Exp Psychol Appl 26(4):692–704 38. Liu P, Yang R, Xu Z (2019) How safe is safe enough for self-driving vehicles? Risk Anal 39(2):315–325 39. Liu P, Du Y, Xu Z (2019) Machines versus humans: people’s biased responses to traffic accidents involving self-driving vehicles. Accid Anal Prev 125:232–240 40. Hong JW (2020) Why is artificial intelligence blamed more? Analysis of faulting artificial intelligence for self-driving car accidents in experimental settings. Int J Hum Comput Interact 36(18):1768–1774 41. Dougherty S, Stowell J, Richards A, Ellen P (2018) Will automated trucks trigger the blame game and socially amplify risks? In: 2018 Engaged management scholarship conference. Philadelphia, PA 42. Waytz A, Heafner J, Epley N (2014) The mind in the machine: anthropomorphism increases trust in an autonomous vehicle. J Exp Soc Psychol 52:113–117

Etiquette Equality or Inequality? Drivers’ Intention …

73

43. Young AD, Monroe AE (2019) Autonomous morals: inferences of mind predict acceptance of AI behavior in sacrificial moral dilemmas. J Exp Soc Psychol 85:103870 44. Schwarting W, Pierson A, Alonso-Mora J, Karaman S, Rus D (2019) Social behavior for autonomous vehicles. Proc Nat Acad Sci USA 116(50):24972–24978 45. Imbsweiler J, Stoll T, Ruesch M, Baumann M, Deml B (2018) Insight into cooperation processes for traffic scenarios: modelling with naturalistic decision making. Cogn Technol Work 20(4):621–635 46. Awad E, Levine S, Kleiman-Weiner M, Dsouza S, Tenenbaum JB, Shariff A, Bonnefon J-F, Rahwan I (2020) Drivers are blamed more than their automated cars when both make mistakes. Nat Hum Behav 4:134–143 47. Özkan T, Lajunen T (2005) A new addition to DBQ: positive driver behaviours scale. Transp Res F Traffic Psychol Behav 8:355–368 48. Taubman-Ben-Ari O, Mikulincer M, Gillath O (2004) The multidimensional driving style inventory—scale construct and validation. Accid Anal Prev 36(3):323–332 49. Chong D, Druckman JN (2007) Framing theory. Annu Rev Polit Sci 10(1):103–126 50. JASP team: JASP (Version 0.11.1) (2020) 51. Hair JF, Black WC, Babin BJ, Anderson RE (2014) Multivariate data analysis, 7th edn. Pearson, London, UK 52. Nunnally JC, Bernstein IH (1994) Psychometric theory, 3rd edn. McGraw-Hill, New York 53. Nass C, Steuer J, Tauber ER (1994) Computers are social actors. In: Proceedings of the SIGCHI conference on human factors in computing systems. Boston, MA 54. Nass C (2004) Etiquette equality: exhibitions and expectations of computer politeness. Commun ACM 47(4):35–37 55. Sheeran P, Webb TL (2016) The intention–behavior gap. Soc Pers Psychol Compass 10(9):503– 518 56. Moody J, Bailey N, Zhao J (2020) Public perceptions of autonomous vehicle safety: an international comparison. Saf Sci 121:634–650 57. Schoettle B, Sivak M (2014) Public opinion about self-driving vehicles in China, India, Japan, the U.S., the U.K., and Australia. UMTRI-2014–30. Transportation Research Institute, University of Michigan, Ann Arbor, MI 58. Liu P, Guo Q, Ren F, Wang L, Xu Z (2019) Willingness to pay for self-driving vehicles: influences of demographic and psychological factors. Transp Res Part C Emerg Technol 100:306–317

Human Collaboration with Advanced Vehicle Technologies: Challenges for Older Adults Joseph Sharit, Dustin J. Souders, and Neil Charness

Abstract In this chapter, we examine the issue of collaboration between older adult drivers and advanced vehicle technologies (AVTs). We highlight some findings from the literature as well as attempt to offer additional perspectives on older drivers and AVTs. In particular, we consider how human information processing limitations and capabilities associated with older adults, tendencies related to trust and risk taking, and opportunities for learning about these systems, as well as the specific nature or design of the AVT define the challenges that older drivers may face when collaborating with these systems. Keywords Advanced vehicle technologies · Older drivers · Human information processing · Risk taking · Perceived trust · Knowledge of automation

1 The Gradual Unfolding of Vehicle Automation As with many jobs that people perform in conventional industries, where tasks are becoming redefined based on advances in automation, technical innovations in vehicles are similarly redefining the operational experiences of many drivers. Although the most noticeable impacts in service and manufacturing industries clearly are those in which functions once entirely performed by humans become completely automated, for many jobs the more realistic scenario is a gradual unfolding of tasks that will become increasingly allocated to intelligent machines. Similarly, vehicles have gradually added layers of aids. These systems, which fall under the umbrella of advanced vehicle technologies (AVTs), are referred to as advanced driver assisted systems (ADAS) at the lower end of the Society of Automotive Engineering’s spectrum of assistive systems (see below) when specifically aiding the driver’s performance, or as automated driving systems (ADS) at higher levels of automation where automated vehicles (AVs) replace the driver in the driving task. Generally, AVTs are “electronic, in-vehicle systems that can perform or assist drivers in performing J. Sharit (B) · D. J. Souders · N. Charness Coral Gables, USA e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Switzerland AG 2023 V. G. Duffy et al. (eds.), Human-Automation Interaction, Automation, Collaboration, & E-Services 11, https://doi.org/10.1007/978-3-031-10784-9_4

75

76

J. Sharit et al.

various behind-the-wheel tasks for which humans may be prone to error and/or complacency” [18].

2 Older Drivers and AVTs AVT systems offer a potential means for addressing the unique vulnerability of aging drivers to crash risks. Figure 1 highlights the need for interventions. Panel A, showing millions of crashes by year and age group, indicates that drivers 15–20 years of age generate more crashes per year than drivers age 65+ years, with numbers relatively stable between 2015 and 2019 for younger drivers, but increasing for older drivers, reflecting in part the growing number of older drivers on the road as a function of general population aging. Panel B, showing percent fatal crashes, indicates that older drivers are about eight times more likely to die when experiencing a crash than younger drivers. An increasing number of crashes over time with much higher death rates per crash make crash risk mitigation for aging drivers an increasingly urgent public health challenge. With the number of senior drivers (aged 65 and older) projected to rise to more than 60 million by 2030 [1], and the fact that many older adults exhibit normal agerelated physical, sensory, and cognitive declines (e.g., declines in vision, memory, divided attention, and slower reaction time) that could impact driving safety [29], AVTs may be able to provide considerable safety benefits. Collaborating with these systems, however, often requires placing trust in these technologies, and trust in technology appears to decrease with age [23]. Knowledge may also be a factor as one’s trust in, and knowledge about technology are closely related constructs; in fact, [29] cited evidence that seniors with less AVT knowledge were more prone to mistrust AVTs or use them in inappropriate driving conditions compared with other age groups. There is also the issue of age-related difficulties in learning new skills and changing well-established behavioral routines [11, 13], which many older drivers might be susceptible to when encountering AVTs. This, in turn, could lead to adverse collaborative arrangements such as overreliance on the assistive technologies or failing to understand their constraints or limitations, and thus the driving contexts under which AVTs may not be reliable. Despite these concerns, older drivers appear willing to adopt AVTs. Based on interviews with older drivers (60–85 years of age) who owned a vehicle with at least two AVTs, evidence suggests that their experiences with these aiding systems led them to believe that AVTs counteracted age-related changes in driving performance that they felt were occurring to them, and that these systems generated increased comfort when driving, and by association, increased safety. Older drivers also described AVTs as generating a sense of comfort behind-the-wheel, where comfort was equated with convenience, ease of use, and increased feelings of safety [19]. In an experimental setting, older drivers placed higher valuations than younger drivers on ADAS such as blind spot monitors [39], which might particularly increase individuals’ awareness

Human Collaboration with Advanced Vehicle Technologies … Panel A

Age 15-20 yr 1.6 1.4

1.41

1.31

77

Age 65+ yr

1.33

1.32

1.2 0.97

0.91

1.0

1.28 1.11

1.05

0.97

0.8 0.6 0.4 0.2 0.0 2015 Panel B

2016

2017

Age 15-20 yr

2018

2019

Age 65+ yr

3.0 2.48

2.48

2.5

2.48 2.24

2.06

2.0 1.5 1.0 0.5

0.29

0.28

0.28

0.27

0.26

0.0 2015

2016

2017

2018

2019

Fig. 1 Panel A shows USA Total Crashes (Millions) by Age Group and Year. Total crashes include fatal crashes, injury crashes only, and property crashes only. Panel B shows USA Percent Crashes that are Fatal Crashes by Age Group and Year. Data from FARS query Dec. 24, 2020 from https:// cdan.dot.gov/query

of other road users in their blind spots when age-related maladies such as arthritis make the task of turning the neck or trunk to check them uncomfortable. Generally, older drivers’ perceptions and preferences regarding AVTs derive from studies involving surveys, interviews, and focus groups, and thus do not account for participants’ actual exposure and experience with these systems. One recent exception is the study by Liang et al. [29], in which drivers’ perceptions were evaluated from three focus groups after substantial driving exposure. Following collection of baseline data from an attitudes to ADAS questionnaire, study participants were each assigned to one of four vehicle models equipped with at least the following four ADAS: blind spot detection (BSD), lane alert (LA), lane keeping assist (LKA), and

78

J. Sharit et al.

adaptive cruise control (ACC). Participants then received extensive multiple-part training that included explanations of these features while the vehicle was parked; sitting beside the experimenter who drove the vehicle and demonstrated when, when not, and how to use the four ADAS; and driving the vehicle, using or experiencing each of the four ADAS based on the experimenter’s verbal guidance. Participants were then asked to drive the vehicle (unaccompanied by the experimenter) as they normally would for a 6-week period, with the attitudes to ADAS questionnaire again administered upon return of the vehicles. During subsequent focus groups participants shared their thoughts about the ADAS through a series of guide questions, and a structural topic modelling approach was used to derive the following five key topics, in order of prevalence: safety; confidence concerning ADAS; ADAS functionality; user interface/usability; and non-ADAS-related features. The implications of ADAS for safety elicited both positive feelings about the safety improvements as well as concerns regarding false alerts and constraints on the range of effective operations provided by ADAS, leading the authors to note that because many current LKA and ACC systems are not capable of handling the full range of road conditions, training may be essential to help older drivers learn about appropriate use of such ADAS as well as their limitations. The knowledge derived from experiencing and using the ADAS while driving the vehicles and from reading the owner’s manual appeared to lead to confidence in the ADAS and in their ability to use them, and compared to their pre-exposure attitudes about ADAS the post-exposure findings indicated positive attitude changes toward ADAS, especially in regard to having a lower concern about false alerts and increased trust in the effectiveness of the systems regarding safety. Overall, the authors of this study emphasized the importance of a training program that includes well-written owner’s manuals for obtaining ADAS basic knowledge and limitations, as well as hands-on experience to: (1) provide real-time operational knowledge and familiarity with ADAS so that the older driver can remain “in-to-the loop” and avoid automation surprises [37], and (2) promote sufficient knowledge and trust that can translate to ADAS adoption, as familiarity is a good predictor of ADAS adoption by older drivers [39]. This might also extend to higher levels of automation, as an interim analysis has shown that experience with an automated shuttle or a simulated automated vehicle increased older drivers’ trust and perceived safety compared to baseline, while experience with both improved their ratings of perceived usefulness [10].

3 Evaluating AVTs AVTs can be categorized into different levels depending on the degree of oversight required from—i.e., the degree of collaboration with—the human driver. According to The Society of Automotive Engineers [36], these vehicle technologies or automation systems range from no automation (Level 0) where the driver is still in complete control of the vehicle, to full automation (Level 5), where the technology has total control of the vehicle (Fig. 2). Examples of AVTs include forward collision

Human Collaboration with Advanced Vehicle Technologies …

79

warning (FCW) or mitigation (FCM) systems, lane departure warning (LDW) or mitigation (LDM) systems, adaptive cruise control (ACC) systems, intelligent speed adaptation (ISA) systems, blind spot detection (BSD) systems, adaptive headlights, parking assists, in-vehicle information systems, and automated driving systems (ADS) beginning at SAE L3 automation. In a scoping review of 324 peer-reviewed mostly recent studies from 25 countries, [18] summarized the evidence on the role of AVTs in improving road safety. Much of this research was based on data derived from high- and low-fidelity driving simulators (70%) and naturalistic driving conditions (including open and closed driving circuits), but in some cases also from large databases containing crash records and fatality data that were used to predict estimates of the effectiveness of certain technologies. Of the 324 studies 51% were mixed ages, 23% had only younger drivers, 3% had only older drivers, and 6% included younger and older drivers; 51 studies (16%) did not report the drivers’ ages. Based on objective outcomes (e.g., driver reaction time, longitudinal control, lateral position, measures of eye movements, vehicle maneuvering, driver’s interaction with AVT, and number of collisions), subjective outcomes (e.g., trust, perceived workload, usefulness, usability, and satisfaction), and the authors’ own conclusions, the AVTs were categorized as positive (safer with the AVT), negative (less safe with the AVT), mixed (mixed positive and negative outcomes for the same AVT), neutral, unclear, and not reported. Overall, while the evidence was generally in favour of AVTs

Fig. 2 Levels of driving automation Source [36]

80

J. Sharit et al.

having a positive effect on driving safety, there were still a considerable number of findings that were classified into the other categories, and the authors conceded that the wide range in classifications could have been due to the large variability in the nature and design of the studies. These authors also noted that their review found that many studies did not specify the demographics of the drivers being tested, which was surprising given the known impact of age on driving behaviour [44]. Although [9] reported that AVTs have moderate to high benefits for older drivers, they cautioned that there is a lack of sufficient evidence to make a determination regarding the relationship between AVTs and safety for this demographic group.

3.1 The Example of Adaptive Cruise Control To exemplify how the terms were operationalized and how findings were classified in the review by Furlan et al. discussed above, we will consider the study by Bianchi Piccinini et al. [5] that examined reactions to a critical situation during driving with Adaptive Cruise Control (ACC) from regular users and non-users of this AVT system. ACC is a system that maintains driver-selected speed and headway to a preceding vehicle, and it is designed primarily with comfort and convenience in mind to make driving easier. According to Bianchi Piccinini et al., because this system presents some limitations that are either partially or totally unknown to the users, many drivers exhibit rudimentary, and thus incomplete and possibly inaccurate mental models of the system, which leads to placing excessive trust in the automated system (e.g., [17, 20] and as a consequence, increases the likelihood for the occurrence of negative effects on road safety. Mental models are “the mechanisms whereby humans are able to generate descriptions of system purpose and form, explanations of system functioning and observed system states, and predictions of future states” [35], and thereby can greatly influence the collaboration between the human and the system. Bianchi Piccinini et al. contend that the driver’s mental model of the system is strictly linked to the driver’s trust in the system, which is a fundamental aspect in determining the human’s usage of automation [32, 33]. Trust was defined, based on [7], “as an attitude resulting from knowledge, beliefs, emotions and other elements, which generates positive or negative expectations concerning the reactions of a system and the interaction with it.” Twenty-six participants, divided into ACC users and non-users, drove twice in a simulated environment—once with the ACC and once manually—with a critical situation (a stationary vehicle stopped in the cruising lane of the highway) experienced during both drives. The objective outcome of the study was average time to collision (TTC), defined as the time required for two vehicles to collide if they continue at their present speed and on the same path (smaller TTC values indicate a higher risk of accidents). Based on the average TTC with and without the ACC, the results indicated that when the ACC system was activated both ACC users and non-users had an increased risk of collision when driving with ACC as compared to driving without it (the baseline manual driving condition). A large negative correlation was also found

Human Collaboration with Advanced Vehicle Technologies …

81

between the driver’s mental model of ACC operation in the critical situation and the safety margins maintained by the ACC users during the same situation. Mental model adequacy (or degree of correctness) for this study was based on the driver’s degree of agreement to statements in a mental model questionnaire (e.g., “The ACC reacts to stationary objects”). The subjective outcome of the driver’s “trust” was also assessed before and after the ACC trial based on the driver’s degree of agreement to statements in a trust questionnaire developed by Jian et al. [25], for example, “I am confident in the system”. The results indicated that compared to before the trial the average trust score did not significantly change after the trial, and that ACC users and non-users did not differ regarding the trust placed in the ACC system. In the Furlan et al. scoping review, the objective outcome from the [5] study (average time to collision) was classified as “negative,” while the subjective outcome (trust in the system) was classified as “neutral.” When combining objective results, subjective results, and the author’s conclusions, Furlan et al. concluded that in the Bianchi Piccinini et al. study safety was worse with ACC than without ACC, resulting in a negative global rating in relation to ACC. This introduction of convenience-oriented (as opposed to safety-oriented) driving automation such as ACC highlights that just because part of the driving task is automated (longitudinal control), driving safety may decrease. As more highly capable SAE L2 systems that piece together both longitudinal and lateral control become more readily available (e.g., Tesla’s AutoPilot) and colloquially thought to increase road safety, care must be taken in the communication of their capabilities, limitations, and appropriate use cases to drivers—particularly drivers who might view such systems as more safety- than convenience-enhancing. “Autonowashing” (i.e., misrepresenting the proper level of human supervision that a semi-automated system requires; [16] by media and marketing interests exacerbates the tendency of human operators to overtrust new, seemingly high-performing, automated driving systems and grow complacent while they are engaged—something that the Bianchi Piccinini et al. study showed that even drivers with prior hands-on ACC system experience were similarly prone to as evidenced by their reduced TTCs when the ACC system was engaged.

3.2 Blind Spot Detection: A Safety Critical and Potentially Demanding Task In the absence of blind spot detection (BSD) capability within vehicles, determining when to change lanes coupled with the execution of that decision, especially at high roadway speeds and traffic density, constitutes a safety critical and demanding information processing task. It requires quickly maneuvering one’s upper body, rapid situational assessment of other vehicles possibly travelling at high speeds, reorienting oneself to the scenario ahead, deciding whether to change lanes, and executing that

82

J. Sharit et al.

decision. BSD essentially automates many components of this potentially anxietyprovoking and physically demanding sequence of activities. The effectiveness of this automation, however, largely depends on how much it is trusted. In the conceptual model proposed by Lee and See [28] of the processes that could influence trust in automation, a critical factor is the degree to which the algorithms underlying automation are transparent to the user, a requirement which clearly becomes more challenging to achieve when the level of automation increases. Even intelligent aids such as parking assist are not likely to be used because of the uncertainty (how it works) and potential vulnerability (the damage that might be accrued if it is not working appropriately). Imposing on older drivers the need for this kind of knowledge in order to mitigate uncertainty related to the automation may be too unrealistic, but is an issue that may need to be resolved if older people are ultimately to trust autonomous vehicles. With regard to BSD, older drivers are known to have less effective useful fields of view (UFOVs; [4], which dictate how much information can be assimilated in a glance, and also have lower perceptual speed, which means that it could take them longer to establish a situational assessment when turning around to assess the traffic environment. Trust of BSD could potentially lead to reducing these demands on older drivers. At the same time, however, automation of BSD could contribute to older drivers’ workload by allocating attention to what the automation is doing—for example, when the automation indicates not to change lines when the driver may have desired to do so—as well as confirming that the automation is correct in its assessment. Trust is therefore a key construct as it could mediate the effectiveness of this human-automation partnership, especially if this AVT feature is tuned to be liberal in detection, perhaps for liability reasons, which runs the risk of high false alarm rates and thus for it being ignored or disabled. There is also concern, especially with older drivers, for invoking a startle response during the progression in the BSD process from a visual signal to an auditory alert if the turn signal is activated when there is something present in the blind spot, which would be contrary to auditory alarm design principles [42] and could induce confusion and unsafe behaviors.

4 Navigating and Using Infotainment Systems in Real Time Another source of potentially excessive demands on resources of attention is use of navigation and infotainment systems while driving. Expensive navigation systems, which also carry repeated outlays of costs for updating, are gradually becoming unnecessary for consumers with the introduction of Apple CarPlay and Android Auto features that enable vehicles to connect to smartphones and display and utilize important smartphone applications. These systems have been found in a younger adult sample (n = 24, average age = 25, age range = 21–36) to reduce workload relative to the infotainment/navigation systems commonly available from the original equipment manufacturers [40]. It may be risky, however, to generalize these results to older adult samples given their lesser familiarity with mobile apps.

Human Collaboration with Advanced Vehicle Technologies …

83

In any case, these features could still place considerable stress on multitasking performance, for example, speaking to the intelligent assistant (e.g., Siri) while driving. Perspectives based on multiple resources of attention theory [41, 42] imply that responding using voice, which constitutes the “verbal code,” draws on different resources or pools of attention than do manually guided responses comprising driving, which constitutes the “spatial code,” and thus should not be as mentally demanding as responding on two tasks that draw on the same resource—in this case, the same code. However, both these responses (giving verbal commands and driving) comprise late stage information processing as opposed to earlier stage (perceptual and central processing) information processing [43], and therefore could prove mentally overloading by competing for the same limited resources of attention available during this stage of information processing. In addition, because perceiving map guidance displays while listening to synthetic speech information both comprise verbal codes as well as an earlier (perceiving) information processing stage, these activities could be especially demanding for older drivers who generally drive less frequently (e.g., due to retirement) and thus are experiencing much less practice and fewer opportunities for skill acquisition and maintenance than their younger counterparts. The earlier recommendation noted by Liang et al. [29] for the need for relatively comprehensive hands-on experience is very relevant for this technological feature.

5 Knowledge and Learning About Vehicular Automation As emphasized, having knowledge about automation in vehicles is a critical consideration for all drivers, and especially older drivers. According to Hoff and Bashir’s [24] three-layered model of trust in automation (dispositional, situational, and learned), users first rely on “dispositional trust” (e.g., individual differences in propensity to trust and pre-existing knowledge such as attitudes/expectations, brand reputation, or experience with similar systems) when using a new automated system. Reliance on pre-existing knowledge that can contribute to false assumptions about how the automation works or how it accomplishes its tasks is the basis for a variety of human errors [37]. For example, the driver may incorrectly understand how the BSD feature becomes activated or deactivated, how sensitive it is to the speed of potentially passing cars from behind, or if the system has adequately considered the time it would take to make a lane change. Having knowledge of how automation works and experience interacting with it can be especially critical for older drivers as it could help them understand the logic underlying the automation’s actions and thus promote more accurate and comprehensive mental models concerning the operation of the AVTs. However, instilling older drivers with this knowledge may come at the risk of imposing added mental load: older drivers may now need to not only understand what automation is available and what it does to support their activities, but also understand how it works as a basis for deciding if the automation is performing incorrectly and thus may need to be “overruled.” Nonetheless, as with other technologies that older adults interact with,

84

J. Sharit et al.

they can be expected to acquire skills with practice and gradual familiarity with the technology [11, 14]. Unfortunately, most research studies on AVTs are not likely to capture this gradual accumulation of knowledge that occurs over longer durations in realistic scenarios. For example, a driver who is uncertain about how ACC works may choose a relatively innocuous context in which to activate this system and then divert extra attention to monitoring how the ACC responds to a vehicle that is being approached. These types of “trial-and-error” experiments are not unlike what many people, including workers, do to discover the parameters that govern the systems they are expected to operate, and are largely how drivers learn how to use the AVTs in their vehicle. Experimentation during driving, however, for the purpose of learning carries risks that many older drivers may not be willing to take. These “experiments” impose timesharing demands whereby the primary task of driving is shared with the secondary task of exploring the behavior of the AVT, which the driver may choose to do during an easy stretch of driving to minimize risk. An important question is whether normal age-related declines in attention capacity and slower reflexes, coupled with tendencies for being more averse to risk and possibly trusting of unfamiliar technology, place older drivers at a disadvantage when it comes to conducting such “controlled driving experiments” for the purpose of learning and skill acquisition of AVTs. Another important consideration is the possibly lesser opportunity for older drivers to learn and rehearse the learned experiences with AVTs. Many older drivers may not be commuting to work or using vehicles on an everyday basis, perhaps due to altered lifestyles. Older drivers may also not accurately perceive their diminishing driving abilities after outsourcing a proportion of their limited car trips to automation, and some older drivers wary of this loss of manual driving practice may be resistant to adopting more fully automated driving systems. A major risk would be the case for renting an unfamiliar vehicle during a vacation where the aging driver needs to contend with potential increased workload from vehicle unfamiliarity, AVT unfamiliarity, and route unfamiliarity.

6 Autonomous Driving Autonomous driving, where the role of the human is shifted from driver or monitor of automation to a passenger who in principle neither has to observe the automation nor the environment, corresponds to SAE levels 4 and 5 (Fig. 1), with most AV developers now focused on safely attaining L4 performance. Although there is evidence that older adults tend to be more concerned with the automation underlying autonomous driving vehicles than younger adults (e.g., [8]), not all studies have found this relationship (e.g., see the review by Golbabaei et al. [21]). Very little is currently known, however, about preferences and attitudes of people based on actual experiences of being driven by autonomous vehicles. In a recent study [31], both trust, based on the [25] scale, and physiological arousal, based on skin conductance, were examined in an exploratory field study in which 11 students

Human Collaboration with Advanced Vehicle Technologies …

85

experienced both the effects of being driven by a human and of being driven by an autonomous vehicle, in close to counterbalanced order, through a realistic route that included a country road, a roundabout, and merging into a motorway. The results indicated higher trust if a human was driving as compared to an ADS, In addition, increased physiological activity corresponding to increased arousal was observed during periods when the automation was controlling driving during specific potentially safety critical events, for instance, during merging onto the motorway. How such findings might translate to older drivers is currently unknown, but it would not appear unreasonable to assume that they would be at least as mistrustful of the automation relative to a human driver as compared to their younger driver counterparts. While there have been numerous tests highlighting the impressive capabilities of AVs, we are still well removed from everyday scenarios where these vehicles are communicating with one another, and it is difficult to foresee the kinds of situations that could emerge when issues confronting the automation of one or more AVs propagate over autonomous vehicular networks. Connected vehicle technologies and communication protocols being developed today between other vehicles (vehicleto-vehicle; V2V), infrastructure (vehicle-to-infrastructure; V2I), or other road users (vehicle-to-x; V2X) might help provide greater alignment and transparency into the actions of a network of AVs, but they themselves open up cybersecurity concerns (e.g., [2]) which vehicle occupants may have difficulty diagnosing. Another concern is for the proficiency with which takeover from an ADS can occur if needed. One of the well-known “ironies of automation” [3] is that when humans are suddenly thrust into the human-machine control loop due to the need for complete or partial takeover of the automated system, they are often ill-equipped to do so as a result of having been insulated from increasingly higher span-of-control processes that potentially interact with each other. In the absence of adequate training, people are not likely to diagnose and recover from AV automation failures or adverse scenarios deriving from the intersection of problems across the autonomous vehicular network. Because these situations may require rapid diagnosis and construction of appropriate mental models needed for supporting decision making, they can impose working memory demands that may be especially challenging for older adults experiencing normal age-related cognitive declines [34]. As [15] has noted, “disinvolvement can create more work.” The possible need for takeover of the vehicle may also be physically demanding for older adults because of age-related physiological changes [27]. Interestingly, a recent meta-analysis of takeover time (not to be confused with takeover quality) after vehicle automation failures found no clear effect of age [45], highlighting the volitionally-determined decision to resume vehicle control’s importance over “biological limitations” associated with aging. As with past research on response times to FCW alerts (e.g., [12, 26, 30]), despite recording slower simple reaction times in the lab, older drivers were just as quick to resume vehicle control as younger drivers. With this in mind, training for older drivers using L2–L3 automation that may require intervention should emphasize early detection of automation failures and the appropriate motor response when resuming manual control of the vehicle.

86

J. Sharit et al.

6.1 To Support or Replace the Older Driver with Automation To this point, we have discussed ADAS that increase the comfort and convenience of driving (e.g., ACC) or detect potential hazards and provide alerts to improve the driver’s own detection of them (e.g., BSD), but one important question to ponder is what role higher functioning AVTs might best play for aging drivers. Much discussion about current vehicle automation has oriented around semi-automated driver assistance systems that blur the SAE level distinctions (e.g., L2 systems like Tesla’s AutoPilot that are often relied on like L3 systems or the newly released “Full SelfDriving” beta test that is ostensibly a L3 system despite its name) and place the driver in a supervisory role that they will likely fail in if given enough time (e.g., [22]), or autonomous L4+ robo-taxi designs and concepts in which the occupant is merely a passenger and not expected to play any role in the supervision of the automation or vehicle control. Obviously, the robo-taxi option lends itself to helping older adults who have retired from driving maintain their community mobility, but issues of acceptance and trust determine its amount of uptake (once readily available), and issues around consumer education, interface design (e.g., scheduling, navigation, payment, etc.), and environmental support (e.g., vehicle ingress/egress, wayfinding once at destination, support for those with sensory, cognitive, or physical disabilities, etc.) will play key roles in how L4+ vehicle automation is utilized by older populations. Conversely, the option of L2 automated driver assistance or adaptive backup might pose several advantages over AVT that wholly replace the older driver. Toyota Research Institute has been developing its Guardian automated driving system that works in parallel with the driver, correcting only when the automation senses the human driver is going to make a catastrophic error [38]. Experimental work on AVTs similar to Guardian is limited, but a recent simulator study by Cabrall et al. [6] investigated the initial viability of different driver monitoring system (DMS) designs that would help make adaptive automated backup control for SAE L2 automation possible. Cabrall and colleagues found that all of the adaptive automated driving conditions investigated in their study significantly reduced the amount of time spent off-road when compared to conventional driving conditions, that an invisible-backup automated driving system was more difficult to over-rely on than a visible interface, and that context-based assessments of distraction (e.g., the DMS might ask the driver if they are looking away too much given the present traffic circumstances) might lessen the negative impacts of false alarms while increasing user satisfaction. As training wheels affixed to young children’s bicycles help them become more comfortable with balance and control, such adaptive automation might help keep older drivers safe while they can still drive themselves and give them enough experience with AVTs and familiarity with their performance to ease the transition away from manual driving if or when it comes for that individual.

Human Collaboration with Advanced Vehicle Technologies …

87

References 1. AAA Exchange (2017) Senior driver safety. https://exchange.aaa.com/safety/senior-driver-saf ety/#.XWR31OhKg2w 2. Amoozadeh M, Raghuramu A, Chuah CN, Ghosal D, Zhang HM, Rowe J, Levitt K (2015) Security vulnerabilities of connected vehicle streams and their impact on cooperative driving. IEEE Commun Mag 53(6):126–132 3. Bainbridge L (1987) Ironies of automation. In: Rasmussen J, Duncan K, Leplat J (eds) New technology and human error. Wiley, New York, pp 273–276 4. Ball KK, Beard BL, Roenker DL, Miller RL, Griggs DS (1988) Age and visual search: expanding the useful field of view. J Opt Soc Am, A, Opt Image Sci 5(12):2210–2219. https:// doi.org/10.1364/JOSAA.5.002210 5. Bianchi Piccinini GF, Rodrigues CM, Leitao M, Simoes A (2014) Reaction to a critical situation during driving with adaptive cruise control for users and non-users of the system. Saf Sci 72:116–126. https://doi.org/10.1016/j.ssci.2014.09.008 6. Cabrall CD, Stapel JC, Happee R, de Winter JC (2020) Redesigning today’s driving automation toward adaptive backup control with context-based and invisible interfaces. Hum Factors 62(2):211–228 7. Cahour B, Forzy JF (2009) Does projection into use improve trust and exploration? the case of a cruise control system. Saf Sci 47(9):1260–1272 8. Charness N, Yoon J-S, Souders D, Stothart C, Yehnert C (2018) Predictors of attitudes towards autonomous vehicles. Front Psychol 18:1–9. https://doi.org/10.3389/fpsyg.2018.02589 9. Classen S, Jeghers M, Morgan-Daniel J, Winter S, King L, Struckmeyer L (2019) Smart in-vehicle technologies and older drivers: a scoping review. OTJR: Occupat Particip Health 39(2):97–107. https://doi.org/10.1177/1539449219830376 10. Classen S, Mason J, Wersal J, Sisiopiku V, Rogers J (2020) Older drivers’ experience with automated vehicle technology: interim analysis of a demonstration study. Front Sustain Cities 2:27. https://doi.org/10.3389/frsc.2020.00027 11. Charness N (2009) Skill acquisition in older adults: psychological mechanisms. In: Czaja SJ, Sharit J (eds) Aging and work: issues and implications in a changing landscape. Johns Hopkins Press, Baltimore, MD, pp 231–258 12. Cotté N, Meyer J, Coughlin JF (2001) Older and younger drivers’ reliance on collision warning systems. In: Proceedings of the human factors and ergonomics society annual meeting 45(4):277–280 13. Craik FIM, Jacoby LL (1996) Aging and memory: Implications. aging and skilled performance: Advances in theory and applications. Lawrence Erlbaum Associates, Mahwah, NJ 14. Czaja SJ, Boot WR, Charness N, Rogers WA (2019) Designing for older adults: principles and creative human factors approaches. CRC Press, Boca Raton, FL 15. Dekker SWA (2005) Ten questions about human error: a new view of human factors and system safety. Lawrence Erlbaum Associates, Hahwah, NJ 16. Dixon L (2020) Transportation research interdisciplinary perspectives autonowashing : the greenwashing of vehicle automation. Transport Res Interdiscip Perspect 5:100113. https://doi. org/10.1016/j.trip.2020.100113 17. Dzindolet MT, Peterson SA, Pomranky RA, Pierce LG, Beck HP (2003) The role of trust in automation reliance. Int J Hum Comput Stud 58(6):697–718 18. Furlan AD, Kajaks T, Tiong M, Lavalliere M, Campos JL, Babineau J, Haghzare S, Ma T, Vrkljan B (2020) Advanced vehicle technologies and road safety: a scoping review of the evidence. Accident Analy Prevent 147. https://doi.org/10.1016/j.aap.2020.105741 19. Gish J, Vrkljan B, Grenier A, Van Miltenburg B (2017) Driving with advanced vehicle technology: a qualitative investigation of older drivers’ perceptions and motivations for use. Accid Anal Prev 106:498–504. https://doi.org/10.1016/j.aap.2016.06.027 20. Goddard K, Roudsari A, Wyatt JC (2012) Automation bias: a systematic review of frequency, effect mediators, and mitigators. J Am Med Inform Assoc 19(1):121–127

88

J. Sharit et al.

21. Golbabaei F, Yigitcanlar T, Paz A, Bunker J (2020) Individual predictors of autonomous vehicle public acceptance and intention to use: A systematic review of the literature. J Open Innov Technol Market, Compl 6:1–27. https://doi.org/10.3390/joitmc6040106 22. Greenlee ET, DeLucia PR, Newton DC (2018) Driver vigilance in automated vehicles: hazard detection failures are a matter of time. Hum Factors J Hum Factors Ergon Soc 60:465–476 23. Ho G, Kiff LM, Plocher T, Haigh KZ (2005) A model of trust and reliance of automation technology for older users. In: AAAI-2005 fall symposium: caring machines: AI in eldercare. November 4–6, 2005, Washington, DC, pp 45–50. AAAI Press, Menlo Park, CA. https://www. aaai.org/Papers/Symposia/Fall/2005/FS-05-02/FS05-02-008.pdf 24. Hoff KA, Bashir M (2015) Trust in automation: integrating empirical evidence on factors that influence trust. Hum Factors 57(3):407–434. https://doi.org/10.1177/0018720814547570 25. Jian J-Y, Bisantz AM, Drury CG (2000) Foundations for an empirically determined scale of trust in automated systems. Int J Cogn Ergon 4(1):53–71. https://doi.org/10.1207/S15327566 IJCE0401_04 26. Kramer AF, Cassavaugh N, Horrey WJ, Mayhugh JL (2007) Influence of age and proximity warning devices on collision avoidance in simulated driving. Hum Factors 49:935–949 27. Kroemer KHE (2009) Ergonomic design of workplaces for the aging population. In: Czaja SJ, Sharit J (eds) Aging and work: issues and implications in a changing landscape. Johns Hopkins University Press, Baltimore, MD, pp 126–143 28. Lee JD, See KA (2004) Trust in automation: designing for appropriate reliance. Hum Factors 46:50–80. https://doi.org/10.1518/hfes.46.1.50.30392 29. Liang D, Lau N, Baker SA, Antin JF (2020) Examining senior drivers’ attitudes toward driver assistance systems after naturalistic exposure. Innovat Aging 4(3). https://doi.org/10.1093/ger oni/igaa017 30. Maltz M, Shinar D (2004) Imperfect in-vehicle collision avoidance warning systems can aid drivers. Hum Factors 46(2):357–366 31. Mühl K, Strauch C, Grabmaeier C, Reithinger S, Huckauf A, Baumann M (2020) Get ready for being chauffeured: passenger’s preferences and trust while being driven by human and automation. Hum Factors 62(8):1322–1338. https://doi.org/10.1177/0018720819872983 32. Muir BM (1987) Trust between humans and machines, and the design of decision aids. Int J Man Mach Stud 27(5–6):527–539. https://doi.org/10.1016/S0020-7373(87)80013-5 33. Parasuraman R, Riley V (1997) Humans and automation: use, misuse, disuse, abuse. Hum Factors 39(2):230–253. https://doi.org/10.1518/001872097778543886 34. Park DC, Lautenschlager G, Hedden T, Davidson NS, Smith AD, Smith PK (2002) Models of visuospatial and verbal memory across the adult life span. Psychol Aging 17(2):299–320 PMID: 12061414 35. Rouse WB, Morris NM (1986) On looking into the black box: prospects and limits in the search for mental models. Psychol Bull 100(3):349–363. https://doi.org/10.1037/0033-2909. 100.3.349 36. SAE (2018) SAE international releases updated visual chart for its “levels of driving automation. Standard for Self-Driving Vehicles. https://www.sae.org/news/press-room/2018/12/saeinternational-releases-updated-visual-chart-for-its-%E2%80%9Clevels-of-driving-automa tion%E2%80%9D-standard-for-self-driving-vehicles 37. Sharit J (2006) Human error. In: Salvendy G (ed) Handbook of human factors and ergonomics, 3rd edn. John Wiley & Sons, New York, pp 708–760 38. Simonite T (2017) Toyota is working on software that could prevent drivers from making deadly mistakes on the road. MIT technology review, Business Insider. https://www.businessi nsider.com/toyota-guardian-software-could-protect-drivers-from-deadly-mistakes-2017-3 39. Souders DJ, Best R, Charness N (2017) Valuation of active blindspot detection systems by younger and older adults. Accid Anal Prev 106:505–514. https://doi.org/10.1016/j.aap.2016. 08.020 40. Strayer DL, Cooper JM et al (2019) Visual and cognitive demands of CarPlay, android auto, and five native infotainment systems. Hum Factors 61(8):1371–13–86. https://doi.org/10.1177/ 0018720819836575

Human Collaboration with Advanced Vehicle Technologies …

89

41. Wickens CD (2008) Multiple resources and mental workload. Hum Factors 50(3):449–455. https://doi.org/10.1518/001872008X288394 42. Wickens CD, Gordon Becker SE, Liu Y, Lee JD (2004) An introduction to human factors engineering, 2nd edn. Prentice-Hall, New York 43. Wickens CD, Carswell M (2006) Information processing. In: Salvendy G (ed) Handbook of human factors and ergonomics, 3rd edn. John Wiley & Sons, New York, pp 111–149 44. Zhan J, Porter MM, Polgar J, Vrkljan B (2013) Older drivers’ opinions of criteria that inform the cars they buy: a focus group study. Accid Anal Prev 61:281–287. https://doi.org/10.1016/ j.aap.2013.02.029 45. Zhang B, de Winter J, Varotto S, Happee R, Martens M (2019) Determinants of take-over time from automated driving: a meta-analysis of 129 studies. Transport Res F: Traffic Psychol Behav. https://doi.org/10.1016/j.trf.2019.04.020

Design for Inclusion and Aged Population in Transportation and Human-Automation Interaction Jimmy Onyedikachi Uba, Jessica Adanma Onwuzurike, Chidubem Nuela Enebechi, and Vincent G. Duffy

Abstract The proportion of the aged population is growing at a faster rate than any other population age group globally (Velkoff et al. 2006). This population’s workforce happens to be one of the largest in many parts of Africa, especially when compared to others worldwide (United Nations 2016). In most African countries, the aged population group takes a significant proportion of the total population, making them crucial to the African economy. However, less attention is being paid to this growing population in terms of their current challenges, especially in the aspect of health care. Information derived from the World Health Organization shows that 28–35% of the elderly fall every year globally; leading to injuries, injury-related disabilities, and death in the aged population (CDC 2014). This paper focuses on shedding light on the health challenges encountered by the older population in several parts of Africa and discusses how these challenges can be combated to enhance their general wellbeing. The approach used in this paper is a literature review from a variety of prestigious journals providing information about various health challenges encountered by the elderly in several countries in Africa. Some of these challenges include falls, mobility loss, vision, and cognitive impairment. From the information gathered from several literature reviews, it was discovered that the aged population would live better with the aid of technologies that are user-friendly to their specific geolocation. The use of J. O. Uba · J. A. Onwuzurike · C. N. Enebechi (B) · V. G. Duffy School of Industrial Engineering, Purdue University, West Lafayette, Indiana 47906, USA e-mail: [email protected] J. O. Uba e-mail: [email protected] J. A. Onwuzurike e-mail: [email protected] V. G. Duffy e-mail: [email protected] J. O. Uba Industrial Engineering and Management, Oklahoma State University, Stillwater, Oklahoma 74078, USA J. A. Onwuzurike Electrical Electronic Engineer, Nile University of Nigeria, Abuja, Nigeria © The Author(s), under exclusive license to Springer Nature Switzerland AG 2023 V. G. Duffy et al. (eds.), Human-Automation Interaction, Automation, Collaboration, & E-Services 11, https://doi.org/10.1007/978-3-031-10784-9_5

91

92

J. O. Uba et al.

modern technology like assistive technology and ergonomically designed accessories can all come in handy in serving as a source of assistance to the elderly population not just in African countries but also in several parts of the world. Given the limitations of technology in various parts of Africa, it can be concluded that more research on improving the aged population’s quality of life quality using design and technology has to be further explored. This will in turn help to enhance their health and retain them in the workforce, thus, increasing the economy of Africa as a whole. Keywords System design · Aged population · Inclusion · Africa

1 Introduction Recent breakthroughs in digital technology have led to more efficient patient care, ease of workflow, and improved public health. Since technology has led to great advancements in the global world at large, it is also essential to ensure that the technological systems being designed should be diverse and inclusive. For example, creating a technological device targeted toward individuals with visual impairment should also take into account the user-friendliness of the devices, especially if it would be used by the older population. This concept of design and inclusion should not only be incorporated in designing devices but also in every facet of the human race. A system is not effective enough if it cannot be efficiently used by humans of various diversity and demographics. On the basis of focusing on design and inclusion, the main aim of this paper is to shed light on the design and Inclusion of technological systems for the elderly in the African Continent. The African continent was specifically chosen because Africa is home to an immense number of developing nations that are constantly rising in population. With an increase in population comes more expectations and adverse changes that have to be made to accommodate the growing population. According to the Central Intelligence Agency, the age structure of a population affects the nation’s socio-economic status. For example, countries with young populations i.e. high percentage of the total population in the age group 15 and below need to invest more in schools while countries with older populations (high percentages 65 and above) need to invest more in the health sector.

1.1 Background According to the United Nations (2016), Africa has the largest aged population proportion in the world and this population is projected to increase from about 5% to 10% by 2050. Africa is the world’s second-largest most populous continent after Asia with a population of over 1.216 billion (UN 2019). A report by the United Nations stated

Design for Inclusion and Aged Population in Transportation …

93

that “Africa’s population has trebled from its estimated 478 million in 1980 to the current estimate if close to 1.2 billion and is projected to increase to 1.5 billion by 2025 and 2.4 billion by 2050” (UN 2019). Eight countries in Africa have at least 1 million people aged 60 and over. According to Velkoff et al. the size of older populations in many sub-Saharan African countries is roughly the same size as those in South Korea and Canada. Therefore, as the continent keeps growing, it is important to take into consideration the adequate ways to efficiently cater to the needs of the continuously rising population. The African countries selected for this paper all projected an increase in the aged population by 2050. The projected increase is determined to be more than twice their current sizes towards the end of 2050. This rise in population can be attributed to the increase and decrease of birth rate and mortality rate respectively [1–3] (United Nations 2015; United Nations 2012; Kagaba et al. 2003). The trend data in Fig. 1 shows that there hasn’t been much information and data available on the topic of design and inclusion for the aged population in the African continent. This paper brings attention to the current situation of health challenges encountered by the elderly in several parts of Africa and uses a systematic approach to discuss how technological infrastructures can be implemented in order to contribute to the advancement in the lives of the aged population in various parts of Africa. This was done through the selection of eight different countries in the African continent, namely; Nigeria, South Africa, Tunisia, Ethiopia, Kenya, Egypt, Botswana, and Rwanda (as shown in Fig. 2). These countries were strategically selected because their locations represented different regions of Africa from the North, South, East, West, and Central Africa.

Fig. 1 Trend graph data with keywords “design and inclusion” “aged population in Africa.” between the year 1998 up until 2020

94

J. O. Uba et al.

Fig. 2 North region: Egypt, Tunisia; South region: South Africa, Botswana; West region: Nigeria, East region: Rwanda, Kenya, Ethiopia (http://geoportal.icpac.net/layers/geonode%3Aafr_g2014_ 2013_0)

Table 1 presents the Author relationship, and Table 2 shows Leading Institutions of the keywords “design and Inclusion” and “aged population in Africa.” The data depicted in Table 2 was pulled from Harzing a software program used to gather and analyze metadata from academic citations during a search in Google Scholar. Table 1 shows the top five authors and publishers on this topic. Table 1 Author relationship table for keywords “design and inclusion” “aged population in Africa” Author name

Publisher

Year

EV Macagnano

witpress.com

2008

A Zins

Elsevier

2016

L Weill

emerald.com

2017

S Chikalipah

Citeseer

2005

HA Taylor Jr

witpress.com

2008

Data was retrieved from Google Scholar using Harzing’s software (https://harzing.com/resources/ publish-or-perish)

Design for Inclusion and Aged Population in Transportation …

95

Table 2 Leading Institutions with keywords “design and inclusion” “aged population in Africa” Institution

Scopus documents

Stellenbosch University

4

University of Witwatersrand

4

Karolinska Institute

3

Harvard T.H. Chan School of Public Health

3

Makerere University

3

Data were derived from the Scopus database (https://www.scopus.com/search/form.uri?display= basic)

The data in Table 2 were derived from Scopus. Scopus is one of the largest citation debases used to analyze and review literature. The information in Table 2 shows the top five institutions with the most relevant materials related to the main goal of this paper. One of the major challenges encountered by the elderly all over the world is health complications due to falling. As stated by Nikolaus and Bach in [4], “falls in older people are a leading cause of disability, distress, admission to intensive care and also death”. A report by the World Health Organization stated that 25–28% of the elderly fall every year globally, which has resulted in injuries, injury-related disabilities, and in some cases death (CDC 2014). A report by the World Health Organization stated that 25–28% of the elderly fall every year globally, which has resulted in injuries, injury-related disabilities, and in some cases death (CDC 2014). When designing a well-rounded and robust system, it is important to incorporate elements of inclusivity especially when designing systems that will be used by the elderly to purposefully address the issue of falls. The age considered as elderly in this paper is 65 and above. It is necessary to pay attention to this population because, in several parts of Africa, they contribute greatly to the economy, and thus neglecting this population would also mean neglecting potential positive impacts that could be implemented in the African community. It should also be noted that the African continent has been recorded to have the largest labor force participation of older individuals in the world. Furthermore, this population contributes to the economic status of not just their various countries, but the African continent at large [5], thus making them a critical aspect of Africa.

2 Methodology Eight Countries were selected from various regions of Africa (North, South, East, West, and Central Africa) to ensure every portion was being represented. Using the systems thinking approach, various factors regarding the aged population in the selected countries were thus examined. Access to viable information regarding

96

J. O. Uba et al.

the situation of the elderly population in the selected countries was also taken into consideration when picking out the specific countries. The health status of the elderly in the selected countries was highlighted to better understand the present state of the aged population before necessary measures can be proposed. Emphasis was laid on the availability of health care, numerous challenges that have caused severe injuries, disability, and death in the elderly. Furthermore, the impact of the challenges that are encountered by the aged population in the selected countries—as it affects the elderly involved, their family, and the economy at large—were elucidated. Based on the literature considered for this paper, current standards for the design and inclusion for the aged population in the selected countries to mitigate the elderly’s challenges were also examined in this paper. The goal is to determine the measures already in place to mitigate these challenges and propose standards for their improvement. Suggestions were made, with the application of ergonomics, on how these challenges can be alleviated or eliminated. Lastly, a bibliometric analysis was also carried out to show the relationship between the keywords “design,” “inclusion,” “aged population,” and “Africa”. The bibliometric analysis was done through software like VOSviewer and Hazing. The terms derived from the bibliometric analysis show how the keywords are entwined with each other; it also brings attention to other fields that can be further explored to enhance the current state of design and inclusion for the aged population in the African countries.

3 Results 3.1 Health Status of the Elderly Although it has been forecasted that there would be an increase in the aged population, less attention is still being paid to this essential population. An example of the apathy being shown to this population is the fact that African countries do not have hospices to take care of the elderly [6]. Despite South Africa having the most sophisticated and advanced medical care system than any other country in Africa, there is still little evidence of their preparedness to provide this medical care for their elderly (Kalula 2013). Even with the proportion of the aged population in this country, the coverage of health insurance is very low (22.9%)—especially for the blacks (6%) and mixedraced individuals (16.6%). Unlike the white population, about 73.5% of them have health insurance. The other populations, which are people from other nationalities, usually have to personally pay for private health care (Peltzer et al. 2012). Furthermore, just like every other selected country for the purpose of this paper, Rwanda does not have any record for having an organized attempt to provide medical

Design for Inclusion and Aged Population in Transportation …

97

services for the aged population (Ntagungira 2005). This situation has led to an escalation of health-related challenges in the aged population. Some of these challenges are:

3.1.1

Falls

Falls are the major cause of injury, injury-related disability, and death in the elderly not just in Africa but also in the world at large (Farombi 2018). As stated earlier, data from the World Health Organization (2016) suggests that 28–35% of the aged population (≥65 years) fall every year globally and this predominantly increases with age; for instance, in Nigeria, the elderly have an increased risk of falling and about 15% of them are likely to fall twice or more times annually (Farombi et al. 2018). 15% of the elderly in Egypt fall at least once a year [7, 8]. Unfortunately, as the population ages in Africa, the frequency of severe sickness, fragility, and disability increases, thus leading to a higher number of falls (Joubert et al. 2006). Wilunda et al. [9] remarked that falls have led to severe cases in the elderly in Kenya and various developing countries. These cases include various forms of impairment—mobility, vision, and cognitive impairment. In Ethiopia, falls have been the major cause of mobility impairment in the elderly, critically affecting their quality of life [10]. In Nigeria, it has been observed that females have a higher tendency than males of sustaining severe injuries from falls (Bekibele et al. 2009). Due to this, it is expedient that the concept of falls is studied especially in Africa.

Causes of Falls (i)

According to WHO (2016), environmental hazards contribute to the frequency of falls in the elderly. These hazards include poor lighting, unsuitable bed rails, bed heights, and wet floors due to urinary incontinence and other forms of spills. According to Ntagungira (2005), falls in the elderly can be attributed to visual impairments that were constituted due to little or no lighting at all, which makes it difficult for elderly persons to negotiate and identify the underlying obstacle. (ii) In Nigeria and most developing countries in Africa, there has been an increase in over-the-counter medication and polypharmacy, leading to an escalated risk of falls [11–14]. (iii) A deficiency of Vitamin D in the system of some elderly in South Africa has caused weakness in their bones, which has led to a higher frequency of falls [15] (WHO 2016; Joubert et al. 2006). (iv) According to Fastbom (2010), medical conditions like diabetes, urinary incontinence, and cardiovascular diseases have led to a major increase in falls. The intake of drugs used to treat these diseases can lead to a rise in the frequency of falls as a result of the physiological changes in the cardiovascular-morbidity and blood pressure-regulation system.

98

J. O. Uba et al.

(v) According to WHO (2016), there is currently no intervention to access and identify the factors to reduce falls in the older population in African countries, unlike in developed countries. Unfortunately, falls will continue to occur, especially in the elderly, until such an approach is made. 3.1.2

Mobility Loss

According to Alsnih et al. (2003), due to the correlation between age and mobility loss, an important consideration is catering to the needs of the elderly who experience mobility loss. An instance of this correlation is observed in South Africa, where mobility loss has affected the elderly in that it has been associated with visual impairment and falls, cognitive impairment, poor quality of life, and an exacerbated mental state [16]. Mobility loss is rampant in the elderly, who perform little or no exercise. In other words, an elderly person who spends a minimum of 150 min in a week doing moderate or light exercise is said to be physically active, thus, having a reduced risk of mobility loss (WHO 2011). Besides, neurological conditions have affected the motor-coordination and mobility of the elderly, leading to falls [17].

3.1.3

Vision Impairment

As stated by Flaxman et al. [18], about 80–90% of the world’s visually impaired elderly are found in low and middle-income countries like Ethiopia. According to the Botswana Health Statistics Unit (2014), these elderly persons have been diagnosed with various forms of vision impairment when compared to the younger ones. The form of vision impairment is majorly glaucoma and cataracts, critically affecting the vision of the diagnosed individual. As compared to all forms of causes of vision impairment, cataracts have been observed to be the major and most common eye condition, causing vision impairment across the African continent [19]. According to Abd-Allah [20], in Egypt, about 47.9% of the aged population has been observed to have low vision, with the major cause of it being cataracts (54.8%), corneal opacity (18.8%), refractive error (7%) and then glaucoma (4.6%).

Cataracts Cataracts are a leading cause of blindness, severe visual impairment, and falls in Africa, affecting an estimated half of the seven million blind people in Africa (Pascolini 2010). Moreover, cataracts are the cause of about 48% of blind individuals worldwide and 50–55% of individuals in Africa (Resnikoff 2004). In a bid to eliminate visual impairment, some African countries have initiated bodies responsible for preventing or assuaging the impacts of cataracts and other

Design for Inclusion and Aged Population in Transportation …

99

forms of visual impairment. For example, the South Africa Bureau for the Prevention of Blindness is responsible for providing cataract surgeries in the rural population in South Africa [21]. Despite the availability of this body, cataract surgeries are still not performed regularly on affected individuals, especially the elderly [19]. When the surgeries are performed, as reported by Kalua [19], they are often done by inexperienced physicians who do not carry out the surgeries efficiently. As a result of this, about 25% of the cataract surgeries performed in developing countries—of which most are found in Africa—end up having severe complications and oftentimes impaired vision [22].

3.1.4

Cognitive Impairment

Cognitive impairment has been the major cause of falls in the elderly, as it leads to the inability of the elderly, to remember the various health and safety measures currently in place to prevent falls. It has also caused disturbed balance and gait [23, 24]. In South Africa and other major parts of Africa, there has been a high level of cognitive impairment in the elderly, signifying that their health needs are currently not being met (Lehohla 2014). Cognitive impairment and decline also increases with age and can be compared to the rate of dementia reported in several African countries [25]. Dementia, a form of cognitive impairment, is highly prevalent in the world, affecting a high percentage of the aged population in developing countries [9]. Although it usually affects the elderly, it should be noted that it is not a normal condition [26]. About 32% of the aged population that do not suffer from dementia in Egypt have a mild cognitive impairment, of which hypertension, depression, and aging are its major associated factors (Amar 2012). According to Jager et al. [26], drugs used to treat dementia and various other forms of cognitive impairment are available in Africa. However, these drugs are unavailable in a large part of Africa and when they are available, they are very expensive for the middle-class.

3.2 Impact on the Elderly In cases where the challenges are prevalent, the elderly have had to improvise. For example, due to the lack of assistive technologies for the visually-impaired elderly in Nigeria, they have had to heavily rely on family and friends in identifying inclusive infrastructure (Okonji et al. 2018). Moreover, there has been a decline in the participation of the aging workforce, as a large percentage of the aged population are forced to stay at home and take care of the younger ones (Freitas et al. 2018).

100

J. O. Uba et al.

3.3 Current Measures to Mitigate Challenges Measures have currently been implemented in various countries in Africa. Although they are not sufficient and most times only a selected portion of the country gets to benefit from these measures, they have still been able to positively control the challenges of the elderly to an extent. They include:

3.3.1

Assistive Technology (AT)

Assistive devices or technology are rehabilitative, adaptive, and assistive devices that are utilized by people with impairments or disabilities to improve their current state. In other words, ATs are used to alleviate the liabilities caused by poor health and improve one’s quality of life. In Ethiopia, older adults who use assistive devices generally have balance and/or mobility problems due to muscle weakness and other comorbid health conditions, making them at risk of falling (Andersen et al. 2007). The assistive device used in a country is dependent on the extent to which the country is technologically inclined. For example; (a) In Botswana, to prevent falls caused by imbalance, of which hearing plays a huge part, a solar-powered hearing aid is being utilized by the elderly (Mayers 2013). (b) When it comes to mobility loss, in Nigeria and most countries in the African continent, canes, wheelchairs, crutches, and walking frames are being used as assistive devices for the elderly to aid their mobility [27]. (c) As regards visual impairment, eyeglasses or spectacles are being used by the elderly to reduce the effect of visual impairment in their daily living. Despite the importance of using assistive technology, many low- and middleincome countries in Africa have little or no access to it. In fact, in Africa, only about 5–15% of individuals who require assistive technology have access to them [28]. Oftentimes, these individuals with access are those from a particular class in the country. For instance, in South Africa, the upper class has access to assistive technologies and severe illness medication unlike the lower class (Lehohla 2014).

3.3.2

Medical Care

Although in recent years there has been no hospice for the elderly, private nursing homes are nascent in Tunisia, but the few available homes are small and play a limited role in making accommodations available and accessible (Formosa 2015). In South Africa, the aged individuals get free public medical care, and recipients who get social help from the state in the structure of a state annuity likewise get free admittance to secondary and tertiary consideration for advanced public medical

Design for Inclusion and Aged Population in Transportation …

101

care. However, the aged individuals dwelling in low-income communities still face difficulties getting to proper health care and may under-use medical care benefits or get deficient care (Peltzer et al. 2012). Based on the information, it is glaringly obvious that there is a need for a better approach and standard procedures to diminish the challenges of the elderly in Africa.

4 Discussion To retain the aged population in the labor force and enhance their daily life activities, proper attention has to be paid to their general wellbeing. “Some of the conditions that need to be highly considered are the elderly’s visual and cognitive health mobility and fall prevention to improve their general health.” The points below expand on what should be done in order to improve the health condition of the elderly.

4.1 Proposed Standards to Enhance Mobility Since some elderly people living in Africa are predicted to have fall-associated incidents [29], factors such as depression, cognitive impairment, unsafe environment, and inadequate nutrition should be managed to reduce the rate of falls in the elderly (Kalula et al. 2017). In 2004, Atoyebi et al. used information technology to predict and detect falls among the elderly. In Cairo, Egypt, Ismail et al. also implemented a health education program for the elderly, which enhanced their awareness and reduced the risk of falls in their homes [7].

4.2 Vision Impairment It is important to reduce the burden and accidents caused by visual impairment since it increases with age [30, 31]. To reduce this burden caused by visual impairments, assistive technology serves as a great tool for the elderly. Nkanga et al. [32] also show that investment in infrastructure, training of locals, and development of systems for early detection and treatment of several retinal diseases is needed in various African countries. As earlier stated in the results, cataracts have been observed to be one of the leading causes of visual impairment in several African countries and can be surgically corrected in most cases. Therefore, it is recommended to increase the frequency of cataract surgeries being carried out in regions with the rampant occurrence of cataracts. However, the rise in the frequency of cataract surgeries should not be the only criteria considered, but also that the medical procedures are efficiently carried out by individuals proficient in their field of medical expertise. An example can be

102

J. O. Uba et al.

seen in 2020 when Javalog et al., performed prolific cataract surgeries in a rural camp in Cameroon which was effective in improving the vision of participating individuals. Apart from cataract surgeries, it is essential that the elderly receive adequate information about cataracts. In the case of Egypt, Abd-Allah et al. conducted an educational program for the elderly who had visual impairment. This approach was successful as it improved their quality of life, knowledge, and reduced dependencies in their daily life activities [20].

4.3 Cognitive Impairments In a bid to reduce the effect of cognitive impairments in the elderly, strengthening education, social ties, and support systems is highly recommended. Furthermore, higher income and increased work efforts have been said to improve the cognitive wellbeing of rural dwellers in some African countries [33]. In 2006, Gamberini et al. used technology in the form of games called Eldergames which was a European-funded project that has been used to improve the cognitive function and social skills of the elderly. More research needs to be done on the impacts of incorporating games to aid the elderly in terms of cognition, especially with the world becoming more technology inclined.

4.4 Ergonomics There is a need for adequate exercise programs, an example of such can be the exergames- which are games with motion sensors and remote control. This game requires the players to move while playing the game [34]. The game is meant for different ranges of the aged population as it reduces the risk of falling. Apart from relying on family members and support systems, the elderly can also rely on medical expertise. For example, Physiotherapists are equipped with knowledge and skills to identify, prevent, and respond to the situation if falls occur. So, including physiotherapists in the designing of technologies can positively contribute to the older population’s predicament in cases involving falls. Environmental hazards can be eliminated through the installation of ergonomically designed accessories. For example, considering anthropometry in the height of the bed, toilet seats, staircase (design of inclusion), etc.

5 Bibliometric Analysis A bibliometric analysis was conducted using the same steps carried out by Enebechi et al. The first step in the analysis was to perform a search in order to produce a

Design for Inclusion and Aged Population in Transportation …

103

Fig. 3 VOSviewer search with keywords “design” and “aged population in Africa” (https://www. vosviewer.com/)

database of papers, this was done through the Web of Science platform. The database used for the bibliometric analysis was produced by using specific search terms on the Web of Science. The terms used for the search were “technology”, “design”, “inclusion”, “aged population”, and “Africa.” These terms were used interchangeably to achieve the most efficient result. A file with metadata was generated from Web of Science, the file was then inserted into the Harzing VOSviewer software to create a network of repeated terms that show how the terms are all connected. The result from the analysis is indicated in Fig. 3, about 54 papers were used for this search. The minimum occurrence of a term was set to five, of the 3267 in the results, only 232 terms met this threshold. Some of the terms derived from the analysis were “education”, “healthcare”, “intervention”, “recommendation”, “outcome” and “prevalence”. These terms show how important it is to find a point where the world of design and inclusion intersects with the needs of the aged population in Africa (Fig. 4).

6 Limitations This paper discusses steps to further dive into the inclusion of the needs of elderly people in the African continent when it comes to technology design. The health challenges associated with the elderly are numerous and not all health problems were considered in this study. The data available for the wellbeing of the older population in several African countries is limited [35]. Information regarding the

104

J. O. Uba et al.

Fig. 4 VOSviewer search with keywords “aged population” and “Africa” (https://www.vosviewer. com/)

elderly was hard to find because a lot of countries in Africa don’t take the health of the aged population seriously. Implementing some of the concepts recommended in this paper might not practically apply to all areas of the African continent since not all African countries are technologically advanced [36].

7 Future Work 7.1 Assistive Technology There is a need for African countries to investigate the current status of the aged population in regards to the affordability and accessibility to different types of ATs in various communities. This is because there is currently no research evidence stating the extent to which these individuals have access to affordable ATs [37, 38]. Analysis of review findings in Marasinghe et al. [28] shows that to increase the current availability of ATs for aging populations in low-income and middle-income countries there is a need for inclusive health and social system. In a study carried

Design for Inclusion and Aged Population in Transportation …

105

out by Rohwerder [39], it was discovered that in developing countries the following factors contribute to the lack of access to assistive devices. 1. 2. 3. 4. 5.

High costs of assistive devices Limited availability of such devices in communities Inadequate awareness of current technologies Insufficiently trained personnel Indifferent government.

Due to this, it is expedient that the factors highlighted above have to be considered and rectified in a bid to ensure ATs are provided for the elderly in Africa. An approach to include value engineering and material science should be considered to enable ATs to be produced at a cheaper cost. Assistive devices for physical and sensory impairment—such as grab bars placed next to showers and other forms of home modifications—should be considered in homes for aged individuals. Furthermore, the aspect of anthropometry should be considered in the design of various equipment in the homes of the elderly. For instance, when designing their beds and staircases, the appropriate height that would not be hazardous to them should be determined and implemented. Since a number of the aged population spend lots of time at home or in aging facilities, there is a need to make adjustments to basic home infrastructures and household equipment to reduce the risk of falls, improve their mobility and assist with visual and cognitive activities. The implementation of such adjustments can be achieved using the concept of smart homes. Smart homes for the aged population consist of assistive devices and technologies that enable them to carry out their daily activities. Some of these technologies include fall prevention and identification systems. Smart homes should be tailored to meet the unique health challenges and needs of different individuals in Africa.

8 Conclusion As the aged population in Africa increases there is a need for ergonomic technological designs to accommodate this rising population. There is still no proper or organized attention being given to the elderly. This insouciance has led to an increase in healthrelated challenges some of which include loss of mobility, vision, cognition, and fall incidents. The main goal of this paper was to elucidate the inclusion of the aged population in technologies and design in Africa. To achieve this goal, 8 countries from different regions of Africa were selected. The current health status, the impact of the challenges faced and ways to improve the quality of living of the elderly were the various factors considered. As stated in the earlier sections of this paper, falls can be linked to cognitive and visual impairment, mobility loss, and lead to injury, reduced mobility, or even death. Environmental hazards, polypharmacy, malnutrition, and some medical conditions

106

J. O. Uba et al.

such as urinary incontinence, cardiovascular diseases, and diabetes increase the risk of falling among the elderly. Mobility loss is very common in the elderly especially those that don’t exercise and catering to their needs is an important fact that should be evaluated. Cataracts are one of the major causes of visual impairment in the elderly in Africa. Dementia, a form of cognitive impairment, is also common in the aged population. The aged population has encountered challenges that have caused a decrease in their quality of life and participation in the workforce. Technology can be used to enhance the living conditions of all age groups including the elderly. Technologies that identify environmental hazards, prevent falls, improve mobility, cognition, and vision of the aged population in numerous parts of Africa are critical in mitigating their health challenges. Health education and proper medical care can improve the well-being of the elderly and prevent some of the health challenges. Future work should include ergonomic designs for the aged population in Africa considering the different educational, social, and economic statuses of the region.

References 1. El Moselhy EA (2016) Aging: the current situation globally and in Egypt. J Gerontol Geriatr Res 5(5):7182. https://doi.org/10.4172/2167-7182.1000e141 2. Charles A, Prb T, Aau I PS, Hailemariam A (2009) Population dynamics, food/nutrition security and health in Ethiopia: delicate balance of vulnerability and resilience. Health (San Francisco), pp 1–6 3. Oyesiku KO (2016) Analysis of mobility pattern and challenges of transportation needs of the elderly in a fast-growing city in Nigeria. Adv Soc Sci Res J 3(3):141–154. https://doi.org/10. 14738/assrj.33.1812 4. Nikolaus T, Bach M (2003) Preventing falls in community-dwelling frail older people using a home intervention team (HIT): results from the randomized falls-HIT trial. J Am Geriatr Soc 51(3):300–305. https://doi.org/10.1046/j.1532-5415.2003.51102.x 5. Douglass R (2016) The aging of Africa: challenges to African development. Afr J Food Agric Nutr Dev 16(1):1–15 6. Jean Pierre PKB (2015) Problematic of aging and managing the elderly in Africa. Anthropology 02(05):1000142. https://doi.org/10.4172/2332-0915.1000142 7. Ismail G, Fahim H, Bakr I, Wassif G, Hamza S (2018) Risk of falls and effect of a health education program in prevention of falls among elderly in geriatric homes in Cairo, Egypt. Egypt J Geriatr Gerontol 5(2):1–7. https://doi.org/10.21608/ejgg.2018.30902 8. Kamel MH, Abdulmajeed AA, Ismail SES (2013) Risk factors of falls among elderly living in urban Suez—Egypt. Pan Afr Med J 14:1–7. https://doi.org/10.11604/pamj.2013.14.26.1609 9. Wilunda B, Ng N, Stewart Williams J (2015) Health and ageing in Nairobi’s informal settlements-evidence from the international network for the demographic evaluation of populations and their health (INDEPTH): a cross sectional study Global health. BMC Public Health 15(1):1–11. https://doi.org/10.1186/s12889-015-2556-x 10. Stewart Williams J, Kowal P, Hestekin H, O’Driscoll T, Peltzer K, Yawson A, Biritwum R, Maximova T, Salinas Rodríguez A, Manrique Espinoza B, Wu F, Arokiasamy P, Chatterji S (2015) Prevalence, risk factors and disability associated with fall-related injury in older adults in low- and middle-income countries: results from the WHO Study on global AGEing and adult health (SAGE). BMC Med 13(1):1–12. https://doi.org/10.1186/s12916-015-0390-8

Design for Inclusion and Aged Population in Transportation …

107

11. Huang AR, Mallet L, Rochefort CM, Eguale T, Buckeridge DL, Tamblyn R (2012) Medicationrelated falls in the elderly: causative factors and preventive strategies. Drugs Aging 29(5):359– 376. https://doi.org/10.2165/11599460-000000000-00000 12. Woolcott (2010) Meta-analysis of the impact of 9 medication classes on falls in elderly persons. (Arch Internal Med (2009) 169(21):1952–1960). Arch Internal Med 170(5):477. https://doi. org/10.1001/archinternmed.2009.510 13. Tinetti ME, Han L, Lee DSH, McAvay GJ, Peduzzi P, Gross CP, Zhou B, Lin H (2014) Antihypertensive medications and serious fall injuries in a nationally representative sample of older adults. JAMA Intern Med 174(4):588–595. https://doi.org/10.1001/jamainternmed.2013. 14764 14. Gillespie LD, Robertson MC, Gillespie WJ, Sherrington C, Gates S, Clemson LM et al (2012) Cochranereviewfalls (1). Coch Database Syst Rev 9 15. de Villiers L, Kalula SZ (2015) An approach to balance problems and falls in elderly persons. S Afr Med J 105(8):695. https://doi.org/10.7196/SAMJnew.8037 16. Swenor BK, Simonsick EM, Ferrucci L, Newman AB, Rubin S, Wilson V (2015) Visual impairment and incident mobility limitations: the health, aging and body composition study. J Am Geriatr Soc 63(1):46–54. https://doi.org/10.1111/jgs.13183 17. Akande-Sholabi W, Ogundipe FS, Adebusoye LA (2020) Medications and the risk of falls among older people in a geriatric centre in Nigeria: a cross-sectional study. Int J Clin Pharm. https://doi.org/10.1007/s11096-020-01140-y 18. Flaxman SR, Bourne RRA, Resnikoff S, Ackland P, Braithwaite T, Cicinelli MV, Das A, Jonas JB, Keeffe J, Kempen J, Leasher J, Limburg H, Naidoo K, Pesudovs K, Silvester A, Stevens GA, Tahhan N, Wong T, Taylor H, Zheng Y (2017) Global causes of blindness and distance vision impairment 1990–2020: a systematic review and meta-analysis. Lancet Glob Health 5(12):e1221–e1234. https://doi.org/10.1016/S2214-109X(17)30393-5 19. Kalua K, Lewallen S, Courtright P (2013) Update on cataract and its management in Africa. Expert Rev Ophthalmol 8(3):297–302. https://doi.org/10.1586/eop.13.11 20. Abd-Allah ES, Zahra SBMG, El-Seoud ARA (2018) Educational program to improve quality of life in elderly patients with visual impairment. Saudi J Nurs Health Care 7921:307–311 21. Kluever H (2006) National-level outreach: South African Bureau for the prevention of blindness. Community Eye Health J 19(58):27–28 22. Yorston D (2008) Cataract complications. Community Eye Health J 21(65):1–3 23. de Ruiter SC, de Jonghe JFM, Jansen RWMM, Germans T, Ruiter JH (2017) Cognitive impairment is very common in elderly patients with syncope and unexplained falls. J Am Med Dir Assoc 18(5):409–413. https://doi.org/10.1016/j.jamda.2016.11.012 24. Segev-Jacubovski O, Herman T, Yogev-Seligmann G, Mirelman A, Giladi N, Hausdorff JM (2011) The interplay between gait, falls and cognition: can cognitive therapy reduce fall risk? Expert Rev Neurother 11(7):1057–1075. https://doi.org/10.1586/ern.11.69 25. Kobayashi LC, Mateen FJ, Montana L, Wagner RG, Kahn K, Tollman SM, Berkman LF (2019) Cognitive function and impairment in older, rural south african adults: evidence from “health and aging in Africa: a longitudinal study of an INDEPTH Community in Rural South Africa.” Neuroepidemiology 52(1–2):32–40. https://doi.org/10.1159/000493483 26. de Jager CA, Joska JA, Hoffman M, Borochowitz KE, Combrinck MI (2015) Dementia in rural South Africa: a pressing need for epidemiological studies. S Afr Med J 105(3):189–190. https://doi.org/10.7196/SAMJ.8904 27. Bekele GT, Allene MD, Getnet MG, Hunegnaw MT, Janakiraman B (2020) Assessing falls risk and associated factors among urban community dwellers older adults in Gondar town, Northwest Ethiopia 2019: a cross sectional study. Int J Surg Open 24:177–184. https://doi.org/ 10.1016/j.ijso.2020.06.002 28. Marasinghe KM, Lapitan JM, Ross A (2015) Assistive technologies for ageing populations in six low-income and middle-income countries: a systematic review. BMJ Innovations 1(4):182– 195. https://doi.org/10.1136/bmjinnov-2015-000065 29. Khater MS, Mousa SM (2012) Predicting falls among Egyptian nursing home residents: a 1-year longitudinal study. J Clin Gerontol Geriatr 3(2):73–76. https://doi.org/10.1016/j.jcgg. 2012.04.005

108

J. O. Uba et al.

30. Umfress AC, Brantley MA (2016) Eye care disparities and health-related consequences in elderly patients with age-related eye disease. Semin Ophthalmol 31(4):432–438. https://doi. org/10.3109/08820538.2016.1154171 31. Mashige KP, Ramklass SS (2020) Prevalence and causes of visual impairment among older persons living in low-income old age homes in Durban, South Africa. Afr J Primary Health Care Family Med 12(1):1–7. https://doi.org/10.4102/PHCFM.V12I1.2159 32. Nkanga D, Adenuga O, Okonkwo O, Ovienria W, Ibanga A, Agweye C, Oyekunle I, Akanbi T (2020) Profile, visual presentation and burden of retinal diseases seen in ophthalmic clinics in sub-saharan Africa. Clin Ophthalmol 14:679–687. https://doi.org/10.2147/OPTH.S226494 33. Payne CF, Kohler IV, Bandawe C, Lawler K, Kohler HP (2018) Cognition, health, and wellbeing in a rural Sub-Saharan African population. Eur J Popul 34(4):637–662. https://doi.org/ 10.1007/s10680-017-9445-1 34. Harrington CN, Hartley JQ, Mitzner TL, Rogers WA (2015) Assessing older adults’ usability challenges using Kinect-based exergames. Lecture notes in computer science (including subseries lecture notes in artificial intelligence and lecture notes in bioinformatics) 9194:488– 499. https://doi.org/10.1007/978-3-319-20913-5_45 35. Audain K, Carr M, Dikmen D, Zotor F, Ellahi B (2017) Exploring the health status of older persons in Sub-Saharan Africa. In: Proceedings of the nutrition society, May 2017, pp 1–6. https://doi.org/10.1017/S0029665117000398 36. Haftu GG (2019) Information communications technology and economic growth in SubSaharan Africa: a panel data approach. Telecommun Policy 43(1):88–99. https://doi.org/10. 1016/j.telpol.2018.03.010 37. Matter RA, Eide AH (2018) Access to assistive technology in two Southern African countries. BMC Health Serv Res 18(1):1–10. https://doi.org/10.1186/s12913-018-3605-9 38. Mji G, Edusei A (2019) An introduction to a special issue on the role of assistive technology in social inclusion of persons with disabilities in Africa: outcome of the fifth African network for evidence-to-action in disability conference. Afr. J. Disabil 8(November):1–4. https://doi. org/10.4102/ajod.v8i0.681 39. Rohwerder B (2018) Assistive technologies in developing countries. Helpdesk report K4D: knowledge, evidence, and learning for development. https://opendocs.ids.ac.uk/opendocs/bit stream/handle/123456789/13599/Assistive_technologies_in_developing_countries.pdf?seq uence=1&isAllowed=y 40. Dobriansky PJ, Suzman RM, Hodes RJ (2007) Why population aging matters—a global perspective. US Department of State, 1–32. papers2://publication/uuid/4B8865DB-5866– 4285-A74D-168F45ED1109 41. Wasfi R, Levinson D, El-geneidy A (2012) B t p s. 6, 8–32 42. Statistics South Africa (2011) Census 2011: profile of older persons in South Africa. http:// www.statssa.gov.za/publications/Report-03-01-60/Report-03-01-602011.pdf 43. Amer M, Mousa S, Khater M, Wahab WA (2012) Prevalence of mild cognitive impairment among older adults living in Mansoura city Egypt. Middle East Current Psychiatry 19(1):3–7. https://doi.org/10.1097/01.XME.0000407821.18381.3c 44. Murphy M (2018) Ageing in sub-Saharan Africa in the context of global development: the multiple indicator survey project (MISA), pp 1–60 45. Kechaou I, Cherif E, Sana BS, Boukhris I, Hassine LB (2019) Traumatic and psychosocial complications of falls in the elderly in tunisia. Pan Afr. Med. J. 32:1–9. https://doi.org/10. 11604/pamj.2019.32.92.16667 46. Community-Dwelling Elderly People in Selected Districts (2005) 47. Lau TM (2019) An inclusive design guideline for designing elderly friendly smart home control device. http://hdl.handle.net/10415/6849 48. Kanyoni M, Phillips JS (2009) Factors associated with physical activity levels among older adults in selected institutions in Rwanda. Jchs 4(1):8–14 49. Roy N, Dubé R, Després C, Freitas A, Légaré F (2018) Choosing between staying at home or moving: a systematic review of factors influencing housing decisions among frail older adults. PLoS ONE 13(1). https://doi.org/10.1371/journal.pone.0189266

Design for Inclusion and Aged Population in Transportation …

109

50. Javaloy J, Signes-Soler I, Moya T, Litila S (2020) Cataract surgery in surgical camps: outcomes in a rural area of Cameroon. Int Ophthalmol 5. https://doi.org/10.1007/s10792-020-01580-5 51. Holzinger A, Ziefle M, Röcker C (2010) Human-computer interaction and usability engineering for elderly (HCI4AGING): introduction to the special thematic session. Lecture notes in computer science (including subseries lecture notes in artificial intelligence and lecture notes in bioinformatics), 6180 LNCS (PART 2), pp 556–559. https://doi.org/10.1007/978-3642-14100-3_83 52. Mannheim I, Schwartz E, Xi W, Buttigieg SC, McDonnell-Naughton M, Wouters EJM, van Zaalen Y (2019) Inclusion of older adults in the research and design of digital technology. Int J Environ Res Public Health 16(19):1–17. https://doi.org/10.3390/ijerph16193718 53. Ridley H (1967) Eye Disease. BMJ 3(5567):728–728. https://doi.org/10.1136/bmj.3.5567. 728-a 54. Gamberini L, Alcaniz M, Barresi G, Fabregat M, Ibanez F, Prontu L (2006) Cognition, technology and games for the elderly: an introduction to ELDERGAMES project. Psych Nology J 4(3):285–308 55. Steyn K, Fourie Jean TN (2016) ISBN: 1-920014-40-3 56. Aging in Sub-Saharan Africa (2006) In aging in sub-Saharan Africa. https://doi.org/10.17226/ 11708 57. Atoyebi OA, Stewart A, Sampson J (2015) Use of information technology for falls detection and prevention in the elderly. Ageing Int 40(3):277–299. https://doi.org/10.1007/s12126-0149204-0 58. Nabalamba A, Chikoko M (2009) Aging population challenges in Africa. Afr Res Bull Econ Financ Tech Ser 46(6):18316A-18316B 59. WHO (2005) A WHO global report on falls among older persons prevention of falls in older persons: Africa case study, pp 1–31 60. Phaswana-Mafuya N, Schneider M, Makiwane M, Zuma K, Ramlagan S, Tabane C (2012) Study of global ageing and adult health (SAGE), 2011 61. Smith R, Turpin M (2017) Design science research and activity theory in ICT4D: developing a socially relevant ICT platform for elderly women in remote rural South Africa. IFIP Adv Inf Commun Technol 504:345–356. https://doi.org/10.1007/978-3-319-59111-7_29 62. Kim DH (2018) Crossing the bridge from research to clinical: possibility to use deep neural network for the individual brain modellings. J Spine Neurosurg 07:9701. https://doi.org/10. 4172/2325-9701-c6-026 63. Okonji PE, Ogwezzy DC (2018) Rehabilitation for independent living: challenges and priorities of visually impaired older people in urban Nigeria. J Niger Optom Assoc 32(3):45–54 64. Marasinghe KM, Lapitan JM, Ross A (2015) Assistive technologies for ageing populations in six low-income and middle-income countries: a systematic review. BMJ Innovations 1(4):182– 195. https://doi.org/10.1136/bmjinnov-2015-000065 65. Pascolini D, Mariotti SP (2012) Global estimates of visual impairment: 2010. Br J Ophthalmol 96(5):614–618. https://doi.org/10.1136/bjophthalmol-2011-300539 66. Odufuwa BO (2006) Enhancing mobility of the elderly in Sub-Saharan Africa cities through improved public transportation. IATSS Res 30(1):60–66. https://doi.org/10.1016/s0386-111 2(14)60156-4 67. VOSviewer (https://www.vosviewer.com/) 68. Web of Science (http://login.webofknowledge.com) 69. Fahimnia B et al (2015) Green supply chain management: a review and bibliometric analysis. Int J Prod Econ 162(C:101–114. https://doi.org/10.1016/j.ijpe.2015.01.003 70. Field Listing: Age structure—The World Factbook—Central Intelligence Agency (n.d.) Retrieved 14 Dec 2020, from https://www.cia.gov/library/publications/the-world-factbook/fie lds/341.html 71. DeSA U (2013) World population prospects: the 2012 revision. Population division of the department of economic and …. http://scholar.google.com/scholar?hl=en&btnG=Search&q= intitle:World+Population+Prospects+The+2012+Revision#1

110

J. O. Uba et al.

72. In Botswana, solar-powered hearing aids uplift hearing impaired | ZDNet (n.d.) Retrieved 14 Dec 2020, from https://www.zdnet.com/article/in-botswana-solar-powered-hearing-aidsuplift-hearing-impaired/ 73. Geographic Information System (GIS) https://uhcl.libguides.com/gis/download

Utilizing Bibliometric Analysis Tools to Investigate Automation Surprises in Flight Automation Systems Evan Barnell

Abstract The use of flight automation systems such as autopilot has proliferated in recent years with the advent and development of new technology that promises to be safe, efficient, and easy to fly. Because of the inherent complexity of modern airframes, these systems are commensurately sophisticated. Unless a pilot spends an inordinate amount of effort and time to fully apprehend the theory and implementation of an autopilot system, there will always be actions that it will take that will not seem intuitive or even comprehensible to the pilot. Conversely, humans often make irrational decisions that contemporary computers can not decipher. This all leads to the potential of the systems performing some unexpected action, termed an “Automation Surprise.” Because of the depth and complexity of research performed related to this topic, this report utilizes various content and bibliometric analysis tools to home in on proper conclusions from large amounts of data that is not manually feasible. Keywords Autopilot · Automation · Surprise · Pilot

1 Introduction and Background My project investigates the use of autopilot systems in aviation and what happens when they do not work as planned. When the automation systems of an aircraft do not interpret inputs from pilots in an expected manner, the result is often an “automation surprise.” This could be due to many different elements to consider, from a physical software or hardware glitch directly changing the intended signal from the pilot to the computer or from the computer to the plane, to the computer misinterpreting the pilot’s intentions or even the pilot’s lack of understanding of the operation, intentions or capabilities of the autopilot systems.

E. Barnell (B) Purdue University, West Lafayette, IN 47906, USA e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Switzerland AG 2023 V. G. Duffy et al. (eds.), Human-Automation Interaction, Automation, Collaboration, & E-Services 11, https://doi.org/10.1007/978-3-031-10784-9_6

111

112

E. Barnell

This report seeks to utilize various analysis tools to characterize the selected topic. Under the category title of “Human-Automation Interaction” and the theme “Manufacturing, Services, and User Experience,” this report investigates the causes and effects of “Automation Surprises” in the context of automation systems in aviation. This topic is related to safety engineering because commercial aviation is one of the most highly regulated forms of transportation, with many high-profile accidents being caused by human error or electronic/mechanical system failure. This report seeks to understand just what happens when a pilot’s expectations and the computer’s operation do not match in addition to how we might work towards improvement. Additionally, this investigation is directly related to Human-Automation Interaction as each new generation of commercial planes includes more and more automation systems that have become incredibly complex and fully integrated with the airframes. I was unable to find sufficient proof of research by the designers that they can perform their functions with minimal chance of becoming a hindrance in a time of need. There are multiple demonstrable and beneficial uses for in-flight automation systems. Thanks to systems like these, cockpits in commercial and military aviation used to have an individual pilot, copilot, flight engineer, and navigator. Nowadays only a pilot and copilot are necessary thanks to the incorporation of such systems. They can take excessive workload off of pilots to allow them to focus on the more important tasks at hand that computers are not yet ready to handle independently. Multiple bibliometric analysis tools were utilized to demonstrate their effectiveness. This content analysis allows easy identification of trends and frequent authors and terms. Having such a clear analysis allows me to effectively explore my chosen topic of automation surprises in autopilot systems. There is an upward trend in research with more and more applications are being discovered year by year.

2 Problem Statement Because of the prominence of flight automation systems in the news and aviation industry today, as well as their complexity, ensuring safety and ease of operation are critical to both increasing confidence in the systems and preventing accidents with the potential for significant negative impacts. For this reason, it is essential to perform analysis into the contemporary state of research on automation surprises to better understand how best to avoid and/or reduce them.

3 Bibliometric Analysis Procedures and Discussion Article Selection An objective of a proper literature review during your research is to choose those that are most related to what you wish to study, but when there are hundreds or thousands

Utilizing Bibliometric Analysis Tools …

113

of articles, which do you choose? The optimal publication could be well cited, has many co-citation reference bursts, and directly relates to your topic. That is often very rare, so being able to narrow down which you read is important. All these resources were chosen with the assistance of at least one of these bibliometric tools. It helped to determine which publications were the most relevant, impactful, and influential today. Using the Purdue Library database access, I sourced my articles from various databases or combinations thereof, including SpringerLink, ResearchGate, Google Scholar, assigned textbooks, Harzing, Scopus, and Web of Science. Limitations of using these tools include that there often is not much data out there on extremely recent publications.

3.1 Harzing’s Publish or Perish My first step was to begin a search of related keywords to my topic using the search tool “Harzing’s Publish or Perish.” The keywords I selected were “autopilot,” “automation,” “flight” and “surprise” to best align the results with my selected topic. This analysis tool utilizes the Google Scholar database and produces documents of all types that contain the entered keywords. I searched only articles that contained all three keywords to help ensure the applicability of the results. I limited the search to the first 400 results as the inquiry processing was extremely time-intensive and I felt this quantity was sufficient for the subsequent data analysis. This tool merely serves to acquire and consolidate search data for further analysis. The first step is to select one of the data source options as seen in Fig. 1. I used “Google Scholar” as it has a broad and developed database of documents. I then saved these search results (seen in Fig. 2) as a Web of Science text file and imported the data into VOSviewer to proceed with the next analysis step. There is little analysis to be done using this step as it only collects a lot of information for further analysis; it is, however, critical and some following tools will not provide meaningful results without it.

Fig. 1 A list of all available data sources for search selection

114

E. Barnell

Fig. 2 Screenshot of search results

3.2 VOSviewer VOSviewer allows the visualization of authors into a network. This aids in determining the connections between authors that may have the most applicable writings on the topic. There are many settings screens to navigate through (like Fig. 3), but I omit most of these in the interest of brevity. The first step in using VOSviewer is to create a new map using the menu on the lefthand side of the program. Then, you must create a map based on bibliographic data (derived from Harzing) as shown above which will import information such as authorship, dates, and publisher. The data output by Harzing will be formatted as a Web of Science (WoS) text file so you must select to read the data from bibliographic database files to have it properly imported and analyzed. You must then locate the saved WoS file from Harzing to enter it into VOSviewer. There are many options available to modify the results, but I used the default settings here. I included all authors from the 400 incorporated documents by setting the minimum number of documents of an author to 1. I did not limit the number of authors included in the inquiry and retained all 561 authors to provide the maximum depth of analysis. You are also able to select and deselect specific authors, but I included all authors present for this analysis. Figure 4 image shows the final output of VOSviewer in the form of network connections. The 38 most prevalent authors in this topic are shown as a different node in the network with the lines connecting each author representing co-authorship that can be used to explore the interactions between authors. It is relatively easy to see

Utilizing Bibliometric Analysis Tools …

115

Fig. 3 Setting screen allowing various author filtering settings

who the most prominent author is, Lance Sherry of the Center for Air Transportation Research at George Mason University in Fairfax, Virginia. Table 1 shows the top four authors discovered in this survey. Therefore, future steps for this research would be to further investigate their published works. A key observation is that two of the top researchers work for the same institution which corresponds to the significant links found between them and many other authors who likely have connections themselves to NASA Ames Research Center. The primary benefit of this tool is in the identification of the most predominant authors on this topic. With minimal effort, it allows the user to pinpoint the authors that potentially have the most relevant writings to their topic, assuming the selected

Fig. 4 Network depicting author connections

116

E. Barnell

Table 1 Table depicting the leading authors among the 561 included in this analysis Author

Years

Institution

Count

Sherry, Lance

1998–2019

Center for Air Transportation Research, George Mason University, Fairfax, VA

23

Palmer, Everett

1991–2017

NASA Ames Research Center, Moffett Field, CA, USA 11

Feary, Michael

1997–2020

NASA Ames Research Center, Moffett Field, CA, USA 11

Mauro, Robert

2014–2020

Decision Research Inc., Eugene, Oregon

10

Active publishing years, affiliated institution and publishing count are included

search keywords are properly chosen. Following this process, the user can then refer to the pertinent writings and authors to analyze their content. Not having to manually search and analyze hundreds of written works saves a great deal of time and effort. I find it straightforward to use and a highly effective tool.

3.3 MaxQDA MaxQDA is the next tool in my arsenal. Its primary purpose in my analysis is to provide a word cloud of text content. I used all fifteen articles listed as references in this report to provide the widest breadth of lexical context for analysis. Figure 5 shows a list of these documents as well as the word cloud button used in this analysis under the “visual tools” tab in the menu.

Fig. 5 Menu for operation selection depicting the word cloud option to the right as well as the list of documents to be analyzed

Utilizing Bibliometric Analysis Tools …

117

Fig. 6 Screenshot depicting the word frequency list used to modify the stop list of excluded terms

Assigning extraneous words to the “Stop List” (as shown in Fig. 6) to exclude them from the word cloud analysis is imperative for useful results. The most important step is filtering out the inconsequential terms that bring no valuable meaning to the analysis. Extraneous words such as “the” and “was” are best left out of the word cloud to clarify the results, not distract the viewer and provide competent interpretation. It is simple to perform this task by entering the word frequency list, selecting the extraneous words, and adding them to the stop list. Figure 7 shows the final output of my use of the MaxQDA software. As can be seen clearly, there are multiple terms shown to form a word cloud. The most frequently used terms are shown in bigger text than the less common terms. “Automation” and “system” appear to be the most frequently used relevant words in these articles which makes sense. The benefit of using this tool allows you to understand what the authors are writing about and guide your continued search from there. Finally, the most common words contained in the articles can be listed. Figure 8’s list is truncated to those with over 200 instances for brevity. These search terms shall help future investigation by giving the most popular terms in existing research. For instance, “automation” and “flight” are among the top terms so I shall use them in subsequent analysis.

3.4 Web of Science The next step is to use the Web of Science database to obtain trend and popularity information on this topic over time. I elected to use search terms similar to the previous steps (“automation,” “flight” and “surprise”) but not utilizing the term

118

E. Barnell

Fig. 7 Word cloud from relevant articles produced by MaxQDA

Fig. 8 Most common words from relevant articles produced by MaxQDA

“autopilot” in order to retain sufficient data quantity and quality for reliable trend analysis. Firstly, you must access the “Web of Science: All Databases” database through the Purdue Library resources and enter the appropriate terms to begin the search resulting in what is seen in Fig. 9. Next, click the “Create Citation Report” button in the top right of the screen to begin the trend analysis.

Utilizing Bibliometric Analysis Tools …

119

Fig. 9 Screenshot depicting the term entry field, search parameters, and results

The final results in Fig. 10 show a marked increasing trend in published works related to the search terms over the past 2 decades or more. This is likely due to the ever-increasing use of flight automation systems in modern airplanes. Further, Table 2 summarizes the results of my searches with how many articles relate to the search terms. “Flight” is the most frequently used term with “automation” in second place. The lack of documentation on “autopilot” is relatively surprising.

3.5 Pivot Chart BibExcel is another tool that allows you to distill large amounts of data into a more manageable form. Using Harzing’s exported data from Google Scholar on the search term “Autopilot Flight,” a list of the most prominent authors is created (see Fig. 11). Once put into pivot chart form using Excel (see Fig. 12), it is nearly trivial to determine where to start your research. The authors are listed by the number of publications they have that relate to your search term. In this case, authors such as Looye, Sharma, and Gast should be investigated as they have the most publications.

120

E. Barnell

Fig. 10 Trend graph results of the search from 1994 to present day Table 2 Search term results with the number of articles

Search term

Location

# of Articles

Autopilot

Web of Science: Core Collection

3513

Automation

Web of Science: Core Collection

99,497

Flight

Web of Science: Core Collection

223,707

Fig. 11 List of leading authors extracted from Google Scholar using Harzing and BibExcel on “Autopilot Flight.”

Utilizing Bibliometric Analysis Tools …

121

Fig. 12 Pivot Chart of leading authors extracted from Google Scholar using Harzing and BibExcel on “Autopilot Flight”

3.6 CiteSpace Clusters Another useful tool is CiteSpace. For this analysis, it serves almost as a hybrid between the VOSviewer network and the MaxQDA word cloud. By using the Web of Science search results produced earlier and the settings shown in Fig. 13, a network of keywords is generated that can be seen in Fig. 14. Some interesting observations result with terms could spark further research into areas not covered in this report. For instance, “AI-Based Control” and a “Bio-Inspired Autopilot” are both intriguing and merit additional research. Then, after extracting labels by keyword, you can produce the following citation burst analysis which lists the top 10 references with the strongest citation bursts corresponding to the most prominently cited resources. The primary results from this CiteSpace analysis are contained in Fig. 15 with “The Top 10 References with the Strongest Citation Bursts.” This one tool provided multiple documents that I read and analyzed further, so it appears to be extremely effective. By clearly listing which publications are the most cited, you can easily decide which are most deserving of investing your time and resources into due to their impact and influence in their corresponding fields.

3.7 Vicinitas Engagement and Trend Analysis Vicinitas is a website/program that produces and extracts data from the social media application, Twitter. It allows analysis of how topical a certain term is today by

122

Fig. 13 CiteSpace cluster analysis menu settings

Fig. 14 CiteSpace cluster analysis results depicted by keyword

E. Barnell

Utilizing Bibliometric Analysis Tools …

123

Fig. 15 CiteSpace cluster citation burst analysis results

producing a metric of how much that term is discussed. It tallies how many users talk about it, in how many posts it appears, how many people have some engagement in it as well as how many people it influences. Figure 16 shows these data points as well as a word cloud of the most common words associated with the search term in the relevant posts. This search term may not be the most effective for this analysis due to its association with the Tesla automotive self-driving technology, but it is still demonstrative of how the tool works and provides some insight on just how relevant autopilot technology is today. Table 3 shows the results of searches for “Autopilot,” “Automation Surprise” and “Flight” in a condensed form listing their user, post, engagement, and influence counts respectively. By far, “flight” is the most prevalent term with “autopilot” not too far behind. “Automation Surprise” on the other hand, has almost no activity on

Fig. 16 Individual Vicinitas Twitter engagement search results for “Autopilot.”

124

E. Barnell

Table 3 Tabulated Vicinitas analysis results Keyword

Users

Posts

Engagement

Influence

Autopilot

1.9k

2.1k

6.5k

6.1M

Automation surprise

12

12

96

46.1K

Flight

19k

2.1k

1.5k

27.5M

Twitter. I believe this is due to a lack of familiarity with the term amongst the general population.

4 Results and Discussion Automation surprises occur when “crews are surprised by actions taken (or not taken) by the automated system” and result in situations where the operator is surprised by the behavior of the automation asking questions like what it is doing now, why it did that, or what is it going to do next. Automation surprise (AS) is the end result of a deviation between expectation and actual system behavior that is only discovered after the crew notices strange or unexpected behavior [1]. When automation is not “adequately designed (or correctly understood by the operator), … so called automation surprises” may occur “that degrade instead of enhanc[ing] the overall performance of the couple (operator, system)” [2]. Automation surprises and “inexplicable glitches with digital systems in the cockpit are experienced commonly enough” [3] that they may no longer truly be a surprise to pilots. According to Combefis et al. “systems are well designed if they can be described by relatively simple mental models for their operators” [4] in order to decrease the amount of thought and effort required to decipher what actions the system is performing at the moment. Furthermore, “additional sensors [may be] necessary” to better approximate the plane’s flight path and increase the predictability of the autopilot systems [5]. Unfortunately, one downside to such incredible automation technology is “automation addiction” which “has degraded pilots’ manual flight skills” [6] due to overreliance on flight aids and lack of practice and experience flying such complex machines without them. With so much automation in modern planes, “a pilot’s real challenge … is pressing the right button at the right time” [7] and not actually flying the plane. When pilots do not understand how an autopilot system works and the mode of confusion is not clear, their attention is taken away from critical functions “resulting in excessive and inefficient visual search patterns” [8] that quite often lead to accidents. Normally, the human pilots are responsible for “tasks … considered [to be] at a high level of abstraction [relative] to work and work objectives, leaving … low-level, repetitive tasks” to be performed by automation [2]. In a human pilot and autopilot system, the “control systems perform different levels of augmentation, while the pilot

Utilizing Bibliometric Analysis Tools …

125

assesses the ease and precision with which these tasks are performed” [9] whether or not they do it consciously. The sign of a well-designed automation system is one that the pilot rarely has to get involved with. One method of increasing a pilot’s awareness of automation surprises is “an alerting system that notifies the user of commands that may lead to unsafe states” [10] to help clarify when the system may not respond expectedly. Research has been performed in relation to an award from the NSF to Ralph Wachter of the University of Illinois with multilevel adaptation for handling large uncertainties. This involves the autopilot sensing a dangerous condition, near or outside of the flight envelope, where the autopilot disengages itself and returns control to the pilot. This could help potentially prevent automation surprises by limiting autopilot’s involvement in situations in which it might fail to operate safely. Additionally, this system may result in automation surprises of its own, so further research shall always be necessary with any sort of flight automation. Another agent of automation surprises is a “difference between commonly held beliefs about the impact of new automation on human and system performance and the actual impact of new technology on the people who must use it in actual work” [11]. Sarter suggests that the designers of these automation systems need to work with more pilots during the design and development stages to gather more information on how the systems will be used in order to create more robust systems. To help reduce automation surprises, “the user-interface [of autopilot systems] should be suitably designed so that the user can interact with the automated system safely and reliably” [12]. Additionally, “in order to reduce the complexity of operation,” increase pilot awareness and ensure they have a fundamental understanding of the automation’s mechanisms, “a more simplified representation of the underlying system is required” [12]. Roger L. Brauer very effectively states the essence of why it is so important to put more emphasis on preventing automation surprise in the design phase. Because of the uncertainty of user knowledgeability and the complexity of modern aviation control systems, “designers have a challenge. They must make technology, … systems, and environments safe. Designers … have responsibilities for many who have little or no understanding of their designs. Recognition of hazards, making judgments, and taking corrective actions cannot be left to untrained users” ([13], 444). According to David L. Goetsch, “strategies for overcoming anticipated future problems include better design of technological systems [and] training and continual retraining” ([14], 522) which correlates to multiple other findings from articles included in this analysis. Airbus Versus Boeing Due to a difference in design philosophy between Airbus and Boeing, two of the main commercial airplane manufacturers, Boeing planes may be more resistant to automation surprises. This is likely due to a difference in warning systems and the transparency of the automation systems. According to the results of de Boer’s study of Dutch pilots, Airbus pilots experienced automation surprises approximately every 10 flights, compared to Boeing pilots every 20 flights and every 30 flights for various

126

E. Barnell

other aircraft. In all, commercial pilots experience an automation surprise event three times annually. In the Airbus A-320, “one of the major problems with pilot-automation interaction is a lack of mode awareness” when the pilots are not clear on the “current and future status and behavior of the automation” [15]. Although Boeing planes are not infallible in this regard, Sarter et al. suggest that this is one of the primary reasons why Airbus planes have more frequent automation surprises.

5 Conclusions and Future Work In general, the bibliometric analysis tools brought much ease to the analysis. In particular, the trend over time of published works produced by Web of Science allowed a great insight into just how popular this topic has gotten in past years. The word cloud produced by MaxQDA allowed effective content analysis to determine which terms were most commonly used. Finally, the nodal network from VOSviewer is the most effective tool to discover the most prominent authors for a given topic; it provides a visualization of the most published author as well as their collaborations. The ability to visualize data in various forms allows faster more reliable, and deeper analysis than would be possible with manual analysis. When an automated system takes surprising actions or decides not to act, the pilots experience automation surprise. This is the “end result of a deviation between expectation and actual system behavior that is only discovered after the crew notices strange or unexpected behavior” [1] and may not be noticeable at the moment, leading to a dangerous situation. Additionally, the frequency of automation surprises as well as the significance during flight is more than one might expect, but most often, automation surprise events have insignificant consequences with only 10% resulting in an undesired aircraft state. This specific topic does not have much published documentation that directly discusses how to detect, avoid and understand automation surprises in an aviation context, but many have useful connections. By performing this bibliometric analysis, I was better able to determine what research is currently out there and what more can and should be done in the future. Neither of this class’s textbooks discussed automation surprises in any detail let alone regarding in-flight automation; there were only a couple of chapters that spoke in general about automation and computers. Investment into further research and development should be guided towards connecting designers and researchers to ensure optimal system designs that reduce automation surprises are produced. Automation surprises seem to stem from the increased complexity of control and interface systems and designs today and by understanding why and how AS occur, we can better work and design to avoid them and their consequences in the future leading to safer flight control systems.

Utilizing Bibliometric Analysis Tools …

127

References 1. de Boer, Robert J, Karel H (2017) Automation surprise: results of a field survey of Dutch pilots. In: Aviation psychology and applied human factors, p 28 2. Palanque P (2020) Ten objectives and ten rules for designing automations in interaction techniques, user interfaces and interactive systems. In: Proceedings of the international conference on advanced visual interfaces (AVI ’20). Association for Computing Machinery, New York, NY, USA, Article 2, pp 1–10. https://doi.org/10.1145/3399715.3400872 3. Woods DD, Erik H (2006) Automation surprises. In: Joint cognitive systems: patterns in cognitive systems engineering. CRC Press, pp 113–142 4. Combefis S, Giannakopoulou D, Pecheur C (2016) Automatic detection of potential automation surprises for ADEPT ,odels. IEEE Trans Human-Mach Syst 46(2):267–278 5. Sujit PB, Saripalli S, Borges Sousa J (2014) Unmanned aerial vehicle path following: a survey and analysis of algorithms for fixed-wing unmanned aerial vehicles. IEEE Control Syst 34(1):42–59 6. Geiselman EE, Johnson CM, Buck DR (2013) Flight deck automation. Ergon Des 21(3):22–26 7. Hersch JJ (2020) The dangers of automation in airliners accidents waiting to happen. S.l.: AIR WORLD 8. Dehais F, Peysakhovich V, Scannella S, Fongue J, Gateau T (2015) “Automation surprise” in Aviation. In: Proceedings of the 33rd annual ACM conference on human factors in computing systems, pp 2525–2534 9. Valavanis KP (2007) Advances in unmanned aerial vehicles state of the art and the road to autonomy, 1st edn, 2007. ed. Intelligent systems, control and automation: science and engineering, 33 10. Ishii D, Ushio T (2016) A bisimulation-based design of user interface with alerts avoiding automation surprises. IEEE Trans Human-Mach Syst 46(2):317–323 11. Sarter NB, Woods DD, Billings CE (1997) Automation surprises. In: Handbook of human factors and ergonomics, pp 1926–1943 12. Adachi M, Ushio T, Ukawa Y (2006) Design of user-interface without automation surprises for discrete event systems. Control Eng Pract 14(10):1249–1258 13. Brauer RL (2016) Human behavior and performance in safety. In: Safety and health for engineers, 3rd edn, pp 435–448 14. Goetsch DL (2015) Computers, automation and robots. In: Occupational safety and health for technologists, engineers, and managers, 8th edn, pp 512–523 15. Sarter NB, Woods DD (1997) Team play with a powerful and independent agent: operational experiences and automation surprises on the airbus A-320. Human Factors 39(4):553–569

HCI in an Automated Vehicle

Human-Computer Interaction in Mobility Systems Heidi Krömker, Cindy Mayas, and Tobias Wienken

Abstract People’s mobility is constantly increasing all over the world, and more and more transport systems are available. This is especially evident in urban transport systems that link individual transport with car and bike sharing or autonomous shuttles. This creates completely new challenges for human-computer interaction, as the complexity of mobility information and the management of intermodal travel becomes more and more sophisticated. A broad spectrum of travelers with different needs must be taken into account. To achieve the acceptance of these systems, human-computer interaction must be completely redesigned. In order to structure this complexity for HCI research, small modules for a theoretical basis have been developed step by step over the last 10 years, most of them were published for the first time at the HCI conferences. In many case studies, method sets and classifications were tested in order to get to know the challenges for HCI better and better. On the basis of case studies method sets and theoretical concepts have been developed. The contribution shows how these results were transferred into the holistic concept of the Mobility Experience. Keywords Mobility systems · Mobility experience · Transportation systems

H. Krömker · C. Mayas (B) Technische Universität Ilmenau, Ilmenau, Germany e-mail: [email protected] H. Krömker e-mail: [email protected] C. Mayas u.Works GmbH, Ilmenau, Germany T. Wienken (B) CodeCamp:N GmbH, Nuremberg, Germany e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Switzerland AG 2023 V. G. Duffy et al. (eds.), Human-Automation Interaction, Automation, Collaboration, & E-Services 11, https://doi.org/10.1007/978-3-031-10784-9_7

131

132

H. Krömker et al.

1 Introduction The design of human-computer interaction in mobility systems is a great challenge. The travelers experience, the ergonomic quality of the mobility system in their individual travel chain when they plan it at home and then carry out their journeys. They use applications on PC at home for planning, different visual and auditory information at stops, in stations, on and in vehicles, and several apps on smartphone for navigation or even for disruption management. What is special about humancomputer interaction in mobility systems is that not only the individual technology has to be designed in a human-centric way, but the interaction of these different tools has to mesh seamlessly. In addition, the environmental conditions are constantly changing. There are phases in the travel chain in which the influences of the environment are rather low, such as during the planning of the trip at home or the orientation about the trip during a long journey in the train. However, there are also situations in which the environmental influences have a very unfavorable impact on the work situation. At transfer points, decisions on the choice of routes and vehicles are required under time pressure and in the hustle and bustle of crowded platforms. Here, travelers often have competing hardware and software tools and information displays at their disposal. Announcements on the loudspeaker have to be compared with the information posted on paper on the platform and on the vehicle, as well as with the information in the individual app on the smartphone. Services have been introduced in recent years to deal with these diverse conditions that can occur in the travel chain. These offer flexibly person-, situation- and taskrelated services that are designed to facilitate the management of the travel chain. These services can be both digital and human. An example is the suggestion of a travel route and a ticket or a concrete human support, e.g. a service person who helps with the transfer of luggage. This paper describes a framework that provides a holistic view on the design of human-computer interaction in travel chains. The framework concept offers a new holistic approach as well as a holistic structuring of the mobility system.

2 Related Work The starting point for the structuring of mobility systems is the individual travel chain, which was already introduced in 2001 in the context of the new electric passenger systems at that time [1]. The authors state that the acceptance of public transport depends not only on attractive transport offers, but especially on providing userfriendly information. Further publications on travel processes from the user’s point of view came into the focus of research only 10 years later with the increase in the possibilities of digital information for the travel chain [2–4].

Human-Computer Interaction in Mobility Systems

133

A systematic description of the classical human-computer interaction in the travel chain can be found in Wienken and Krömker [5], who introduces the connection between utility, usability, user and mobility experience. In order to make the multitude of rapidly developing tools, information displays and support possibilities in a travel chain usable, the concept of services was used. Here, a special role is played by the customer-induced orchestration of services, as described by Winter et al. [6]. The core idea here is that the ubiquitous availability of information strengthens the users’ desire for self-determination. Users want to select and compile the services themselves. Since the almost unmanageable multitude of services remains manageable for the user if the services are offered in dependence of the user’s context, the concepts of context-awareness by Dey and Abowd [7] were used. Abowd and Dey define context to be the user’s physical, social, emotional or informational state. This is an important help to structure the so much varying contexts of the traveler in the travel chain. In order to make the complexity of the mobility system manageable, systems engineering [8] approaches are also relevant. They make it possible to define the elements that are relevant for the fulfillment of user tasks during the journey. With the principles of systems thinking, it enables both the definition of system boundaries and the modelling of system levels from the user’s point of view. These theories are essential foundations for the development of the mobility experience approach in travel chains.

3 Research Question The research gap lies in linking and extending the above approaches for a holistic description of human-computer interaction in mobility systems. A framework should be developed that allows all stakeholders of the mobility system to develop a holistic user-centered view. Typical stakeholders in mobility systems are e.g. mobility planners, transport providers or mobility service providers. They all can contribute to a user-centered development of the mobility system. The research question is therefore: Which model can be used to describe the human-computer interaction in the travel chain holistically?

4 Research Procedure In order to validate and refine the above theoretical relationships for HCI in mobility systems, a series of case studies were conducted. Eisenhardt [9] describes this research strategy as an approach that focuses on understanding the dynamics of specific situations.

134

H. Krömker et al.

Fig. 1 Case studies, many of which were published for the first time at HCI International [2, 4, 10–14]

The individual case studies were designed to be theory-driven so that the added value of the theories for understanding the situations could be empirically demonstrated. Hypotheses derived from theory were used to measure multiple constructs and test relationships between variables. Figure 1 provides an overview of which research questions have been explored through case studies. Most of the results have been presented at HCI International Conference over the last 10 years.

5 Results 5.1 HCI Relevant Description of the Mobility System HCI in Mobility. Human-computer interaction is defined by the relationships between users, tasks, tools and context [15]. In terms of mobility, users are represented by travelers moving from a starting point to a destination. To reach the destination, tasks arise for the users, which can be represented along the travel chain. The journey of the users is supported by tools provided by the mobility system. These are used in a specific travel context that influences the overall relationship between users, tasks and mobility system (Fig. 2). Mobility System. The mobility system is hierarchically subdivided into different subsystems, Fig. 3 shows the theoretical approach to transform the system engineering according to Haberfellner et al. [8] to mobility systems. In organizational terms, a distinction can be made between the mobility subsystems of the transportation companies. These subsystems provide tools to support mobility, such as vehicles, which in turn can be subdivided into further interaction elements. The interaction elements include bidirectional human-computer interaction, e.g. with vehicles or ticket vending machines, and human-human interaction, e.g. with driving or

Human-Computer Interaction in Mobility Systems

135

Fig. 2 HCI relevant factors in mobility according to Shackel [15]

service personnel, as well as uni-directional communication channels, e.g. information displays. This results in the following three parts representing the interaction elements in the mobility system: ● Hardware includes all real objects, such as information panels or push buttons on doors and vending machines. ● From the user’s point of view, software includes all digital elements, such as information displays or mobility platforms. Fig. 3 Mobility system with subsystems according to Haberfellner et al. [8]

136

H. Krömker et al.

● Persons include humans who are relevant for the mobility system, such as ticket inspectors, ticket sellers or other service providers. Other elements of the mobility system that influence interaction but do not cause direct interaction are summarized in the section Travel Context. In this way, the user perceives the mobility system as a combination of different interaction elements. This individual combination of interaction elements during a trip leads to very heterogeneous images of the mobility system for the user’s perception. A special challenge arises for the user at the boundaries of the mobility subsystems of different transportation companies with regard to the consistency of presentation and content of the information offered across different mobility subsystems. Users. The aim of describing users is to take a holistic view. For this purpose, it is recommended to consider behavior, skills and capabilities, environment, feelings and attitudes [16, 17]. These four areas can be operationalized for mobility users as follows [2]: ● The behavior of mobility users is determined by their goals, their frequency and regularity of use, and their flexibility in terms of travel time, choice of route and mode of transport. ● The user skills are characterized by their knowledge of the place and the system, which describe the prior knowledge to describe the start and destination places as well as the route choice. In addition, the trip may be limited by the capabilities such as walking or eyesight or luggage. ● The usage environment, which is individually determined by the user, refers not only to the time of day of traveling, but also to the individually available travel capabilities such as tickets. ● Furthermore, attitudes and feelings are represented by a user’s personal travel experiences, transportation preferences and expectations. For example, an established method for describing users with these variables are personas [17], which construct stereotypical users from the different goals and behaviors of real users [2]. Tasks along Travel Chain. The goal of the description of tasks is a solutionindependent consideration of the task performance process, which is influenced by the characteristics of the user [18]. The task “travel” can be subdivided into further tasks [19]. Important categories are preparation, travel and dealing with disturbances [3]. Another possibility to structure the tasks temporally along their sequence during a trip is the travel chain [1]. This results in 8 main tasks, which can be subdivided into further subtasks (see Fig. 4). Context. Along the travel chain, the user passes through different contexts. The description of the context includes all elements that can influence the interactions [7]. This influence can affect both the interacting people and systems. In the field of mobility, the contexts that occur are particularly diverse, as they include contexts in vehicles, at bus stops, and also footpaths that may be located outside the bus stop. The

Human-Computer Interaction in Mobility Systems

137

Fig. 4 Hierarchical task examples of traveling

following five categories are necessary to describe the context of mobility systems [20]: ● Spatial context includes for example current position, routes of vehicles and distances to go. ● Temporal context includes for example current time, timetable times, planned departure time, duration to go. ● Environmental context includes for example social conditions, properties and equipment of stations, vehicles or routes, and physical conditions such as light, temperature, noise. ● Social context is determined by travel companions, among others. ● Informational context, includes for examples disturbance information or tariff information. The combination of the many factors influencing HCI for mobility, including users, tasks, tools, and context, results in a high level of system complexity, which requires new approaches to be considered holistically.

5.2 Framework Mobility Experience Mobility services are increasingly transforming into complex systems. Numerous providers are cooperating with each other to expand the scope of services for users [6]. The desired goal is to network the elements of the mobility systems with each other and thereby provide comprehensive services for the individual needs of the users. This is evident both in the core of a mobility service (transportation), with the networking of different means of transportation, and in the integration of supplementary services, such as ticket booking or providing of passenger information. This development is not only driven by the providers. Rather, the networking approaches are a reaction to the changed usage habits of customers. Today, users interact with different providers and generate an individual user journey with numerous contact points between users

138

H. Krömker et al.

Fig. 5 Mobility experience and touchpoints along travel chain

and providers (so-called touchpoints) [21]. In line with these habits, the perception of service quality is changing alongside the usage process. Instead of isolated fragments, users perceive the service chain as an overall construct. Thus, the assumptions and perceptions about usage, the perception of actual usage, and subsequent processing [22] are shaped by the whole service chain. For this reason, a paradigm shift must also be made in the design of mobility services. From the isolated consideration of a single human-computer interaction to a holistic consideration of the entire service chain. Based on this challenge, the approach of the Mobility Experience was developed. The mobility experience defines the passenger’s perceptions and responses resulting from any direct or indirect contact and anticipated contact with the service provider along the journey [5]

In order to apply the approach systematically to concrete issues in the analysis and design of mobility systems, the mobility experience can be broken down into individual dimensions (see Fig. 5). The three inner levels of the layer model describe the human-technology interaction in isolation for a touchpoint. The quality of the interaction is operationalized by the three established dimensions: Utility, Usability, and User Experience [23–25]. In contrast to these selective dimensions, the fourth level addresses the holistic perspective and considers the service across all involved touchpoints of a journey as a process.

5.3 Methods for Analyzing and Designing the Mobility Experience Mobility Service Diaries. The method of Mobility Service Diaries [26] aims at the holistic, empirical collection, analysis and evaluation of mobility services before, during and after a trip. The Mobility Service Diaries method is based on the Mobility Diaries method of national travel surveys [27–30] and extends it to empirically identify the interaction with other services during a trip. Mobility services to support the execution of a trip can be, besides the classical transport services, for example ticketing services, information services or comfort services for food and drinks, hygiene, security or entertainment. The Mobility Service Diary consists of two parts for each trip: a questionnaire to collect trip data and a diary sheet to record the services used. Depending on the goal

Human-Computer Interaction in Mobility Systems

139

and prior knowledge of the survey, the survey fields can be standardized or openended to answer descriptive or exploratory research questions. Also, the questionnaire on travel data can be supplemented with another questionnaire to collect data from the traveler of a Mobility Service Diary. Table 1 shows the four main categories of travel data to be collected—trip duration, motive, companion, and familiarity. The travel diary consists of four additional categories for each mobility service use—time, place type, service type, and touchpoint, shown in Table 2. Whether the locations and services are collected only as types or with specific names is also dependent on the research objective and the data protection framework. The Mobility Service Diary can be supplemented flexibly with further questions on the Mobility Experience of individual services and the overall trip. The variety of data and data relationships collected with the Mobility Service Diaries poses special challenges for the analysis and presentation of results in order to uncover the complex relationships and draw conclusions for the holistic design of the mobility system. Mobility Experience Map. The Mobility Experience Map is a method for the holistic analysis, design, and evaluation of complex mobility systems and helps to implement the principles of the mobility experience. Specifically, the experience map shows which situations and activities users have to cope with on their journey and to what extent positive as well as negative emotions are triggered [31]. In order to draw a comprehensive picture of the experiences, the experience map is supplemented by the users’ cognitive, affective, and physical reactions [31]. As a result, the user Table 1 Basic elements of a mobility service diary questionnaire Element

Description

Example

Travel duration Difference between start and end time The travel duration can be collected on condition that the travel does not by detailed time data or in categories, include more than one date. Otherwise such as “less than 10 min” more than one date must be considered Motivation

Reasons for mobility

The reasons can be related to the destination, such as “way to work” and “way for shopping and errands” or to the context such as “vacation trips”

Companions

Persons, who accompany the traveler in parts or the complete travel

The companions can be described by number, for example “2” or with additional information, for example “adult” or “children” to define the additional effort

Familiarity

Previous knowledge of the route, Travelers might describe previous which might influence the information experiences on the route, frequency of needs of the traveler use or caterories of familiarity such as “I’m familiar with the whole journey.” or “I’m not familiar with the majority of the journey.”

140

H. Krömker et al.

Table 2 Basic elements of a mobility service diary Element

Description

Time

Start time of a service use on the day of The data gathering should have travel with an accuracy of about five defined formats, such as “08:05” or minutes. Services, which are used “before day of travel” before or after the day of travel, e.g. travel planning or billing, are documented without a concrete time

Example

Location type Generalized description of the place of Exemplary categories of location service use without specifying the place types are home, bus stop, airport or sharing station Service type

General extent of the service used without further details

Exemplary categories of service types are information, transport, food, hygiene or entertainment

Touchpoint

General description of the service provider

Exemplary categories of touchpoints are restaurant, bus, sanitary facilities or website

experience along an individual travel chain can be analyzed and designed selectively as well as holistically. The central starting point of an experience map is the usage process. The process subdivides the service and defines which tasks users must pursue in the individual phases. To accomplish these goals during a journey, users must perform a series of interactions, such as purchasing a ticket on their smartphone. In the context of the Mobility Experience Map, these interactions are described in more detail through the interplay of user model, task, and touchpoint. Analogous to the HCI model shown in Sect. 5.1, the touchpoints describe the technology and the surroundings available for the interaction or the contact with a person. Each of these interactions triggers a reaction from the user. These reactions can be of different nature and serve as an instrument for qualitative evaluation. The reactions are captured in the experience map in the form of cognitive and physical activities as well as emotions. The research project Digitalized Mobility—the Open Mobility Platform (DiMoOMP) [32] can be used to illustrate the use and potential of a Mobility Experience Map. The goal of the project was to develop a reference software architecture for mobility platforms. The novel mobility platforms combine a variety of transport modes, such as buses and trains as well as bike and car sharing, and allow users to plan, book and pay for intermodal trips from a single source. However, this technological expansion from the classic information and travel planning system to a holistic mobility platform also entails changes on the user side. For the users, the tasks that they have to solve with the system change first of all. As a consequence, a new usage process is emerging. New information and functionalities must be integrated, and existing information must be made available at other times. In addition, the touchpoints at which customers and providers interact with each other change. With the help of the experience map, the current challenges and thus potential

Human-Computer Interaction in Mobility Systems

141

Fig. 6 Mobility experience map of a casual user for intermodal travel [36]

usage problems along the travel chain were identified and analyzed. Figure 6 shows the intermodal journey by bus and bike sharing for an opportunity user (Table 3).

6 Discussion The results of the case studies showed the need for a holistic way of thinking in order to understand the human-computer interaction in mobility systems. The concept of Mobility Experience provides a framework for understanding, analyzing and designing individual travel chains. However, the diversity of user needs also means that the evaluation criteria have to be compiled and weighted again and again. Consequently, the conception of the Mobility Diaries and Mobility Experience Maps also depends on the cognitive interest of the respective case study. The qualitative results provide starting points for more in-depth studies. It should be noted, however, that these studies require a great deal of time and effort. In addition, travelers are confronted with mobility offers of organizationally completely separate mobility providers, such as public transport and individual transport. This makes it difficult to harmonize the design of these mobility offerings.

142

H. Krömker et al.

Table 3 Basic elements of a mobility experience map [31] Element

Description

Example

Usage process and its phases A usage process is any purposeful activity or group of activities that result in an outcome [33]

For the mobility sector, the travel chain is an archetypal process that is relevant across all means of transport and describes the user tasks for every single phases

User model

User descriptions contain stereotypical users and embody their motivations, expectations, attitudes, and skills that are relevant to the use of the services [17]

Typical user types for the mobility sector are, for example, commuters or occasional users of the mobility systems

Touchpoint

A touchpoint is defined as any possible contact point between users and providers. These interactions take place human-human, and human-machine, but also occur indirectly via third parties, such as reviews from other users [34]

For public transport, the mobile passenger information application or the timetable at the bus stop can be mentioned as examples

Context

The term of context can be used to characterize the situation of an interaction between users and service providers. In the sector of mobility, touchpoints are closely interwoven with the context, which shows to what extent a prevailing situation influences the perception and interaction of the users

In order to precisely identify and analyse these situational influences, the context for mobility can be defined in five categories: Spatial, temporal, environmental, social, and informational context [20]

User activities

The user activities contain all the actions that users actually perform to achieve a goal during the mobility service

To purchase tickets via smartphone, users have to perform numerous activities, such as selecting a ticket, entering credit card details and initiating the transaction, etc

User emotions

User emotions generally involve the physiological response of the brain and body to threats and opportunities [35]. Emotions have a prominent cause, occur and decrease rapidly and thus are relatively intense and clear cognitive contents [35] (e.g. Anger over train failure)

Negative emotions arise in a journey, such as anger when the travel chain breaks and users miss the connecting train

(continued)

Human-Computer Interaction in Mobility Systems

143

Table 3 (continued) Element

Description

Example

User thinking

User thinking indicates which actions a user thinks through before deciding to execute them. In doing so, the user thinking provides insights into a user’s mental model

For the mobility sector, user thinking provide insights into a user’s mental model what is defined as a rich and elaborate structure, reflecting the user’s understanding of what a mobility service contains, how it works, and why it works that way

However, it is positive to note that the mobility providers have taken up the results of these studies with great interest and that parts of them have also been translated into binding recommendations [37–39] for the mobility industry.

7 Future Work To implement this holistic view in the practice of mobility providers, it is important to establish responsible persons at the organizations. They must continuously improve the human-computer interaction in the individual travel chains on the basis of this holistic view. But users must also become aware that they have a holistic demand for the ergonomic quality of their travel chain and demand it. Only if both mobility providers and users systematically address these challenges can human-computer interaction in travel chains be improved in the medium term.

References 1. Verband Deutscher Verkehrsunternehmen (VDV) (2001) Telematics in public transport in Germany. Alba Fachverlag GmbH + Co. KG, Düsseldorf 2. Mayas C, Hörold S, Krömker H (2013) Meeting the challenges of individual passenger information with personas. In: Stanton NA (ed) Advances in human aspect of road and rail transportation. CRC Press, Boca Raton, pp 822–831 3. Hörold S, Mayas C, Krömker H (2013) Identifying the information needs of users in public transport. In: Stanton NA (ed) Advances in human aspect of road and rail transportation. CRC Press, Boca Raton, pp 331–340 4. Wienken T, Mayas C, Hörold S, Krömker H (2014) Model of mobility oriented agenda planning. In: Kurosu M (ed) Human-computer interaction. Part I, HCII 2014, LNCS, vol 8512. Springer International Publishing Switzerland, pp 537–544 5. Wienken T, Krömker H (2018) Designing for mobility experience - towards an understanding. In: Stopka U (ed) Mobilität and kommunikation. Winterwork, Borsdorf, pp 17–27

144

H. Krömker et al.

6. Winter A, Alt R, Ehmke J, Haux R, Ludwig W, Mattfeld D, Oberweis A (2012) Paech A: manifest—kundeninduzierte orchestrierung komplexer dienstleistungen. Informatik Spektrum 35(6):399–408 7. Dey AK, Abowd GD (2000) Towards a better understanding of context and context-awareness. In: Proceedings of the CHI 2000 workshop on the what, who, where, when, and how of context-awareness. Georgia Institute of Technology, Atlanta, pp 304–307 8. Haberfellner R, de Weck O, Fricke E, Vössner S (2019) Systems engineering—fundamentals and applications. Birkhäuser, Basel, pp 3–26 9. Eisenhardt KM (1989) Building theories from case study research. The Acad Manage Rev 14(4):532–550 10. Hörold S, Mayas C, Krömker H (2013) Analyzing varying environmental contexts in public transport. In: Kurosu M (ed) Human-computer interaction. human-centred design approaches, methods, tools, and environments. LNCS, vol 8004. Springer, Heidelberg, pp 85–94 11. Mayas C, Hörold S, Krömker H (2015) Workflow-based passenger information for public transport. In: Kurosu M (ed) Human-computer interaction: design and evaluation. HCI 2015. LNCS, vol 9169. Springer, Cham, pp 381–389 12. Wienken T, Krömker H, Spundflasch S (2017) Agenda planning—design guidelines for holistic mobility planning. In: Kurosu M (ed) Human-computer interaction. Interaction contexts. HCI 2017. LNCS, vol 10272. Springer, Cham, pp 713–720 13. Mayas C, Steinert T, Krömker H (2018) Interactive public displays for paperless mobility stations. In: Kurosu M (ed) Human-computer interaction. Interaction in context. HCI 2018. LNCS, vol 10902. Springer, Cham, pp 542–551 14. Mayas C, Steinert T, Krömker H (2019) Towards an integrated mobility service network. In: Krömker H (ed) HCI in mobility, transport, and automotive systems. HCII 2019. LNCS 11596. Springer, Cham, pp 430–440 15. Shackel B (1991) Usability—context, framework, definition, design and evaluation. In: Shackel B, Richardson S (eds) Human factors for informatics usability. University Press, Cambridge, pp 21–37 16. Goodwin K (2009) Designing for the digital age: how to create human-centered products and services. Wiley, Indianapolis 17. Cooper A, Reimann R, Cronin D (2007) About face 3: the essentials of interaction design. Wiley, Indianapolis 18. Hackman JR (1969) Toward understanding the role of tasks in behavioral research. In: Acta psychologica 31. North-Holland Publishing Co., Amsterdam, pp 97–128 19. Annet J (2005) Hierarchical task analysis (HTA). In: Handbook of human factors and ergonomics method. CRC Press, Boca Raton, FL, pp 33-1–33-7 20. Krömker H, Wienken T (2015) Context elicitation for user-centered context-aware systems in public transport. In: Kurosu M (ed) HCI 2015, vol 9170. LNCS. Springer, Cham, pp 429–439 21. Sangiorgi D, Prendiville A, Ricketts A (2014) Mapping and developing service design research. Lancaster, SDR Service Design Research UK Network 22. Sarodnick F, Brau H (2006) Methoden der usability evaluation: wissenschaftliche grundlagen und praktische anwendung, 2nd edn. Hans Huber, Bern 23. Nielsen J (1993) Usability engineering. Academic Press, San Diego, CA 24. International Organization for Standardization (1998) Ergonomics of human-system interaction: part 11: ergonomic requirements for office work with visual display terminals (VDTs) 25. International Organization for Standardization (2010) Ergonomics of human-system interaction: part 210: human-centred design for interactive systems 26. Mayas C (2020) Implications of mobility service diaries on adaptive mobility platforms. In: Ahram T, Karwowski W, Pickl S, Taiar R (eds) Human systems engineering and design II, IHSED 2019. Advances in intelligent systems and computing, vol 1026. Springer, Cham, pp 47–52 27. Department for Transport Website. Statistical release, National Travel survey England. https:// assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment_data/file/ 729521/national-travel-survey-2017.pdf. Last accessed 1 March 2019

Human-Computer Interaction in Mobility Systems

145

28. Ecke L, Chlond B, Magdolen M, Eisenmann C, Hilgert T, Vortisch P. Deutsches Mobilitätspanel (MOP)—Wissenschaftliche Begleitung und Auswertungen, Bericht 2017/2018: Alltagsmobilität und Fahrleistung, https://daten.clearingstelle-verkehr.de/192/162/Bericht_MOP_17_18. pdf. Last accessed 1 March 2019 29. Infas Institut für angewandte Sozialwissenschaft GmbH: Mobilität in Deutschland, Kurzreport. http://www.mobilitaet-in-deutschland.de/pdf/infas_Mobilitaet_in_Deutschland_2 017_Kurzreport.pdf. Last accessed 1 March 2019 30. Federal Highway Administration Website. National household travel survey. https://nhts.ornl. gov/. Last accessed 1 March 2019 31. Wienken T, Krömker H (2018) Experience maps for mobility. In: Kurosu M (ed) Humancomputer interaction. Interaction in context. HCI 2018. LNCS, vol 10902. Springer, Cham, pp 615–627 32. Dohmen C et al. DiMo-OMP—Digitalisierte Mobilität—Die Offene Mobilitätsplattform. https://www.tib.eu/de/suchen/id/TIBKAT:1664971661/. Last accessed 15 Jan 2021 33. Haksever C, Render B (2013) Service management: an integrated approach to supply chain management and operations. FT Press, New Jersey 34. Stickdorn M, Schneider J (2012) This is service design thinking. BIS Publishers, Amsterdam 35. Jeon M (2017) Emotions and affect in human factors and human-computer interaction: taxonomy, theories, approaches, and methods. In: Jeon M (ed) Emotions and affect in human factors and human-computer interaction. Academic Press, London, pp 3–26 36. Wienken T, Ullrich M, Krömker H, Steinert T (2019) Digitalisierte Mobilität: Mobilitätsplattformen—Herausforderungen auf dem Weg zum ganzheitlichen Mobilitätsservice. Nahverkehr 37(5):36–42 37. Verband Deutscher Verkehrsunternehmen (VDV) TRIAS Website. https://www.vdv.de/projektip-kom-oev-ekap.aspx. Last accessed 15 Jan 2021 38. Verband Deutscher Verkehrsunternehmen (VDV) (2014) VDV-Mitteilung 7036: User Interface Design für die elektr. Aushanginformation. Beka, Köln 39. Verband Deutscher Verkehrsunternehmen (VDV) (2018) VDV-Mitteilung 7046: Definition und Dokumentation der Nutzeranforderungen an eine offene Mobilitätsplattform. Beka, Köln

Cognitive Analysis of Multiscreen Passenger Vehicles Alex Krochman and Thorsten Kuebler

Abstract As passenger car design continues to develop into the coming era, designers attempt to one up the Apple and Tesla aesthetic. More screens, bigger screens, touch screen everything, all written into design language with the expectation that soon, a car will drive us, as opposed to the other way around. Designers are attempting to create home theatres on wheels, that will waft us from destination to destination while we barely lift a finger. Unfortunately for those designers, and the consumers who yearns for these vehicles, autonomous driving has yet to reach where it needs to be for that dream to come true. We look at companies such as Byton and Cadillac, that are working on a cross car display, attempting to span the width of the vehicle with their screens. Following hotly on their heels are companies like Sony, with the Vision-S concept, and Kia with their 21 screen concept car, the later of which is a tongue in cheek take on their competitors visions of cross car displays. However, all of these screens have the potential to be a significant distraction to the driver. Mazda recently vowed to remove all touch screens from their lineup because they believe they are too distracting for the drivers to safely utilize. However, the driving majority of consumers have spoken, and screens are here to stay for the foreseeable future.

1 Introduction As we begin to increase the number of light sources in vehicles, especially those with nonstandard geometry, we increase the presence of reflections from these light sources onto the interior of the car. The greatest issue with this in vehicles comes from reflections off glass and high gloss surfaces, such as the ubiquitous piano black trim. Anyone who has driven at night with a passenger playing with their phone or A. Krochman (B) Nikola Motors, Phoenix, United States e-mail: [email protected] T. Kuebler Human Solutions of North America, Inc., Morrisville, NC 27560, USA e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Switzerland AG 2023 V. G. Duffy et al. (eds.), Human-Automation Interaction, Automation, Collaboration, & E-Services 11, https://doi.org/10.1007/978-3-031-10784-9_8

147

148

A. Krochman and T. Kuebler

tablet has seen a “phantom” screen reflected in the vehicle’s windows. Where these phantom reflections are can potentially block the driver’s ability to see what is beyond them, be it something benign or critical. A benign occlusion may be a trashcan on the side of the curb as the vehicle drives past it, knowing that it is these will have little to no effect on how the driver operates the vehicle.

2 Case Observations A critical occlusion would be if the screen(s) in the vehicle produce phantom images between the driver and a pedestrian that is crossing the road in front of them, or between the driver and the side mirror of the vehicle, preventing them from seeing another vehicle or cyclist that they may turn in front of. While a critical occlusion of this magnitude can be avoided in initial vehicle setup using SAE eye ellipses, that assumes that the templates can accurately represent not only the body proportions of a global set of drivers, but also predict how they are likely going to sit in the vehicle. In this regard, templates fall short, and actual ergonomic analysis needs to be performed. Taking a look at the Byton M-byte interior, in Fig. 1, we can see that the car has three screens in the front occupant zone alone. Looking at the initial reflection analysis, we can see in Figs. 2a–c that the reflection from the three screens present in the Byton M-Byte is not in the same place for all drivers. The reflection in the driver window will “move” depending on the size of the driver as well as the drivers seating position. Analyzing the location of these reflections further, we can predict that the fifth percentile US female is expected to have some occlusion of the driver side exterior mirror, as can be seen in Fig. 2a. Further analysis through Ramsis Cognitive validates that there will be a minor occlusion of the viewable area of the exterior mirror by the reflection from one of the screens. However, since this blocked area is in the most upper, inboard section of the mirror, it was deemed that this area of the mirror would not present a critical occlusion if the driver has the mirrors adjusted correctly.

3 Discussion Thus far we have focused on only the driver side window, but what about the windscreen? While Tesla has gone to great lengths in their vehicles to ensure that any reflections from the screens are up and out of the way of the driver’s line of sight, what of other vehicles that have a larger screen. Vehicles such as the Byton M-Byte, the 2021 Cadillac Escalade, or the Kia 21 screen concept? We can see in Fig. 3a–c the reflected images from the Byton M-Byte on the windscreen from their cross-car display. For shorter drivers, it does not pose any significant blockage, but when we look at the ninety fifth percentile US male, we can see that the reflection is predicted to be in their line of sight looking forward out of the vehicle. There are a few ways

Cognitive Analysis of Multiscreen Passenger Vehicles

149

Fig. 1 Byton M-byte Interior

Fig. 2 a–c 5th, 50th, and 95th percentile driver window reflection

this can be mitigated. First of which is to physically move the screen, in the case of Fig. 3c, it would be desirable to move the screen rearward in car. If that is not possible, it would be desirable to rotate the top of the screen rearward in vehicle, with the rotational axis at the bottom of the screen. If neither of these options are possible, then other mitigating techniques must be employed. One of the most popular techniques to combat screen reflections is active brightness control, working in concert with a “Dark mode”. Recently, Google has rolled out dark mode for many of its applications, citing reduced eye strain, increased contrast ratios, and lower battery consumption. This can be easily adapted to automotive displays in order to significantly reduce reflection areas on windows that may block critical information from the drivers. However, a vehicle needs to have the ability to switch back and forth

150

A. Krochman and T. Kuebler

Fig. 3 a–c 5th, 50th, and 95th percentile US driver’s windscreen reflection

from dark mode to light mode depending on the ambient light conditions, as well as have rapid control over the brightness levels of the screen. A driver passing through a light tunnel at night may not be able to discern the information displayed on the screen if the ambient lighting suddenly increases to the point where the information is overpowered by the light difference. For this, precision tuned active brightness control will be required, using ambient light sensors to gauge how bright the screen should be, just as cell phones do today. Now that we have committed to executing the concept of a cross-car display, we have a challenge. Where do we put the speedometer, tell tales, and display other critical driver information? While this question of gauge placement is common in the automotive industry, it has, traditionally been placed between the center and upper hoop of the steering wheel. However, Byton’s unique vehicle architecture, and attempt to make the dream of a 48'' wide curved display a reality, meant that we had to investigate some less traditional solutions. It was readily apparent that the steering wheel was going to block some of the screen in front of the driver, but where? Ramsis cognitive application was able to assist us here, as can be seen in the Fig. 4a–c. Additional information about the use of Ramsis software in industry and education is shown in other literature [1, 2]. In accordance with the inputs given to Ramsis, we can see that the predicted screen blockage area shifted upwards as the driver size increased, with fifth percentile female being the lowest on the screen, and ninety fifth percentile male being the highest. This initially presented a certain level of difficulty in placing the driver information in line with the driver, as opposed to a central display, similar to the Mini Copper when the vehicle was relaunched in the early 2000s. Looking at the figures below, we can see that the central area of the screen could perform that function, but then what do you put in the area of the screen in front of the driver, knowing that some of it is blocked? This was not a suitable place for navigation, infotainment did not seem appropriate to put front and center. The discussions continued through all the other options we had with a central layout for the driver critical information. Ultimately, it was decided that since we have the screen, and it is a simple software change to go from left hand drive to right hand drive configuration, the main advantage of a central display was invalidated, as we didn’t have to create a second set of tools for that section of the dash in order to incorporate a handed gauge cluster. Knowing that we would have to put the driver’s information on their section of the screen (left side or right side respectively) we ran a physical trial with a range of test subjects from 5' 0'' to 6' 4'' , representing 5th percentile US female

Cognitive Analysis of Multiscreen Passenger Vehicles

151

Fig. 4 a–c 5th, 50th, and 95th percentile screen blockage prediction

through 95th percentile US male. We instructed test subjects to enter the car, set up the seat and steering wheel in a comfortable positions for them to drive, and ensure that they could see an exterior target in front of them which represented an obstacle that they would encounter in everyday driving. With this setup in place, we outfitted the drivers with motion tracking glasses and divided the screen into one-inch square grid and instructed the drivers to tell us which sections were completely clear, partially blocked, or completely blocked by the steering wheel. The data that we received from this physical evaluation was somewhat different from what Ramsis had predicted, which was at first puzzling. Both the fifth percentile female and the ninety fifth percentile male had their steering wheel positions closer to neutral than was initially predicted in our digital simulations. Specifically, the fifth percentile females preferred to have their steering wheels higher than expected, sitting in a more commanding driving position. The 95th percentile males, by contrast, preferred to recline the seat more than expected and lowered their steering wheel to match in order to provide greater than expected clearance to the headliner. With this study completed, we reran our digital simulation utilizing the new data on the drivers preferred seating position and the digital simulation matched what we found in our physical evaluation. We then elected to place the driver’s information as you see it today in Fig. 5, with the driver’s critical information above the steering wheel, and the ADAS feature in the traditional gauge location, viewed between the center and the rim of the steering wheel. Now that we have an idea of where we want to put the driver’s information on the screen, and have decided some of the location of our other screens on this vehicle, we need to assess what other issues may arise from our desired layout. First of which is understanding how long it will take for the driver to be able to discern what is happening on the screen. For this, we can utilize another feature of Ramsis Cognitive, gaze change and visual shadow. These features enable us to determine where we want to fine tune our layout, and can alert us to any issues that may not have been caught until this point. As we can see in the Fig. 6, the driver is able to read the speedometer in 0.03 s, the ADAS readout in 0.05 s, and the drivers table in 0.2 s. Assuming out vehicle is traveling at 100 kph, the driver will be able to look down at the information, register it, and refocus on the road while having travelled 1.67 m, 2.78 m, and 11.12 m respectively. We can also see, how a competitor with a central, low mounted screen may perform with the driver needing to avert their eyes further from the road in order to control their infotainment system. We can assume that this

152

A. Krochman and T. Kuebler

Fig. 5 95th Percentile screen blockage physical validation

might take between 0.1 and 0.5 s to view, depending on the location of the desired button on the screen and the size of the driver, although a proper analysis would need to be run to confirm these assumptions. While todays driver assist functions are attempting to prevent collisions, from distracted drivers, it is still a best practice to reduce the amount of time that the driver removes their eyes from the road to operate the basic features of their vehicles. Returning to the reflection analysis feature of Ramis Cognitive, we can wrap up our virtual validation of the screens as well as the screen layout. For those of you reading this paper that are fond of road trips with other passengers, it is likely

Fig. 6 M-byte screen cognition time, heat map and competitor screen heat map

Cognitive Analysis of Multiscreen Passenger Vehicles

153

in this day and age that one of your passengers, while playing with their phone, tablet, or any reflective device, has managed to reflect the sun into your eyes while driving. Now, this can be anything from mildly irritating to downright infuriating and dangerous, and we can only hope that your passenger did it accidentally. If they did it intentionally, I would recommend leaving them home from now on. However, this brings up an important question as vehicles are equipped with more and more screens, how will they handle reflection from outside light sources? What information might the exterior light obscure on the screens? Will you be able to drive at night with a semi-truck following you and still be able to read your speed off of the screen in front of you? The reflection feature in Ramsis is able to validate this for us, by switching the input and reflecting surfaces. Care should be exercised when adding blocking elements to the analysis to ensure that they are placed in the correct sequence of inputs. All blocking elements must occur between the light source and the reflecting geometry. However, for some analysis, we may want to place the blocking element between the reflecting surface and the driver. Ramsis reflection application cannot compute this yet, but there is a simple trick to work around this. First, we will perform the reflection analysis as we did earlier, with no blocking elements utilized. From there, we can either hide or delete all of the vision rays until we are left with the reflected “image” on the reflecting surface. Next, we will perform an occlusion analysis to the “image” on the reflecting surface and add any blocking elements that we wish to review. This is the technique that was utilized in order to understand what areas that the driver might see a ghosted image on the windscreen from the cross-car display with the interior rear-view mirror blocking a sizeable portion of the reflected area from the driver, as was initially seen in Fig. 3a–c. Reverting back to exterior light sources reflecting on our cross-car display, we are able to see the locations on the screen where light from the rear window, all passenger side windows, and the sunroof will reflect. Due to the geometry of the screen and its position to the driver, no driver side windows had any reflection with respect to the driver. As can be seen in Fig. 7, the locations of reflection on the cross-car display do not interfere with any critical driver information. Many of the possible locations for the light to reflect off the cross-car display also require very specific circumstances. For the sunroof, it is only present for mere minutes during sunset or sunrise, with the vehicle driving away from the light. For the passenger windows, due to the position that the light must originate from with respect to the car, they are also low instance reflections. The greatest likelihood for a reflection off of the cross car display comes from light entering the rear window, which we can see in Fig. 8 below must come in from high on the rear glass on the passenger side of the vehicle, and must work their way past a middle or passenger side rear occupant, and or any front passenger that may have their seat adjusted back on its rails. With this analysis completed, it is believed that any reflection from outside light on the cross-car display will be momentary, and with a low chance of occurrence.

154

A. Krochman and T. Kuebler

Fig. 7 Reflections off of screen with relation to driver

Fig. 8 Light ingress location to reflect off cross-car display

4 Conclusions As consumers continue to demand that the latest technology be present in vehicles, we are likely to see more and larger screens for infotainment purposes, and we are likely to see the driver need to interact less and less with the vehicle in order to pilot it from one location to another. Fully autonomous driving will be here soon, and vehicles will need to be prepared to take advantage of these advances in order to provide customers with new, unique, and exciting traveling experiences. However, it is not yet here, no matter the promises that the show cars make at auto shows and

Cognitive Analysis of Multiscreen Passenger Vehicles

155

consumer electronics shows. Until then, we need to remain vigilant in our approach to vehicle architecture and ergonomics to ensure that the occupants are able to safely pilot the vehicles that we design.

References 1. Duffy VG (2021) Digital human modeling in design. In: Salvendy G (eds) Handbook of human factors and ergonomics, 5th ed. Wiley, New Jersey 2. Sinchuk K, Hancock AL, Hayford A, Kuebler T, Duffy VG (2020) A 3-Step approach for introducing computer-aided ergonomics analysis methodologies. In: Duffy VG (ed) Digital human modeling and applications in health, safety, ergonomics and risk management. Posture, Motion and Health. International Conference on Human-Computer Interaction, pp 243–263. Springer, Cham

Systematic Review on the Emergence of Kan-sei Engineering as a Human Factors Method Daniel J. Tillinghast and Suhas G. Aekanth

Abstract Kansei Engineering (KE) is a longstanding method to utilize affective measures for the production of more user-centric product designs. KE focuses on the use of specific words to classify users’ feelings about a product or their desires for its function and feel. With the expansion of KE’s application to service design and in conjunction with computing, designers can expand upon the original KE method to produce results that best satisfy end users’ desires. With the purpose of exploring the areas and nature of KE’s emergence, this study utilizes bibliometric analysis and data mining tools to generate insights on prominent authors, sub-topics, and regions involved in the study of KE. Analyses are shown in the form of descriptive figures exported from software tools used for bibliometric analysis. Insights from these figures and study of prominent KE articles reveal an opportunity for review and reappraisal of KE as a topic. A review of recent literature supports the idea that KE is a growing topic of interest in the field of ergonomics, with an increasing set of applications stretching beyond traditional product design. Keywords Kansei engineering · Emotion · Service design · Product design · Systematic literature review

1 Introduction and Background Kansei Engineering (KE) describes an ergonomic design method that is consumeroriented yet closely associated with computer science techniques. KE aims to translate consumers’ emotions toward and perceptions about a product into actionable design decisions. Specifically, KE centers on the classification of specific words to inform designers’ objectives. This classification is considered by Nagamachi [1], a pioneer in developing the KE method, as the Type 1 style of KE. Type 2 KE involves D. J. Tillinghast (B) · S. G. Aekanth Purdue University, West Lafayette, IN 47907, USA e-mail: [email protected] S. G. Aekanth e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Switzerland AG 2023 V. G. Duffy et al. (eds.), Human-Automation Interaction, Automation, Collaboration, & E-Services 11, https://doi.org/10.1007/978-3-031-10784-9_9

157

158

D. J. Tillinghast and S. G. Aekanth

the use of a “computer system… to transfer the consumer’s feeling and image to the design details” [1, p. 5]. Type 3 KE utilizes a mathematical model to “reason the appropriate ergonomic design” (1). KE has been used in the early development of prominent vehicle models such as the Mazda Miata and Ford Taurus and continues to influence current work in the discipline of ergonomics. KE represents an important yet overlooked area in human factors and ergonomics. A cursory review of the Handbook of Human Factors and Ergonomics, Fourth Edition reveals that while much of the literature focuses on areas like human psychology, human-system compatibility, and engineering design, relatively little is covered in the area of affective engineering and design (Salvendy). KE, and design to satisfy human emotion in general, represents a rather useful method growing in popularity for designers of products and services. Nagamachi’s article provides a foundation for new work by present-day scholars who are building upon KE to make advances in affective design and engineering.

2 Purpose of Study While Kansei Engineering is not a new concept, its principles provide particular relevance as researchers have recently sought to use techniques such as advanced computing methods to conduct more accurate and advanced Kansei evaluations [2, p. 1]. This study applies bibliometric analysis and data mining tools to determine areas of greatest emergence in the literature related to Kansei Engineering.

3 Procedure To analyze the area of Kansei Engineering, a host of bibliometric analysis and data mining tools are used to extract insight from the current body of literature. Measures of engagement and emergence, as well as trend analysis, co-citation analysis, coauthor analysis, and categorization of leading authors and locales, will be discussed.

3.1 Identification of Emergence To gain an initial understanding of the topic of Kansei Engineering in the literature, a keyword search was conducted in SpringerLink and in Google Scholar through Harzing’s Publish or Perish software [3]. These two searches allowed for the identification of specific authors, who were then queried in ResearchGate. The range of KE applicability quickly arose: an article from SpringerLink by Zhang et al. [4] emphasized garment design with KE, while the Google Scholar search returned a 2017 paper of interest using text mining alongside KE to analyze online content for

Systematic Review on the Emergence of Kan-sei Engineering …

159

the international E-Commerce industry. Yu Hsiang Hsiao, a contributor to the 2017 paper, was then queried in ResearchGate, resulting in the discovery of a 2019 article he also co-authored discussing KE applied toward dental services [5]. Each of these articles could contribute to practical business functions in the present day. Finally, a lexical search within the Handbook of Human Factors and Ergonomics, Fourth Edition for “kansei engineering” led to the discovery of a chapter titled “Affective Engineering and Design” by Martin Helander and Halimahtun Khalid (2012), which includes a section reviewing KE. New applications for KE discovered in recent literature paired with the time-tested basis from Helander and Khalid’s work suggest KE has both validity and emergence as a research area.

3.2 Data Collection In support of each analysis conducted in this study, specific keyword searches were conducted to provide appropriate data. To collect data for co-author and co-citation analysis and to identify a trend, searches in Web of Science and in Harzing’s Google Scholar feature were conducted [6]. Table 1 indicates search terms and settings used along with the number of articles returned. Additional searches referenced later in this study are also included in Table 1. Table 1 Summary of keyword searches conducted to extract data in support of analyses presented in this study Database

Search keywords

Search settings

Harzing Google Scholar Search

kansei engineering

Year .7 [8]. For effect sizes, Cohen’s d was reported.

2.4 Participants Overall, 160 participants took part in the study. The participants’ age ranged between 19 and 79 years (M = 31.4, S D = 12.0). The sample consisted of more women (55.0%) than men (45.0%) and was well educated according to 69.4% university graduates. The majority held a driving license (93.8%) and already experienced one or more driver assistance systems themselves (85.0%). Experience with driving automation was rather low (15.0%). According to self-reports, a few of the participants experienced driving automation in public transport (e.g. subway), autonomous bus and shuttle services in pilot projects and on test tracks, or working contexts (research, automotive industry). Considering attitudes and dispositions, assessments were slightly positive in regard to technology commitment (M = 2.6, S D = 0.8), the propensity to trust technology (M = 2.7, S D = 0.6), and interpersonal trust perceptions (M = 2.6, S D = 0.7). The average decision on the willingness to take risks was on the middle of the scale (i.e. undecided: neither approval nor rejection, M = 2.0, S D = 0.9).

3 Results Results are reported according to the following structure: First, general use case evaluations are presented in regard to acceptance issues, perceptions and attitudes toward the ASS, and the participants’ current state of knowledge on this. Then, user-specific trust and distrust perceptions and conditions are outlined.

3.1 Perception and Evaluation of the Autonomous Shuttle Service The participants were generally willing to accept the ASS in public transport: They indicated interest (M = 2.7, S D = 1.0) and pleasure (M = 2.7, S D = 1.0) for this new form of mobility and, in tendency, they reported that they would like to use

Crazy Little Thing Called Trust—User-Specific Attitudes and Conditions …

243

it (M = 2.6, S D = 1.1). Also, the idea of facing the ASS in road traffic revealed open-minded perceptions and attitudes. Affective evaluations in terms of perceived surprise (M = 2.6, S D = 0.9) and joy (M = 2.3, S D = 0.9) were slightly positive. The participants were not feared (M = 1.3, S D = 1.0). They rather did not believe that something bad could happen if road traffic were dependent on the ASS (M = 1.6, S D = 1.1) or that human-controlled driving would be safer (M = 1.9, S D = 1.0). They tended to assume that the ASS would increase road safety (M = 2.4, S D = 1.6). The thought of boarding an autonomous shuttle caused nervousness for some of the participants, but not for all (M = 2.0, S D = 1.0). Being asked about their current state of knowledge on the ASS, the participants indicated interest in the topic (M = 2.5, S D = 1.0), but were not very familiar with it so far (M = 1.7, S D = 1.1) and also did not feel well informed about the state of the art (M = 1.7, S D = 1.2).

3.2 User-Specific Trust in the Autonomous Shuttle Service Human trust in the ASS was measured using two validated scales with 14 items in total (see Sect. 2.1). In order to identify trust and distrust factors, we performed a Principal Component Analysis (PCA). Prerequisites for performing a PCA were verified (Kaiser-Meyer-Olkin measure of sampling adequacy, Bartlett’s test of sphericity). Only factors with eigenvalues ≥ 1 were considered. PCA with varimax rotation revealed a two-factor solution as the best solution accounting for 63.0% of the variance. Table 1 presents the two identified factors “ASS trust” and “ASS distrust” including related items and descriptive statistics. A comparison of mean values showed that attitudes of trust (M = 2.3, S D = 0.7) and distrust (M = 1.6, S D = 0.7) differed significantly (t (159) = 7.29, p < .001, d = .577): The participants were generally trustful toward the ASS. Trusting Ones Versus Skeptics To get a broader understanding of ASS trust and distrust, a deeper investigation of the participants’ evaluation was necessary. For this purpose, a hierarchical cluster analysis was chosen to identify user segments based on their evaluations of perceived ASS trust and distrust. The hierarchical cluster analysis (ward method, euclidean distance) identified a two cluster segmentation as the most interpretable solution in the data set with n = 106 (66.3%) participants in Cluster 1 and n = 54 (33.8%) participants in Cluster 2. T-tests and Mann-WhitneyU-tests for independent samples confirmed the validity of the cluster segmentation as the two clusters significantly differed in regard to diverse user factors, ASS trust, distrust, and acceptance (see Table 2). With regard to demographic characteristics, inference statistical analyses revealed that the two clusters differed significantly regarding their age and education: Participants of Cluster 1 were younger and, with a proportion of 77.4% academics, exceptionally well educated, though, the level of education in Cluster 2 was also

244

H. Biermann et al.

Table 1 Rotated component matrix (varimax with Kaiser normalization) of the PCA and descriptive statistics of the two factors Factor 1 “ASS trust” Factor 2 “ASS distrust” Items I think the ASS* would be trustworthy I think the ASS would be reliable I would rely on the ASS I think the ASS would provide security In general, I would trust the ASS In need of help, relying on the ASS is a good idea The ASS would help me to solve many problems I think the ASS would work perfectly I think the ASS would be misleading I think ASS’s actions might have negative effects I would distrust the decisions of the ASS I would not trust information provided by the ASS I would have to be careful using the ASS I think the ASS would behave obscurely Mean value (SD)

.788 .757 .753 .711 .693 .692 .652 .624 .807 .727 .703 .682 .681 .678 2.3 (0.7)

1.6 (0.7)

* autonomous shuttle service

above average (cf. Federal Statistical Office [36]). In contrast, the clusters were not influenced by the participants’ gender, nor factors of mobility behavior. The majority of both clusters hold a driver license, was experienced in driving assistance systems, and only a minority had already gained experience with driving automation (with a higher proportion in Cluster 1). As regards attitudes and dispositions, results showed significant differences. Overall, the participants in Cluster 1 showed a greater technology commitment and were more willing to take risks. Trust in technology and interpersonal trust perceptions were positive in both clusters, but particularly high in Cluster 1. As further criteria, the participants’ evaluation of ASS trust and distrust and their perceived acceptance including their interest, pleasure, and willingness to use the ASS were investigated with regard to both clusters. The results revealed significant

Crazy Little Thing Called Trust—User-Specific Attitudes and Conditions …

245

Table 2 User characteristics, perceptions of trust, distrust, and acceptance toward the ASS* of the segmented clusters Cluster 1 (N = 106) Cluster 2 (N = 54) P “Trusting ones” “Skeptics” Age Gender Education Licensed driver Driving assistance Driving automation Willingness to take risks Technology commitment Technology trust Interpersonal trust ASS trust ASS distrust ASS acceptance *

M = 29.2 (SD = 10.2) 50.9% f; 49.1% m 77.4% academics 92.5% 86.8% experienced 17.9% experienced M = 2.1 (SD=0.8)

M = 35.8 (SD = 14.0) 63.0% f; 37.0% m 53.7% academics 96.3% 81.5% experienced 9.3% experienced M = 1.7 (SD = 0.9)

p < .01 n.s. p < .01 n.s. n.s. n.s. p < .05

M = 2.8 (SD = 0.6)

M = 2.2 (SD = 0.9)

p < .001

M = 3.0 (SD = 0.4) M = 2.8 (SD=0.6) M = 2.7 (SD = 0.4) M = 1.3 (SD = 0.4) M = 3.1 (SD = 0.8)

M = 2.3 (SD = 0.6) M = 2.3 (SD = 0.7) M = 1.6 (SD=0.5) M=2.2 (SD = 0.6) M=1.9 (SD = 0.9)

p p p p p

< .001 < .001 < .001 < .001 < .001

Autonomous shuttle service

differences indicating higher levels of acceptance and trust toward the novel mobility service in Cluster 1, whereas the participants in Cluster 2 showed a rather negative attitude and expressed stronger distrust in this context. Hence, Cluster 1 was called “Trusting ones” and Cluster 2 “Skeptics”. Conditional Trust For a better understanding of the cluster members’ perceptions and attitudes, we compared conditions (identified in preceding interviews) under which “Trusting ones” (Cluster 1) vs. “Skeptics” (Cluster 2) would more or less trust the ASS. Figure 2 shows the evaluation of conditions that increase trust differentiating between the two clusters. Apart from taking the vehicle control, which achieved similar agreements in both clusters, there were significant differences in the evaluation of trust-increasing conditions: particularly as regards sustainable propulsion technology (e.g. electric motor) (t (127.5) = 4.71, p < .001, d = .738), which was rejected in Cluster 2 (M = 1.9, S D = 0.9), but appreciated in Cluster 1 (M = 2.7, S D = 1.1), and automation of the entire transport infrastructure (i.e. no mixed traffic of autonomous and conventional vehicles) (t (158) = 4.59, p < .001, d = .767), which received a neutral (i.e. undecided) evaluation in Cluster 2 (M = 2.0, S D = 1.2), but high agreements in Cluster 1 (M = 2.9, S D = 1.1). Further evaluations showed common patterns as in both clusters the same items were accepted or rejected as conditions to trust the ASS. However, significant differences became apparent according to the level of agreement or disagreement: Responses in Cluster 2 tended to be more restrained, while Cluster 1 often expressed strong consent.

246

H. Biermann et al.

Fig. 2 User-specific evaluation of trust-increasing conditions with regard to the ASS (mean values; * corresponds to p < .05, ** corresponds to p < .01, *** corresponds to p < .001)

Regarding Cluster 1, particularly verified vehicle safety (e.g. statistics on lower accidents compared to conventional vehicles) (M = 3.5, S D = 0.7) and that the service operator would be liable in the event of an accident (M = 3.4, S D = 0.8) were considered as trust-increasing conditions. Also, vehicle appearance (M = 3.3, S D = 0.8), receiving information while driving (M = 3.3, S D = 0.7), real life demonstrations (M = 3.3, S D = 0.8), and the possibility to contact a service center (M = 3.2, S D = 0.9) were evaluated as trust conditions. Cluster 2 voted for similar conditions to increase trust, but with different priority and less agreement overall. The highest approval was given to the liability of the service operator (M = 3.0, S D = 1.0), followed by the need for vehicle control (M = 2.9, S D = 1.1). Also, shuttle appearance (M = 2.8, S D = 0.9), receiving information while driving (M = 2.8, S D = 0.8), and verified vehicle safety (M = 2.8, S D = 0.9) were perceived as trust conditions. Camera surveillance inside the vehicle was rejected as a trust-increasing condition in both clusters (Cluster 1: M = 1.9, S D = 1.2; Cluster 2: M = 1.4, S D = 1.0; t (158) = 2.24, p < .05, d = .372). Figure 3 shows the evaluation of trust-decreasing conditions of the two clusters. Overall, no significant evaluation differences were found: Participants in both clusters replied to the items in question in a similar way. In particular, no guarantee for safety was perceived as trust-decreasing in Cluster 1 (M = 3.5, S D = 0.8) and in Cluster 2 (M = 3.3, S D = 1.0). Further, aspects relating to an unstable appearance of the shuttle, questions of data and information

Crazy Little Thing Called Trust—User-Specific Attitudes and Conditions …

247

Fig. 3 User-specific evaluation of trust-decreasing conditions with regard to the ASS (mean values, n.s.)

distribution, and negative publicity were considered as trust-decreasing conditions with moderate agreements in both clusters, whereas seating against the direction of driving and the possibility of other passengers taking the vehicle control received comparatively less approval (slightly above the threshold value, i.e. center of scale) (see Fig. 3). In contrast, not being involved in the development of the shuttle service (Cluster 1: M = 1.7, S D = 1.0; Cluster 2: M = 1.9, S D = 1.0), taking the vehicle control (Cluster 1: M = 1.7, S D = 1.2; Cluster 2: M = 1.6, S D = 1.2), and no camera surveillance inside the shuttle (Cluster 1: M = 1.9, S D = 1.2; Cluster 2: M = 1.7, S D = 1.1) did not decrease trust attitudes in both clusters.

4 Discussion The aim of this study was to investigate the perspective of potential users toward an on-demand ASS that can be used by single persons or for ride sharing. The focus was on its general perception and evaluation, user-specific attitudes of trust and distrust, and conditions that increase and decrease trust in the ASS. In the following, we discuss the results obtained with special regard to aspects of human trust and distrust in automation. Also, limitations and implications for future research are considered.

248

H. Biermann et al.

4.1 Public Perception, Evaluation, and (Dis)trust In line with previous research [4, 29], the participants were open-minded toward the ASS. They showed high levels of interest and indicated their willingness to use the novel mobility service if it was available in public transport. The self-reported state of knowledge in regard to the ASS was low and only a minority of the participants already experienced driving automation. To evaluate the influence of expertise and knowledge on the perception, evaluation, and trust (as in other studies, e.g. Gold et al. [9]) in regard to our use case scenario, it could be helpful to include experts vs. laypeople in future studies and to directly contrast their opinions and perspectives. Participants were trustful toward the ASS. Though distrust assessments were significantly lower, there were no extreme ratings in terms of overall high trust and low distrust. Supporting Lewicki et al. [20], it is highly probable that both feelings of trust and distrust co-existed with an average tendency toward ASS trust, but perceived uncertainties were not completely out of question. To look into the identified dimensions of ASS trust and distrust in more detail (cf. Table 1): Items related to ASS trust referred to the vehicle’s trustworthiness, security, and functionality as well as to issues of reliance, perceived support and relief; whereas items related to ASS distrust included statements of fear and suspicion in regard to the shuttle service as a whole, but also in regard to single features, e.g. automated decision-making and information distribution. To follow up on this, future work may investigate the influence of these factors on shuttle trust and distrust in more detail, for example, using qualitative methods (e.g. interview, focus group), to find out in which situations and under which conditions these are particularly relevant and also, how they can be encouraged or compensated for. Key findings could then be quantified, e.g. using Conjoint analysis, to explore trade-offs related to ASS trust and distrust and also “must have’s” or “no go’s” for its use as basis for the technical development, public education, and information concepts.

4.2 Of Trusting Ones and Skeptics We showed that attitudes of trust and distrust toward the ASS were different for diverse users, divided into two groups: “Trusting ones” and “Skeptics”. In line with Brell et al. [5], technology commitment was higher for trustful participants; contrasting to further results of that research team, impacts of gender could not be found here: Both groups contained more women than men which, however, may be explained in regard to our sample composition, since overall more women than men participated in the present study. Also, different from e.g. Gold et al. [9], “Trusting ones” were younger than “Skeptics”. As our sample was rather young (on average), follow-up studies should explicitly address older persons who may be mobility- and/or healthimpaired in order to investigate the influence of a possibly rediscovered “freedom”

Crazy Little Thing Called Trust—User-Specific Attitudes and Conditions …

249

and flexibility in daily life, but also a perceived necessity in the use of autonomous mobility services on trust attitudes. Next to varying demographics in the sample construction to identify further trust types in the given use case context, it is important to consider another point in more detail here, namely that trust is not equal, but apparently dependent on multiple factors. With regard to our groups, both showed trust for fellow humans and technology in general. But: Whereas members of the trustful group also indicated their trust toward the ASS, enthusiasm, and a high willingness to use the novel mobility service, it was suspiciously rejected by skeptical participants. Since (at least) distrust does not appear to be a generic trait, but was rather directed exclusively against the ASS (in the group of “Skeptics”), other factors will need to be surveyed, e.g. people’s willingness to use innovation vs. fear of innovation, control dispositions, and also relations between establishing human trust toward trustees of diverse “natures” (other humans, commonly known technology, novel automation, etc.) in order to find reliable predictors of trust and distrust in this context. Obtained conditions to trust the ASS may serve as a further basis to find relevant predictors. Considering trust-increasing conditions, evaluations of the two groups significantly differed in regard to levels of (dis)agreement and priority. Key conditions to trust for the “Trusting ones” were vehicle safety and the manufacturer’s liability in the event of damage, which also received high agreements among the group of “Skeptics”. For these, moreover, the possibility of vehicle control in particular was decisive. Considering trust-decreasing conditions, no significant evaluation differences were found as both groups showed a similar voting behavior. Particularly a missing safety guarantee, an unstable vehicle design, lacking information and communication about the service in general and during the journey itself, the use of passenger data for commercial purposes, and negative media or publicity decreased trust attitudes. We can draw three conclusions from this: First, factors and conditions that constitute users’ (dis)trust in the presented ASS are similar to those in other application areas of autonomous mobility (e.g. passenger cars), particularly as regards safety and control (cf. Sect. 1.2). Second, in this sample, potential users agreed on what inhibits ASS trust (these conditions should be prevented in technical development), but significantly differed in their opinions on what promotes ASS trust (different user needs should be integrated into technical development and communication strategies). Hence, and third, the dividing point seems to be what trust is—rather than what it is not (regardless of the fact that both are important). These findings (again) point to the multifaceted nature of trust in automation and underline the importance of considering both trust and distrust dimensions individually to capture them in their full depth and range. In particular, ASS trust-increasing conditions need to be investigated in more detail as they seem to be discussed diversely—in order to take into account individual and highly personal needs and demands.

250

H. Biermann et al.

4.3 Limitations and Future Research Some limitations and outlooks on future research were already given in Sects. 4.1 and 4.2—with regard to this study’s sample (concerning factors of demography, personality, and mobility behavior to be improved) and dimensions related to trust and distrust in automation to be considered in more detail. Still, a few further points remain relating to the presented use case and the context in which this study was conducted. The introduction and evaluation of the ASS was scenario-based due to the Covid19 pandemic. As the survey was conducted in early 2020, real-life surveys were inconceivable with regard to significant infection risks. In this context, ride sharing options within the given scenario may have (implicitly) influenced the participants’ trust attitudes in a negative way. As soon as the situation improves, face-to-face simulations and experiments are recommendable, also to investigate trust factors prior and during human-shuttle interaction (cf. Hoff und Bashir [11]). As this study was conducted in Germany, key results are to be understood against the background of culturally determined norms, values, and attitudes, for example, in terms of the perceived importance and meaning of a car, which may be different in other countries and cultures. Also, culturally shaped attitudes with respect to the accepted role of technology in general, but also the public attitudes toward automation and the role automation may play in societies, might be an issue. Hence, a crosscultural and cross-national validation of our key findings would be of great help to identify and quantify further details and dimensions regarding trust in automation as well as perceptions and attitudes toward the use and implementation of innovative mobility services. Acknowledgements The authors thank all participants for their openness to share opinions on novel, autonomous mobility services. Special thanks are given to Ines Güldenberg for research assistance. This work has been funded by the Federal Ministry of Transport and Digital Infrastructure (BMVI) within the funding guideline “Automated and Networked Driving” under the project APEROL with the funding code 16AVF2134C.

References 1. Beierlein C, Kemper C, Kovaleva A, Rammstedt B (2012) Kurzskala zur Messung des zwischenmenschlichen Vertrauens: Die Kurzskala Interpersonales Vertrauen (KUSIV3). GESISWork Papers 22:1–26 2. Beierlein C, Kovaleva A, Kemper CJ, Rammstedt B (2014) Eine single-item-Skala zur Erfassung von Risikobereitschaft: Die Kurzsskala Risikobereitschaft-1 (R-1). GESIS-Work Papers 34:1–28 3. Biermann H, Philipsen R, Brell T, Ziefle M (2020) Rolling in the deep. User perspectives, expectations, and challenges of data and information distribution in autonomous driving. HumanIntell Syst Integr 4. Biermann H, Philipsen R, Brell T, Ziefle M (2020) Shut up and drive? User requirements for communication services in autonomous driving. In: Krömker H (ed) HCI in mobility, transport,

Crazy Little Thing Called Trust—User-Specific Attitudes and Conditions …

5.

6. 7. 8. 9.

10.

11. 12. 13.

14. 15. 16. 17. 18. 19. 20. 21.

22. 23. 24. 25. 26.

251

and automotive systems. Automated driving and in-vehicle experience design (HCII 2020). LNCS 12212. Springer, Switzerland, pp 3–14 Brell T, Biermann H, Philipsen R, Ziefle M (20190) Trust in autonomous technologies. A contextual comparison of influencing user factors. In: Moallem A (ed) HCI for cybersecurity, privacy and trust (HCII 2019). LNCS 11594. Springer, Switzerland, pp 371–384 Choi JK, Ji YG (2015) Investigating the importance of trust on adopting an autonomous vehicle. Int J Hum Comput Interact 31(10):692–702 Dzindolet MT, Peterson SA, Pomranky RA, Pierce LG, Beck HP (2003) The role of trust in automation reliance. Int J Hum Comput Stud 58(6):697–718 Field A (2009) Discovering statistics using SPSS, 3rd edn. Sage Publications Ltd., London Gold C, Körber M, Hohenberger C, Lechner D, Bengler K (2015) Trust in automation—before and after the experience of take-over scenarios in a highly automated vehicle. Procedia Manuf 3:3025–3032 Helldin T, Falkman G, Riveiro M, Davidsson S (2013) Presenting system uncertainty in automotive UIs for supporting trust calibration in autonomous driving. In: Proceedings of the 5th international conference on automotive user interfaces and interactive vehicular applications (AutomotiveUI 2013), pp 210–217 Hoff KA, Bashir M (2015) Trust in automation: integrating empirical evidence on factors that influence trust. Hum Fact 57(3):407–434 Iclodean C, Cordos N, Varga BO (2020) Autonomous shuttle bus for public transportation: a review. Energies 13(11):2917 Jessup SA, Schneider TR, Alarcon GM, Ryan TJ, Capiola A (2019) The measurement of the propensity to trust automation. In: Chen J, Fragomeni G (eds) Virtual, augmented and mixed reality. Applications and case studies (HCII 2019). LNCS 11575. Springer, Switzerland, pp 476–489 Jian JY, Bisantz AM, Drury CG (2000) Foundations for an empirically determined scale of trust in automated systems. Int J Cogn Ergon 4(1):53–71 Kaur K, Rampersad G (2018) Trust in driverless cars: investigating key factors influencing the adoption of driverless cars. J Eng Technol Manage 48(April):87–96 König A, Grippenkoven J (2020) Travellers’ willingness to share rides in autonomous mobility on demand systems depending on travel distance and detour. Travel Behav Soc 21:188–202 König M, Neumayr L (2017) Users’ resistance towards radical innovations: the case of the self-driving car. Transport. Res. Part F 44:42–52 Krueger R, Rashidi TH, Rose JM (2016) Preferences for shared autonomous vehicles. Transport. Res. Part C 69:343–355 Lee J, See K (2004) Trust in automation: designing for appropriate reliance. Hum Fact 46(1):50– 80 Lewicki R, McAllister D, Bies R (1998) Trust and distrust: new relationships and realities. Acad Manage Rev 23(3):438–458 Li M, Holthausen BE, Stuck RE, Walker BN (2019) No risk no trust: investigating perceived risk in highly automated driving. In: Proceedings of the 11th international ACM conference on automotive user interfaces and interactive vehicular applications (AutomotiveUI 2019), pp 177–185 Merritt SM, Ilgen DR (2008) Not all trust is created equal: dispositional and history-based trust in human-automation interactions. Hum Fact 50(2):194–210 Michałowska M, Ogłozi´nski M (2017) Autonomous vehicles and road safety. In: International conference on transport systems telematics. Springer, Cham, pp 191–202 Muir BM (1994) Trust in automation: Part I. Theoretical issues in the study of trust and human intervention in automated systems. Ergonomics 37(11):1905–1922 Muir BM, Moray N (1996) Trust in automation. Part II. Experimental studies of trust and human intervention in a process control simulation. Ergonomics 39(3):429–460 Neyer FJ, Felber J, Gebhardt C (2012) Entwicklung und Validierung einer Kurzskala zur Erfassung von Technikbereitschaft. Diagnostica 58(2):87–99

252

H. Biermann et al.

27. Parasuraman R, Riley V (1997) Humans and automation: use, misuse, disuse, abuse. Hum Fact 39(2):230–253 28. Pavone M (2015) Autonomous mobility-on-demand systems for future urban mobility. In: Maurer M, Gerdes J, Lenz B, Winner H (eds) Autonomes Fahren. Springer, Berlin Heidelberg, pp 399–416 29. Philipsen R, Brell T, Ziefle M (2019) Carriage without a driver—user requirements for intelligent autonomous mobility services. In: Stanton N (ed) 9th International conference on applied human factors and ergonomics (AHFE 2018), AISC 786. Springer, Berlin, pp 1–12 30. Raats K, Fors V, Pink S (2020) Trusting autonomous vehicles: an interdisciplinary approach. Transport Res Interdiscip Perspect 7:100201 31. SAE (2018) Taxonomy and definitions for terms related to driving automation systems for on-road motor vehicles. J3016 32. Salonen AO (2018) Passenger’s subjective traffic safety, in-vehicle security and emergency management in the driverless shuttle bus in Finland. Transp. Policy 61:106–110 33. Schaefer KE, Straub ER (2016) Will passengers trust driverless vehicles? Removing the steering wheel and pedals. In: IEEE International multi-disciplinary conference on cognitive methods in situation awareness and decision spport (CogSIMA), pp 159–165 34. Schneider T, Jessup S, Stokes C, Rivers S, Lohani M, McCoy M (2017) The influence of trust propensity on behavioral trust. In: Poster session presented at the meeting of Association for Psychological Society, Boston 35. Söderström E (2009) Trust types: an overview. In: Dhillon G (ed) Proceedings of the 8th annual security conference. Discourses in security assurance & privacy, pp 1–12 36. Statistisches Bundesamt (Destatis): Bildungsstand (2020) 37. Tenhundfeld NL, de Visser EJ, Ries AJ, Finomore VS, Tossell CC (2020) Trust and distrust of automated parking in a Tesla model X. Hum Fact 62(2):194–210 38. Ward C, Raue M, Lee C, D’Ambrosio L, Coughlin JF (2017) Acceptance of automated driving across generations: the role of risk and benefit perception, knowledge, and trust. In: Kurosu M (ed) Human-computer interaction. User interface design, development and multimodality (HCII 2017), vol LNCS 10271. Springer, Cham, pp 254–266 39. Waytz A, Heafner J, Epley N (2014) The mind in the machine: anthropomorphism increases trust in an autonomous vehicle. J Experim Soc Psychol 52:113–117

From Trust to Trust Dynamics: Combining Empirical and Computational Approaches to Model and Predict Trust Dynamics In Human-Autonomy Interaction X. Jessie Yang, Yaohui Guo, and Christoper Schemanske Abstract Trust in automation has been identified as one central factor in effective human-autonomy interaction. Despite active research in the past 30 years, most studies have used a “snapshot” view of trust and evaluated trust using questionnaires administered at the end of an experiment. This “snapshot” view does not fully acknowledge that trust is a dynamic variable that can strengthen and decay over time. With few exceptions, we have little understanding of the temporal dynamics of trust formation and evolution, nor of how trust changes over time as a result of momentto-moment interactions with autonomy. In this chapter, we present and synthesize the results of two studies examining trust dynamics in human-autonomy interaction. In study 1, we identify and define three properties of trust dynamics, namely continuity, negativity bias, and stabilization. The three properties characterize a human agent’s trust formation and evolution process de facto. In study 2, we propose a computational model of trust dynamics that adheres to the three properties and evaluate the computational model against existing trust inference models. Results show that our model provides superior prediction performance, and moreover, guarantees good model explainability and generalizability. Keywords Trust dynamics · Bayesian inference · Trust in automation · Human-robot interaction · Human-automation interaction

We use both “automation” and “autonomy” in this chapter. “Autonomy” is mainly used to reflect the recent advances in autonomous systems. “Automation” is occasionally used in order to be consistent with prior literature. X. J. Yang (B) · Y. Guo · C. Schemanske Department of Industrial and Operations Engineering, University of Michigan, 1205 Beal Avenue, Ann Arbor, MI, USA e-mail: [email protected] Y. Guo e-mail: [email protected] C. Schemanske e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Switzerland AG 2023 V. G. Duffy et al. (eds.), Human-Automation Interaction, Automation, Collaboration, & E-Services 11, https://doi.org/10.1007/978-3-031-10784-9_15

253

254

X. J. Yang et al.

1 Introduction Recent advances in autonomous systems, such as autonomous vehicles and collaborative robots, have the potential to improve every sector of our economy and improve how people live and work. However, realizing the full economic, safety, and health potential of these technologies is only possible if people establish appropriate trust in them. Trust in automation, defined as the “attitude that an agent will help achieve an individual’s goals in situations characterized by uncertainty and vulnerability [1]”, has been identified as one central factor in effective human-autonomy interaction. Despite active research in the past 30 years, existing research on trust in automation is subject to one major research gap: Most studies have used a “snapshot” view and evaluated trust with questionnaires administered at the end of an experiment (Fig. 1). More than two dozen factors have been identified to influence (snapshot) trust in automation. These factors can be broadly categorized into three groups: individual factors such as culture [2] and age [3], system factors such as reliability [4] and presentation of uncertainty information [5], and environmental factors such as multi-tasking requirement [6] and task emergency [7] (See [8, 9] for a full list of factors). However, this “snapshot” view does not fully acknowledge that trust is a dynamic variable that can strengthen and decay over time. With few exceptions (e.g., [10–14]), we have little understanding of the temporal dynamics of trust formation and evolution, nor of how trust strengthens or decays over time as a result of moment-to-moment interactions with autonomy [10, 14–16]. The goal of this chapter, therefore, is to present and synthesize the results of two studies that have examined issues pertinent to trust dynamics in human-autonomy interaction. In study 1, we identify and define three major properties of trust dynamics: continuity, negativity bias, and stabilization. In study 2, we propose a computational model of trust dynamics that adheres to the three properties identified in study 1, and evaluate the computational model against existing trust inference models.

Fig. 1 The static “snapshot” view versus the dynamic view of trust. If taking a snapshot at time t, both agents have the same trust level. However, their trust dynamics are fairly different

From Trust to Trust Dynamics: Combining Empirical …

255

2 Study 1: Three Properties of Trust Dynamics 2.1 Study 1: Method 60 university students with an average age of 23.0 years participated in Study 1, wherein they performed 40 trials of two-alternative forced-choice (2AFC) memory recognition tasks with the aid from an imperfect autonomous decision aid. The autonomous decision aid had three levels of reliability: 70, 80, and 90%, and thus had 12, 8, and 4 autonomy failures randomly located in the 40 trials, respectively. Each participant was randomly assigned to one of the three levels. The experimental testbed was adapted from the memory recognition task of Tulving [17]. Each participant first viewed a series of 150 pictures consisting of 60 target pictures and 90 buffering items (Fig. 2), followed by an interpolated task. After that, participants performed the recognition test, consisting of 40 trials of 2AFC memory recognition task (Fig. 3). For each recognition trial, participants identified the target picture when it was presented with a distractor. They first made the initial recognition selection entirely by themselves. After that, they were presented with a recommendation from an autonomous decision aid and were asked to make their final recognition selection. Once the participants made their final recognition selection, they received immediate feedback on the correctness of the recommendation given by the autonomous decision aid and of their final choice. At the end of each recognition trial, participants were asked to indicate their trust towards the autonomous decision aid using a visual analogue scale, with the leftmost point labelled “I don’t trust it at all.” and the rightmost point “I trust it completely.” The visual analog scale was then converted to a 0-100 scale. The dependent variable of interest was the trust adjustment calculated as Δti = ti − ti−1 , where ti indicates the trust after the ith trial.

Fig. 2 Participant views a series of pictures

(1)

256

X. J. Yang et al.

Fig. 3 Participant performs the recognition task. The target picture (left) is presented with a distractor (right)

2.2 Study 1: Results and Discussion A combination of the participant’s initial recognition, the recommendation from the autonomous decision aid, and the participant’s final recognition resulted in 8 (2 × 2 × 2) possible patterns. Table 1 shows the number of participants displaying each performance pattern and the mean and standard deviation (SD) of trust adjustment corresponding to each pattern. Due to the extremely low number of occurrences for patterns 1 and 6, the two patterns were discarded from the data analysis. In Study 1, we conducted a series of analyses using the linear mixed models. First, we investigated the direction of trust adjustment upon autonomy successes versus failures by comparing patterns 2, 3, and 7 (i.e., correct recommendations) against zero (i.e., no trust adjustment), and patterns 0, 4, and 5 (i.e., wrong recommendation) against zero. The mixed model analysis showed that autonomy successes (i.e., correct recommendation) increased trust (Pattern 2: F(1, 59) = 37.39, p < 0.001; Pattern 3: F(1, 60) = 137.35, p < 0.001; Pattern 7: F(1, 59) = 71.83, p < 0.001) and autonomy failures (i.e., wrong recommendations) reduced trust (Pattern 0: F(1, 60) = 33.99, p < 0.001; Pattern 4: F(1, 61) = 60.46, p < 0.001; Pattern 5: F(1, 61) = 45.78, p < 0.001). Second, we examined the magnitude of trust adjustment upon autonomy successes versus autonomy failures. We conducted two pairs of planned comparisons: the magnitude of pattern 0 versus that of pattern 2, and the magnitude of pattern 5 versus that of pattern 7. Note that the only difference between each pair was the correctness

From Trust to Trust Dynamics: Combining Empirical …

257

Table 1 8 (2 × 2 × 2) possible performance patterns (Decimal: Binary) based on combinations of human operator’s initial recognition, the recommendation provided by the autonomous decision aid, and the operator’s final recognition Initial Recommendation Final # of Trust Pattern recognition recognition Participants adjustment (decimal: binary) 0: 000 1: 001 2: 010 3: 011 4: 100 5: 101 6: 110 7: 111

Wrong (0) Wrong (0) Wrong (0) Wrong (0) Correct (1) Correct (1) Correct (1) Correct (1)

Wrong (0) Wrong (0) Correct (1) Correct (1) Wrong (0) Wrong (0) Correct (1) Correct (1)

Wrong (0) Correct (1) Wrong (0) Correct (1) Wrong (0) Correct (1) Wrong (0) Correct (1)

54 1 59 57 51 52 2 60

−4.6(6.1) NA 1.8 (2.3) 2.0 (1.3) -5.4 (5.4) -3.9 (4.5) NA 1.3 (1.2)

of the decision aid’s recommendation. The linear mixed model analysis revealed a larger magnitude of trust decrement in pattern 0 compared to the magnitude of trust increment in pattern 2 (F(1, 58) = 13.33, p = 0.001), and a larger magnitude of trust decrement in pattern 5 compared to the magnitude of trust increment in pattern 7 (F(1, 59) = 27.80, p < 0.001). Third, we were interested in investigating how trust adjustment changed as a function of interaction experience (i.e., during trials in the present study). We hypothesized that a rational human’s trust in the same autonomy would stabilize over time— their magnitude of trust adjustment will decrease as they gain more experience interacting with the same autonomy. As described earlier, a participant was randomly assigned to one of the three conditions: 4 autonomy failures and 36 successes, 8 autonomy failures and 32 successes, and 12 autonomy failures and 28 successes. We constructed two mixed linear models and the analysis results showed that as participants experienced more autonomy failures, the magnitude of their trust decrement became smaller (F(1, 33) = 38.61, β = −0.6, p < .001), i.e., an additional autonomy failure reduced the magnitude of trust decrement by 0.6. Similarly, as participants experienced more autonomy successes (F(1, 95) = 92.74, β = −0.08, p < .001), the magnitude of their trust increment became smaller, i.e., an additional autonomy success reduced the magnitude of trust increment by 0.08. Discussion. Based on the results of Study 1, we identify and define three properties of trust dynamics as a result of moment-to-moment interactions with autonomy: continuity, negative bias, and stabilization. Please note that the three properties are identified by analyzing all human participants on average. It is likely that a specific individual may not display all the three properties. • Continuity: Trust at the present moment i is significantly associated with trust at the previous moment i − 1. Results of Study 1 revealed that an autonomy success leads to a positive trust adjustment relative to trust at the previous moment (i.e.,

258

X. J. Yang et al.

Δti = ti − ti−1 > 0) while an autonomy failure results in a negative trust adjustment. The continuity property was also observed in previous work showing that ti−1 is a stronger predictor of ti [11]. • Negativity bias: Negative experiences due to autonomy failures have a greater influence on trust than positive experiences due to autonomy successes. Manzey et al. [12] proposed that there are two feedback loops in a person’s trust adjustment process, namely a positive feedback loop triggered by the experience of autonomy success and a negative feedback loop activated by autonomy failures. In addition, the negative feedback loop engenders a much stronger influence on trust adjustment than the positive feedback loop. Results of Study 1 provided empirical evidence for the negativity bias, that the magnitude of trust decrement due to autonomy failures is significantly larger than that of trust increment due to autonomy successes. • Stabilization: A person’s trust will stabilize over repeated interactions with the same autonomy. In our previous work [10], we showed that “trust of entirety”, calculated as the average of “area under the trust curve (Fig. 1)” would stabilize over time. Results of Study 1 further showed that the stabilization property is applicable to a person’s trust ti . Because a person’s moment-to-moment trust adjustment will become smaller as s/he gains more interaction experience, his or her trust ti will eventually stabilize.

3 Study 2: Computational Model of Trust Dynamics In study 1, we identify and define three properties of trust dynamics. In study 2, we propose a computational model that adheres to the three properties and evaluate the performance of the model.

3.1 Study 2: Computational Model We propose to use the Beta distribution to model a person’s temporal trust and show mathematically that the Beta distribution formulation adheres to the three properties identified in study 1. In the formulation, after the autonomous system completes the i th task, the human’s temporal trust ti follows a Beta distribution: ti ∼ Beta(αi , βi ).

(2)

The predicted trust tˆi is calculated by the expectation of ti tˆi = E(ti ) =

αi . αi + βi

(3)

From Trust to Trust Dynamics: Combining Empirical …

259

αi and βi are updated by  αi =

αi−1 + w s , if pi = 1 , , if pi = 0 αi−1

 βi−1 + w f , if pi = 0 βi = , if pi = 1 βi−1

(4)

where pi is the performance of the autonomous system on the i th task, αi and βi are the parameters of the Beta distribution, and ws and w f are the gains due to the human’s positive and negative experiences with the autonomous system. In other words, an autonomy success causes an increase in αi by ws and a failure causes an increase in βi by w f . The superscript s stands for success and f stands for failure. Next we explain how the model adheres to the three properties of trust dynamics. First, it is clear in Eq. (4) that the present trust is influenced by the previous trust, which satisfies the first property. Second, we calculate the difference between trust increment caused by autonomy success and trust decrement caused by autonomy failure at time i: (tˆi | pi =1 − tˆi−1 ) − (tˆi−1 − tˆi | pi =0 )   (5) w f αi−1 1 w s βi−1 − , = s f D+w D D+w where D = αi−1 + βi−1 . If αi−1 and βi−1 are close, Eq. (5) indicates that the autonomous system’s failure will lead to a greater trust change compared to its success when w f > w s . More s s f w , the autonomous system’s failures will have a greater precisely, when βα > ww f D+w D+ws w f impact. Figure 4 shows that within the white region the autonomous system’s failure would lead to a larger trust change. In [14] we show that w f > w s is true for most human participants, such that the second property will be satisfied when the value of w s and w f are appropriately chosen. We assume the autonomous system has a constant reliability r . After n tasks, the autonomous system accomplishes n s tasks and fails n f tasks. Then tn ∼ Beta(α0 + n s w s , β0 + n f w f ).

(6)

When n → ∞, tn will be a point mass distribution centered at α0 + n s w s r ws = , f f s s s α0 + β0 + n w + n w r w + (1 − r )w f

(7)

which is a constant and it means trust stabilizes with repeated interactions. Therefore, the proposed model satisfies the three properties of trust dynamics.

260 Fig. 4 In the white region, the autonomous system’s failure would have a greater impact on trust than the its success. Here we set w f = 50 and ws = 20

X. J. Yang et al. 300 250 200

Region where failures have larger impact on trust

150 100 50 50

100 150 200 250 300

3.2 Study 2: Method In this section, we describe the experimental dataset used to test the computational model and the method used to infer the model parameters. We used data of 39 participants from the dataset in Yang et al. [10]. Participants in the study had an average age of 24.3 years (SD = 5.0 years). All participants performed a simulated surveillance task with the aid of four drones. Each participant performed two tasks simultaneously (Fig. 5): controlling the drones using a joystick and detecting potential threats in the images captured by the drones. The participant was able to access only one task at any time and had to switch between the controlling and the detection tasks. The drones were able to detect potential threats. They would report ‘danger’ when a threat was detected. Due to environmental noises, the threat detection was

Fig. 5 Dual-task environment in the simulation testbed. The two images show displays from the simulation testbed for the tracking (left) and detection (right) tasks respectively. Participants could access only one of the two displays at a time, and could switch between them

From Trust to Trust Dynamics: Combining Empirical …

261

imperfect. The system reliability of the autonomous threat detection system was set as 70, 80, and 90%. Each participant completed 100 trials of the surveillance task with the aid from the threat detection system. After each trial, participants reported their trust, denoted as ti . The self-reported trust was used as the ground truth to train and test the proposed computational model. According to Eq. (6), we can predict a human’s trust at any moment as long as we know the values of the parameter set   θ = α0 , β0 , w s , w f .

(8)

In study 2, we use Bayesian inference to calculate the parameters. We consider a scenario where an autonomous system is to aid a new human operator on a series of tasks. We denote the autonomous system’s performance on the i th task (i.e., trial) as pi ∈ {0, 1}, where pi = 1 indicates a success and pi = 0 indicates a failure. The reliability of the autonomous system, r ∈ [0, 1], is defined as the probability that the autonomy can succeed in the task. We assume that the autonomy has the same reliability throughout the interaction experiences. At time i, after observing the autonomous system’s performance pi , the new human operator will update his or her current trust ti ∈ [0, 1] according to the autonomy’s performance history { p1 , p2 , . . . , pi }, where ti = 1 means the new human operator completely trusts the autonomous system and ti = 0 means s/he does not trust it at all. We assume that before the new human operator, k other old human operators have worked with the autonomous system, and each of the old human operator finished n tasks. Each old human operator reported his or her trust at the end of each task, so j j his or her trust history T j = {t1 , . . . , tn } and the autonomous system’s performance j j j history P = { p1 , . . . , pn } are fully available, j = 1, 2, . . . , k. Before performing a real task, the new human operator receives a training session consisting of l tasks (see Fig. 6). In the training session, the new human operator reports his or her trust after every task. After the training session, the new human operator is to perform real tasks, during which s/he can choose whether to report his or her trust towards the autonomous system occasionally at their own discretion.

Fig. 6 The new human operator receives a training session before performing the real tasks. During the training, the operator reports his or her trust after every interaction. When performing the real tasks, the operator reports his or her trust occasionally at their own discretion

262

X. J. Yang et al.

The objective of the trust prediction problem is to predict the new human operator’s trust tm after s/he finishes the m th task, based on the autonomous system’s performance history Pm = { pi |i = 1, 2, 3, . . . , m}, trust history during the training session Tmt = {ti |i = 1, 2, 3, . . . , l}, occasionally reported trust Tmo = {ti |i ∈ Om , Om ⊂ {l + 1, l + 2, . . . , m − 1}}, and the data T j and P j from the k old human operators, j = 1, 2, ..., k. Here, Om is an indicator set: Om = Om−1 ∪ {m − 1} if the user choose to report his trust after the m − 1th task, otherwise Om = Om−1 . We define trust history at time m as Tm = Tmo ∪ Tmt . Personalizing the trust model for the new human operator means finding the best θ for him or her. Here, we use the maximum a posteriori estimation (MAP) to estimate θ , which is to maximize the posterior of θ , given the autonomous system’s performance Pm , trust history Tm and autonomy reliability r . First, we have P(θ | Pm , Tm , r ) ∝P(Pm , Tm , r | θ ) P(θ ) = P(Tm | θ, Pm , r ) P(Pm , r | θ ) P(θ ) = P(Tm | θ, Pm ) P(Pm | r, θ ) P(r | θ ) P(θ ) = P(Tm | θ, Pm ) P(Pm | r ) P(r ) P(θ ) Π Beta(ti ; αi , βi ) · P(θ ). ∝

(9)

ti ∈Tm

Then θ = argmax P(θ | Pm , Tm , r ) θ Π Beta(ti ; αi , βi ) · P(θ ) = argmax θ

= argmax θ

(10)

ti ∈t

Σ

log(Beta(ti ; αi , βi )) + log P(θ ).

ti ∈Tm

The above equation shows that θ will be updated only when the human agent provides a new trust report. As P(θ ) is unknown, the model needs to learn P(θ ) first. This prior can be estimated by the empirical distribution of the parameters of the k old human agents who have previously worked with the same autonomous system. The parameter θ j of agent j is estimated via the Maximum Likelihood Estimation (MLE): θ j = argmax P(T j | θ, P j ) θ

= argmax θ

j

j

n Π

j

j

j

Beta(ti ; αi , βi ),

i=1

where αi , βi , i = 1, 2, ..., are determined by Eq. (4).

(11)

From Trust to Trust Dynamics: Combining Empirical …

263

3.3 Study 2: Results and Discussion We compared the proposed model with two existing trust inference models: the online probabilistic trust inference (Optimo) model [18] and the auto-regression moving average vector (ARMAV) [11] model. Since the Optimo and the ARMAV models use different sets of variables, we modify them so all three models use the autonomy’s performance history, but not other behavioral variables (e.g., human agent’s intervention behaviors [18]). We use root mean square error (RMSE) to evaluate the difference between the predicted value and the ground truth. The smaller the RMSE, the more accurate the prediction. For each participant h, we calculate his or her RMSE using each prediction model g / Σ100  ˆg g i=l+1 ti − ti R M S Eh = , (12) 100 − l where ti is the self-reported trust, tˆi is the predicted trust calculated using method g (i.e., our proposed model, ARMAV, and Optimo), and l is the length of the personmodel is calculated as alized training session. The RMSE for each trust prediction g 1 Σ39 the average of all the 39 participants: RMSEg = 39 h=1 RMSEh . Table 2 details the mean and standard deviation of the RMSE values of the three models when the training duration l was set to 10 and afterwards the human operator reported his or her trust every 10 trials (Fig. 6). To compare the performance of the three trust prediction models, we conducted a repeated-measure Analysis of Variance (ANOVA), followed by pairwise comparisons with Bonferroni adjustments. The omnibus AN-OVA revealed a significant difference among the three models (F(2, 76) = 21.64, p < .001). Pairwise comparisons revealed that our proposed model significantly outperforms ARMAV with a medium-large effect size (t(39) = 3.9, p < .001, Cohen’s d = 0.63), and Optimo with a large effect size (t(39) = 5.7, p < .001, Cohen’s d = 0.91). Figure 7 compares the performance of the three models. Discussion. The superior performance of our proposed model over the Optimo and the ARMAV models could have been due to two reasons: First, the proposed model captures the nonlinearity of trust dynamics, that trust stabilizes over repeated interg

Table 2 Mean and standard deviation (SD) of the RMSE values of the three models when the human operator received a training session of 10 trials and reported his or her trust every 10 trials afterwards RMSE Mean SD Proposed method ARMAV Optimo

0.072 0.101 0.139

0.053 0.052 0.080

264

X. J. Yang et al.

Fig. 7 Mean and standard error (SE) of RMSE for the three models. The error bar indicates the standard errors

action with the same autonomous system. In other words, the effect on trust due to a success or a failure from the autonomous system is dependent on the human’s interaction experience. While the first task failure from the autonomous system may cause a dramatic drop in trust, an autonomy failure after the human operator gains enough experience may not. On the contrary, the ARMAV and Optimo models employ a linear rule for updating the predicted trust, assuming consistent changes irrespective of the interaction experience. Second, although the three models define trust on a bounded interval [0,1], only our proposed method guarantees the predicted value to be bounded. The predicted trust value from ARMAV or Optimo needs to be truncated if it exceeds the defined boundary.

4 Conclusion The present work makes significant contributions to the literature in multiple ways. First, we emphasize the dynamic view of trust and examine how trust changes due to moment-to-moment interaction with autonomy. Second, we identify and define from empirical research three properties of trust dynamics, namely continuity, negativity bias, and stabilization. The three properties characterize a human’s trust development process. Third, we propose and test a computational model that adheres to the three properties. This computational model shows superior prediction performance. Furthermore, because it adheres to how people’s trust formation and evolution process de facto, it guarantees good model explainability and generalizability. The computational model offers a means to measure a human agent’s trust in autonomy in real-time without querying the human, which can be applied to design adaptive autonomy. Acknowledgements This work was partially funded by ARL Cooperative Agreement Number W911NF-20-2-0087. The views and conclusions contained in this document are those of the authors and should not be interpreted as representing the official policies, either expressed or implied, of the Army Research Laboratory or the U.S. Government. The authors would also like to thank Christine Searle and Kevin Li for developing the simulation testbeds.

From Trust to Trust Dynamics: Combining Empirical …

265

References 1. Lee JD, See KA (2004) Trust in automation: designing for appropriate reliance. Hum Fact 46(1):50–80 2. Rau PP, Li Y, Li D (2009) Effects of communication style and culture on ability to accept recommendations from robots. Comput Hum Behav 25(2):587–595 3. Ezer N, Fisk AD, Rogers WA (2008) Age-related differences in reliance behavior attributable to costs within a human-decision aid system. Hum Fact 50(6):853–863 4. Wickens CD, Dixon SR (2007) The benefits of imperfect diagnostic automation: a synthesis of the literature. Theor Issues Ergon Sci 8(3):201–212 5. Du N, Huang KY, Yang XJ (2020) Not all information is equal: effects of disclosing different types of likelihood information on trust, compliance and reliance, and task performance in human-automation teaming. Hum Fact 62(6):987–1001 6. Zhang MY, Yang XJ (2017) Evaluating effects of workload on trust in automation, attention allocation and dual-task performance. Proc Hum Fact Ergon Soc Ann Meet 61(1):1799–1803 7. Robinette P, Li W, Allen R, Howard AM, Wagner AR (2016) Overtrust of robots in emergency evacuation scenarios. In: Proceedings of the 11th ACM/IEEE international conference on human-robot interaction (HRI’16). ACM, pp 101–108 8. Hoff KA, Bashir M (2015) Trust in automation: integrating empirical evidence on factors that influence trust. Hum Fact 57(3):407–434 9. Hancock PA, Billings DR, Schaefer KE, Chen JYC, de Visser E, Parasuraman R (2011) A meta-analysis of factors affecting trust in human-robot interaction. Hum Fact 53(5):517–527 10. Yang XJ, Unhelkar VV, Li K, Shah JA (2017) Evaluating effects of user experience and system transparency on trust in automation. In: Proceedings of the 2017 ACM/IEEE international conference on human-robot interaction (HRI’17). ACM, pp 408–416 11. Lee JD, Moray N (1992) Trust control strategies and allocation of function in human-machine systems. Ergonomics 35(10):1243–1270 12. Manzey D, Reichenbach J, Onnasch L (2012) Human performance consequences of automated decision aids: the impact of degree of automation and system experience. J Cogn Eng Decis Making 6(1):57–87 13. Yang XJ, Wickens CD, Hölttä-Otto K (2016) How users adjust trust in automation: contrast effect and hindsight bias. Proc Hum Fact Ergon Soc Ann Meet 60(1):196–200 14. Guo Y, Yang XJ (2020) Modeling and predicting trust dynamics in human-robot teaming: a Bayesian inference approach. Int J Social Robot 2020. [Online]. Available: https://doi.org/10. 1080/00140130802680773 15. de Visser EJ, Peeters MM, Jung MF, Kohn S, Shaw TH, Pak R, Neerincx MA (2020) Towards a theory of longitudinal trust calibration in human-robot teams. Int J Soc Robot 12(2):459–478 16. Azevedo-Sa H, Jayaraman SK, Esterwood CT, Yang XJ, Robert LP, Tilbury DM (2020) Realtime estimation of drivers’ trust in automated driving systems. Int J Soc Robot 2020. [Online]. Available: https://doi.org/10.1007/s12369-020-00694-1 17. Tulving E (1981) Similarity relations in recognition. J Verbal Learn Verbal Behav 20(5):479– 496 18. Xu A, Dudek G (2015) OPTIMo: an online probabilistic trust inference model. In: Proceedings of the 10th annual ACM/IEEE international conference on human-robot interaction (HRI’15). ACM, pp 221–228

Calibration of Trust in Autonomous Vehicle Seul Chan Lee and Yong Gu Ji

Abstract Trust has been regarded important determinant for human-automation interactions. The importance of trust in autonomous vehicles (AVs) is even higher. If there is a mismatch between the perceived belief and actual capability of AVs, overtrust and mistrust can be built, which can lead to critical traffic accidents. Therefore, it is necessary to understand the process of building and calibrating an appropriate level of trust in AVs. This chapter reviewed related studies on trust in AVs to provide a deeper understanding of it. It also presented human-machine interfaces for building and calibrating trust in AVs from the perspective of Human Factors and Human-Computer Interaction. What and how information is provided through HMIs were described. Also, design considerations related to implementing HMIs from a building and calibrating trust perspective were discussed. We hope that this research will help to design HMIs of AVs and lead to a safer road environment in an era of AVs. Keywords Autonomous vehicle · Trust · Calibration of trust · Human–machine interface

1 Introduction Automobile industries have endeavored to develop autonomous vehicles (AVs), which are expected to become a key component of future transportation systems. AVs will provide benefits in daily life as well as in logistics, such as extending the mobility of those who were not able to use personal mobility, utilizing traverse time for other activities, and implementing novel services. The final goal of the development of automated driving systems is to enable them to drive without the driver’s S. C. Lee Department of Industrial and Systems Engineering, Gyeongsang National University, Jinju, Republic of Korea e-mail: [email protected] Y. G. Ji (B) Department of Industrial Engineering, Yonsei University, Seoul, Republic of Korea e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Switzerland AG 2023 V. G. Duffy et al. (eds.), Human-Automation Interaction, Automation, Collaboration, & E-Services 11, https://doi.org/10.1007/978-3-031-10784-9_16

267

268

S. C. Lee and Y. G. Ji

intervention, which frees drivers from driving tasks. However, it is expected that it will take more time to have a perfect version of AVs. This means that we need to spend more time in a vehicle as a driver. In other words, until we arrive at the era of fully AVs (FAVs), people as a driver should interact with different levels of automated driving systems. In general, automation refers to a machine or computing system that can perform tasks instead of human operators, such as information acquisition, analysis, decision making, and action [1]. However, an automation system is not always fully capable of handling all situations. The automation level is defined depending on the degree to which an automation system performs the tasks instead of an operator. Parasuraman et al. [1] proposed ten levels from non- to full-automation based on the replacement of the automation system over the human operator. In the automotive domain, the US Department of Transportation’s National Highway Traffic Safety Administration (NHTSA) and the Society of Automotive Engineers (SAE) respectively defined the level of AV from levels 0 to 4 and 0 to 5 with higher-level representing the increased capability of the automated driving system [2, 3]. Before the introduction of the perfect version of AV, a partially automated driving system will be able to perform only parts of driving tasks, implying that drivers still need to observe driving environments, perform maneuvering operations, check the status of AV, etc. For example, drivers are necessary to regain control of AV if the AV is unable to control the vehicle in a specific driving situation. This transition of control from the vehicle to the driver is called takeover, which has been investigated by many researchers in the field of Human Factors and Human-Computer Interaction [4–7]. In other words, drivers are required to collaborate on driving tasks with automated driving systems in a partially AV (PAV). One of the important considerations of collaborative work is trust in automated driving systems. This is because trust influences the whole process of collaborative work with automated systems [8]. As trust is a result of the dynamic process [9, 10], it changes during the use phase of an automated system depending on various factors and processes [11]. Due to this characteristic, it has been reported that the effect of trust is found in the actual use phase as well as in the early stage of adoption [12, 13]. Therefore, understanding the building and calibration of trust in AVs is important for enabling future transportation systems. In this chapter, we will review related studies on trust issues in AVs to provide a deeper understanding of it. Specifically, influential factors for trust, how to calibrate trust, and human-machine interface (HMIs) for building and calibrating trust in AVs were examined. We hope that this research will help lead to a safe road environment in an era of AVs.

Calibration of Trust in Autonomous Vehicle

269

2 Determinants and Calibration of Trust in Autonomous Vehicles 2.1 Determinants of Trust According to Oxford Learner’s Dictionary, trust is “the belief that somebody/something is good, sincere, honest, etc. and will not try to harm or trick you” The definition emphasized the characteristics of belief and attitude towards somebody/something, the intention of behavior, and results from behavior. Due to many facets of trust, it has been defined in different ways depending on various perspectives. These attributes of trust are similarly found in human-automation interaction. Lee and See [8] defined trust in automation as “the attitude that an agent will help achieve an individual’s goals in a situation characterized by uncertainty and vulnerability” That is to say, trust in automation is the expectation or belief that automation technology can help achieve the objectives in various situations as a collaborative system. Under the definition and assumption of trust, considerable research has tried to understand the role of it in automation. Especially, trust is regarded as one of the major factors in determining whether to accept AVs in the pre-use phase [12, 14–20]. Most studies have shown that a higher level of trust in AVs leads to the acceptance and adoption of them. These consistent results on the relations between trust and AVs can be seen as resulting from the fact that a single failure of dealing with driving situations can lead to fatal accidents for all people in a traffic situation. We also need to examine what determinants affect trust-building in AVs. As part of this effort, researchers have investigated various determinants of trust and acceptance of AVs including personal and sociotechnical characteristics [12, 15, 21–23]. For example, Golbabaei et al. [24] conducted a systematic review study on determinants of acceptance and intention to use of AVs. After fully reviewing 80 articles, they reported that demographics, psychological factors, and mobility behaviours are key predictors.

2.2 Calibration of Trust As explained before, trust is regarded as a key determinant of the adoption of AVs, and many variables influence the process of building trust. The questions raised here are what level of trust is appropriate and what happens if the level of trust is not built at an appropriate level. People with higher trust in AVs at an early phase of adoption are more likely to accept the technology. However, perceived awareness of the technology in this phase is likely to be based on the indirect experience, which could not be accurate [25]. According to Rousseau et al. [26], trust can be defined as “a psychological state comprising the intention to accept vulnerability based upon positive expectations of the intentions or behavior of another” Also, Mayer et al.

270

S. C. Lee and Y. G. Ji

[27] described trust as “the willingness of a party to be vulnerable to the actions of another party based on the expectation that the other will perform a particular action important to the trustor, irrespective of the ability to monitor or control that other part” In other words, there is a possibility that the capability of AVs does not match what users expect before using it because people assume the capability of the automation based on their expectations and experiences. It implies that people’s subjective beliefs and actual capability can be mismatched when it is moved to the actual use phase after accepting AVs. The problem is that the mismatch possibly leads to decreasing task effectiveness and efficiency, and this can be much more serious in the context of driving. Therefore, how to build and calibrate trust in AVs at an appropriate level is an important question. The calibration process of trust is to adjust the subjective belief on the automation system to the actual capability of the automation system [28, 29]. There are three states of trust calibration in automation systems: overtrust, optimal status, and distrust (Fig. 1). If the level of trust of an operator corresponds to the actual capability of automated systems (i.e., optimal status), the appropriate forms of collaboration can be developed. However, when the calibration of trust fails, it can lead to one of two discrepancies, i.e., overtrust and distrust. If an operator has overconfidence in the performance of an automation system than its actual capability (i.e., overtrust), then he/she will heavily depend on the system even when it is hard to function properly. On the other hand, if an operator distrusts automated systems and thereby discontinue using the automated systems, then it will be difficult to achieve the optimal level of performance through the human-automation collaborations. Because these processes of calibration of trust repeatedly continue to affect the process [12], p. 367, the performance of human-automation collaboration changes depending on overtrust and distrust [30]. The more trust operators have in the automation system, the more they rely on it. Complacency occurs in this situation, which in turn leads to difficulties in situation awareness and proper responses. If the situation continues, the performance of human-automation collaboration keeps decreasing. Fig. 1 Trust level depending on the relation between perceived automation capability and its actual reliability

Calibration of Trust in Autonomous Vehicle

271

Eventually, the ability of operators to detect and analyze the situation will decrease, and it falls into a vicious circle of complacency. On the other hand, operators who distrust the automation system will not rely on it regardless of its performance, and they will try to deal with every situation by themselves. As a result, the opportunity to increase task performances through collaboration with the automation system will be lost. These mechanisms (overtrust and distrust) can be more critical in transportation because driving is a safety-prioritized context. Distrust in automation leads drivers to turn off automated driving systems, which makes them lose the benefit of humanvehicle collaborative driving. In this case, the driver may abandon the possibility of performing the driving task efficiently, but it does not directly cause safety problems. However, overtrust in AVs can cause a situation that is more serious than losing the advantage of collaborative driving. The first fatal accident caused by an Uber selfdriving car was reported in March 2018 (Arizona, USA). The Uber AV was not able to detect the victim as a pedestrian and did not slow down the speed even though the pedestrian was on the road. It was revealed that the driver boarding the vehicle did not fully pay attention to driving situations [31]. If the driver had concentrated on the driving situation as in normal driving, the probability of the accident would have been lower. Since the Uber AV was not a FAV, the driver should have paid attention to the driving. Nevertheless, the driver was distracted because of overtrust in the automated driving system. Despite the inability of the Uber AV to cope with all situations perfectly, it could have driven without much difficulty in general situations, which leads to overtrust in the vehicle than its actual capability. This case indicates the importance of calibration between the capability of AVs and the level of trust so that keeping the driver in the loop. To help calibrate trust, it is necessary to understand the dimensions that contribute to the calibration process of trust. According to the finding of Lee and See [8], purpose, process, and performance are key dimensions of calibrating trust. Purpose refers to the intention of the designer in an automated system and the degree to which automation is being used as intended. Process describes the steps and algorithms of the automation to achieve its purpose. Performance refers to the current and previous level of achievement of purpose in terms of reliability, predictability, and ability. The process of building and calibrating trust in automation has been articulated based on these three dimensions. First, designers and developers establish the goal of collaborative tasks that should be achieved through automated systems. After implementing an automated system, it is important to ensure that it works properly to achieve the purpose as intended. While working with the automated system, the user will evaluate whether the system contributes to the achievement of tasks appropriately. The trust level will be built on the contribution of the system for achieving the goal at the expected level. Then, the trust level will be adjusted and changed depending on whether the user can understand how the system works in the process and why the result happens. Trust in the system will decrease if the user cannot even identify the causes in situations where the system does not function well. However, even though the system does not work well, the user can understand what is going on if the system communicates to the user about the system status and how the system works. In sum,

272

S. C. Lee and Y. G. Ji

trust in an automation system can be built at an appropriate level when the system clearly communicates to users about the purpose, process, and performance of it.

3 Human–Machine Interfaces for Calibrating Trust in AVs The next question is how we can successfully calibrate trust in AVs. The development of advanced automated driving systems is being pursued to increase the capability of AVs, thereby satisfying the high level of expectation and trust. However, even though AVs have the same capability and performance, the calibration of trust can be different depending on how HMIs are designed [32]. What should be discussed from the perspective of Human Factors and Human-Computer Interaction is how to calibrate the level of trust through HMIs, type and communication strategy of HMIs, and design space.

3.1 In-Vehicle HMIs Versus External HMIs There are two categories of HMIs depending on the involved users and physical location. The first type is in-vehicle HMIs that have been used from the context of manual driving. In earlier days, simple in-vehicle systems were available, such as playing music and adjusting temperature. With the advance of in-vehicle intelligent systems, vehicles provide drivers with various functions. Accordingly, in-vehicle HMIs have become complex to operate functions and present information. The traditional information display (e.g., instrument cluster) primarily presented information about vehicle status (e.g., speed, RPM, and fuel), mechanical issues (e.g., system malfunction warnings), and functional feedback (e.g., directional light). Recently, different types of visual displays, e.g., cluster display and navigational display, have been considered to present various information related to vehicle status, climates, multimedia, entertainment, and navigation. Also, the number and types of controllers have been increasing to operate in-vehicle systems [33]. Only simple push buttons or knobs were used in the past, but in recent years, various types of controllers are utilized, e.g., complex. Further, the use of touchscreen-based displays has increased following the success of the smartphone. In the manual driving context, the focus of researchers in designing in-vehicle HMIs was to enable drivers to perform non-driving related tasks without having negative effects on driving task performances [34–38]. According to the gradual change to FAVs, in-vehicle HMIs are expected to use for more diverse purposes including the calibration of trust in AVs. Many vehicles with partially automated driving functions adopt HMIs to provide warning messages if drivers are out-of-theloop because of overtrust in automated driving systems. The vehicle requires a driver to return to driving tasks by providing repeated warning messages (Fig. 2). If a driver does not return to driving tasks regardless of warning messages, the vehicle turns

Calibration of Trust in Autonomous Vehicle

273

Fig. 2 Examples of steering wheel hands off alert (image source: left—https://www.extremetech. com/extreme/161832-2014, right—http://www.kniro.net/warning-472.html)

off automated driving functions to prevent critical situations. It is confirmed that such systems to inform drivers about the capability of the automated driving system are effective. Helldin et al. [39] reported that drivers who were reported about the uncertainty of automated driving system showed faster responses when necessary, which comes from the less trust in the driving system. In other words, presenting information about the uncertainty of the automated driving system leads to a more proper level of trust calibration. Also, researchers seek to increase trust in FAVs by utilizing different in-vehicle HMIs [40–42]. Oliveira et al. [41] investigated the influence of system transparency on trust and user experiences in FAVs. They conducted surveys and interviews on trust after the ride of FAVs with different HMIs. Although not all measures showed a significant difference compared to the baseline condition, the augmented reality display and animated representation of the environment interface was favored. Also, Niu et al. [43] reported that anthropomorphizing information can increase trust in FAVs after conducting a driving simulation experiment with or without anthropomorphic interfaces. Ruijten et al. [44] also found a similar result that people experiencing the autonomous driving journey with anthropomorphic HMIs that present driving-related information showed a higher level of trust in FAVs. The second type is external human-machine interfaces (eHMIs), which are installed outside of vehicles rather than inside of vehicles. The concepts of eHMIs were newly proposed to replace communications between drivers and road users. In the manual driving context, drivers directly interact with road users, for example, flickering emergency light to deliver messages “sorry” or “thank you” This way of communications is observed in all over the world even though the meaning of interaction can be somewhat different across cultures [45]. However, once FAVs are available, people in the vehicle will not focus on driving environments anymore. Accordingly, FAVs will take the responsibility of interaction with road users, which is important for calibrating trust in AVs from the view of road users. One of the representative scenarios is communicating with pedestrians. FAVs deliver the driving intention or vehicle status through eHMIs [46, 47]. Recent discussions on eHMIs have expanded to the interaction with various road users such as other vehicles [48, 49].

274

S. C. Lee and Y. G. Ji

3.2 Strategy for Calibrating Trust in AVs: System Transparency The target user of in-vehicle HMIs and eHMIs is different. The former focuses on boarding passengers and the latter is for interacting with road users. Nevertheless, both types of HMIs take the same strategy in terms of pursuing trust calibration by increasing the transparency of AVs. To calibrate trust in AVs, the concepts of HMIs have been proposed according to the dimension of trust mentioned earlier (purpose, process, and performance). First, it has been reported that in-vehicle HMIs are effective to increase trust and lead to positive user experiences in FAVs. This is because the driving mechanism of AVs based on machine learning algorithms is completely different from that of human drivers [50]. However, not all people understand how AVs can receive surround information, respond to driving situations, and operate the vehicle. A misunderstanding could happen, and it can increase anxiety and lead to distrust in AVs. Thus, reducing this misunderstanding can reduce distrust in AVs. To achieve this, AVs present information about what they perceive, driving intentions, system status, etc. For example, the autopilot systems of Tesla presented presents detected surrounding vehicles, lanes, and near-future driving plans. Empirical evidence of these systems supported the effectiveness of presenting purpose [43], process [41], and performance [39, 44]. Investigating the effect of eHMIs has also been conducted with the same strategy [48, 51–55]. First, delivering the AV’s driving intention or status to road users is a way of communication. Then, road users can decide what to do based on the information about whether the AV will pass by, slow down, or recognize pedestrians. Another way of communication is to present messages that directly give instructions or guidance instead of delivering the AV’s driving intention or status. Lastly, researchers also investigated eHMIs presenting information that allows road users to decide for their next behavior. For example, a laser or projection display installed on an AV projects a crosswalk pattern on the front floor of the vehicle. Through this, people can guess that the vehicle will stop from a metaphor of crosswalk, and it will help decide the next step.

3.3 Design Considerations of HMIs for the Calibration Process of Trust It has been confirmed that system transparency is an effective way of communications to calibrate trust in AVs. The next question is how to design HMIs for system transparency. In this section, design considerations of HMIs for the calibration process of trust are discussed. In-vehicle HMI. Different design considerations are depending on the context of PAVs and FAVs. In the context of PAVs, it is necessary to contemplate how we should introduce the idea of the level of AVs to users. The level of the automation

Calibration of Trust in Autonomous Vehicle

275

system is generally categorized depending on the degree to which an automation system performs different tasks instead of users [1]. In the same line, the level of AVs is defined by NHTSA or SAE [2, 3]. The concept of automation level is helpful for researchers and designers to have a blueprint for automation systems and to translate it into actual functions. However, drivers do not perform driving tasks in different stages such as information acquisition, information processing, and action. All information stages are simultaneously and dynamically processed as a whole task. It implies that it would be difficult for drivers to accept the idea that the part of tasks are being replaced by automated driving systems because the tasks cannot be partially replaced like a puzzle piece. In addition, it is often difficult for drivers to judge whether it can be operated or not depending on the road conditions. For example, the highway driving assistance system is designed only in highway conditions. However, the condition “in highway” is not sometimes clear for drivers. In other words, the intention of designer and the understanding of actual drivers about partially automated driving systems could differ, which have an unexpected impact on trust in AVs and drivers’ behaviors. As an example, much research supported that 7 s. is a minimum requirement for drivers to safely take over the control from PAVs in emergent situations [56]. However, according to the findings of Frison et al. [57], drivers were unable to safely resume the control within 7 s even though the vehicle speed is relatively low (30 km/h). It suggests that the understanding of AVs and building trust in AVs of drivers could be different from what developers and designers expect, which could lead to critical issues. Therefore, before having a perfect version of AVs, it should be considered how we should deliver the concept of automated driving systems to drivers to build an appropriate level of trust and make proper actions. The focus of designing in-vehicle HMIs for trust in FAVs is to find out effective ways to enhance trust. Even if a vehicle can deal with all driving situations, people still try to protect themselves against uncertainties. This is why increasing system transparency leads to an increase in trust in FAVs. In this regard, a comprehensive discussion is made on design variables to find out more effective ways to present information about autonomous driving systems [41, 43, 58–61]. eHMI. Since eHMI is a new type of user interface that has not existed before, researchers and practitioners have tried to understand the overall picture of the design space. Dey et al. [62] analyzed seventy existing eHMI concepts and elicited eighteen dimensions related to eHMIs. Actually, researchers have investigated the effects of these dimensions and design variables of eHMIs such as modality [63], color [64], nature of message [54, 55], placement [48, 65], vehicle status [55], and communication strategy [53, 66]. In this chapter, we did not cover the detailed results of the empirical findings from each study. Instead, we discussed the importance of understanding the holistic picture of interaction with eHMIs. Although the initial purpose of eHMI may focus on communication with pedestrians, more complex factors need to be considered at the actual use of the interface. There are many road users whom AVs need to

276

S. C. Lee and Y. G. Ji

interact with other than pedestrians, such as other vehicles, bicyclists, etc. Interaction with these road users is different from that with pedestrians in terms of information contents, type, and presentation methods. For example, eHMIs installed on the front side of AVs cannot be used for the interaction with side or rear side vehicles. Further, there will be side effects that are not intended to. The advanced in-vehicle systems provide drivers with various functions and information related to driving, vehicle status, entertainment, etc. Although these systems are helpful in a way, these systems increase interface complexity. The increased level of visual and manual complexity has a negative effect on driving performance and visual distraction [33, 35, 67, 68]. Similarly, it should be considered that the newly introduced eHMIs could make unintended consequences. Many AVs with eHMIs on the road can cause visual distractions of other drivers, and problems such as glare due to eHMIs can arise in night driving.

4 Summary This chapter reviewed related studies on trust in AVs to provide a deeper understanding of the topic. Specifically, influential factors for trust, calibration process of trust, and HMIs for building and calibrating trust in AVs were examined. We found that the trust building process and HMIs for calibrating trust has been researched extensively. However, as research on this issue is in a nascent stage, we do not have a holistic picture of future driving scenarios yet. Recently, several studies have reported the limitation of the takeover HMI in PAVs [57] and adverse consequences of eHMIs [69]. These findings indicate that the effects of the introduction of AVs and new HMIs on the interaction with users could be different from our expectations. Therefore, given that transportation context is closely related to safety issues, more elaborate efforts to precisely understand the process of building and calibrating trust from more diverse perspectives are required. Acknowledgements This work was supported under the framework of international joint research program managed by the National Research Foundation of Korea (NRF-2022K2A9A2A08000167).

References 1. Parasuraman R, Sheridan TB, Wickens CD (2000) A model for types and levels of human interaction with automation. IEEE Trans Syst Man, Cybern Part A Syst Humans Publ IEEE Syst Man Cybern Soc 30:286–97 (2000). https://doi.org/10.1109/3468.844354 2. SAE international: Taxonomy and definitions for terms related to driving automation systems for on-road motor vehicles (2018) 3. Administration NHTS. Preliminary statement of policy concerning automated vehicles, Washington DC

Calibration of Trust in Autonomous Vehicle

277

4. Yoon SH, Lee SC, Ji YG (2021) Modeling takeover time based on non-driving-related task attributes in highly automated driving. Appl Ergon 92:103343. https://doi.org/10.1016/j.ape rgo.2020.103343 5. Yoon SH, Kim YW, Ji YG (2019) The effects of takeover request modalities on highly automated car control transitions. Accid Anal Prev 123:150–158. https://doi.org/10.1016/j.aap. 2018.11.018 6. Lee SC, Yoon SH, Ji YG (2020) Effects of non-driving-related task attributes on takeover quality in automated vehicles. Int J Human–Comput Inter 1–9 (2020). https://doi.org/10.1080/ 10447318.2020.1815361 7. Yoon SH, Ji YG (2019) Non-driving-related tasks, workload, and takeover performance in highly automated driving contexts. Transport Res F: Traffic Psychol Behav 60:620–631. https:// doi.org/10.1016/j.trf.2018.11.015 8. Lee JD, See KA (2004) Trust in automation: designing for appropriate reliance. Hum Factors 46:50–80. https://doi.org/10.1518/hfes.46.1.50_30392 9. Lee J, Moray N (1992) Trust, control strategies and allocation of function in human-machine systems. Ergonomics 35:1243–1270. https://doi.org/10.1080/00140139208967392 10. Cohen MS, Parasuraman R, Freeman JT (1998) Trust in decision aids: a model and its training implications. In: Proceeding of command and control research and technology symposium 11. Mcknight DH, Cummings LL, Chervany NL (1998) Initial trust formation in new organizational relationships. Acad Manag Rev 23:473–490 12. Choi JK, Ji YG (2015) Investigating the importance of trust on adopting an autonomous vehicle. Int J Human-Comput Inter 31:692–702. https://doi.org/10.1080/10447318.2015.1070549 13. Hartwich F, Witzlack C, Beggiato M, Krems JF (2019) The first impression counts—a combined driving simulator and test track study on the development of trust and acceptance of highly automated driving. Transport Res F: Traffic Psychol Behav 65:522–535. https://doi.org/10. 1016/j.trf.2018.05.012 14. Xu Z, Zhang K, Min H, Wang Z, Zhao X, Liu P (2018) What drives people to accept automated vehicles? Findings from a field experiment. Transport Res Part C: Emerg Technol 95:320–334. https://doi.org/10.1016/j.trc.2018.07.024 15. Molnar LJ, Ryan LH, Pradhan AK, Eby DW, St. Louis RM, Zakrajsek JS (2018) Understanding trust and acceptance of automated vehicles: an exploratory simulator study of transfer of control between automated and manual driving. Transport Res Part F: Traffic Psychol Behav 58:319– 328 (2018). https://doi.org/10.1016/j.trf.2018.06.004 16. Zhang T, Tao D, Qu X, Zhang X, Lin R, Zhang W (2019) The roles of initial trust and perceived risk in public’s acceptance of automated vehicles. Transport Res Part C: Emerg Technol 98:207– 220. https://doi.org/10.1016/j.trc.2018.11.018 17. Lee J, Lee D, Park Y, Lee S, Ha T (2019) Autonomous vehicles can be shared, but a feeling of ownership is important: examination of the influential factors for intention to use autonomous vehicles. Transport Res Part C: Emerg Technol 107:411–422. https://doi.org/10.1016/j.trc. 2019.08.020 18. Dirsehan T, Can C (2020) Examination of trust and sustainability concerns in autonomous vehicle adoption. Technol Soc 63. https://doi.org/10.1016/j.techsoc.2020.101361 19. Zhang T, Tao D, Qu X, Zhang X, Zeng J, Zhu H, Zhu H (2020) Automated vehicle acceptance in China: social influence and initial trust are key determinants. Transport Res Part C: Emerg Technol 112:220–233. https://doi.org/10.1016/j.trc.2020.01.027 20. Panagiotopoulos I, Dimitrakopoulos G (2018) An empirical investigation on consumers’ intentions towards autonomous driving. Transport Res Part C: Emerg Technol 95:773–784. https:// doi.org/10.1016/j.trc.2018.08.013 21. Lee JD, Kolodge K (2020) Exploring trust in self-driving vehicles through text analysis. Hum Factors 62:260–277. https://doi.org/10.1177/0018720819872672 22. Gold C, Körber M, Hohenberger C, Lechner D, Bengler K (2015) Trust in automation—before and after the experience of take-over scenarios in a highly automated vehicle. Procedia Manuf 3:3025–3032. https://doi.org/10.1016/j.promfg.2015.07.847

278

S. C. Lee and Y. G. Ji

23. Buckley L, Kaye SA, Pradhan AK (2018) Psychosocial factors associated with intended use of automated vehicles: a simulated driving study. Accid Anal Prev 115:202–208. https://doi. org/10.1016/j.aap.2018.03.021 24. Golbabaei F, Yigitcanlar T, Paz A, Bunker J (2020) Individual predictors of autonomous vehicle public acceptance and intention to use: a systematic review of the literature. J Open Innov Technol Market Compl 6:1–27. https://doi.org/10.3390/joitmc6040106 25. Gallivan MJ (2001) Organizational adoption and assimilation of complex technological innovations: development and application of a new framework. Data Base Adv Inf Syst. https://doi. org/10.1145/506724.506729 26. Rousseau DM, Sitkin SB, Burt RS, Camerer C (1998) Not so different after all: a cross-discipline view of trust. Acad Manag Rev 23:393–404 27. Mayer RC, Davis JH, Schoorman FD (1995) An integrative model of organizational trust. Acad Manag Rev 20:709–734 28. Muir BM (1994) Trust in automation: Part I. Theoretical issues in the study of trust and human intervention in automated systems. Ergonomics 37:1905–1922. https://doi.org/10.1080/001 40139408964957 29. Lee JD, Wickens CD, Liu Y, Boyle LN. Designing for people: an introduction to human factors engineering. CreateSpace 30. Karrer K, Roetting M (2007) Effects of driver fatigue monitoring—an expert survey. In: Lecture notes in computer science (including subseries lecture notes in artificial intelligence and lecture notes in bioinformatics), pp 324–330. Springer Verlag. https://doi.org/10.1007/9783-540-73331-7_35 31. News B. Uber’s self-driving operator charged over fatal crash, https://www.bbc.com/news/tec hnology-54175359 32. Körber M, Baseler E, Bengler K (2018) Introduction matters: manipulating trust in automation and reliance in automated driving. Appl Ergon 66:18–31. https://doi.org/10.1016/j.apergo. 2017.07.006 33. Lee SC, Ji YG (2019) Complexity of in-vehicle controllers and their effect on task performance. Int J Human-Comput Inter 35:65–74. https://doi.org/10.1080/10447318.2018.1428263 34. Lee SC, Yoon SH, Ji YG (2019) Modeling task completion time of in-vehicle information systems while driving with keystroke level modeling. Int J Ind Ergon 72:252–260. https://doi. org/10.1016/j.ergon.2019.06.001 35. Lee SC, Kim YW, Ji YG (2019) Effects of visual complexity of in-vehicle information display: age-related differences in visual search task in the driving context. Appl Ergon 81:102888. https://doi.org/10.1016/j.apergo.2019.102888 36. Jeon M, Walker BN, Gable TM (2015) The effects of social interactions with in-vehicle agents on a driver’s anger level, driving performance, situation awareness, and perceived workload. Appl Ergon 50:185–199. https://doi.org/10.1016/j.apergo.2015.03.015 37. Zahabi M, Kaber D (2018) Effect of police mobile computer terminal interface design on officer driving distraction. Appl Ergon 67:26–38. https://doi.org/10.1016/j.apergo.2017.09.006 38. Harvey C, Stanton NA, Pickering CA, MacDonald M, Zheng M (2011) A usability evaluation toolkit for in-vehicle information systems (IVISs). Appl Ergon 42:563–574. https://doi.org/10. 1016/j.apergo.2010.09.013 39. Helldin T, Falkman G, Riveiro M, Davidsson S (2013) Presenting system uncertainty in automotive UIs for supporting trust calibration in autonomous driving. In: Proceedings of the 5th international conference on automotive user interfaces and interactive vehicular applications—automotiveUI’13, pp 210–217. ACM Press, New York, New York, USA. https://doi. org/10.1145/2516540.2516554 40. Zihsler J, Hock P, Walch M, Dzuba K, Schwager D, Szauer P, Rukzio E (2016) Carvatar: increasing trust in highly-automated driving through social cues. In: Proceedings of the 8th international conference on automotive user interfaces and interactive vehicular applications adjunct—automotive’ui 16, pp 9–14. ACM Press, New York, New York, USA. https://doi.org/ 10.1145/3004323.3004354

Calibration of Trust in Autonomous Vehicle

279

41. Oliveira L, Burns C, Luton J, Iyer S, Birrell S (2020) The influence of system transparency on trust: evaluating interfaces in a highly automated vehicle. Transport Res F: Traffic Psychol Behav 72:280–296. https://doi.org/10.1016/j.trf.2020.06.001 42. Ekman F, Johansson M, Sochor J (2018) Creating appropriate trust in automated vehicle systems: a framework for HMI design. IEEE Trans Human-Mach Syst 48:95–101. https:// doi.org/10.1109/THMS.2017.2776209 43. Niu D, Terken J, Eggen B (2018) Anthropomorphizing information to enhance trust in autonomous vehicles. Human Factors Ergon Manuf 28:352–359. https://doi.org/10.1002/hfm. 20745 44. Ruijten P, Terken J, Chandramouli S (2018) Enhancing trust in autonomous vehicles through intelligent user interfaces that mimic human behavior. Multimodal Technol Inter 2:62. https:// doi.org/10.3390/mti2040062 45. Lee SC, Stojmenova K, Sodnik J, Schroeter R, Shin J, Jeon M (2019) Localization vs. internationalization: research and practice on autonomous vehicles across different cultures. In: Proceedings of the 11th international conference on automotive user interfaces and interactive vehicular applications adjunct proceedings—automotiveUI ’19, pp 7–12. ACM Press, New York, New York, USA. https://doi.org/10.1145/3349263.3350760 46. Faas SM, Kao AC, Baumann M (2020) A longitudinal video study on communicating status and intent for self-driving vehicle—pedestrian interaction. In: Proceedings of the 2020 CHI conference on human factors in computing systems, pp 1–14. ACM, New York, NY, USA. https://doi.org/10.1145/3313831.3376484 47. Ackermann C, Beggiato M, Schubert S, Krems JF (2019) An experimental study to investigate design and assessment criteria: What is important for communication between pedestrians and automated vehicles? Appl Ergon 75:272–282. https://doi.org/10.1016/j.apergo.2018.11.002 48. Kim YW, Han JH, Ji YG, Lee SC (2020) Exploring the effectiveness of external human-machine interfaces on pedestrians and drivers. In: Adjunct proceedings—12th international ACM conference on automotive user interfaces and interactive vehicular applications, automotiveUI 2020, pp 65–68. https://doi.org/10.1145/3409251.3411725 49. Rettenmaier M, Albers D, Bengler K (2020) After you?!—use of external human-machine interfaces in road bottleneck scenarios. Transport Res F: Traffic Psychol Behav 70:175–190. https://doi.org/10.1016/j.trf.2020.03.004 50. Li L, Ota K, Dong M (2018) Humanlike driving: empirical decision-making system for autonomous vehicles. IEEE Trans Veh Technol 67:6814–6823. https://doi.org/10.1109/TVT. 2018.2822762 51. Nuñez Velasco JP, Farah H, van Arem B, Hagenzieker MP, Velasco JPN, Farah H, van Arem B, Hagenzieker MP (2019) Studying pedestrians’ crossing behavior when interacting with automated vehicles using virtual reality. Transport Res Part F: Traffic Psychol and Behav 66:1–14. https://doi.org/10.1016/j.trf.2019.08.015 52. Daniels PT, Share DL (2018) Writing system variation and its consequences for reading and dyslexia. Sci Stud Read 22:101–116. https://doi.org/10.1080/10888438.2017.1379082 53. Song YE, Lehsing C, Fuest T, Bengler K (2018) External HMIs and their effect on the interaction between pedestrians and automated vehicles. Adv Intell Syst Comput 722:13–18. https://doi. org/10.1007/978-3-319-73888-8_3 54. Kooijman L, Happee R, de Winter JCFF (2019) How do eHMIs affect pedestrians’ crossing behavior? A study using a head-mounted display combined with a motion suit. Information 10. https://doi.org/10.3390/info10120386 55. Faas SM, Mathis LA, Baumann M (2020) External HMI for self-driving vehicles: Which information shall be displayed? Transport Res F: Traffic Psychol Behav 68:171–186. https:// doi.org/10.1016/j.trf.2019.12.009 56. Zhang B, de Winter J, Varotto S, Happee R, Martens M (2019) Determinants of take-over time from automated driving: a meta-analysis of 129 studies. Transport Res F: Traffic Psychol Behav 64:285–307. https://doi.org/10.1016/j.trf.2019.04.020 57. Frison A, Wintersberger P, Schartmüller C, Riener A (2019) The real T(h)OR: evaluation of emergency take-over on a test track. In: Proceedings of the 11th international conference

280

58.

59.

60.

61.

62.

63.

64.

65.

66.

67.

68.

69.

S. C. Lee and Y. G. Ji on automotive user interfaces and interactive vehicular applications adjunct proceedings— automotiveUI ’19, pp 478–482. ACM Press, New York, New York, USA. https://doi.org/10. 1145/3349263.3349602 Lee SC, Sanghavi H, Ko S, Jeon M (2019) Autonomous driving with an agent: speech style and embodiment. In: Proceedings of the 11th international conference on automotive user interfaces and interactive vehicular applications adjunct proceedings—automotiveUI ’19, pp 209–214. ACM Press, New York, New York, USA. https://doi.org/10.1145/3349263.3351515 Miglani A, Diels C, Terken J (2016) Compatibility between trust and non-driving related tasks in UI design for highly and fully automated driving. In: Proceedings of the 8th international conference on automotive user interfaces and interactive vehicular applications adjunct—automotive’UI 16, pp 75–80. ACM Press, New York, New York, USA. https://doi.org/10.1145/300 4323.3004331 Large DR, Burnett G, Harrington K, Clark L, Luton J, Thomas P, Bennett P (2019) It’s small talk, jim, but not as we know it. Engendering trust through human-agent conversation in an autonomous, self-driving car. In: Proceedings of the 11th international conference on automotive user interfaces and interactive vehicular applications—automotiveUI ’19. https://doi.org/ 10.1145/3342775.3342789 Large DR, Harrington K, Burnett G, Luton J, Thomas P, Bennett P (2019) To please in a pod: employing an anthropomorphic agent-interlocutor to enhance trust and user experience in an autonomous, self-driving vehicle. In: Proceedings of the 11th international conference on automotive user interfaces and interactive vehicular applications—automotiveUI ’19, pp 49–59. ACM Press, New York, New York, USA. https://doi.org/10.1145/3342197.3344545 Dey D, Habibovic A, Löcken A, Wintersberger P, Pfleging B, Riener A, Martens M, Terken J (2020) Taming the eHMI jungle: a classification taxonomy to guide, compare, and assess the design principles of automated vehicles’ external human-machine interfaces. Transport Res Interdisc Perspect 7. https://doi.org/10.1016/j.trip.2020.100174 Florentine E, Ang MA, Pendleton SD, Andersen H, Ang MH (2016) Pedestrian notification methods in autonomous vehicles for multi-class mobility-on-demand service. dl.acm.org. 387– 392. https://doi.org/10.1145/2974804.2974833 Li Y, Dikmen M, Hussein TG, Wang Y, Burns C (2018) To cross or not to cross: urgency-based external warning displays on autonomous vehicles to improve pedestrian crossing safety. In: Proceedings of the 10th international conference on automotive user interfaces and interactive vehicular applications—automotiveUI ’18, pp 188–197. ACM Press, New York, New York, USA. https://doi.org/10.1145/3239060.3239082 Eisma YB, van Bergen S, ter Brake SM, Hensen MTT, Tempelaar WJ, de Winter JCF (2020) External human-machine interfaces: the effect of display location on crossing intentions and eye movements. Information (Switzerland) 11. https://doi.org/10.3390/info11010013 Löcken A, Golling C, Riener A (2019) How should automated vehicles interact with pedestrians? A comparative analysis of interaction concepts in virtual reality. In: Proceedings— 11th international ACM conference on automotive user interfaces and interactive vehicular applications, automotiveUI 2019, pp 262–274. https://doi.org/10.1145/3342197.3344544 Lee SC, Hwangbo H, Ji YG (2016) Perceived visual complexity of in-vehicle information display and its effects on glance behavior and preferences. Int J Human-Comput Inter 32:654– 664. https://doi.org/10.1080/10447318.2016.1184546 Hwangbo H, Lee SC, Ji YG (2016) Complexity overloaded in smart car: How to measure complexity of in-vehicle displays and controls? In: Adjunct proceedings of the 8th international conference on automotive user interfaces and interactive vehicular applications, pp 81–86. https://doi.org/10.1145/3004323.3004332 Kaleefathullah AA, Merat N, Lee YM, Eisma YB, Madigan R, Garcia J, de Winter J (2020) External human-machine interfaces can be misleading: an examination of trust development and misuse in a CAVE-based pedestrian simulation environment. Hum Factors. https://doi.org/ 10.1177/0018720820970751

Human-Automation Interaction for Semi-Autonomous Driving: Risk Communication and Trust Jing Chen, Scott Mishler, Shelby Long, Sarah Yahoodik, Katherine Garcia, and Yusuke Yamani

Abstract Autonomous vehicles (AVs) are expected to play an increasingly important role in future transportation systems as a promising means of improving road safety and efficiency by eventually replacing human-driven vehicles. Semiautonomous vehicles (semi-AVs; SAE Level 2 and Level 3) feature automatic lateral and longitudinal control of the vehicle with human drivers required to supervise the system at all times (Level 2) or prepared to resume control when requested (Level 3). As these definitions reveal, semi-AVs still require human oversight and intervention to fully ensure safety. Humans are required to monitor and be ready to take over control when the vehicle fails to recognize or respond to hazardous events. Thus, it is essential to ensure effective human-automation interaction and collaboration for semi-AVs. This book chapter will discuss the critical challenges for effective human-automation interaction for semi-autonomous driving, including communicating potential risks to human drivers and maintaining proper driver trust in the semi-AV. Risks in the current context are moving or stationary objects and road environments that impose imminent threats to drivers, including overt hazards such as road obstacles, a pedestrian crossing the road, and an intruding vehicle, or covert hazards such as a pedestrian that is about to cross but is occluded by a parked truck or a roadway structure. We discuss the design of effective risk communication mechanisms to convey these risks to the human driver, which helps maintain the driver’s situation awareness and facilitate the driver’s actions when needed. In addition, the effectiveness of this risk communication can be influenced not only by the characteristics of the driver and the semi-AV, but also their interaction. Finally, we will discuss factors that affect drivers’ trust in semi-AVs and subsequently how it affects effective risk communication in semi-AV driving. Keywords Semi-autonomous driving · Trust · Risk communication Autonomous vehicles (AVs) are expected to be the future of surface transportation on the roadways. A multitude of assistive driving technologies have already been J. Chen (B) · S. Mishler · S. Long · S. Yahoodik · K. Garcia · Y. Yamani Old Dominion University, Norfolk, VA, USA e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Switzerland AG 2023 V. G. Duffy et al. (eds.), Human-Automation Interaction, Automation, Collaboration, & E-Services 11, https://doi.org/10.1007/978-3-031-10784-9_17

281

282

J. Chen et al.

integrated into many new vehicles such as lane keeping systems, forward collision warning systems, and adaptive cruise control, just to name a few. AVs seek to integrate these assistive driving technologies into a cohesive package, which gradually relegates the human driver away from a primary operative position and into a role of a supervisor or fallback system. The eventual goal of AVs is to eliminate the need for the human driver altogether. However, full driving automation, as with most promises of complete automation for complex tasks, is a long way off. The state-of-art research and design of semi-AVs requires the human driver to remain an important aspect of the driving system.

1 Levels of Automation for AVs There are six different classifications of driving systems (i.e., levels of automation; LOA) ranging from Level 0—no driving automation, to Level 5—full driving automation [1]. On Level 0, the driver performs all aspects of the driving task, which is common in conventional vehicles. Many of the singular driver assistance systems such as lane-keeping or adaptive cruise control fall into the Level 1 designation where the vehicle only executes one of the subtasks of lateral or longitudinal vehicle motion control. The more advanced partial driving automation systems, such as Tesla’s Autopilot, Cadillac’s Super Cruise, and Volvo’s Pilot assist, are categorized as Level 2. On this LOA, the vehicle has complete control of the lateral and longitudinal vehicle motion control, but the driver is still responsible for detection of objects and events and therefore avoiding any hazards as well as supervising the driving system. The definitions get a bit more complex and difficult to distinguish at Level 3, conditional driving automation. By definition, the vehicle should now be fully capable of object and event detection and response in its operational design domain, and the human driver is the fallback for the dynamic driving task. However, the human driver is required to intervene when prompted by the vehicle, which requires the driver’s situation awareness of the driving environment [2]. Some of these previously mentioned systems like Tesla’s Autopilot are already capable of at least minor forms of detecting objects and events and performing responsive actions to these hazards. The system might not be fully capable of managing the entire task, but it is performing some of the task and assisting the human driver at higher levels than would be strictly defined under Level 2. Additionally, the human driver may think the vehicle has higher capabilities than its specified LOA or may not fully understand the vehicle’s actions, leading to much confusion. On Levels 4 and 5, the vehicle is expected to carry the entire driving task without the human driver’s intervention in its operational design domain (L4) or unconditionally (L5). This chapters focuses on the state-of-the-art LOAs of Levels 2 and 3 (i.e., semi-AVs), for which effective human-automation interaction and collaboration are vital. To understand how people may engage with various levels of AVs and their conceptualization of definitions associated with AVs, it is helpful to survey the general public about their understandings. For example, Kyriakidis and colleagues

Human-Automation Interaction for Semi-Autonomous Driving: Risk…

283

[3] examined driver opinions on automated driving and specifically what secondary, non-driving related tasks (e.g., reading or watching a movie) drivers reported they would do at various LOAs. A common trend emerged in their results showed that drivers were more willing to engage in secondary tasks as the LOA rose from manual to partially, highly, and fully automated driving systems. Note that a good portion of the respondents indicated that they would engage in secondary tasks that would prevent them from safely performing their share of the driving task at the lower LOAs. Although the respondents did not actually interact with any systems or exhibit actual driving behaviors, this result demonstrates a mismatch between the public’s understanding of the AV’s capabilities and their actual capabilities. Consequently, this mismatch may lead to the human driver’s low awareness of the risks in the driving environment. Therefore, it is critical to communicate the capabilities of AVs and potential risks in the driving environment to provide the maximum benefits and safety.

2 Risk Communication in Semi-AVs Risk communication is generally defined as “the exchange of information among interested parties about the nature, magnitude, significance, or control of a risk” ([4], p. 359). There are three main elements of risk communication: the message containing information from an organization or sender, the medium the message is relayed through, and the audience that the message is targeted to [5, 6]. Risk communication is associated with threat sensing and assessment [7]. The risk communication messages can be organized into four categories based on their primary objective: information and education, behavior change and protective action, disaster warnings and emergency information, and joint problem solving and conflict resolution [8]. These messages are then dispatched to the public through methods such as mass communications, community engagements, media, and even social media platforms such as Twitter [9]. Risk communication is based on the assumption that the public needs to know about possible hazards and risks and should be able to make informed decisions accordingly [7]. It is imperative for the sender to draft the risk message with the focus on the receiver, who will be receiving, interpreting, and acting on the message. In the context of semi-AVs, risks include both overt and covert hazards in the driving environment [10, 11]. Overt hazards include moving or stationary objects on the road that pose an issue for the driver. For example, an object or a pedestrian on the road, in the driver’s field of view, would be an overt hazard. Covert hazards include road hazards that are not immediately visible, such as a pedestrian who is about to cross the road but is currently occluded by a parked car. With driver assistance systems, partial driving automation, and conditional automated driving, the human driver needs to be aware of the risks in the driving environment. The higher the LOA, the more likely the driver is to decrease their situation awareness and vigilance over time because the automation is doing more of the driving task

284

J. Chen et al.

[12–14]. In driving scenarios involving either overt or covert hazards, it is critical that the vehicle communicates existing and potential risks to the driver so that the drivers may anticipate such hazard and respond appropriately. Various risk communication mechanisms can be used to convey these risks to drivers in semi-AVs.

3 Effective Warning Design as a Risk Communication Mechanism Designing effective warnings can help the human driver understand risks in the driving environment. The direction, content, and timing of warnings can affect the effectiveness of semi-AV warnings. Moreover, an important aspect of warnings in semi-AVs is to ensure that it is properly tailored to the LOA. In this section, we discuss research on lateral directional warnings, semantic versus non-semantic warnings, and warnings of different time to collision (TTC). These different design characteristics of warnings in semi-AVs should be considered at different LOAs given the unique human-vehicle interaction status expected at each LOA. Lateral warnings are warnings that utilize the direction of warnings to indicate hazards that there are at the sides of the vehicle to the human drivers. These hazards can be either overt or covert, which the driver needs to be aware of and be ready to take action to avoid them. An effective lateral warning needs to quickly direct the driver’s attention to the potential hazard so that the driver can decide what action needs to be taken [15–17]. Because the human driver might be more disengaged and slower to react at the higher level compared to at the lower levels, warnings need to be designed differently. At lower LOAs (Levels 1 and 2), this warning would help orient the driver in the driving task and bring the hazard to their attention, which they might not have noticed otherwise (e.g., a pedestrian crossing the road). Because the human driver is mostly in control of the dynamic driving task at the lower levels, the lateral warning acts as a lookout and advisor where the human can easily react because they already mostly have control. However, when it comes to the higher LOA of Level 3, a lateral warning would act as more of an alert to the human that they should pay attention to something highly critical and make a takeover decision when necessary. Besides the direction of warnings, the warning content can also affect their effectiveness in communicating risks to drivers. Non-semantic warnings (e.g., simple auditory tones) may be adequate to orient a driver toward an object at lower LOAs, but might not provide adequate information at higher levels with reduced driver situation awareness and vigilance. They could even result in a startle effect, confusing or surprising the driver [18–20]. Even simple auditory tones may be presented differently to drivers depending on how far away the hazard is or direction in which the hazard is moving [17]. Instead of simple auditory tones, warnings can be semantic, in spoken words such as, “car in blind spot” to give more contextual information to the driver [21]. However, implementation of these semantic messages is dependent

Human-Automation Interaction for Semi-Autonomous Driving: Risk…

285

on the time to the potential collision, how long the warning message takes, and even the background noise in the environment. Furthermore, semantic warnings are expected to be especially effective for warning latent hazards. The ability to anticipate latent hazards, hazards that have not yet materialized but may materialize in any moment in the future, is a crucial skill for safe driving [22]. Although current semi-AVs at Levels 2 and 3 using local LIDAR and radar sensors and cameras are not fully capable of detecting hazards that are out of the vehicles’ direct line of sight, connected vehicle (CV) technology could provide a partial solution to mitigate the risks of latent hazards and fill the gap. CVs use short-range radio signals to transmit information from vehicle-to-vehicle or from vehicle-to-infrastructure [23], which can help provide a more complete picture of the dynamic road environment and subsequently warnings to drivers about latent or developing hazards. For example, a driver would receive a warning that there is stopped traffic ahead, a latent hazard that may be visually obscured by road geometry characteristics such as a hill or curve. Semantic warnings would be more suitable than non-semantic warnings to communicate this kind of information. With effective semantic warnings, drivers would be able to allocate their attention effectively to the objects that are “cued” to them and likely better anticipate them. In a simulated driving study, for example, participants were shown a visual head-up display warning that alerted participants to a possible, upcoming pedestrian and vehicle latent hazards [24]. The warnings were effective at increasing the number of drivers who fixated on the hazard. In another driving simulator study examining different hidden hazard warning schemes, participants were better able to avoid crashes when the warnings contained specific information regarding the placement of the hazard (distance or direction) than when they received general warnings [25]. In fact, human drivers’ latent hazard anticipation was found to be worse when driving a simulated Level 3 vehicle than when driving a simulated Level 0 vehicle [26]. Thus, drivers of higher LOAs are more likely to possess an incomplete picture of the immediate, dynamically changing road environment, thereby limiting their latent hazard anticipation abilities. As a result, semantic warnings can be more helpful indicating latent hazards to drivers at higher LOAs. The timing of the warning is crucial for the warning to be effective so that the human driver has sufficient time to perceive a hazard and take action [6, 24]. Intuitively, it is beneficial to warn the human driver of a potential hazard as early as possible. However, the semi-AV may not always be able to do so due to limitations of sensors, and early warnings might be compromised in terms of its accuracy. The timing of a warning is typically defined in terms of TTC, which is the time from the onset of the warning to the time of a potential collision [15]. Lodinger and Delucia [27] examined whether automation affected drivers’ braking responses to a hazard as well as their visual perception measured by TTC judgment. They found that, compared to manual driving, automation freed up participant’s resources and facilitated their process of visual information to use for more accurate TTC judgment. Research on manual driving showed that TTC modulated the effect of the direction of lateral warnings on their effectiveness. For example, Straughn and colleagues [15] found that with a shorter TTC of 2 s, warnings signaling the avoidance direction were

286

J. Chen et al.

more effective than warnings signaling the collision direction, but this effect reversed with a longer TTC of 4 s. However, there has been mixed results found for automated driving. Cohen-Lazry and colleagues [28] found the avoidance-direction warnings to be more effective than the collision-direction warnings with a 4-s TTC, whereas Petermeijer and colleagues [29] found no difference between these two types of warnings with a longer TTC of 7 s. Chen and colleagues [17] found that for a pedestrian starting from one side of the road and walking towards the center, collision-direction warnings are more effective than the avoidance-direction warnings, and the advantage of the former increases with greater TTCs. It is clear, though, faster responses were yielded when the TTC was shorter, possibly due to shorter TTCs reflecting higher urgency of the situation to respond to the hazard.

4 Human Trust in Semi-AVs Proper human trust in semi-AVs is one of the most important aspects of ensuring proper cooperation and efficiency between the semi-AV and the human driver. Trust is a multifaceted and complex concept that many may be nominally familiar with but applying trust to automation adds more complexity. Lee and See [30] have defined human trust in automation as “the attitude that an agent will help achieve an individual’s goals in a situation characterized by uncertainty and vulnerability” (p. 51). Researchers built upon theories of interpersonal trust to establish theories of human-automation trust [31–33]. Specifically, Muir and colleagues [32, 33] adopted a triadic model of trust from theories of interpersonal trust (e.g., [34, 35]) to describe trust development. She asserted that automation trust develops from predictability, dependability, and faith progressively. Predictability is defined as the perceived consistency and desirability of behaviors of the machine, dependability is defined as the extent to which the stability of machine behaviors is based on accumulation of behavioral evidence, and faith is defined as the expectation that the machine performs beyond the current situation that operators gathered behavioral evidence that can generalize to future situations. Lee and colleagues [30, 31] further developed this triadic model, asserting that performance, process, and purpose are the three dimensions responsible for automation trust development. Performance is defined as what the automation does according to past and present operation, process is defined as how the automation operates in terms of its programmed algorithm, and purpose is defined as why the automation was developed, representing the automation designer’s goal. As trust is considered a critical factor for successful human-automation interaction, researchers have begun to explore trust in the context of semi-AVs. For example, Choi and Ji [36] examined the role of trust in initial adoption of AVs through a survey. They identified that trust is one of the major determinants for automation use. System transparency, technical competence, and situational management positively impacted trust, whereas perceived risk negatively impacted trust [36]. In another survey-based study, Zhang and colleagues [37] explored constructs of the Technology Acceptance

Human-Automation Interaction for Semi-Autonomous Driving: Risk…

287

Model, including trust, of a Level 3 AV. Regarding trust, the authors found that initial trust was the most critical factor for positive system attitudes. Additionally, they assert that trust can be improved by reducing perceived safety risk and improving perceived usefulness. More recently, Lee and Kolodge [38] conducted a study exploring qualitative ratings for level of trust in semi-AVs using structural topic modelling. They found topics that align with the three proposed dimensions of trust (e.g., “safer than a human” relating to the performance dimension of trust, “technology improving” relating to the process dimension of trust, “hacking and glitching” relating to the purpose dimension of trust). In terms of trust development, we consider the following three main categories or phases: dispositional, situational, and learned trust [39]. Dispositional trust is an individual’s likelihood or propensity to trust anything based on their age, culture, or personality. It can change over time but is slow to adapt and is usually relatively static throughout one’s life. Situational trust considers the factors that may influence a persons’ trust in a system both internally (e.g., self-confidence, expertise, mood, attention) or externally (e.g., system type, complexity, task difficulty). Learned trust is split into initial and dynamically learned trust, referring to the timing of your interaction with the system. In the context of semi-AVs, if a driver has yet to interact with an semi-AV system, they still have an initial learned trust in that system, even if they have never used it. Perhaps a driver has never driven a Tesla or used their Autopilot feature, but they have heard of the company’s brand and their reputation or they have experienced a similar technology and might understand some of the basic concepts. All of these factors can influence the initial learned trust in the system before the driver even experiences it. Once the driver begins to interact with the system, they continue the trust learning and development process in what is referred to as dynamically learned trust. The system’s performance and design features can influence the trust of the driver while using the system. Dynamically learned trust is the section where trust is actively calibrated according to this specific system and can greatly vary as experience mounts. Over time, operators can learn the system capabilities (e.g., system reliability, types of errors) and properly calibrate their trust. With a reliable automation system, the level of trust should increase to match an equivalent level of reliability [40, 41]. Established trust can be impaired by automation errors, one of the most influential factors affecting trust, depending on the severity of the error. For minor errors, trust decrement might be small and inconsequential. However, major errors like a vehicle crash could be much more potent [42, 43]. These results are expected and indicate proper calibration of trust [30, 44–46]. However, the decline of trust could hurt the performance of the human-automation system if the level of trust does not rebound from the decrease. If the human driver does not understand why an error occurred, they could lose trust in the automation because they do not know the conditions under which automation was unable to perform or what caused the error. Increased transparency of the semi-AV can provide the driver a better understanding of limitations of the driving system and causes of errors. Trust can recover after an error with no intervention if no more errors occur and performance continues to be good [39, 47, 48]. However, it may take a long time to reach the previous level of

288

J. Chen et al.

trust and may cause confusion about the system’s capabilities if no repair strategy or transparency is employed [43]. There are different strategies for repairing human trust in automation after a system error [48], similarly to the best human-human teams trying to intentionally build rapport and fix trust if any problems occur. There is limited work in the area of trust repair for human-automation systems; however, de Visser and colleagues appraised studies about trust repair and provided a list of several active trust repair strategies (e.g., apologize, deny, empathize, explain). These strategies seem to be promising, but many are new or require more testing. Some of the most obvious methods, such as to apologize and explain, can be seen as transparency measures demonstrating that the vehicle is trying to work as a team and communicate the issue. Other solutions (e.g., deny) can be less obvious to implement in a vehicle, but are often employed by humans such as denying that an error occurred or assuring someone else that it was not as big of a deal as it seems. However, in the interest of promoting transparency and teamwork, negative approaches toward repair could further damage trust in the automated system and impact safety later.

5 Risk Communication and Trust in Semi-AVs The importance of risk communication is also underscored by the relationship between perceived risk and trust. Innate within Lee and See’s [30] definition of trust is the necessity of uncertainty and vulnerability for trust to develop, both of which are characteristics of risk. Mayer and colleagues [49] outline that trusting behavior without risk is best represented by other constructs, such as cooperation, confidence, or predictability. Recently, researchers have explored risk as a variable in studies on trust in automation. For example, Sato and colleagues [50] found that, in a multitasking scenario with varied task load and risk, participants with higher levels of perceived risk reported more trust than those with lower levels of perceived risk, suggesting that risk is a critical factor that influences trust development. Proper risk communication from the semi-AV to the human driver is closely related to the driver’s trust in semi-AV. Effective risk communication can improve system transparency, which in turn increases driver’s trust in semi-AVs. In addition, the more helpful the human driver finds the risk messages, the more they would trust the semi-AV as a whole. Repeated false alarms or unhelpful warnings may be seen as annoying and could decrease trust and increase response time to hazards [40, 51, 52]. Specifically for higher LOAs, a takeover request (TOR) may be required when the system can no longer perform the driving task due to some limitation or condition change. This TOR can be viewed as means of communicating risks to the driver. For example, some roadways or weather conditions might not be conducive or suitable for the semi-AV and it may be able to let the human driver know well in advance about the upcoming issue as well as the need for a takeover [53, 54]. However, other times takeover may occur in an emergency situation due to unforeseen road events or system issues [55, 56]. TORs are usually not considered as system errors because

Human-Automation Interaction for Semi-Autonomous Driving: Risk…

289

they are intended actions of the semi-AV [1]. However, research has shown mixed effects of TORs on driver’s trust. On the one hand, TORs can affect the human driver’s trust negatively, possibly due to driver seeing a TOR as incapability of the semi-AV [43, 57]; on the other hand, TORs are found not to affect trust, likely because of the specific situation and timing of TOR implementation or when trust is measured [55, 58].

6 Conclusion and Future Considerations Human-automation interaction is essential for systems that require collaboration between the human and the automation. Semi-AV is such a system that requires close collaboration between the human driver and the vehicle. The challenges for successful collaboration include how to effectively communicate risks and enhance proper trust between the parties. We have discussed risk communication mechanisms from the vehicle to the driver and human trust in automation in semi-AVs. Apparently, both concepts can be bidirectional. A further consideration is to investigate mutual risk communication and trust between the human and the automation system in semiAVs. We have also only focused on driver-vehicle communication. Another future consideration is to incorporate the interactions between the semi-AV and pedestrians, other vehicles, and the infrastructure from a system-of-systems perspective. The ultimate goal of considerating the interactions surrounding semi-AVs is to ensure system safety and security before AVs become fully autonomous.

References 1. SAE (2016) Taxonomy and definitions for terms related to driving automation systems for on-road motor vehicles, pp 1–12 2. Endsley MR (1995) Toward a theory of situation awareness in dynamic systems. Hum Factors 37(1):32–64 3. Kyriakidis M, Happee R, de Winter JC (2015) Public opinion on automated driving: results of an international questionnaire among 5000 respondents. Transport Res F: Traffic Psychol Behav 32:127–140 4. Covello VT (1992) Risk communication: an emerging area of health communication research. Ann Int Commun Assoc 15(1):359–373 5. Telg R (2010) Risk and crisis communications: When things go wrong. Agricultural education and communication department, Florida cooperative extension service, institute of food and agricultural sciences, University of Florida. WC093. pp 1–6 6. Chen J (2020) Risk communication in cyberspace: a brief review of the information-processing and mental models approaches. Curr Opin Psychol 36:135–140 7. Reynolds B, Seeger MW (2005) Crisis and emergency risk communication as an integrative model. J Health Commun 10(1):43–55 8. Covello VT, Slovic P, Von Winterfeldt D (1986) Risk communication: a review of the literature. National Emergency Training Center 9. Panagiotopoulos P et al (2016) Social media in emergency management: twitter as a tool for communicating risks to the public. Technol Forecast Soc Chang 111:86–96

290

J. Chen et al.

10. Sun L, Hua L (2019) Effects of hazard types on drivers’ risk rating and hazard response in a video-based hazard perception task. PLoS ONE 14(3):e0214226 11. Wetton MA et al (2010) The development and validation of two complementary measures of drivers’ hazard perception ability. Accid Anal Prev 42(4):1232–1239 12. Endsley MR (2017) From here to autonomy: lessons learned from human-automation research. Hum Factors 59:5–27 13. Endsley MR, Kiris EO (1995) The out-of-the-loop performance problem and level of control in automation. Hum Factors 37(2):381–394 14. Molloy R, Parasuraman R (1996) Monitoring an automated system for a single failure: vigilance and task complexity effects. Hum Factors 38:311–322 15. Straughn SM, Gray R, Tan HZ (2009) To go or not to go: stimulus-response compatibility for tactile and auditory pedestrian collision warnings. IEEE Trans Haptics 2(2):111–117 16. Wang D-YD, Proctor RW, Pick DF (2007) Coding controlled and triggered cursor movements as action effects: Influences on the auditory Simon effect for wheel-rotation responses. J Exp Psychol Hum Percept Perform 33(3):657 17. Chen J et al, Effectiveness of lateral auditory collision warnings: should warnings be toward danger or toward safety? Human Factors, in press 18. Banks VA et al (2017) Is partially automated driving a bad idea? Observations from an on-road study. Appl Ergon 2018(68):138–145 19. Naujoks F, Mai C, Neukum A (2014) The effect of urgency of take-over requests during highly automated driving under distraction conditions. In: Proceedings of the 5th international conference on applied human factors and ergonomics AHFE, pp 2099–2106 20. Sarter NB, Woods DD, Billings CE (1997) Automation surprises 21. Šabi´c E, Chen J, MacDonald J, Towards a better understanding of in-vehicle auditory warnings and background noise. Human factors, in press 22. Pradhan AK et al (2005) Using eye movements to evaluate effects of driver age on risk perception in a driving simulator. Hum Factors 47(4):840–852 23. Administration NHTS (2017) Fact sheet: improving safety and mobility through vehicle-tovehicle communication technology 24. Hajiseyedjavadi F et al (2018) Effectiveness of visual warnings on young drivers hazard anticipation and hazard mitigation abilities. Accid Anal Prev 116:41–52 25. Schwarz F, Fastenmeier W (2018) Visual advisory warnings about hidden dangers: effects of specific symbols and spatial referencing on necessary and unnecessary warnings. Appl Ergon 72:25–36 26. Samuel S, Yamani Y, Fisher DL (2020) Understanding drivers’ latent hazard anticipation in partially automated vehicle systems. Int J Human Factors Ergon 7(3):282–296 27. Lodinger NR, DeLucia PR (2019) Does automated driving affect time-to-collision judgments? Transport Res F: Traffic Psychol Behav 64:25–37 28. Cohen-Lazry G et al (2019) Directional tactile alerts for take-over requests in highly-automated driving. Transport Res F: Traffic Psychol Behav 65:217–226 29. Petermeijer S et al (2017) Take-over again: investigating multimodal and directional TORs to get the driver back into the loop. Appl Ergon 62:204–215 30. Lee JD, See KA (2004) Trust in automation: designing for appropriate reliance. Hum Factors 46(1):50–80 31. Lee J, Moray N (1992) Trust, control strategies and allocation of function in human-machine systems. Ergonomics 35(10):1243–1270 32. Muir BM (1994) Trust in automation: Part I. Theoretical issues in the study of trust and human intervention in automated systems. Ergonomics 37(11):1905–1922 33. Muir BM, Moray N (1996) Trust in automation. Part II. Experimental studies of trust and human intervention in a process control simulation. Ergonomics 39(3): 429–460 34. Barber B ()1983 The logic and limits of trust 35. Rempel JK, Holmes JG, Zanna MP (1985) Trust in close relationships. J Pers Soc Psychol 49(1):95

Human-Automation Interaction for Semi-Autonomous Driving: Risk…

291

36. Choi JK, Ji YG (2015) Investigating the importance of trust on adopting an autonomous vehicle. Int J Human-Comput Inter 31(10):692–702 37. Zhang T et al (2019) The roles of initial trust and perceived risk in public’s acceptance of automated vehicles. Transport Res Part C: Emerg Technol 98:207–220 38. Lee JD, Kolodge K (2020) Exploring trust in self-driving vehicles through text analysis. Hum Factors 62(2):260–277 39. Hoff KA, Bashir M (2015) Trust in automation: Integrating empirical evidence on factors that influence trust. Hum Factors 57(3):407–434 40. Parasuraman R, Riley V (1997) Humans and automation: Use, misuse, disuse, abuse. Hum Factors 39(2):230–253 41. Chen J et al (2018) The description-experience gap in the effect of warning reliability on user trust and performance in a phishing-detection context. Int J Hum Comput Stud 119:35–47 42. de Visser EJ et al (2016) Almost human: Anthropomorphism increases trust resilience in cognitive agents. J Exp Psychol Appl 22(3):331–349 43. Mishler S (2019) Whose drive is it anyway? Using multiple sequential drives to establish patterns of learned trust, error cost and non-active trust repair while considering daytime and nighttime differences as a proxy for difficulty, (Master’s thesis) 44. Lee JD, Moray N (1994) Trust, self-confidence, and operators’ adaptation to automation. Int J Hum Comput Stud 40(1):153–184 45. McGuirl JM, Sarter NB (2006) Supporting trust calibration and the effective use of decision aids by presenting dynamic system confidence information. Hum Factors 48(4):656–665 46. Muir BM (1987) Trust between humans and machines, and the design of decision aids. Int J Man Mach Stud 27(5–6):527–539 47. Mishler S, Jeffcoat C, Chen J (2019) Effects of anthropomorphic phishing detection aids, transparency information, and feedback on user trust, performance, and aid retention. In: Proceedings of the human factors and ergonomics society 63rd international annual meeting 48. de Visser EJ, Pak R, Shaw TH (2018) From ‘automation’to ‘autonomy’: the importance of trust repair in human–machine interaction. Ergonomics 61(10):1409–1427 49. Mayer RC, Davis JH, Schoorman FD (1995) An integrative model of organizational trust. Acad Manag Rev 20(3):709–734 50. Sato T et al (2020) Automation trust increases under high-workload multitasking scenarios involving risk. Cogn Technol Work 22(2):399–407 51. Chancey ET et al (2017) Trust and the compliance-reliance paradigm: the effects of risk, error bias, and reliability on trust and dependence. Hum Factors 59(3):333–345 52. Singer J, Lerner N (2015) Auditory alerts in vehicles: effects of alert characteristics and ambient noise conditions on perceived meaning and detectability. In: 24th International technical conference on the enhanced safety of vehicles (ESV) 1:15–0455 53. Dogan E et al (2017) Transition of control in a partially automated vehicle: Effects of anticipation and non-driving-related task involvement. Transport Res Part F: Traffic Psychol Behav 46(A):205–215 54. Eriksson A, Stanton NA (2017) Takeover Time in highly automated vehicles: noncritical transitions to and from manual control. Hum Factors 59(4):689–705 55. Körber M, Baseler E, Bengler K (2018) Introduction matters: manipulating trust in automation and reliance in automated driving. Appl Ergon 66:18–31 56. Mok B et al (2015) Emergency, automation off: unstructured transition timing for distracted drivers of automated vehicles. In: IEEE Conference on intelligent transportation systems, proceedings, ITSC, 2015, pp 2458–2464 57. Hergeth S et al (2015) Effects of take-over requests and cultural background on automation trust in highly automated driving 331–337 58. Gold C et al (2015) Trust in automation—before and after the experience of take-over scenarios in a highly automated vehicle. Procedia Manuf 3(AHFE):3025–3032

A Systematic Literature Review of Human Error and Machine Error in Accident Investigation Nathan B. Rowland Miller

Abstract The presence of error in systems has been an essential part of analyzing accidents and incidents when investigations occur. More importantly, it is imperative to look at the overall impacts of human error and machine error systems and how they impact the investigation and accident. For this study of the effects and implications of both, literature reviews were conducted, and tools like Purdue Databases, MAXQDA, Vicintas, VOSviewer, and Harzing’s Publish or Perish were used to gather data. These tools analyzed the most prevalent articles in the study through co-citation analysis, and significant trends were mapped and tracked. From this, the conclusion is found that the relationship between human error and machine error is dependent on one another, as the presence of one can cause the other. Additionally, in this analysis, consideration was given when looking into the relationship that machine AI and accidents and errors have. The conclusion reached is that while a connection is evident between the two points and there are systems in the works, these systems need to be implemented in a more robust fashion and with more stability in this practice. Keywords Human error · Machine error · Accident investigation · Literature review · Harzing · VOSviewer · MAXQDA · Vicintas · Mendeley

1 Introduction Accidents can occur in a whole variety of areas and for an entire array of reasons. While accidents should be avoided as any response will be primarily reactive and not proactive, it is critical to have procedures and resources available to respond to and analyze the accident promptly [1]. More specifically, it is vital to have systems to work with and analyze the root causes of the situation and accident that occurred. While there can be many different reasons for an accident to occur, the main problems often boil down to one of two issues: human error or machine error. While there are N. B. Rowland Miller (B) School of Industrial Engineering, Purdue University, West Lafayette, IN 47906, USA e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Switzerland AG 2023 V. G. Duffy et al. (eds.), Human-Automation Interaction, Automation, Collaboration, & E-Services 11, https://doi.org/10.1007/978-3-031-10784-9_18

293

294

N. B. Rowland Miller

reliable models that exist that allow for work and investigations to be performed, there can exist a disparity between the expected results and the actual concluded answers. Accident and incident investigations often are framed in the engineering scope of identifying where a machine or technology may have failed, and the human factors error was the root cause [2]. In programming and software design, the opposite is true where human error is considered before looking at machine or technology issues [3]. When looking at the human and machine aspects, there is often no direct comparison or relationship that is examined, only that one was the main root cause, and the other was an underlying factor. Newer studies show that there does exist a mutual dependence between humans and machines, as often a human error can lead to the machine malfunctioning, or an error on the machine can cause a human to make a mistake [4]. There can be more to gain from understanding this relationship.

2 Purpose of Study 2.1 Justification in Relation to Job Design The primary purpose of this study is to perform a literature review of a variety of articles that focus on human and machine errors, using tools like Harzings’s Publish or Perish or Purdue Databases to gather reports and then tools like VOSviewer, MAXQDA, and BibExcel to conduct analyses on the results. With this project’s focus on errors in accident investigation, there is a connection to job design as human factors are essential to accident investigation and necessary to design and facilitate controls for machine development [1, 5].

2.2 Relevance to Human-Automation Interference The overall aim is to illustrate how human and machine errors aid and harm one another and continuing to show how, with this knowledge, engineers and system designers can put practical solutions in place to prevent further injuries or deaths that may take place. This process has been investigated before and taken up, addressing errors and using cognitive models in nuclear plants and, more recently, using a human factors analysis and classification system (HFACS) to assess maritime accidents and the root relationships behind the causes [6, 7]. To help take stock of all articles used and aid in the citation of items (as not all pieces are taken from Web of Science which includes citations), Mendeley is used to cite and create a bibliography at the end of the paper [8].

A Systematic Literature Review of Human Error …

295

3 Research Methodology 3.1 Data Collection To start the literature review and gather data for the analyses, articles needed to be compiled and reviewed. This data was collected through searches through various databases. These databases—Web of Science, Scopus, and Google Scholar—pulled information such as author, title, source, and abstract. However, cited sources were gathered only for those items from the Web of Science and Scopus databases, where the information collected from Google Scholar did not have citation data. Therefore, they needed either Mendeley to assist if the report used the information or other tools analyzed it. For the articles from Web of Science, VOSviewer, a tool by Nees Jan van Eck and Ludo Waltman, generated a co-citation analysis to help visualize data and show the importance and relevance of the data. VOSviewer was also used to conduct a content analysis on keywords common in the articles gathered from the Google Scholar search. The “Results” section will discuss the specifics of these points. The overall purpose of this data collection is to start the investigation process on the topic discussed of the relationship between human error and machine error in accident investigation. This gathered data from both sites would provide a scope for the paper and discussion moving forward and aid the development of visuals, references, and usage of documents moving forwards. For all kinds of database searches, the keywords used were “accident investigation” and “human error and machine error.” This split was essential as the search combining both terms yielded 22 results, which is not a very diverse or effective data pool to use. This separation also separates to focus on the investigation part of the analysis and the individual relationship between human and machine error. “Incident investigation” was also left out of the search as the results followed a very similar trend to the “accident investigation” search. Still, both are used almost interchangeably in certain documents found. For the term “accident investigation,” Web of Science yielded 9070 articles found, and Google Scholar yielded 980 articles, shy of the upper limit on Harzing’s Publish or Perish of 1000 articles. For the search “human error and machine error,” the Web of Science search yielded a total of 5046 articles, and Google Scholar yielded 990 articles found.

3.2 Web of Science Analysis There are two main comparisons and items that are looked at through the Web of Science database. The first, trend analysis, is conducted using the data gathered from the Web of Science searches. These figures are used to show the overall trends of data and relevance as time has gone on. The second of these, source title analysis, is done to see the overall trends when looking at the organizations and publications releasing and citing the articles searched. This process is necessary as it shows the overall relevance of the data in different fields and where these articles may focus

296

N. B. Rowland Miller

the critical items or topics. Web of Science searches on both sets of data gathered for “accident investigation” and “human error and machine error.” The first analysis is done on the search term “accident investigation.” This term set the groundwork for the comprehensive searches and garnered a good deal of information to aid future searches. The first article cited that matched the keywords listed above was found in 1901, but it quickly dipped down after that back to 0. The fluctuation trend occurred from then until 1962 when the trend started to show an increase without a decrease to 0 again. This trend continued, increasing considerably in 1991 and continuing the sharp increase until the present. Thus, the trend does seem to follow a steady increase, with notable exceptions being spikes in 2005 and 2014 and then a more considerable jump in 2017. As it currently stands, the year 2021 has 137 articles related to the topic and only for the first quarter of the year, so readers can assume that the number of articles published this year will be on track with the trend data. This trend gathered from the Web of Science data can be seen in Fig. 1. The source titles for the search “accident investigation” are shown in Fig. 2. This figure shows the top 12 sources of titles. This analysis, taking a look at the publication information, shows that the articles come from a wide variety of different publications. Most notably, there is a trend of articles coming from nuclear engineering and accident-focused sources. The highest number of articles comes from the Nuclear Engineering and Design publication, with 259 published pieces. The second analysis done is on the search term “human error and machine error.” Due to the higher number of words and specificity that this phrase has, the number was lower and had a lower consistent number of articles released every year. Web of Science found the first recorded article in 1969. It did not start having a consistent number of articles published starting in 1991, similarly to the trend that Web of Science noticed for accident investigation. However, the articles relating to this phrase had a steady incline from 1991 on, experiencing a spike in articles starting in 2019. Currently, there are 177 articles published in the year 2021 in the first quarter, showing an accurate representation of an increase that would match the trend by the end of the year. This trend is shown in Fig. 3.

Fig. 1 Trend analysis of articles on “accident investigation” [9]

A Systematic Literature Review of Human Error …

297

Fig. 2 Analysis of source titles for “accident investigation” [9]

Fig. 3 Trend analysis of articles on “human error and machine error” [9]

The top twelve source titles for the search “human error and machine error” are shown in Fig. 4. The articles found in this analysis are more straightforward, showing that a majority of the articles come from industrial engineering-focused publications. Other publications focus on computer science and artificial intelligence. It is important to note that with a lower total number of articles selected than in “accident investigation,” the overall record count for these sources is lower by a little more than half, barely breaking 100 records for the top source title. This top source is titled Lecture Notes in Computer Science.

298

N. B. Rowland Miller

Fig. 4 Analysis of source titles for “human error and machine error” [9]

4 Results 4.1 Co-citation Analysis Now that the data has been gathered and general trends are ready to go, the next step is to process and analyze those results. The first of the ways is through the use of VOSviewer to assess this data through a co-citation analysis. Co-citation analysis is an effective tool to show the connections that may exist between two or more articles. It is vital because this connection can show how removed or centralized the ideas discussed may be and whether it is an avenue worth going down. This analysis also is an excellent way to identify other works of literature that can guide the overall discussion. In both cases of keyword searches, the most recent 500 articles published were used not to make data analysis happen for a total of roughly 14,000 articles gathered from the Web of Science database. Fig. 5, shown below, shows the overall co-citation analysis for the term “accident investigation.” The search performed to generate the given graph was limited to giving articles with nine or more citations, which left four articles remaining as the highest quoted. Out of these, the article by Rasmussen had the highest with 15 citations, while the three other articles showed nine citations. The co-citation for the keywords “human error and machine error” is shown in Fig. 6. The requirements were much more divisive, requiring a minimum of 23 citations to be considered in the top four articles. The highest number of citations, 36, is for the article by Breiman, with the other articles being cited 30, 24, and 23 times respectively. sThe articles found here were considered for further use in the next steps of analysis.

A Systematic Literature Review of Human Error …

299

Fig. 5 Co-citation analysis for “accident investigation” [10]

Fig. 6 Co-citation analysis for “human error and machine error” [10]

4.2 Content Analysis The overall content of the articles needs to be validated to ensure that the articles gathered through the co-citation analysis are beneficial. Content analysis will be performed using the information gathered from Harzing’s Publish or Perish tool that takes the articles from Google Scholar. These articles are reviewed in the title and abstract fields, removing copyright statements and structured abstract labels. Once the information is processed and sorted, the information is turned into a word cluster representing the number of times an article used a word and how it connects with other similar words around the topic at hand. VOSviewer will use the word clusters to identify the highest sorted words and be used in the future analysis of the articles to verify their validity and relationship to the keywords at hand. A cluster map for the term “accident investigation” is shown in Fig. 7. This cluster map shows the most prominent information taken from Herzing’s Publish or Perish search of Google Scholar for the selected term. A term had to occur more than ten times to be considered for the generation of the figure. After operators had put the limit into place, 153 terms remained as reaching the threshold. Out of these terms, VOSviewer gave an option to include a relevance score. This option brought the list down to about 60% of the original list, or in this case, about 92 words total. Out of these 92 words, the most common word used was “report,” at 193 occurrences. The other most prevalent words are shown in Table 1.

300

N. B. Rowland Miller

Fig. 7 Cluster analysis for “accident investigation” [10]

Table 1 Keyword selection for “accident investigation” from VOSviewer [10]

Keyword

Occurrences

Report

193

Aircraft accident investigation

71

Information

69

Reconstruction

55

Issue

54

in Fig. 8, the cluster map is generated for the phrase “human error and machine error.” This map also requires a minimum of 10 occurrences of keywords to be considered for the map. This limitation left 145 words to be considered. Out of those 145, only 87 remained after following the relevance score that would cut to 60% of the results for more accurate readings. Out of these remaining 87 words, the most common response was “human error,” followed by “machine learning,” with 191 and 102 occurrences, respectively. The remaining top five keywords are indicated in Table 2.

A Systematic Literature Review of Human Error …

301

Fig. 8 Cluster analysis for “human error and machine error” [10]

Table 2 Keyword selection for “human error and machine error” from VOSviewer [10]

Keyword

Occurrences

Human Error

191

Machine learning

102

Classification

67

Evaluation

62

Algorithm

62

4.3 MAXQDA Content Analysis Results Given the highlighted keywords through the VOSviewer analysis of Harzing’s Publish or Perish, operators can do the next step of the data analysis and results by combining the co-citation results and the cluster mapping results. Using another tool, MAXQDA, a word cloud (seen in Fig. 9) can be generated of the keywords found in all articles. These keywords generated are the all-encompassing keywords, being integral in the studies overall understand and context. It is important to note that while the co-citation analysis brought the documents by the World Health Organization as significant, the contents skewed the results to more traffic-themed errors and thus were discounted for this specific word cloud.

302

N. B. Rowland Miller

Fig. 9 Word cloud of combined terms and articles from MAXQDA [11]

4.4 Leading Tables and Pivot Graphs Generated through BibExcel The final form of analysis on the results was through using the BibExcel tool generated by Olle Persson. This tool looks at articles and data collected and used when looking at the bibliometric data. This process is similar to the step done in Sect. 4.1 with the co-citation analysis. Still, this process used for this specific analysis takes a look at the authors found through the Harzing Publish or Perish tool and finds authors with the most citations in works. This tool is effective as it allows for analysis of the authors from the Google Scholar search and accounts for the authors cited at higher rates than others. There were two different analyses performed on the two different search terms given. The first, performed on the term “accident investigation,” is shown below. Once Web of Science had done cleaning the data and had transformed the file into a usable format, the cleaned data sorted the authors selected and prepared in descending order based on the frequency of appearance. When BibExcel did this, the points that met or exceeded the limit of 4 occurrences were copied into Excel. As seen in Fig. 10, 147 articles had no author listed or no accessible information, so operators discounted the results from the analysis. After gathering the information, operators added authors with a frequency of at least four appearances to an Excel spreadsheet. This data led to the creation of a Pivot Chart of crucial information for frequency and author name. This graph is another way to visualize the information gathered. For example, Fig. 11 shows that

A Systematic Literature Review of Human Error …

303

Fig. 10 Leading Table from BibExcel search of “accident investigation” for author frequency

Waterson, P is the more prominent author when looking at accident investigations, with 12 overall appearances in the search. When it comes to the second term of “human error and machine error,” a similar cleaning and preparation process was performed. As shown in Fig. 12, this search has 276 iterations of the authors not being found or notably indicated, so the chart disregarded the number. The authors that had returns of 4 or more in the frequency test were taken and used in future consideration. With the data imported into Excel, the Pivot Chart created showed the values necessary and in comparison with one another for overall appearances. For example, for “human error and machine error,” Woods, DD is the most prominent author with nine appearances, as seen in Fig. 13. Prominent Authors in Accident Investigation Searches from Google Scholar 14

Total

12 10 8 6 4 2

Fig. 11 Pivot Chart from Excel for the search of “accident investigation”

Zhou Z

Wang H

Waterson P

Stoop JA

Underwood P

Stanton NA

Rivers RW

Salmon PM

Kletz TA

Manca D

Katsakiori P

IV JC Marsh

Johnson CW

Fu G

Hollnagel E

Dechy N

Fagerlind H

Bil C

Brambilla S

Baker JS

0

304

N. B. Rowland Miller

Fig. 12 Leading Table from BibExcel search of “human error and machine error” for author frequency

10 9 8 7 6 5 4 3 2 1 0

Total

Anguita D Castilho S Chen J Grundkiewicz R Hoc JM Hollnagel E Junczys-Dowmunt… Lavie A Leape LL Li Z Liu X Liu Y Moray N Ney H Norman DA Och FJ Oneto L Popović M Rasmussen J Reason J Sarter NB Schwartz R Sennrich R Stymne S Swerts M Toral A Vanderhaegen F Wang H Wang Z Woods DD Zhang Y

Prominent Authors in "Human and Machine Error" searches on Google Scholar

Fig. 13 Pivot Chart from Excel for the search of “human error and machine error”

5 Discussion 5.1 Accident Investigation Over the years, accident investigation has become a steadily more interesting and significant part of the conversation when it comes to an understanding where errors may occur. As discussed earlier, the overall trend shows an increase in the articles written and produced surrounding this topic. Outside of academia, there is a high amount of interaction and engagement with the topic. Vicinitas allows for this concept to be explored. As seen below in Fig. 14, the term has seen higher numbers of

A Systematic Literature Review of Human Error …

305

Fig. 14 Interaction numbers, word cloud, and trend charts for “accident investigation” on Vicinitas [12]

engagement in the last ten days, increasing from 0 engagements to peaking at the time of the selection at about 2200 engagements. This term also had a total of 26.9 million influences throughout analyses. The most common words seen in this analysis would be accident investigation. However, the results prove not to be an effective measure of relevance to this study. For the Vicinitas results, other keywords of interest include “officers,” “police,” and “vehicles.” When Vicinitas conducted this trend, the news broke about an incident concerning a police shooting and its investigation. For the results of the co-citation analysis, VOSviewer skewed the results of the top articles in favor of the World Health Organization’s releases of their 2015 and 2018 global status reports of road safety. Due to the dense, qualitative information present in these two sources, the trend skewed towards favoring traffic accidents and investigations surrounding their causes. While this helps tackle one area of investigation, it does not address overall concerns with industry-specific accidents or other integral areas. Some developments come from the searches and aid in the developments of the accident investigations. Accidents are aided through lack of interaction vertically in a work environment and by poor adaptation to new environments by static systems [13]. Human factors analysis and classification system (HFACS) is an instrumental

306

N. B. Rowland Miller

tool in aiding and developing robust insights for latent and active failures on all system levels [1]. Steps are being taken to round out a holistic approach to investigations, but more standardization for general solutions is necessary.

5.2 Human Error and Machine Error Similar to the analysis for accident investigation, articles relating to “human error and machine error” have also steadily increased according to the Web of Science trend analysis. According to Vicinitas for the period analyzed, the trend is consistent as well. As shown in Fig. 15, the overall engagements with tweets involving human and machine error peaked after an increase over the ten days at around 5700 engagements. The overall influence did sit lower than that of accident investigation, averaging about 8.3 million impressions made. This relationship does make sense as no influential world events were happening around this topic during the analyzed time frame. While the searches and articles associated with this yielded more informational results than the accident investigation search, some trends still influenced the outcome. A good number of the sources focused more on software development or machine learning sides, specifically reducing errors in New methods are developed to address human factors issues, but not as much address the errors posed by

Fig. 15 Interaction numbers, word cloud, and trend charts for “human error and machine error” on Vicinitas [12]

A Systematic Literature Review of Human Error …

307

machine learning [14]. Error analyses exist for human and machine systems, but the focus tends to determine innovation flaws and how to adjust from the human factors setting [5]. Many articles focus on one point or another, but not enough tackle the split between both and how they interact. The only article that discusses this dependence directly discusses limitations faced because of the new nature of the concept [4]. Development of this idea is essential before new ideas or theories can be pulled from and developed.

6 Conclusion The overall trend shown in the research performed is that the scope is too wide to narrow down and focus on the dependence that human and machine errors have on one another. For example, when searching for accident investigation, results tend to lead to more information on traffic accidents and incidents, not focusing on machine errors in production settings [15]. While this information is essential for identifying important points and considerations, the overall focus is on a specific set of instances about a global experience [16]. When searching human error and machine error, the scope focused primarily on programming and software development and how machines may experience errors in executing the systems created [17]. Some articles support the overall scope of the tested searches, but the total number is too few to support the current state substantially.

7 Future Work While there have been a few studies showing the overall relationship and dependence of human error and machine error on one another, not much has been done to progress that search and development of standing ideas. However, Carnegie Mellon University is currently performing a study titled Preventing Human Errors in Cyber-human Systems with Formal Approaches to Human Reliability Rating and Model Repair that examines this relationship closely. Awarded funding through the National Science Foundation, this study, started in 2019 and set to conclude in 2023, takes a look at human-automation interaction (HAI) and will assess different work stations and different situations and how the methods used or not used aid in the outcome and errors that were observed [18]. Additionally, while the focus of the report is more on medical systems and machine learning tools, another study focuses on the biases that may arise due to machine error and how this can be either mitigated or reduced to better aid patients and professionals in medical spaces [19]. Though it does not have the same scope as general engineering practices, this will continue to look at the interactions machine learning and humans have.

308

N. B. Rowland Miller

References 1. Dempsey PG (2012) Accident and incident investigation. In: Handbook of human factors and ergonomics, pp 1083–1091. Wiley, Hoboken, NJ, USA. https://doi.org/10.1002/978111813 1350.ch38 2. Reinach S, Viale A (2006) Application of a human error framework to conduct train accident/incident investigations. Accid Anal Prev 38(2):396–406. https://doi.org/10.1016/j.aap. 2005.10.013 3. Endres A (1975) An analysis of errors and their causes in system programs. IEEE Trans Softw Eng SE-1(2):140–149. https://doi.org/10.1109/TSE.1975.6312834 4. Che H, Zeng S, Guo J (2019) Reliability assessment of man-machine systems subject to mutually dependent machine degradation and human errors. Reliab Eng Syst Saf 190:106504. https:// doi.org/10.1016/j.ress.2019.106504 5. Sharit J (2012) Human error and human reliability analysis. In: Handbook of human factors and ergonomics, pp 734–800. Wiley, Hoboken, NJ, USA. https://doi.org/10.1002/978111813 1350.ch26 6. Cacciabue PC (1988) Evaluation of human factors and man-machine problems in the safety of nuclear power plants. Nucl Eng Des 109(3):417–431. https://doi.org/10.1016/0029-549 3(88)90287-7 7. Chauvin C, Lardjane S, Morel G, Clostermann JP, Langard B (2013) Human and organisational factors in maritime accidents: analysis of collisions at sea using the HFACS. Accid Anal Prev 59:26–37. https://doi.org/10.1016/j.aap.2013.05.006 8. Mendeley Reference Manager (n.d). Accessed April 30, 2021. https://www.mendeley.com/ref erence-manager/library/all-references 9. Web of Science [v.5.35]—Web of Science Core Collection Basic Search (n.d). Accessed April 29, 2021. https://apps-webofknowledge-com.ezproxy.lib.purdue.edu/WOS_GeneralSearch_i nput.do?product=WOS&search_mode=GeneralSearch&SID=6EKEdLuDT2UBSrhNcCA& preferencesSaved= 10. VOSviewer—Visualizing Scientific Landscapes (n.d). Accessed April 30, 2021. https://www. vosviewer.com/ 11. MAXQDA | All-In-One Tool for Qualitative Data Analysis & Mixed Methods (n.d). Accessed May 1, 2021. https://www.maxqda.com/ 12. Vicinitas : Twitter Analytics Tool for Tracking Hashtags, Keywords, and Accounts (n.d). Accessed May 1, 2021. https://www.vicinitas.io/ 13. Rasmussen J (2021) General rights risk management in a dynamic society a modelling problem. Downloaded from Orbit.Dtu.Dk On. https://doi.org/10.1016/S0925-7535(97)00052-0 14. Cacciabue PC (2004) Human error risk management for engineering systems: a methodology for design, safety assessment, accident investigation and training. Reliab Eng Syst Saf 83(2):229–240. https://doi.org/10.1016/j.ress.2003.09.013 15. WHO | Global Status Report on Road Safety 2015 (2018). WHO. http://www.who.int/violence_ injury_prevention/road_safety_status/2015/en/ 16. Global Status Report on Road Safety 2018 (n.d). Accessed May 1, 2021. https://www.who.int/ publications/i/item/9789241565684 17. Lecun Y, Bengio Y, Hinton G (2015) Deep learning. Nature. Nature Publishing Group. https:// doi.org/10.1038/nature14539 18. NSF Award Search: Award # 1918140—FMitF: Collaborative Research: Track I: Preventing Human Errors in Cyber-Human Systems with Formal Approaches to Human Reliability Rating and Model Repair (n.d). Accessed April 29, 2021. https://nsf.gov/awardsearch/showAward? AWD_ID=1918140&HistoricalAwards=false 19. Pot M, Kieusseyan N, Prainsack B (2021) Not all biases are bad: equitable and inequitable biases in machine learning and radiology. In: Insights into imaging. Springer Science and Business Media Deutschland GmbH. https://doi.org/10.1186/s13244-020-00955-7

Safety, Maintenance and Physical Modeling of Vehicle Packaging and Assembly

Systematic Literature Review of Safety Management Systems in Aviation Maintenance Operations Natalie Zimmermann and Vincent G. Duffy

Abstract Safety, and by extension safety management systems (SMS), are increasingly popular notions and practices in a variety of industries, including the aviation industry. Historically, SMS concepts in aviation were primarily applied in the context of flight crew management, flight safety, and air traffic management. Nevertheless, safety management practices are now similarly increasingly applied to aviation maintenance operations, commonly referred to as MRO (maintenance, repair, and overhaul). To understand the progress in this area and to identify emerging trends together with potential future developments, a detailed and structured literature review was performed. The results of the analysis are used to identify emerging themes and areas for future work. Additionally, factors hindering the application of SMS concepts in the area of aviation maintenance operations are discussed. Similarly, elements required for further implementation thereof are presented. Furthermore, emerging themes such as communication, teamwork, organizational structure and effort, and human factors are described. Lastly, the results of the literature review are tied to— and discussed within the context of—the impact and application of SMS to aviation maintenance operations with human-automation interaction. Keywords Safety management systems · Aviation maintenance · Human-automation interaction · Bibliometric analysis · Systematic literature review

N. Zimmermann (B) · V. G. Duffy Purdue University, West Lafayette, IN 47907, USA e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Switzerland AG 2023 V. G. Duffy et al. (eds.), Human-Automation Interaction, Automation, Collaboration, & E-Services 11, https://doi.org/10.1007/978-3-031-10784-9_19

311

312

N. Zimmermann and V. G. Duffy

1 Introduction and Background In the past, safety was managed reactively. This means that organizations focused on preventing accidents from reoccurring, rather than reducing the risks of accidents occurring in the first place [1]. Additionally, the investigation of organizational accidents focused on identifying the errors and the individuals that committed said errors. The managerial aspects, workplace culture, and the working environment were rarely assessed [1]. Nevertheless, the safety focus has shifted over the years from a reactive to a proactive approach. To compete in the nowadays global and competitive market, a safety-first corporate culture is a critical element [2]. A safety-first culture is reflected by how an organization, namely the managers and employees, act and perform in a variety of scenarios [2]. This approach includes the establishment and implementation of a safety management system (SMS), as these safety management systems integrate safety and health into the management of an organization and its associated operations [3].

1.1 Safety Management Systems in Aviation Safety has always been a prevalent factor in aviation. Safety issues in the aviation industry are highlighted in the form of accidents, and thus, are comparatively visible to the public. Therefore, accident investigation techniques have been continuously developed and the findings thereof are applied to the ultimate goal of increasing safety [4]. Additionally, human factors significantly impact the operation of the aviation industry, and as such, further influence its safety [5]. These human factors can relate to the characteristics of the individuals in the system, as well as to the working conditions that affect the performance of said individuals [6]. A variety of approaches, systems, and safety tools are applied to aviation operations and accidents to understand the human factors and their influence. Examples include crew resource management—CRM [7], as well as the human factors analysis and classification system—HFACS [8]. Additionally, safety management systems are continuously introduced and applied to the various aspects of the industry, ranging from air traffic management to maintenance operations [5], even in small-scale operations [9]. A variety of organizations control, determine and mandate the safety requirements and regulations of aviation operations. Examples of said organizations include the International Civil Aviation Organization (ICAO), the European Union Aviation Safety Agency (EASA), and the United Kingdom Civil Aviation Authority (CAA) [10]. Even though with the same goal, each of the organizations provides a different regulatory safety framework, laws, standards, and recommendations [10]. Current literature in the area specifically focuses on the evaluation of the effectiveness of the SMS systems in place [11].

Systematic Literature Review of Safety Management …

313

Safety Management Systems in Aviation Maintenance Operations As a specific area of the aviation industry, maintenance operations—also commonly referred to as maintenance, overhaul, and repair (MRO)—are subject to human errors and associated safety issues. Elements such as trust [12], maintenance documentation [13], human error control [14], safety audits [15], as well as errors and contributing factors [16] are analyzed in literature with relation to their impact on safety. Additionally, an array of studies focusing on the evaluation of the implementation of safety management systems in specific maintenance organizations are performed. For example, McDonald et al. [17] analyzed the relationship between safety management systems and safety culture. Similarly, Jaiswal et al. [18] analyzed the safety culture and safety management systems in maintenance organizations in the United Arab Emirates. Furthermore, tools for the implementation of safety management systems are explored, such as the REPAIRER human factors safety reporting system [19].

1.2 Topic Emergence and Engagement The popularity, emergence, and engagement of the topical area related to safety management systems in the aviation industry are reflected in the form of an upward trend of scholarly and research publications. A search in Authormapper [20] reflected a 2.57 growth in the area of SMS in aviation maintenance operations. This indicator is obtained through a comparison between the number of articles published in the topic area in 2019 and 2015, respectively. Furthermore, in the designated five-year timeframe, 489 authors published 290 articles, reviews, and books in the area of SMS in aviation maintenance operations. The results of the Authormapper [20] search are summarized in Table 1.

2 Problem Statement The systematic review performed outlines and assesses the evolution of the literature in the area of safety management systems and the application thereof to aviation maintenance operations and activities. Through the literature review, the current state Table 1 Topic emergence indicators based on AuthorMapper [20] literature search Emergence indicators

Emergence values

Growth (Articles in 2019/Articles in 2015)

2.57

Network

489 Authors

Persistence

290 Articles

314

N. Zimmermann and V. G. Duffy

of SMS implementation in maintenance activities is evaluated. Additionally, factors that hinder additional implementation together with limitations and inefficiencies of existing applications are identified. Lastly, emerging elements and keywords that are identified in the literature as factors influencing further implementation of SMS in maintenance activities are described. Through the systematic literature review, future applications and developments in the area can be devised.

3 Procedure The systematic literature review procedure was adapted from Fahimnia et al. [21]. Various databases were searched by using predefined keywords. The results of these searches were analyzed to identify leading authors, institutions, and journals. Similarly, the citations were reviewed to identify authors as well as key and core articles in the topic area. Additionally, key terms in the literature are identified through content analysis. The results from the analyses are discussed in terms of emerging keywords and elements for further development of SMS in aviation maintenance organizations. Lastly, areas for future work are outlined based on the aforementioned analyses as well as emerging areas furthered by the National Science Foundation [22].

3.1 Keywords The first step in the systematic literature review, as outlined by Fahimnia et al. [21], consists of identifying the keywords for the literature searches. For the topic area of SMS in aviation maintenance organizations, the following keywords were used: “SMS”, “safety management”, “safety”, “aviation”, “aircraft maintenance”, and “aviation maintenance”. The exact combination of keywords used was dependent on the purpose of each analysis step and the database used for each search.

3.2 Data Collection Combinations of the aforementioned keywords were used for searches in the following databases: Web of Science [23], Scopus [24], AuthorMapper [20], SpringerLink [25], ResearchGate [26], Google Scholar [27], and Google Scholar through Harzing [28]. Table 2 highlights the exact keywords used to search each database and the number of publications in each database that match the keywords. The highest number of publications—specifically 9111—were found in SpringerLink [25] with the respective keyword string shown in Table 2. The lowest number of publications— specifically 189—were obtained from the Google Scholar search through Harzing [28], using the keywords and Boolean operators highlighted in Table 2. The number

Systematic Literature Review of Safety Management …

315

Table 2 Keywords and databases used for the literature search and number of publications found Database

Keywords

Web of science [23]

(“SMS” OR “safety management” OR “safety”) 234 AND (“aviation maintenance” OR “aircraft maintenance”)

Web of science [23]

(“SMS” OR “safety management” OR “safety”) 4960 AND (“aviation maintenance” OR “aircraft maintenance” OR “aviation”)

Scopus [24]

(“SMS” OR “safety management” OR “safety”) 429 AND (“aviation maintenance” OR “aircraft maintenance”)

AuthorMapper [20]

(“SMS” OR “safety management” OR “safety”) 215 AND (“aviation maintenance” OR “aircraft maintenance”)

SpringerLink [25]

(“SMS” OR “safety management” OR “safety”) 9111 AND (“aviation maintenance” OR “aircraft maintenance”)

ResearchGate [26]

(“SMS” OR “safety management” OR “safety”) AND (“aviation maintenance” OR “aircraft maintenance”)

Google scholar [27]

(“SMS” OR “safety management” OR “safety”) 793 AND (“aviation maintenance” OR “aircraft maintenance”)

Harzing—Google scholar [28] (“SMS” AND (“aviation maintenance” OR “aircraft maintenance”))

Publications

+ 100

189

of publications obtained from the ResearchGate [26] was inconclusive, as the search display was limited to a maximum of 100 results.

3.3 Trend and Content Analysis A trend analysis was performed based on the Scopus [24] search, using the according keyword string shown in Table 2. The graph in Fig. 1 [24] reflects the trend of publications in the topic area. The first articles were published in the 1960s, however, a steady upward trend was not established until the late 1990s and early 2000s. Even though on certain individual years the number of publications in the topic area dropped, an overall upward trend is discerned. The drop observed in 2020 is likely due to the search being performed at the beginning of the year 2020, and thus not including a representative timeframe for 2020, rather than due to a decrease in the interest of the topic area. The same Scopus [24] search was used to identify the leading authors and institutions in the topic area. Table 3 provides a list of the top five leading authors contributing to the literature together with the respective leading keywords. The

316

N. Zimmermann and V. G. Duffy

Fig. 1 Safety management systems in aviation maintenance literature trend analysis [24]

authors added between six and seven publications to the literature in an eight- to 23-year timeframe. The literature of the identified top authors presents similar or overlapping keywords. Furthermore, amongst the top authors, the decades of the 1990s, 2000s, and 2010s are observed to be fruitful periods, matching the trend depicted in Fig. 1. Leading authors were also identified from the Google Scholar search through Harzing [28]. The parameters for this search were adjusted to only include literature published between 2010 and 2020 with the keywords highlighted in Table 2. The reduced and modern publishing timeline for this search was chosen to identify more recent literature. Table 4 provides an overview of the top five leading authors from this search. One author, namely William B. Johnson, is reflected as a leading author in both searches. The remaining four authors do not overlap. It is important to note, however, that the publication rate is higher amongst the more recent authors identified in Table 4. For example, Ender Gerede (Table 4) published seven articles in two years, while Colin G. Drury (Table 3) published the same number of articles that matched the Table 3 Leading authors contributing to the literature based on Scopus [24] search Author

Years

Leading keywords

Count

Drury, C. G

1990–2013

Ergonomics, Aviation Maintenance, Inspection, Accident Prevention, Job Analysis

7

Johnson, W. B.

1990–2013

Human Engineering, Aviation, Aviation Maintenance, Maintenance, Accident Prevention

7

Taylor, J. C

1990–2017

Accident Prevention, Airframes, Aviation Maintenance, Maintenance, Repair

7

Hobbs, A.

2002–2010

Aircraft, Human, Maintenance, Accident Prevention, Occupational Accident

6

Williamson, A.

2002–2015

Aircraft, Human, Maintenance, Safety, Accident Prevention

6

Systematic Literature Review of Safety Management …

317

Table 4 Leading authors contributing to the literature based on Harzing [28] search Author

Years

Count

Gerede, E.

2017–2019

7

Johnson, W. B.

2011–2017

7

Avers, K.

2011–2014

5

Bagan, H.

2017–2019

5

Cromie, S.

2013–2017

4

search parameters in 23 years. The increasing publication rate can be interpreted as a topic emergence indicator. The top five leading institutions based on the aforementioned Scopus [24] search are outlined in Table 5. The United States is the country with the most contributing institutions in the top five, with four institutions. The greatest individual contributor is the University at Buffalo, NY, with 10 publications. Trinity College Dublin, Ireland, and Clemson University (in the United States) share second place with nine publications each. However, the Trinity College Dublin in Ireland is the single greatest contributor to literature outside of the United States. A list of the top journals and proceedings at which the literature is published or presented, respectively, is shown in Table 6. This table was compiled with the data from the Harzing [28] search, with literature published between 2010 and 2020. The leading conference proceeding is the International Conference on Applied Human Factors and Ergonomics, with four publications. The place as the leading journal is shared by the Aircraft Engineering and Aerospace Technology journal and the Cognition, Technology & Work journal, with three publications each. Table 5 Leading institutions contributing to the literature based on Scopus [24] search Institution

Country

Leading keywords

Count

University at Buffalo, The State University of New York

U.S.A

Ergonomics, Inspection, Aviation Maintenance, Human Error, Accident Prevention

10

Trinity College Dublin

Ireland

Accident Prevention, Human Engineering, Aircraft, Safety, SMS

9

Clemson University

U.S.A

Aircraft, Maintenance, Accident Prevention, Air Transport, Aircraft Maintenance

9

Embry-Riddle Aeronautical University

U.S.A

Maintenance, Aviation, Aviation Maintenance, Safety Engineering, Aircraft Maintenance

8

NASA Ames Research Center

U.S.A

Aircraft, Human Errors, Maintenance, Accident Prevention, Ergonomics

7

318

N. Zimmermann and V. G. Duffy

Table 6 Top journals/proceedings for literature in the topic area based on Harzing [28] search Journal or proceeding name

Count

International Conference on Applied Human Factors and Ergonomics

4

Aircraft Engineering and Aerospace Technology

3

Cognition, Technology & Work

3

2018 Advances in Science and Engineering Technology International Conferences (ASET)

2

Journal of Air Transport Management

2

Proceedings of the Human Factors and Ergonomics Society Annual Meeting

2

4 Results The results from the trend and content analyses aforementioned were further analyzed in terms of authorship, reference, and citation information, as well as keywords.

4.1 Authorship and Citation Analysis A co-authorship analysis was performed using the data retrieved from the Google Scholar search through Harzing [28], as outlined in Table 2. The search retrieved literature published between 2010 and 2020. Figure 2 was compiled through VOSViewer [29] and represents an author map based on co-authorship. Specifically, only authors with three or more publications were included, resulting in a cluster or 23 publications. CiteSpace [30] was used to identify the leading 10 publications with the strongest citation bursts. For this analysis, the 4,960 publications obtained from the Web of Science [23] search with the keywords highlighted in Table 2 were used. The result

Fig. 2 VOSViewer [29] co-authorship analysis map based on Harzing [28] search

Systematic Literature Review of Safety Management …

319

thereof is shown in Fig. 3. The articles with the strongest citation bursts were all published on, or before, the year 2000, and experienced their citation bursts before the 2010s. The chart in Fig. 4 was built based on the data obtained from the Harzing [28] search, with publications between 2010 and 2020. The most cited article from this search is “The multitasking myth: Handling complexity in real-word operations” by Barshi et al. [31]. A co-citation analysis map was created to perform an analysis thereof using the search results from Web of Science [23] with the respective keywords reflected in Table 2. The co-citation map was created using VOSViewer [29] and is shown in Fig. 5. Specifically, the threshold for publications to be included in the map was set to be two citations, resulting in 21 publications to be reflected therein. Key articles were selected from the produced map (shown in Fig. 5) and listed in Table 7. A series of overlapping authors between the top publications from the co-citation analysis (Table 7) and the leading authors in Tables 3 and 4 are discerned. Specifically, the following authors emerge in both analyses: Alan Hobbs, Ann Williamson, and Colin G. Drury.

Fig. 3 References with the strongest citation bursts [30] based on Web of Science [23] search

Fig. 4 Top 10 most cited publications based on Harzing [28] search

320

N. Zimmermann and V. G. Duffy

Fig. 5 VOSViewer [29] co-citation analysis map based on Web of Science [23] search

Table 7 Key articles from co-citation analysis Authors

Title and publication information

Year

Wiegmann, D. A., and Shappell S. A

A human error approach to aviation accident analysis: the human factors analysis and classification system. Ashgate

2003

Hobbs, A., and Williamson, A

Association between errors and 2003 contributing factors in aircraft maintenance. Human Factors: The Journal of the Human Factors and Ergonomics Society

Chang, Y. H., and Wang, Y. C

Significant human risk factors in aircraft maintenance technicians. Safety Science

2010

Drury, C. G., Guy, K. P., and Wenner, C. A Outsourcing aviation maintenance: human 2010 factors implications, specifically for communications. The International Journal of Aviation Psychology Federal Aviation Administration (FAA)

Aviation maintenance technician handbook. 2018 U.S. Department of Transportation

Systematic Literature Review of Safety Management …

321

4.2 Keyword and Cluster Analysis The literature was also reviewed through content analysis tools, specifically to identify keywords and clusters in the published work. For these analyses, VOSViewer [29], CiteSpace [30], and MaxQDA [32] were used. The literature content obtained from the 2010–2020 Harzing [28] search was mapped through VOSViewer [29], as shown in Fig. 6. The map was adjusted to include words that were repeated five or more times. Additionally, words not related to safety management systems or tied thereto were manually deselected and removed. Key clusters from the map in Fig. 6 are shown in Table 8. Specifically, three main clusters are selected: aviation organizations, human influence, and safety elements. Each cluster reflects critical keywords related to the implementation of safety management systems in aviation maintenance organizations and activities. An additional cluster analysis was performed using CiteSpace [30], as shown in Fig. 7. For this analysis, 4960 publications from the aforementioned Web of Science

Fig. 6 VOSViewer [29] keyword map based on Harzing [28] search

Table 8 Keywords clusters based on key term information extracted from VOSViewer [29] Aviation organizations

Human influence

Safety elements

ICAO

Human errors

Safety culture

Federal Aviation Administration

Human factors

Risk management

Aviation maintenance organization

Fatigue

Safety management system

322

N. Zimmermann and V. G. Duffy

[23] search were considered. Key clusters obtained through the analysis include “critical incidents”, “human error”, and “collective mindfulness”. The cluster “critical incidents” refers to the study and analysis of accidents and incidents that have occurred to identify causal factors and prevent future occurrences thereof. As previously discussed, relating to the “human error” cluster, the human element is an important factor in the area of safety. Consequently, it is crucial to understand human errors and their impact on an operation. Lastly, the cluster “collective mindfulness” indicates the importance of team- and group-work for safety purposes, highlighting that safety in an organization is not an individual task, but rather a group effort. Therefore, collective mindfulness is a critical item discussed in the literature. Key publications retrieved from the various database searches afore described, as well as from the co-citation, co-authorship, and content analyses were combined to form a Wordcloud using MaxQDA [32] software. Specifically, the following references were included in the Wordcloud: Zimolong and Elke [1], Brauer [3], Landry [5], Flin et al. [7], Wiegmann and Shappell [8], Müller and Drax [10], Yeun et al. [11], Chatzi et al. [12], Drury and Johnson [13], Chen and Zhou [14], Hsiao et al. [15], Hobbs and Williamson [16], McDonald et al. [17], Jaiswal et al. [18], Miller and Mrusek [19], and Fahimnia et al. [21]. The resulting Wordcloud is shown in Fig. 8, highlighting the following keywords: “safety”, “management”, “aviation”, “system”, “human”, and “factors”. Furthermore, keywords previously identified from leading authors and institutions (Tables 3 and 5), the VOSViewer [29] map (Fig. 6), and the associated cluster analysis (Table 8) are also reflected in the Wordcloud (Fig. 8).

Fig. 7 CiteSpace [30] cluster map and analysis based on Web of Science [23] search

Systematic Literature Review of Safety Management …

323

Fig. 8 MaxQDA [32] Wordcloud based on key terms of selected literature

Examples of these keywords include “safety culture”, “organizations”, “accidents”, “errors”, and “hazards”.

5 Discussion There is a clear and defined upward trend in the literature related to safety management systems in the area of aviation maintenance activities and organizations. Key concepts and terms focus primarily on safety elements, such as risk management and the investigation of maintenance-related accidents, as well as on human elements, such as human errors. These trends are reflected, for instance, in the research conducted by Reason [6], Wiegmann and Shappell [8], Chen and Zhou [14], as well as Hobbs and Williamson [16]. Nevertheless, the current implementations of safety management systems in aviation maintenance organizations require some degree of improvement to achieve their maximum and ideal efficiency. According to Yeun et al. [11], the audits currently in place to evaluate the effectiveness of a safety management system are not an absolute assessment tool. Consequently, further evaluation tools and methods need to be developed to adequately and functionally assess the effectiveness of SMS frameworks [11]. By extension, variations in SMS implementation across individual organizations are possible. One of the factors affecting the variations includes cultural differences, dependent on the country each organization is based. For instance, Jaiswal et al. [18] studied safety attitudes and safety management systems in maintenance organizations in the United Arab Emirates. The research [18] showed that the overall safety culture and the implementation of safety management systems were positive. Nevertheless, room for improvement was evident, specifically in terms of reporting culture, safety training, and safety meetings [18].

324

N. Zimmermann and V. G. Duffy

Table 9 Leading project terms and equivalent terms in theoretical frameworks Project terms

Theoretical framework

Communication

Safety promotion

Teamwork

Teamwork approach to promoting safety

Organizational effort

Safety-first corporate culture

Aviation oversight agencies

Legislation and regulation

Human errors

Human factors

The literature review further provided insight into elements and factors that impact the effective development and implementation of SMS into aviation maintenance activities specifically. The identified factors are centered around elements of communication, teamwork, organizational efforts, oversight agencies, and human errors. Table 9 outlines how these five elements relate to theoretical frameworks discussed by experts in the topic area.

5.1 Communication, Teamwork, and Organizational Effort Communication and teamwork can be tied to the organizational effort, which, when combined, reflect the collective mindfulness cluster of the literature identified in Fig. 7. As outlined by Brauer [3], SMS is an organizational approach to the management of safety, and thus involves every individual in an organization, ranging from front-line workers, i.e. the aviation maintenance technicians, to the managerial levels. Communication, teamwork, and organizational effort further tie to safety promotion, the teamwork approach to promoting safety, and to safety-first corporate cultures, respectively. According to Goetsch [2], one aspect of promoting safety includes communicating safety expectations with every individual involved in an organization’s operation. Thus, communication is a key element for the promotion of safety. One method of safety promotion is the so-called teamwork approach. This approach consists of providing cross-functional teams with the task of improving safety [2]. Through the teamwork approach, an organization can reinforce the concept of safety being a team effort, and thus, that everyone in an organization—together—is responsible for safety. The ultimate organizational effort is reflected in a safety-first corporate culture, where the entire organization shares safety-first attitudes, believes, expectations, and behaviors [2].

5.2 Aviation Oversight Agencies In the realm of safety, especially in terms of occupational safety, a vast regulatory and legislative body is in place. In the United States, the Occupational Safety and Health

Systematic Literature Review of Safety Management …

325

Agency (OSHA) is the primary occupational safety regulatory and oversight agency [2]. Amongst others, OSHA Standards also apply to—and consequently regulate— aviation maintenance activities. However, aviation-specific oversight and regulatory agencies play a further critical role. The involvement of aviation regulatory agencies, such as the aforementioned ICAO, FAA, CAA, and EASA, is critical for the implementation of safety management systems. These agencies establish the SMS frameworks that are to be implemented [10], and thus impact how maintenance organizations manage safety.

5.3 Human Errors Aviation maintenance operations are highly influenced by human factors, such as fatigue and stress [5]. According to Goetsch [2], human factors greatly impact the safety of a workplace, and as such, must be taken into account and managed through the implementation of safety management systems. Landry [5] highlights that even though the area of human factors in aviation maintenance operations is critical, it is scarcely researched. A variety of provisions can be adopted to better manage and potentially reduce the impact of human factors and, by extension, human errors in aircraft maintenance activities. Examples of these provisions include designing activities in a way that hazards are eliminated, introducing safety and warning devices, as well as furthering existing safety provisions and training [2].

5.4 Automation and Safety The interaction of humans and automated systems is integral to the operation of the aviation industry overall and specifically crucial to the safety thereof. The inclusion of automated systems—especially in the cockpit—is cited as a large contributor to the outstanding recent safety record of the industry [33]. Commonly thought of examples of automation in aviation include the autopilot as well as automatic flight plan generation together with alerts and monitoring processes for flight and air traffic control, respectively [34]. Similar to SMS, the implementation of automated systems is not limited to the flight and air traffic control elements of the aviation industry, but also extends into the aviation maintenance operations. The traditional maintenance activities—such as inspections and repairs—heavily rely on manual and non-automated work [35]. Instead, the automation is included in maintenance support activities. Specifically, automation is employed in two main areas. First, to provide up-to-date maintenance information to mechanics, such as maintenance task cards with instructions, parts tracking, and job scheduling. Second, in troubleshooting and testing of aircraft systems and components. A common automated tool is the built-in-testing (BITE) used to troubleshoot avionics components [35].

326

N. Zimmermann and V. G. Duffy

As aforementioned, the introduction of automation has positively impacted the overall safety of the aviation industry. Nevertheless, with automation integration, new threats to safety emerge. First, with the addition and implementation of automated systems, established ways of operating the aviation systems have changed, ultimately requiring new procedures to be established [33]. Second, the automated systems do not operate independently and do not replace the human element. As such, a certain level of human-automation interaction is required, and methods for the adequate and safe integration thereof are needed [34]. Furthermore, with a specific focus on maintenance activities, the limitations of the human element need to be considered to avoid implementing automation in a way that over- or undermines the human component [35]. As phrased by Drury [35]: “Automation does not reduce the importance of human factors. Instead, it makes more vital the integration of human workers with automated systems at all stages of the design process” (p. 5). Consequently, when designing, implementing, and evaluating the introduction of SMS in aviation organizations and operations—and specifically within the realm of aviation maintenance activities—automation-related elements need to be considered. Safety management systems need to account for the threats to safety posed by automated systems, especially concerning the human-automation interaction, by providing methods to systematically recognize and correct said threats and dangers.

6 Conclusion and Future Work The literature in the area of safety management systems in aviation maintenance activities and organizations covers both, the theoretical framework of safety management in maintenance activities as well as practical studies focusing on the effectiveness of the implementation thereof. Through the systematic literature review, leading authors, institutions, and journals together with conference proceedings in the topic area were identified. Additionally, the results obtained from the searches and leading publications were used to perform authorship, citation, and content analyses. Through these techniques, a variety of key terms related to the implementation of SMS in aviation maintenance organizations and for future development thereof were identified. As the current literature has moved from theoretical frameworks to practical applications, future work in the area of SMS in aviation maintenance activities can be focused on the integration of safety education in aviation maintenance technician training as well as on the incorporation of automation and its interaction with maintenance personnel. The educational effort can focus specifically on SMS principles, the application thereof, and its associated importance and benefits. A large number of the National Science Foundation [22] awards in the area of aviation are focused on new teaching and educational methods, specifically for aviation maintenance technicians. Additionally, collaboration projects between industry and university educational programs are emerging, as in the example provided by Genis et al. [36]. These methods primarily focus on the integration of new technology in training. However,

Systematic Literature Review of Safety Management …

327

they can be expanded to include safety principles to ultimately improve and increase the safety of the industry.

References 1. Zimolong BM, Elke G (2006) Occupational health and safety management. In: Salvendy G (ed) Handbook of human factors and ergonomics, 3rd edn. Wiley, Hoboken, NJ, pp 671–707 2. Goetsch DL (2019) Occupational safety and health for technologists, engineers, and managers, 9th edn. Pearson, New York, NY 3. Brauer R (2016) Safety and health for engineers, 3rd edn. Wiley, Hoboken, NJ 4. Helmreich RL (2000) On error management: lessons from aviation. BMJ 320(7237):781–785 5. Landry SJ (2012) Human factors and ergonomics in aviation. In: Salvendy G (ed) Handbook of human factors and ergonomics, 4th edn. Wiley, Hoboken, NJ, pp 1667–1688 6. Reason J (2000) Human error: models and management. BMJ 320(7237):768–770 7. Flin R, O’Connor P, Mearns K (2002) Crew resource management: Improving team work in high reliability industries. Team Perform Manag 8(3/4):68–78 8. Wiegmann DA, Shappell SA (2001) Human error analysis of commercial aviation accidents: application of the human factors analysis and classification system. Aviat Space Environ Med 72:1006–1016 9. Goglia J (2019) Torqued: the case for voluntary SMS for small operators. Aviation International News. https://www.ainonline.com/aviation-news/blogs/torqued-case-voluntary-smssmall-operators 10. Müller R, Drax C (2014) Fundamentals and structure of safety management systems in aviation. In: Müller R, Wittmer A, Drax C (eds) Aviation risk and safety management. Springer, Cham, Switzerland, pp 45–55 11. Yeun R, Bates P, Murray P (2014) Aviation safety management systems. World Rev Intermodal Transp Res 5(2):168–196 12. Chatzi AV, Martin W, Bates P, Murray P (2019) The unexplored link between communication and trust in aviation maintenance practice. Aerospace 6(6):1–18 13. Drury CG, Johnson WB (2013) Writing aviation maintenance procedures that people can/will follow. Proc Hum Factors Ergon Soc 57(1):997–1001 14. Chen N, Zhou C (2011) Civil aviation maintenance human error control assessment based on evidence theory. In: Yan X, Yi P, Wu C, Zhong M (eds) ICTIS 2011: multimodal approach to sustained transportation system development: Information, technology, implementation. American Society for Civil Engineers, Reston, VA, pp 2122–2128 15. Hsiao Y-L, Drury C, Wu C, Paquet V (2013) Predictive models of safety based on audit findings: part 1: model development and reliability. Appl Ergon 44(2):261–273 16. Hobbs A, Williamson A (2003) Associations between errors and contributing factors in aircraft maintenance. Hum Factors 45(2):186–201 17. McDonald N, Corrigan S, Daly C, Cromie S (2000) Safety management systems and safety culture in aircraft maintenance organisations. Saf Sci 34(1–3):151–176 18. Jaiswal K, Al-Mahadin A, Verma S, Singh B (2018) Safety culture in aircraft maintenance organizations of United Arab Emirates. In: 2018 Advances in science and engineering technology international conferences (ASET). IEEE, pp 1–8 (2018) 19. Miller M, Mrusek B (2019) Implementing the REPAIRER human factors safety reporting system through MRM(MxF) to meet SMS compliance in aviation maintenance. In: Arezes PM (ed) Advances in safety management and human factors. Springer, Cham, Switzerland, pp 46–57 20. AuthorMapper Homepage, https://www.authormapper.com/. Last accessed 24 April 2020 21. Fahimnia B, Sarkis J, Davarzani H (2015) Green supply chain management: a review and bibliometric analysis. Int J Prod Econ 162:101–114

328

N. Zimmermann and V. G. Duffy

22. 23. 24. 25. 26. 27. 28.

National Science Foundation Homepage, https://www.nsf.gov/. Last accessed 28 April 2020 Web of Science Homepage, https://www.webofknowledge.com/. Last accessed 24 April 2020 Scopus Homepage, https://www.scopus.com/. Last accessed 20 April 2020 SpringerLink Homepage, https://link.springer.com/. Last accessed 27 April 2020 ResearchGate Homepage, https://www.researchgate.net/. Last accessed 20 April 2020 Google Scholar Homepage, https://scholar.google.com/. Last accessed 20 April 2020 Harzing’ s Publish or Perish Homepage, https://harzing.com/resources/publish-or-perish/. Last accessed 24 April 2020 VOSViewer Homepage, https://www.vosviewer.com/. Last accessed 17 April 2020 CiteSpace Homepage, http://cluster.ischool.drexel.edu/~cchen/citespace/download/. Last accessed 22 April 2020 Loukopoulos LD, Key Dismukes R, Barshi I (2009) The multitasking myth: handling complexity in real-world operations. Routledge, London, United Kingdom MaxQDA Homepage, https://www.maxqda.com/. Last accessed 20 April 2020 Chialastri A (2012) Automation in aviation. In: Kongoli F (ed) Automation. IntechOpen, pp 79–102 Billings CE (1997) Aviation automation: the search for a human-centered approach. CRC Press, Boca Raton, FL Drury CG (1998) Automation. In: Maddox M (ed) Human factors guide for aviation maintenance. Federal Aviation Administration, Washington, DC Genis V, Vyas S, Midora T, Bottari R (2010) Industry-university collaboration in course development and instruction. In: Conference for industry and education collaboration. ASEE

29. 30. 31. 32. 33. 34. 35. 36.

Multimodal Interactions Within Augmented Reality Operational Support Tools for Shipboard Maintenance Victoria L. Claypoole, Clay D. Killingsworth, Catherine A. Hodges, Jennifer M. Riley, and Kay M. Stanney

Abstract Large ships, both military and commercial, are dependent on many complex and interdependent systems, necessitating notoriously intensive maintenance regimens. For instance, low event rates, ambiguous problem presentation, temporal stressors, and high working memory demands are common challenges for shipboard maintenance and repair personnel. Operational support systems are often employed in this context to supplement or fill training gaps at the point-of-need. Such support, however, has traditionally been heavily dependent on insight and input from subject matter experts, which places a steep premium on personal experience. Recent advances in augmented reality (AR), especially head-worn displays (HWDs), present a promising avenue for improving the efficacy of operational support by leveraging multimodal interactions and displays. Multimodal solutions provide an opportunity to present support information, in a more veridical form, and, if carefully designed, without increasing demand on operators. Real-world spatialization of such information sources via AR is one technique that can be used to increase the level of operational support while simultaneously reducing short-term memory demands imposed by traditional operational support tools, the latter of which require mental transformation from the medium of the support tool (e.g., a technical manual) to the V. L. Claypoole · C. D. Killingsworth · C. A. Hodges · J. M. Riley · K. M. Stanney Design Interactive Inc, Orlando, Florida, USA e-mail: [email protected] J. M. Riley e-mail: [email protected] K. M. Stanney e-mail: [email protected] C. D. Killingsworth University of Central Florida, Orlando, Florida, USA e-mail: [email protected] V. L. Claypoole (B) Dynepic, Inc., Reno, Nevada, USA e-mail: [email protected] C. D. Killingsworth Guidehouse, McLean, Virginia, USA © The Author(s), under exclusive license to Springer Nature Switzerland AG 2023 V. G. Duffy et al. (eds.), Human-Automation Interaction, Automation, Collaboration, & E-Services 11, https://doi.org/10.1007/978-3-031-10784-9_20

329

330

V. L. Claypoole et al.

operator’s environment. The broad range of capabilities of AR HWDs also affords tailoring operational support tools to the unique sensory needs of each use case and, in conjunction with adaptive training techniques and advances in AI, to the needs of each operator. Herein, we provide an overview of how multimodal AR can be implemented within operational support tools; best practices for the design and development of multimodal interactions and displays within AR are discussed. Keywords Operational support · Job performance aid · Maintenance training

1 Introduction and Background The maritime industry is critical for global operations; over 50 percent of imported and exported goods are transported via large ships [11]. It is imperative that global operations, such as commerce transportation and military missions, are executed successfully and on-time. One of the key processes for ensuring high reliability of ship operations and improved onboard system performance is maintenance operations [5]. In fact, the single most influential factor in successful maritime transportation is ship maintenance and repair [61]. However, these large ships are dependent on many complex and interdependent systems, necessitating notoriously intensive maintenance regimens that require experienced personnel.

1.1 Shipboard Maintenance Maintenance activities, and the knowledge, skills, and abilities required to successfully execute them, are multifaceted and tend to vary from organization and ship. In the Navy, the required range of maintenance skills is vast and far surpasses the requirements of commercial ships [46]. One example is the varied age range of Naval ships, which make standard maintenance training difficult. Naval ships range from the newly commissioned to those with over 20 years of service—and the required maintenance skills for each ship within the Fleet, and the equipment, systems, and components onboard each one are unique [46]. While potentially less complex, commercial shipping fleets similarly need effective maintenance operations, as delays in servicing and inspection of ships can result in undetected problems that lead to downtime, shortened machine life expectancy, and operations disruption, among other concerns. The bottom-line… if effective maintenance operations are not maintained, both Navy and commercial fleet readiness may be compromised [74]. The significance of maintenance operations can be seen by examining associated investments and manpower allocations. For example, maintenance departments comprise the largest expense and workforce in shipping companies [20]. Likewise, up to 40% of maritime shipping operation costs have been attributed to maintenance [2]. In this vein, it is readily apparent how critical ship maintenance is; from cost to

Multimodal Interactions Within Augmented Reality Operational …

331

loss of life, maintenance impacts every aspect of ship performance [49]. In particular, maintenance failures pose considerable costs, both monetary and otherwise [64]. Failures can lead to major accidents, impose safety concerns by endangering crew, result in business losses, damage the ship itself, and even threaten operators and their environments [49, 64]. Unfortunately, maintenance operations are particularly susceptible to error as associated tasks are typically complex, require high levels of skills, and are commonly executed in harsh working conditions under extreme time pressure [56]. Moreover, inadequacy of training and support documentation for operators can also lead to hampered maintenance operations [56]. One report noted that maintenance manuals and support documentation are extremely lacking, with documents oftentimes being poorly presented, without style, void of clarity, and even error prone [12].For example, NASA operations are highly dependent on maintenance manuals yet information-related document errors have been implicated in a multitude of maintenance errors, with the most problematic documentations issues being incompleteness, incorrectness interpretation difficultly, conflicting information, and obsolescence [48]. Taken together, such maintenance manual deficiencies can result in operators violating protocols and contributing to human error. Furthermore, Military maintenance operations are experiencing additional, unique challenges. For example, the new Ford class carriers are reduced crew environments—meaning, fewer Sailors are required to man and operate the ships. However, these new carriers contain many new, complex, and unique systems that still require skilled operators. Coupled with a higher working tempo and fewer opportunities to performance scheduled maintenance [73], maintenance delays, errors, and failures have increased within the Navy [47]. Previous research has advocated for the use of operational support tools to overcome the challenges associated with maintenance errors, failures, and delays in maintenance operations.

1.2 Historical Methods for Operational Support As maintenance operations become increasingly more complex and streamlined, providing just-in-time, operational support to operators will be critical. In the past, operational support has been referred to in the literature as a host of different names. For example, the terms job aids, performance aids, job performance aids (JPAs), Electronic Performance Support Systems (EPSS), performance support, job guide, skill performance aid, procedural guide, and others have been used almost interchangeably [8, 14, 37, 63]. For the purposes of this chapter, these tools and methods will be discussed as one construct—operational support. Operational support refers to an umbrella of tools and methodologies aimed at providing point-of-need assistance to human users. In this vein, any tool or methodology that assists a person in completing a task more effectively and efficiently would fall under the umbrella of operational support. An operational support tool includes any artifact that improves performance by facilitating or guiding operators in their

332

V. L. Claypoole et al.

pursuit of executing work-related tasks [14]. Thus, the nomenclature provided above is meant to be exemplary, not prescriptive. Operational support enables consistent task performance while simultaneously minimizing error. These tools assist workers by storing task performance information in an easily accessible form, ultimately reducing long-term memory and working memory constraints as the amount of information to be remembered is minimized [8]. In fact, though operational support can be traced back to World War II, the best-known operational support tool was first introduced as the “pilot’s checklist”, which stemmed from the need to reduce human error associated with the crash of the Boeing Model 299 Aircraft [27, 29, 50]. Since the advent of the pilot’s checklist, operational support tools have assumed many formats. In fact, operational support can range from a simple note to a large electronic database [37]. The most common formats are informational aids, procedural aids, and decision-making aids [37, 62]. Informational aids provide basic facts and concepts related to systems or subsystems and ensure accuracy of the work, procedural aids present logical, step-wise information of procedures affording increased efficiency and accuracy, finally decision-making aids guide and assist with making the correct decision (e.g., determining correct outcome of a troubleshooting test). These three categories have been further distilled into six key formats: step, worksheet, array, decision table, flow chart, and checklist [8]. Irrespective of the format, operational support tools provide important job-related information (e.g., maintenance, troubleshooting, operation, etc.) before and during task performance. The use of operational support tools has resulted in immense benefits. Operational support has been found to be a valuable tool for novice workers, simplifying task requirements and enabling them to perform more like experts [27]. Moreover, it has been found that operational support affords the transference of skills from training to the workplace, ultimately resulting in standardization of job performance and product development [37]. In this vein, operational support tools have been found to be a cost-effective supplement to formalized training as they support distributed knowledge, revolutionizing workplace support and learning by providing opportunities for knowledge gains in new contexts—not just the classroom [63]. They have even been touted as an enhancement to workforce development [8]. In terms of performance, those who use operational support tools have been found to be more productive, commit fewer errors, and execute tasks quicker [14]. Reviews of operational support [8, 10, 22, 63] have noted several additional advantages for leveraging these tools, including, reduced training time, consistently high performance, less downtime of operations, increased safety via fewer accidents and mishaps, decreased completion time of work-related tasks, improved productivity in terms of both quantity of tasks completed and quality of those tasks, uniform product quality, and decreased need for managerial supervision of and assistance with complex tasks. Similar advantages have been observed within the military and maintenance domains. For example, one review determined that operational support tools made significant and positive contributions to the military, especially in terms of cost savings and performance gains [21]. A plethora of research has indicated that improved maintenance performance has been demonstrated within Air Force maintainers, regardless of their experience level [18, 40]. The use of operational support

Multimodal Interactions Within Augmented Reality Operational …

333

with military maintainers has been shown to result in reduced repair times, thousands of saved manpower hours, a reduction labour costs, and improved performance of novice maintainers [40]. Of utmost important to the military domain is an increase in overall readiness, derived from improved maintainer performance, decreased downtime and increased aircraft availability. Thus, it is clear how beneficial operational support tools can be—independent of operational context or organization type. With the technological advancements of computing capabilities during the last century, operational support tools have been developed in electronic formats in addition to the traditional paper-based formats. Both computer- and paper- based operational support tools have been shown to enhance job performance [67]. Conversely, other studies have suggested that better comprehension and retention of operational support may be demonstrated when information is presented in paper-based as opposed to computer-based mediums [69]. In fact, recommendations for operational support development have largely advocated paper-based artifacts due to the cost and development time associated with developing computer-based products [8]. However, new technologies are constantly emerging and improving every aspect of human life, resulting in the modernization of society before our eyes. One such technology that has great potential to improve and modernize operational support is Augmented Reality (AR).

2 Modernizing Operational Support Augmented Reality (AR) is a technology that allows for overlaying the real environment with computer-generated content, which can take the form of two- and three-dimensional objects, multimodal cues, and other digital entities that can be seamlessly superimposed on to the real world [54]. Typically, digitized AR content is tied to specific geo-spatial locations or pre-programmed activities. Put simply, AR overlays virtual objects on to the real world [41, 43, 44]. In recent years, AR has been used in a variety of domains, including, education, training, military, entertainment, and e-commerce [1, 26, 68, 83]. AR has recently been applied to operational support—with great success. For instance, one study reported preliminary evidence that AR-based operational support tools are just as effective as traditional paperbased tools; novice workers demonstrated fewer errors and reduced time, a finding consistent with classical operational support research [39]. Beyond comparable effectiveness, AR technology provides the capability for widespread distribution across an organization, instantly putting operational support tools in the hands of those who need it. Currently, there is a noticeable lack of use of AR-based operational support tools within the maritime industry, both commercial and military. This is unfortunate, as the pedagogical and operational advantages conveyed by AR, along with the technology’s recent emergence in consumer markets, have led to a rise in adoption for education, training, and on-the-job support in other domains. For example, existing

334

V. L. Claypoole et al.

applications in smart manufacturing and logistics leverage AR to provide point-ofneed operational support by allowing operators to monitor assembly line processes and access contextualized user manual content [19], and tools are currently being developed to facilitate the transfer of implicit, institutional knowledge from experienced maintenance technicians to trainee recruits via AR solutions [15]. As at-sea deployments take Sailors and maritime operators into operational environments for up to 9 months at a time, providing ample time for knowledge loss and skill decay [42], a need exists for operational support at the point-of-need for shipboard maintenance. The maritime sector can leverage the aforementioned innovative technology to support critical shipboard maintenance operations via AR-based operational support.

2.1 Implementing Augmented Reality (AR) for Operational Support The potential advantages of AR are immense. First, the concomitance of the real environment overlaid with virtual objects allows for visualization of abstract concepts and complex spatial relationships [3], which are critical for enabling novices to perform like experts. It also provides the possibility to experience phenomena that are not possible, or safe to perform, in the real world [41]. For example, when integrated with digital twin technologies, AR-based operational support tools can show operators the outcomes of potential actions, thereby facilitating decision-making, reducing error, and increasing safety. With respect to implementation, computer-based operational support may, in theory, occur over any sensory channel by which operators perceive or transmit information (i.e., the five senses). In practice, however, most such solutions involve only a few modalities (visual, auditory, and tactile). But even these channels are limited in what, or how much, they can transmit. Despite the utility of the eyes and ears for gathering spatial information—especially when working in concert—visual and auditory signals from a computer-based support tool carry relatively little spatial information owing to the physical constraints of screens and speakers. Yet, AR-based operational support tools can fill this spatialization gap, which may prove critical when conducting maintenance operations on a physical system such as machines found in a ship’s engine room. In addition to manual input vectors, modern AR head worn displays (HWDs) may receive audio inputs via voice commands and visuospatial inputs via gaze and eyetracking. Further, ongoing development seeks to add other input modalities including olfaction [23] and gustation [52]. Novel output modalities, including active haptic signaling [82], are being added to the repertoire of AR applications seeking to increase the fidelity and engagement of these systems. Notable among cutting-edge output modalities are the multimodal interaction effects pursued in efforts to alter inputs of one channel indirectly by manipulating a separate channel. For example, in noisy operational environments, tactile information (e.g., vibration on left hip) could be

Multimodal Interactions Within Augmented Reality Operational …

335

used to convey spatial information [24], such as the direction to focus on during an inspection. Thus, AR has the potential to afford the use of novel multimodal interactions, thereby extending and enhancing the type of support provided.

2.2 AR-Based Multimodal Interactions for Operational Support Previous research has noted that operational support tools can impose additional, unwanted cognitive load demands [72]. However, designing for multimodal interaction helps to minimize the cognitive load imposed by using an operational support tool in two ways. First, because cognitive processing resources are limited, and more specifically because each sensory channel possesses limited resources, which may be further diminished by the stress associated with operational context [4, 79, 80], designing an operational support tool such that it distributes the informational load across multiple modalities reduces the risk of overloading any individual channel [33]. Overloading one (or more) channels, even when other channels are below their capacities, can increase cognitive load, produce operator distress, and increase errors [34], which would diminish the efficacy of an operational tool. For example, consider the task of picking an item from a warehouse for shipment. Navigating the facility may place demands on spatial working memory [28], while selecting the correct item may require recalling the part number. If both navigation instructions and part number are conveyed verbally—as is likely if conveyed by a human operator—the worker must depend on verbal working memory resources for both. If, however, the navigation instructions are presented in an AR-based operational support tool via visual-spatial cues (e.g., highlighting the path to follow) and the part number is conveyed auditory, the task remains the same while task demands are distributed across multiple (semi-independent) resource pools, thereby reducing cognitive load. Second, multimodal interaction may reduce cognitive load by removing the need for users to translate between modalities, preserving the ability to deal with inputs and outputs in their native forms. The warehouse picker, when presented with verbal navigation guidance, must translate from the modality of the transmission to one they may use to complete the task; in this case, from auditory-verbal guidance to visualspatial directional information. If an AR system can present the same information in a visual-spatial mode, as with a visual way finding overlay that highlights the path to follow, the operator needs to complete at least one fewer cognitive step to use the information, resulting in faster comprehension [35]. Studies of navigation efficiency and warehouse worker performance confirm that AR HWD job aids improve performance when presenting information in a form immediately usable by the operator, with use of multimodal AR aids resulting in fewer navigation errors [51], fewer missed signals in identification tasks [57] and faster task completion [6]. Subjective assessments of cognitive load are also lower when AR aids present navigation information visually or via non-verbal audio [6]. The next sections describe how

336

V. L. Claypoole et al.

four interaction types (i.e., visual, auditory, haptic, and olfactory) can be designed to facilitate AR-based multimodal interaction within operational support, ultimately enhancing performance and benefiting the maritime maintenance sector. Visual. Visual cues are best suited for delivering spatial information to users [78]. Indeed, this is the main benefit of using AR operational support tools—to see procedural aids or task-relevant information in context (e.g., [9]). Visual cues also serve as an efficient output modality in communication exchanges between operator and system as they allow the operator to see large amounts of information at once (e.g., step-by-step instructions), freeing up valuable working memory resources by keeping information “in the world” [13]. When coupled with auditory cues, synergistic intersensory facilitation effects can be achieved [70, 71]. For example, combining a visual marker with a spatialized auditory cue can be an effective way to direct attention to a point of interest and ground operators in their environment. While visual cues are most appropriate for these types on interactions, the effects they have on operators’ cognitive load cannot be ignored. Visual cues are more discernable than cues of other modalities, but they require more pre-frontal cortex processing to interpret and react to [30]. Overusing such cues, or making them persistent in the environment, can clutter an operator’s working environment and decrease situational awareness. In these cases, other modalities—or combinations of modalities—may be more appropriate. Auditory. Auditory cues are best at relaying temporal or time-sensitive information, as well as enhancing spatial awareness [78]. They can be used to guide visual search during inspection and navigation. Audio stimuli can also be useful to enhance fidelity of interactions that are difficult to replicate in AR, such as vection [59, 60]. Vection is defined as “an illusion of self-motion” [55]. An applied example of such acoustic illusions could be playing subtle sound effects of wind rushing by to simulate a sailor walking across a ship deck to an inspection area. These cues could assist the sailor in estimating the time it would take to traverse to the area in which maintenance operations need to be performed. Related to auditory cues, voice as an input is efficient at communicating from operator to system, and allows for hands-free exchanges. Tactile (or Haptic). The use of haptics in an augmented operational support tool is primarily suited to providing spatial information (e.g., left hip vibration) and sensory substitutions for virtual object manipulation as to increase interaction fidelity. For AR-based operational support, this could mean simulating a virtual nut tightening to the correct tension by increasing the intensity of vibrational feedback as it approaches the correct position. While most modern mobile devices that support AR have haptic capabilities (e.g., smartphone with vibration functions), some applications may require the use of haptic gloves. In cases where the system is used with a HWD, pseudo-haptics (i.e., the use of visual and auditory stimuli to simulate the experience of touch) can be a substitution [17]. It should also be noted, however, that offloading information to tactile channels from more heavily used modalities, such as vision, holds promise for managing cognitive load [38]. For example, White and Hancock [77] demonstrated substantial performance gains in target detection for

Multimodal Interactions Within Augmented Reality Operational …

337

directional cues delivered via a tactile belt compared to similar visual or auditory cues. Olfactory. Recent research has shown that the use of “e-noses” in AR-supported maintenance environments can be used as a substitution for identifying and discriminating smells [23]. While wide-spread use of olfactory cues in AR operational support tools is still highly experimental, there has been much research on its potential benefits. Distinct odors tend to linger in memory long term, whereas visual text information gradually disappears [36]. Scents also appear to relate strongly to emotions “Scents are extremely evocative, and can shift attention, add novelty, enhance mental state [58].” Smell has also been described as a “near sense” meaning that all the information and qualities a smell has can be perceived regardless of distance, even if the smell itself becomes fainter. Olfactory cues are also easily distinguishable, allowing humans to discriminate between most odors [45]. In the context of training, odors can be used to manipulate mood, increase vigilance, decrease stress, and improve retention and recall of learned materials [81]. More research is needed to understand whether olfactory cues are synergistic with visual/auditory/haptic cross-modal interactions.

3 Best Practices for Design and Development of Multimodal Interactions and Displays There are many aspects of design that must be considered with care for any operational support system. However, the shift from a paper-based to 2D computer-based to a 3D AR-based system brings additional design considerations that, if ignored, will drastically impact the effectiveness of the support being received. The triedand-true principles of human factors still apply, but their implementation must be adapted to support the needs of operators executing their task, in their environment. The following is a best-practices guide for designing multimodal AR-based operational support for maintenance. Special consideration is applied to the pillars of User Centered Design (UCD).

3.1 Designing Interactions for the Environment The success of any system, operational or not, is hampered by its worst micro interaction. Charles Eames’ famous quote “The details are not the details. They make the design” applies not only to the creation of sleek furniture, but to the design world at large. As AR operational support tools move into operators’ real-world spaces, the unique environment of large ships, for example, imposes itself onto the smallest interaction. Therefore, the success of any interaction, and by extension the AR operational support tool, will depend on to what degree the interactions consider

338

V. L. Claypoole et al.

the operator’s environment. As important as ensuring the effectiveness of support, is ensuring the safety of Sailors and operators within their environment. Since AR systems overlay digital multisensory information, a poorly designed interaction that does not consider the operator’s capabilities and the environment, could impede an operator’s situational awareness, increase cognitive load, and lead to accidents. The following aspects of operators’ environments must be considered in the design of AR operational support tools. Lighting Conditions. There are dramatic differences in light levels in operational environment, such as onboard a ship, ranging from the bright sunlit deck to florescent lit windowless rooms. To further complicate conditions, a room’s illuminance and the reflective surfaces within are typically configured to specific standards for the task that will be completed inside [75]. The light level of working areas can thus drastically impact an operator’s ability to see AR system elements. Because of the effects lighting conditions have on the contrast, color shift, and stereo acuity of AR HWDs [25], some tasks may not be appropriate to be supported by an AR-based operational support tool, especially those concerning color or depth perception. Thus, the lightning environment of each working area must be considered when determining how much a of multi-modal interaction depends on visual cues. An operator working above deck in natural sunlight will be able to discern less contrast between visual elements than one working in a darker room; consequently, any interactions an operational support tool implements must rely strongly on multimodal auditory and haptic cues. Inversely, a sailor working in a room with consistent illuminance levels below deck could perceive visual elements much more readily, but perhaps be in a louder environment (e.g., an engine room). In this case, interactions must rely more on visual cues than auditory cues. Regardless of lightning condition, visual elements of an AR system should always use highly contrasting colors. However, it is important to note that most AR HWDs do not recommend use of pure white for any overlay elements, as they will appear too bright and intense on the eye. Conversely, very dark colors are also not recommended, as darker colors are perceived as being transparent [76]. Thus, the known lightening conditions must be considered in the design and implementation of AR-based operational support tools. Sound Levels. Much like lighting conditions, sound levels in operational environments, such as those onboard a ship, vary widely. Operators can experience noise levels ranging from 86 to 101 decibels during a 12–14 h extended work shift [66]. Noise, ambient or otherwise, can severely affect the design of AR systems intended for operational use. For example, one study indicated that sailors onboard a Naval aircraft carrier reported difficulty perceiving auditory instructional content of an ARbased training system, even with the device sounds levels set to max [16]. However, the authors posited that for significantly loud operational environments, auditory instructional content should be used as a supplement to visual instructional content, and not the main sensory mode of delivery. For example, in one AR-based knowledge capture tool currently being used in maritime environments, sailors are given guidance on what to capture with visual text while auditory narration of the instructions is played in the background [15]. Such audio-visual multimodal presentation results in better learning retention and transfer of learning to operational tasks [31,

Multimodal Interactions Within Augmented Reality Operational …

339

53]. Even though the on-ship environment experienced by sailors is likely to be too loud for audio narration, auditory modalities should not be wholly discounted. Audio cues, including spatial audio cues, can still be used for intersensory facilitation effects [70, 71], including system feedback, directing attention, and grounding operators spatially. Alignment between auditory and visual cues can increase operators’ ability to navigate [7], and improve their performance on search tasks [65]. For example, an interaction for directing an operator’s attention towards a specific piece of equipment could include a digital spotlight illuminating the target area along with a spatialized auditory cue. Even in a loud environment, the interaction is expected to be effective as it is not important that the operator interpret the “meaning” of the sound, only that it is heard. The example interaction above is doubly effective in that it reduces the need for world-cluttering AR overlay elements, increasing an operator’s situational awareness of their real-world environment.

3.2 Designing Interactions for the Task The fidelity of multimodal interactions in an AR operational support tool should be informed by the tasks being supported. Two methodologies that can inform what level of fidelity a multimodal interaction should have is Cognitive Task Analysis (CTA) and Sensory Task Analysis (STA). Both start with finding a relevant Subject Matter Expert (SME) and rely on observations and interviews with that expert. However, different components of a task are gleamed from each analysis. The objective of a CTA is to “capture a description of the explicit and implicit knowledge that experts use to perform complex tasks” [32]. The end result of a CTA is the understanding of an expert’s mental model, decision-making and problem-solving processes, and critically, when high levels of cognitive load can be expected in a procedure. The STA is used to identify the critical multimodal cues experts gather and use to determine their next course of action [32]. By gathering this information, a clearer picture of which sensory modality should be used, and when, can be deduced and used to inform the design of AR-based operational support tools. For example, these analyses may conclude in order for an operator to make a critical decision in a particular step in troubleshooting an issue, a switch needs to be flipped and then a real-world auditory cue needs to be detected. This information suggests that to support this interaction via an AR tool, both visual and spatial auditory cues should be used to direct attention to the switch that needs flipping. This design would allow the operator to listen for the real-world sound and reduce cognitive load while processing task information that would inform their next decision. Afterwards, the AR tool should be designed to detect step completion (either by object and/or gesture recognition, or manual operator input), and immediately remove both the visual and auditory stimuli to avoid cognitive overload.

340

V. L. Claypoole et al.

4 Conclusions Expertly executed maintenance operations are critical to the successful implementation of maritime activities, both commercial and military. As maintenance operations become increasingly more complex and demanding, it will be imperative to support operators with state-of-the art operational support. Operational support tools have a long history of improving job performance, enabling expert performance, facilitating productivity, and reducing errors—among a whole host of other benefits [8, 10, 22, 63]. With the rise of recent technological advancements, particularly in the realms of spatial computing and augmented reality, outdated paper-based operational support tools can be treated to a twenty-first century update. Existing AR-based operational support tools have been found to allow operators to experience phenomenon not possible in the real world [41] and significantly improved work performance [51]. Unfortunately, given the novelty of this technology, best-practices for the design and development of AR-based operational support tools are not readily available. This chapter argues that those creating these next-gen operational support tools must leverage the available multimodal interactions afforded by this technology in order to see the greatest order of magnitude gains. Multimodal AR-based operational support tools can be used to provide point-of-need assistance and shoulder some of the cognitive load that currently falls to maintainers in ship-based and other operational environments. AR tools can be designed to receive inputs from operators in a natural way (e.g., affect-dependent voice commands, gestures, eye tracking) and provide operators with information in a form or forms that afford easy comprehension and application (e.g., auditory and tactile cues used in visually cluttered operational environment). Such multimodal design is anticipated to offload overloaded channels (e.g., visual) and present information in a form in which it will be readily used by operators. As technologies like object recognition and eye tracking mature and become more seamlessly integrated with AR HWD’s, the line separating training and operational support may begin to blur. If an AR-based operational support tool is capable of effectively monitoring an operator’s task performance while donning an HWD by capturing where his eyes are looking and which objects are focused on, it will be possible to provide performance feedback. This presents the ability for AR operational support to fill one of the greatest needs for training skilled work: personalized feedback.

References 1. Al-Azawi M (2019) The application of eye-tracking in consumer behaviour. Int J Eng Technol 8:83–86 2. Alhouli Y, Ling D, Kirkham R, Elhag TM (2009) On the factors affecting maintenance planning in the mercantile industry. In: COMADEM2009, 22nd international congress on condition monitoring and diagnostic engineering management, San Sebastian, Spain 3. Arvanitis TN et al (2009) Human factors and qualitative pedagogical evaluation of a mobile augmented reality system for science education used by learners with physical disabilities. Pers

Multimodal Interactions Within Augmented Reality Operational …

341

Ubiquit Comput 13:243–250 4. Baddeley A (2000) The episodic buffer: a new component of working memory? Trends Cogn Sci 4(11):417–423 5. Bayer D, Aydın O, Celik M (2018) An ICOR approach towards ship maintenance software development. Int J Maritime Eng 160:A11–A20 6. Beitzel S, Dykstra J, Huver S, Kaplan M, Loushine M, Youzwak J (2016) Cognitive performance impact of augmented reality for network operations tasks. In: Advances in human factors in cybersecurity, pp 139–151 7. Bormann K (2005) Presence and the utility of audio spatialization. Presence: Teleoperators Virt Environ 14:278–297 8. Campbell JP, Gasser MB, Oswald FL (1996) The substantive nature of job performance variability. In: Murphy KR (ed) Individual differences and behavior in organizations. Jossey-Bass 9. Carl B, Bopp M, Saß B, Voellger B, Nimsky C (2019) Implementation of augmented reality support in spine surgery. Eur Spine J 28(7):1697–1711 10. Chalupsky AB, Kopf TJ (1967) Job performance aids and their impact on manpower utilization. https://eric.ed.gov/?id=ED015316 11. Chambers M, Liu M (2012) Maritime trade and transportation by the numbers. https://www. bts.gov/archive/publications/by_the_numbers/maritime_trade_and_transportation/index 12. CHIRP: marine operating and maintenance manuals—Are they good enough? www.chirp. co.uk (2006) 13. Clark A, Chalmers D (1998) The extended mind. Analysis 58(1):7–19 14. Clark DR (1999) Big dog and little dog’s juxtaposition homepage, http://nwlink.com/~don clark/design/design_models.html. Last Accessed on 22 Jan 2021 15. Claypoole VL et al (2020) Leveraging emerging technologies for shipboard training and sail-or support: recent advancements and current operational challenges. In: Proceedings of the fleet maintenance and modernization symposium 16. Claypoole VL, Stanney KM, Padron CK, Perez R (2020) Enhancing naval enterprise readiness through augmented reality knowledge extraction. In: Proceedings of the interservice/industry training, simulation, and education conference (I/ITSEC) Annual Meeting 1 17. Collins K, Kapralos B (2019) Pseudo-haptics: leveraging cross-modal perception in virtual en-vironments. Sens Soc 14:313–329 18. Curtis C, Whited V, Kancler D, Burneka C (2006) Analyzing requirements for and designing a collaborative tool based on functional and user input. In: International symposium on collaborative technologies and systems (CTS’06), pp 220–225 19. Deac CN, Deac GC, Popa CL, Ghinea M, Cotet CE (2017) Augmented reality in smart manufacturing. In: Proceedings of the 28th DAAAM international symposium, pp 727–732 20. Dikis K, Lazakis I, Theotokatos G (2015) Dynamic reliability analysis tool for ship machinery maintenance. In: Towards green marine technology and transport, pp 639–646. CRC Press 21. Duncan CS (1985) Job performance aids. Job aids really can work: a study of the military application of job aid technology. Performance + Instruction 24:1–4 22. Egger J, Masood T (2020) Augmented reality in support of intelligent manufacturing—a systematic literature review. Comput Ind Eng 140:106195 23. Erkoyuncu J, Khan S (2020) Olfactory-based augmented reality support for industrial maintenance. IEEE Access 8:30306–30321 24. Erp JBFV, Veen HAHCV, Jansen C, Dobbins T (2005) Waypoint navigation with a vibrotactile waist belt. ACM Trans Appl Percept 2:106–117 25. Fidopiastis CM, Meyer C, Fuhrman CA, Rolland JP (2003) Quantitative assessment of visual acuity in projective head-mounted displays. In: Proceedings of SPIE 5079, helmet- and headmounted displays VIII: technologies and applications, pp 399–406 26. Flavián C, Ibáñez-Sánchez S, Orús C (2019) The impact of virtual, augmented and mixed reality technologies on the customer experience. J Bus Res 100:547–560 27. Fosshage E (2014) The effect of job performance aids on quality assurance. In: Proceedings of the human factors and ergonomics society annual meeting, vol 58, pp 1959–1963

342

V. L. Claypoole et al.

28. Garden S, Cornoldi C, Logie RH (2002) Visuo-spatial working memory in navigation. Appl Cogn Psychol Official J Soc Appl Res Memory Cogn 16(1):35–50 29. Gawande A (2009) The checklist manifesto: How to get things right. Metropolitan Books, New York, NY 30. Gazzaley A, Rissman J, Cooney J, Rutman A, Seibert T, Clapp W, D’Esposito M (2007) Functional interactions between prefrontal and visual association cortex contribute to top-down modulation of visual processing. Cereb Cortex 17:i125–i135 31. Grant K, Greenberg S (2001) Speech intelligibility derived from asynchronous processing of auditory-visual information. In: Proceedings of the workshop on audio-visual speech processing 32. Hale KS, Stanney KM, Milham LM, Carroll MAB, Jones DL (2009) Multimodal sensory information requirements for enhancing situation awareness and training effectiveness. Theor Issues Ergon Sci 10:245–266 33. Hancock PA, Warm JS (1989) A dynamic model of stress and sustained attention. Human Factors 31(5):519–537 34. Hancock PA (2013) In search of vigilance: the problem of iatrogenically created psychological phenomena. Am. Psychol 68(2):97 35. Harrison SM (1995) A comparison of still, animated, or nonillustrated on-line help with written or spoken instructions in a graphical user interface. In: Proceedings of the SIGCHI conference on human factors in computing systems. Addison-Wesley Publishing Co, pp 82–89 36. Herz RS (1998) Are odors the best cues to memory? A crossmodal comparison of associative memory stimuli. Ann N Y Acad Sci 855:670–674 37. Jackson RD (2012) Performance aids. In: The encyclopedia of human resource management. John Wiley & Sons, Ltd, Hoboken, NJ, pp 348–352 38. Jerome CJ, Witmer B, Mouloua M (2006) Attention orienting in augmented reality environments: effects of multimodal cues. Proc Human Factors Ergon Soci Ann Meet 50(17):2114– 2118 39. Jetter J, Eimecke J, Rese A (2018) Augmented reality tools for industrial applications: What are potential key performance indicators and who benefits? Comput Hum Behav 87:18–33 40. Kancler D, Curtis C, Burneka C, Bachmann S (2006) Design consideration tests: mid-project verification and validation. Proc Human Factors Ergon Soc Ann Meet 50:2507–2511 41. Klopfer E, Squire K (2008) Environmental detectives-the development of an augmented reality platform for environmental simulations. Educ Technol Res Dev 56:203–228 42. Kluge A, Frank B (2014) Counteracting skill decay: four refresher interventions and their effect on skill and knowledge retention in a simulated process control task. Ergonomics 57(2):175– 190 43. Kolivand H, Sunar MS (2015) An intelligent application for outdoor rendering taking sky color and shadows into account 513:178–190 44. Kolivand H, Sunar MS, Selamat A (2015) Real-time light shaft generation for indoor rendering 532:487–495 45. Köster E (2002) The specific characteristics of the sense of smell. Olfaction, Taste and Cognition 27–44 46. Kunjumon A (2017) Challenges in naval ship maintenance. In: Proceedings of the international maritime conference 47. Larter D (2009) The US military ran the largest stress test of its sealift fleet in years. It’s in big trouble. Defense News 48. Lattanzio D, Patankar K, Kanki BG (2008) Procedural error in maintenance: a review of research and methods. Int J Aviat Psychol 18:17–29 49. Lazakis I, Dikis K, Michala AL, Theotokatos G (2016) Advanced ship systems condition monitoring for enhanced inspection, maintenance and decision making in ship operations. Transport Res Procedia 14:1679–1688 50. Meilinger PS (2004) When the fortress went down. Air Force Magazine 51. Meneghetti C, Labate E, Toffalini E, Pazzaglia F (2019) Successful navigation: the influence of task goals and working memory. Psychol Res 1–15

Multimodal Interactions Within Augmented Reality Operational …

343

52. Nakano K, Horita D, Sakata N, Kiyokawa K, Yanai K, Narumi T (2009) DeepTaste: augmented reality gustatory manipulation with GAN-based real-time food-to-food translation. In: 2019 IEEE international symposium on mixed and augmented reality (ISMAR), pp 212–223 53. Oviatt S, Coulston R, Lunsford R (2004) When do we interact multimodally? Cognitive load and multimodal communication patterns. In: Proceedings of the 6th international conference on multimodal interfaces, pp 129–136 54. Palmarini R, Erkoyuncu JA, Roy R, Torabmostaedi H (2018) A systematic review of augmented reality applications in maintenance. Robot Comput-Integr Manuf 49:215–228 55. Palmisano S, Allison RS, Schira MM, Barry RJ (2015) Future challenges for vection research: definitions, functional significance, measures, and neural bases. Front Psychol 6 56. Pennie DJ, Brook-Carter N, Gibson WH (2007) Human factors guidance for maintenance. In: Proceedings of the human factors in ship design and operation 57. Ren G, Wei S, O’Neill E, Chen F (2018) Towards the design of effective haptic and audio displays for augmented reality and mixed reality applications. Adv Multimedia 58. Richard E, Tijou A, Richard P, Ferrier J-L (2006) Multi-modal virtual environments for education with haptic and olfactory feedback. Virtual Reality 10:207–225 59. Riecke BE, Feuereissen D, Rieser JJ (2009) Auditory self-motion simulation is facilitated by haptic and vibrational cues suggesting the possibility of actual motion. ACM Trans Appl Percept 6(20):1–20:22 60. Riecke B (2010) Compelling self-motion through virtual environments without actual selfmotion: using self-motion illusions (’Vection’) to improve VR user experience. Virtual Reality 61. Roshandeli M (2007) Pathology of maritime industries in country. Trans Ind J 62. Rossett A, Gautier-Downs J (1991) A handbook of job aids. Wiley & Sons, Hoboken, NJ 63. Rossett A, Schafer L (2006) Job aids and performance support: moving from knowledge in the classroom to knowledge everywhere, 2nd edn. Pfeiffer, Hoboken, NJ 64. Rothblum A (2000) Human error and marine safety. In: Proceedings of the maritime human factors conference, Linthicum, MD 65. Rumi´nski D (2015) An experimental study of spatial sound usefulness in searching and navigating through AR environments. Virtual Reality 19 66. Schaal NC, Majar M, Hunter A (2019) Sound level measurements in berthing areas of an aircraft carrier. Ann Work Exposures Health 63:918–929 67. Shriver EL, Hart FL (1975) Study and proposal for the improvement of military technical in-formation transfer methods, Contract No DAAD05–74-C-0783, AD-A-023409, US army human engineering laboratory, Aberdeen proving grounds, MD 68. Sirakaya M, Alsancak Sirakaya D (2018) Trends in educational augmented reality studies: a systematic review. Malays Online J Educ Technol 6:60–74 69. Staff (1988) Keep it short and simple. Interactive Courseware User’s Group (ICUG) Bull 8:1–2 70. Stanney K et al (2004) A paradigm shift in interactive computing: deriving multimodal design principles from behavioral and neurological foundations. Int J Human-Comput Inter 17:229– 257 71. Storms RL (2002) Auditory-visual cross-modality interaction and illusions. In: Handbook of virtual environments: design, implementation, and applications, pp 455–469. Lawrence Erlbaum Associates, Mahwah, NJ 72. Swaak J, de Jong T (2001) Learner vs. system control in using online support for simulationbased discovery learning. Learn Environ Res 4:217–241 73. Sydow KR (2008) Shipboard maintenance: what do surface warfare officers need to know—and when do they need to know It? Nav Eng J 120:89–98 74. Tambe S, Bayoumi A-ME, Cao A, McCaslin R, Edwards T (2015) An extensible CBM architecture for naval fleet maintenance using open standards. In: Proceedings of the intelligent ship symposium 75. U.S. Department of Defense (2019) DoD design criteria standard for human engineering (MILSTD-1472G-CHG1) 76. Vitazko M (2018) Color, light, and materials. Microsoft Docs https://docs.microsoft.com/enus/windows/mixed-reality/design/color-light-and-materials

344

V. L. Claypoole et al.

77. White TL, Hancock PA (2020) Specifying advantages of multi-modal cueing: quantifying improvements with augmented tactile information. Appl Ergon 88:103146 78. Wickens CD, Gutzwiller RS, Vieane A, Clegg BA, Sebok A, Janes J (2016) Time sharing between robotics and process control: validating a model of attention switching. Hum Factors 58(2):322–343 79. Wickens CD (2008) Multiple resources and mental workload. Hum Factors 50(3):449–455 80. Wickens CD (2002) Multiple resources and performance prediction. Theoret Issues Ergon Sci 3(2):159–177 81. Youngblut C, Johnson RE, Nash SH, Weinclaw RA, Will CA (1996) Review of virtual environment interface technology IDA paper P-3186. Chapter 8, pp 209–216, http://www.hitl.was hington.edu/scivw/IDA/ 82. Yu X, Xie Z, Yu Y, Lee J, Vazquez-Guardado A, Luan H, ... Rogers JA (2019) Skin-integrated wireless haptic interfaces for virtual and augmented reality. Nature 575(7783):473–479 83. Yuen SC-Y, Yaoyuneyong G, Johnson E (2011) Augmented reality: an overview and five directions for AR in education. J Educ Technol Dev Exchange 4:23

Task Simulation Automation via Digital Human Models: A Case Study on Cockpit Fire and Smoke Emergencies Mihir Sunil Gawand and H. Onan Demirel

Abstract This research introduces a Digital Human Modeling (DHM) based early design framework to automate task analysis and workplace ergonomics for emergencies and provides a proof-of-concept demonstration of the automation framework within the context of cockpit fire and smoke case study. DHM brings significant advantages to explore human-centered design issues by enabling human-product or human-environment interaction analysis within a computer or virtual environment. However, a substantial problem in designing with DHM is the reliance on manual methods during simulation setup and ergonomics analysis. A majority of the DHM workflow requires user input via controllers such as a keyboard, mouse, or wand. Manually creating DHM simulation involves tedious object orientation and task manipulations, which add extra time and effort and decrease the usefulness of DHM as an early design solution. Moreover, DHM design studies within the transportation domain often focus on ergonomics evaluations of tasks that occur in ideal or nearly ideal settings. There is a lack of DHM design tools that concentrate on emergencies. The case study depicted in this paper aims to implement a proactive ergonomics approach via the automation framework to assess reach gap and percent loss in luminance issues in cockpit smoke and fire emergencies. Overall, this research demonstrates how the proposed automation framework automates existing DHM toolkits and extends the capabilities of DHM through third-party technology integration. Keywords Digital human modeling · Design automation · Emergency · Cockpit packaging · Ergonomics · Fire and smoke · Design theory and methodology

M. S. Gawand · H. O. Demirel (B) School of Mechanical, Industrial and Manufacturing Engineering, Oregon State University, Corvallis, OR 97330, USA e-mail: [email protected] M. S. Gawand e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Switzerland AG 2023 V. G. Duffy et al. (eds.), Human-Automation Interaction, Automation, Collaboration, & E-Services 11, https://doi.org/10.1007/978-3-031-10784-9_21

345

346

M. S. Gawand and H. O. Demirel

1 Introduction and Background Fire and smoke in the cockpit are some of the most hazardous and life-threatening situations that happen during the flight. No matter how sophisticated the sensors and suppression systems are and how much training the pilots and crew receive, emergencies due to fire and smoke often occur in commercial aircraft. Even with the introduction of state-of-the-art design and manufacturing practices, fire-resistant materials, and insulation techniques, a plane is prone to smoke, fire, and fume (SFF) emergencies because of the presence of onboard combustibles—primarily the large quantities of flammable fuel stored in the aircraft, miles long electric cables, and also payload in the enclosed cargo and cabin. During the past forty years, more information becomes available on the occurrence of the SFF events. For example, according to Federal Aviation Administration (FAA), between 1981 and 1990, twenty percent of the total fatalities on U.S. transport airlines were due to fire events [1]. A review extracted from the Service Difficulty Report (SDR) database also indicates that 964 recorded smoke and fire events occurred during 1999 [2]. Around 382 of these incidents were related to high-temperature events, which happened in the passenger cabin, cockpit, galley, or lavatory areas. According to Fire, Smoke or Fumes Occurrence (FSF) Database, between 2002 and 2011, around 17,751 records were considered related to smoke and fire events [3]. During 2019 and 2020 alone, SFF events caused more than 50 in-flight emergencies [4]. Aside from the possibility of causing catastrophic loss of aircraft, non-fatal SFF events are also associated with unscheduled landing, damage to airplane components, delays, and cancellations [3], which cause disruptions in the flight and ground activities and increase the overall operational cost. During a fire in the cockpit, smoke accumulation can lead to poor visibility and spatial disorientation, which can cause pilots and flight crew to perform poorly [5]. The vision obscuration coupled with cognitive stressors increases the likelihood of possible human error while the pilot performs tasks in nonnormal conditions. In these scenarios, performance not only depends on the pilots’ experience, skills, physical and cognitive abilities but also the appropriate task and workplace design. An aircraft’s performance and safety can be enhanced when humans become an essential consideration when designing complex systems. In practice, many emergency assessment methods rely on physical prototypes and human subject experiments, which lack the ability to pinpoint failures earlier in design. Often, complex interactions of humans (e.g., pilot and crew) and the emergency conditions (e.g., onboard fire and smoke) are represented in full-scale physical mockups. This conventional prototyping approach is costly, time-consuming, and does not allow quick design changes [6]. It also has severe limitations in the context of data collection for emergency studies. For example, collecting human-machine interaction data on physical mockups bring the risk of environmental and health hazard in fire and smoke-related design studies [7]. This research introduces a Digital Human Modeling (DHM) based early design framework to automate task analysis and workplace ergonomics for emergencies

Task Simulation Automation via Digital Human …

347

and provides a proof-of-concept demonstration of the automation framework within the context of cockpit fire and smoke case study. The design framework also automates the repetitive task simulations and ergonomic evaluations, typically performed manually by the designer. Manually creating DHM simulation involves tedious object orientation and task manipulations, which add extra time and effort and decrease the usefulness of DHM as an early design solution [5, 8]. The case study depicted in this paper aims to implement a proactive ergonomics approach via the automation framework to assess reach gap and percent loss in luminance issues in cockpit smoke and fire emergencies. Overall, this research demonstrates how the proposed automation framework automates existing DHM toolkits and extends the capabilities of DHM through third-party technology integration.

2 Problem Statement 2.1 Cockpit Fire and Smoke Emergencies Onboard smoke or fire events in the cockpit are among the most hazardous emergencies and require immediate action. In the event of SFF incidents, pilots need to perform necessary steps (e.g., going through the manufacturer’s checklist) with autopilot’s aid if the auto-flight system’s functioning is not affected by the fire [9]. Engaging autopilot during such SFF events helps pilots go over the checklist and identify the root cause of the fire and smoke while planning the best strategies. However, many fire and smoke events grow rapidly, are coupled with multiple system failures, and require emergency landing. This chain of events often requires diversion and immediate landing, where pilots disengage from the autopilot and manually operate the aircraft [5]. One of the most daring issues during a fire in a cockpit event is the accumulation of smoke, which adversely affects the pilots’ ability to perform the necessary tasks [10]. Adequate visibility within the cockpit is essential for safe and sustainable flight. In the event of a fire, the smoke buildup can quickly diminish the visibility and deteriorate pilots’ vision. The denser the smoke levels in the cockpit, the harder it gets for pilots to go through the checklist and fly the aircraft. This is particularly life-threatening when pilots need to navigate the plane manually. The presence of the smoke occlusion can quickly tax the pilot, both physically and cognitively, and hinders the successful execution of emergency procedures by obstructing instrument and flight deck panels.

348

M. S. Gawand and H. O. Demirel

2.2 Modeling Cockpit Fire and Smoke Emergencies Computational modeling and simulation of human-machine interaction have gained popularity in product and system design in past decades due to the flexibility of running “what-if” scenarios before actual events occur in actual use conditions [11]. Most human factors engineering (HFE) work in this area focuses on specific problems and offers a conventional approach—extensive human subject data collection via full-scale physical mockups [12]. Although the conventional approach is a preferred method when exploring or developing novel methods and algorithms, it is slow and costly for conceptual design activities. It is especially problematic for early design HFE work in transportation design. For example, physically prototyping large vehicles (e.g., submarine) and collecting human subject data is infeasible, involves a high risk of injury (e.g., crash testing), or propose environmental hazard (e.g., fuel spill) [7]. Overall, the time and cost associated with early design HFE in the transportation domain require a need to bring an integrated representation of humans and the work environment with modeling and simulation toolkits. With the advancements of computational tools and data collection technologies, DHM became a significant asset for early design ergonomics research [13]. DHM refers to the technology of visualizing, simulating, and analyzing musculoskeletal and cognitive attributes of humans within a Computer-Aided Design (CAD) or virtual environment [11, 13]. The early human manikins were initially developed in the aeronautics domain—many pioneering DHM research dates back to the 1970s, including Boing Man (Boeman) [14], Crash Victim Simulator (CVS) [15], and ThreeDimensional Static Strength Predictions (3DSSPP) [16]. These software packages were standalone applications and focused mostly on research activities for large scale projects. Within the past few decades, along with the introduction of modern human data collection technologies (e.g., motion capture) and Computer-Aided Engineering (CAE) platforms, more integrated DHM packages (e.g., Siemens Jack, SANTOS) have introduced with CAD-based comprehensive ergonomics analysis capabilities, including force-influenced posture prediction, clearance, interference detection, reach envelope creation, and comfort assessment [17, 18]. Considerable design work has been performed in the transportation domain with the aid of DHM simulation and analysis tools; however, there is a lack of studies that focus on automating workplace design and task analysis for emergencies or abnormal work conditions. DHM studies are traditionally applied to ergonomics evaluations of ideal or nearly ideal conditions, with environmental considerations that do not represent emergencies [5]. There is a need for developing DHM based concept design evaluation methodologies that incorporate abnormal work conditions (such as emergencies). This paper’s primary contribution is to demonstrate how DHM research can extend ergonomics analysis for emergencies. Although DHM brings significant advantages to early engineering design activities by enabling human-product or human-environment interaction modeling and analysis within a CAD-based interface, a substantial issue in designing with DHM is the reliance on manual scenario setup and ergonomics analysis. A majority of

Task Simulation Automation via Digital Human …

349

the DHM workflow requires user input via controllers such as a keyboard, mouse, or a wand [5]. Manually creating DHM simulation setup involves tedious object orientation and task manipulations such as altering manikin posture, assigning weights, creating reach assignments, changing CAD positions. Typically, this manual point-and-click-based workflow relies on repetitive keystrokes and precision mouse control, which adds extra time and effort and increases the simulation error, particularly within early design applications [10]. In such cases, users generally make changes in the environment, manikin, or task setup until abstract ideas are converted into computational concept models. Overall, the lack of automation workflow available in DHM research diminishes the returns of injecting DHM in early design. Therefore, this paper’s other contribution is to demonstrate a DHM-based automation framework to simulate DHM task analysis.

3 Methodology This section provides the building blocks of the DHM-based automation framework through a cockpit packaging case study. Cockpit packaging involves understanding and considering a wide range of design elements, including pilots and flight crew’s physical and cognitive attributes. The safe and comfortable journey relies on how the workplace design accommodates user needs, abilities, and limitations. The automation framework proposed in this research mainly focuses on early design ergonomics for cockpit smoke and fire emergencies. The methodology section describes the case study’s motivation, then introduces the simulation model and design variables, and finally sums up each building block to layout the automation framework.

3.1 Case Study Aircraft cockpits are unique work environments where a complex set of humanmachine interactions are intertwined with each other. For example, reaching control surfaces for making minor adjustments on aircraft’s attitude requires monitoring information presented on multiple displays since data shifts dynamically due to flight and weather-related changes. The same happens continually, even when the autopilot is engaged. Therefore, during the conceptualization stage, the cockpit packaging variables must be screened carefully as the design elements directly influence pilot performance. For example, designers should evaluate whether protruding control surfaces obscure displays on the instrument panel before committing to final designs. One must note that these design studies should also consider non-normal work conditions and emergencies. For example, SFF emergencies are common events that degrade the pilot performance as obscuration due to smoke accumulation reduces the instrument panel’s visibility.

350

M. S. Gawand and H. O. Demirel

In this study, the ergonomics design problem is formulated based on a design exploration of a commercial cockpit packaging study for smoke and fire emergencies. We used a simplified version of a generic Boeing 767 aircraft cockpit model throughout the study. Based on the SFF smoke accumulation scenario, the cockpit packaging case is evaluated for two ergonomics outcomes: (1) reach gap and (2) percent loss in luminance. This case study’s primary design variables are the location of the control knobs on the front instrument panel, pilot’s seat position, change in anthropometry, and smoke accumulation within the cockpit. Some of this case study elements and inspiration to model emergencies are borrowed from our previous work published earlier [5, 10, 19].

3.2 Simulation Model Hierarchal Task Analysis (HTA). Although not every emergency scenario is identical due to the faulty events’ root-cause and progression, the SFF events and pilots’ actions overlap in many cases. In this study, we built an emergency scenario that represents many overlapping events and task sequences during a fire emergency, followed by a heavy smoke build-up in a cockpit. The automation for DHM-based simulation started with identifying tasks that pilots execute during fire and smoke emergencies. Although some of the steps during such emergencies require manufacturerspecific guidelines, in this study, we used the smoke and fire emergency procedure checklist provided by the National Transport Safety Board (NTSB) [20]. In general, in-flight smoke and fire scenario management include communication with Air Traffic Control (ATC), diverting the plane, and assessing the situation and available resources. This particular scenario represents an unmanageable fire and smoke emergency where pilots perform immediate descent or immediate landing of the fire detected [5, 9, 21, 22]. The following list of tasks represents what pilots typically need to execute in an actual SFF scenario. These steps do not depict all the tasks or reflect the actual guideless in a step-by-step fashion. Instead, we represented some of the subsets of tasks in an ascending order that a pilot might execute according to guidelines, manufacturer recommendations, and industry best practices [5, 9, 21, 22]. Thus, some of the tasks and associated pilot posture and motion would vary slightly in actual conditions. However, even if different cockpit settings are considered, the task sequence represented in the HTA list depicts some of the common actions pilots perform, including wide-area visual screening and upper body movements when monitoring and reaching controls surfaces. The sequence of hypothetical tasks selected for this case study is as follows: 1. 2. 3. 4.

Look at the crew warning and alerting control Reach the warning control Look at the vertical speed indicator Reach the vertical speed control

Task Simulation Automation via Digital Human …

351

5. Look at the Engine Indication and Crew Alerting System (EICAS) display screen 6. Reach the Engine Indication and Crew Alerting System (EICAS) display screen. We injected the above HTA list into the automation framework where each manikin followed the same sequence. Before the automation sequence starts, within the HTA design, it was also assumed that pilots have already donned the oxygen mask on. Reference Manikin for Posture and Position Calibration. A reference digital manikin was illustrated with a neutral posture, which represented a comfortable but attentive pilot posture—upright pose, hand on the joke, feet on rudder pedals (see Fig. 1). This posture described the pilot’s posture immediately after donning the masks and right before executing the HTA sequence. We used the Comfort Assessment toolbox in Siemens Jack’s Occupant Packaging toolkit for calibrating the posture by adjusting upper and lower body joint angles within acceptable comfort ranges based on Porter and Gyi [23]. As the automation framework goes through each manikin composed of different anthropometries, this reference posture is used for calibration to eliminate the positioning and posture setup related errors. Manikins and Anthropometry. Exhaustive use of anthropometry data to cover a wide range of user populations is vital in cockpit packaging assessments to build a work environment that serves pilots of different sizes. In this study, we used different anthropometry data for two ergonomics outcomes (See Table 1). In reach gap analysis, U.S. ANSUR II data based was used for five selected manikins (1st Female, 5th Female, 50th Female, 95th Male, and 99th Male) and three target points (Target 1, Target 2, and Target 3) in ideal conditions. Since this study focused on establishing the automation workflow and the physical reach inverse

Fig. 1 The reference posture setup used for calibration and the differences in seat adjustments due to change in anthropometry

352

M. S. Gawand and H. O. Demirel

Table 1 Task conditions and simulation output associated with the case study Task conditions Design study

Population

Reach gap

ANSUR II

Output Manikin

Environment

1% female

Ideal

Target Target 1

5% female

Target 2

50% female

Target 3

Distance (cm)

95% male 99% male Visibility

ANSUR II

1% male, female

Ideal

Target 1

Japanese

5% male, female

Light smoke

Target 2

Korean

50% male, female

Medium smoke

Target 3

Canadian

95% male, female

Heavy smoke

Chinese

99% male, female

Luminance (%)

kinematics (IK) scheme does not consider smoke accumulation, we only used ideal cockpit conditions representing normal work settings (non-emergency, no smoke accumulation). In percent luminance loss study, five target population databases (U.S. ANSUR II, Japanese, Korean, Canadian, and Chinese), five standard population percentile cuts (1st, 5th, 50th, 95th, and 99th) both for male and female genders, four different environmental conditions (ideal, light smoke, medium smoke, and heavy smoke) and three targets (Target 1, Target 2, and Target 3) were considered. Work Environment—Cockpit and Smoke Model. In this study, the work environment was composed of the computational cockpit and the smoke model. The cockpit model was based on a generic three-dimensional (3D) Boeing 767 CAD cockpit, including rudder pedals, yokes, front-facing instruments panel, central control console, oxygen masks. In addition to the CAD cockpit model, a smoke model was created in Blender 3D modeling software. The smoke source was generated where the extreme-right hand air vents are located. The scenario was set to simulate smoke build-up as the fan’s direction blows the smoke towards the pilot in command in increasing time steps. Three smoke meshes representing the gradual smoke accumulation (from light to heavy smoke) in the cockpit were modeled via Computational Fluid Dynamics (CFD) solvers in Blender (See Fig. 2). Also, a no smoke condition was set to represent the ideal work conditions where no fire and smoke emergency exists.

Task Simulation Automation via Digital Human …

353

Fig. 2 Smoke mesh models (no smoke, light smoke, medium smoke, and heavy smoke) generated in Blender and imported into Jack represent time-based smoke accumulation in cockpit

3.3 Automation Framework The automation framework used in this study was developed in Siemens Jack software using Jackscript module, a Phyton-based integrated development environment (e.g., Spyder, PyCharm) composed of simulation and ergonomics analysis scripts that enable designers to automate DHM operations. We applied the HTA depicted earlier to generate Jackscript functions. Tool Commending Language (TcL) module was utilized to call the Jackscript codes and automatically execute task simulations, calculate ergonomics assessments, and save analysis on a CVS file (See Fig. 5). Methodologies and assumptions to automate ergonomics outcomes are summarized as follows: Reach Gap. We used the ReachHold IK scheme when measuring the gap between the end-effector and target areas. RachHold function is part of the Siemens Jack application programming interface (API) and calculates arm position and takes the palm-center as the default end-effector when used in reaching the specific point in space. We also used the forearm end-effector with the palm-center to replicate fully extended arm reach to generate typical reach postures executed by pilots. Initially, using the thumb-tip with the palm-center was considered an ideal end-effector combo; however, the ReachHold IK scheme with the thumb-tip end-effector generated less realistic (awkward) postures. The end-effector (palm-center) and the pre-defined control targets (Target 1, 2, and 3) located on the front flight instrument panel represent a 3D vector in space. Therefore, the Euclidian distance method (Eq. 1) was used to measure the distance (e.g., the gap between palm-center and vertical speed knob) between two vectors (See Fig. 3).

354

M. S. Gawand and H. O. Demirel

Fig. 3 The image illustrates the location of the targets on the instrument panel. The image in the red box shows manikin reaching Target #2 with “locked torso” kinematic constraint selected

d( A, B) =

/

(xa − xb )2 + (ya − yb )2 + (z a − z b )2

(1)

where Palm-center: A = (xa , ya , z a ). Target: B = (xb , yb , z b ). In addition to IK scheme above, within the DHM simulation model, we used the “locked torso” kinematic constraint to represent usual static postures during gazing and reaching activities associated with the tasks explained in HTA. The “lock torso” kinematics option enables the manikin to gaze and reach only from its shoulder and torso with the waist constrained. Therefore, body postures that do not comply with typical gaze and reach postures while operating an aircraft were eliminated.

Task Simulation Automation via Digital Human …

355

Percent Loss in Luminance. The automation code also collects Eye Window screen captures from manikins’ binocular field-of-view within the DHM interface (Fig. 4). These screenshots represent what the manikin is looking at within the DHM environment. Each screenshot collected in Siemens Jack was stored in a folder, and MATLAB’s image processing module was used to calculate the luminance values (Eq. 2). The batch functions converted true-color RGB screenshots into grayscale images provided below. This approach eliminated hue and saturation data while retaining the luminance. L = mean2(rgb2gray(im))

(2)

r esults.L = L

(3)

The loss in visibility in this study was described in terms of loss in luminance. The luminance rate reduction method from Yuki et al. [24] was adopted to calculate percent loss in luminance as shown below: Percent Loss in Luminance =

|L 1 − L 2 | × 100 L1

(4)

where L1 = Luminance of the object in ideal condition L2 = Luminance of the object after smoke adhesion.

Fig. 4 Binocular field-of-view screenshots captured from DHM interface and sent to MATLAB to measure percent loss in luminance

356

M. S. Gawand and H. O. Demirel

Fig. 5 Flowchart illustrates a snippet of the manual DHM tasks automated by Jackscript

The mean luminance values of each scenario in ideal conditions (no smoke buildup), based on the manikin’s binocular field-of-view screenshots, were compared with the luminance values associated with varying smoke conditions (light, medium, heavy) (Fig. 5).

4 Results In this section, the DHM-based design automation framework was applied to the cockpit packaging case study introduced in Sect. 3.1. Two ergonomics outcomes, reach gap and percent loss in luminance, were summarized in their contribution to the pilot performance. Although many DHM software already includes several toolkits for measuring the reach gap, there is a lack of automation since

Task Simulation Automation via Digital Human …

357

existing toolkits require designers to build simulations and collect data manually. Therefore, in the reach gap measures part of this study, we focused on presenting our automation framework’s accuracy by comparing results between manual and automated simulations. In contrast, none of the existing vision analysis toolkits offer visibility (e.g., luminance) related ergonomics measures. Visibility toolkits often focus on obscuration zones due to solid or opaque CAD models and ignore how visibility diminishes due to translucent medium (e.g., smoke build-up). Thus, in Sect. 4.2, we summarized how the automation framework brought new assessment capabilities to DHM-based design.

4.1 Reach Gap Measures One of the goals of the automation framework was to provide an accurate quickand-dirty ergonomics assessment based on computational tools and data available in the concept development stage. In this section, the framework’s performance was analyzed in terms of the ergonomics output accuracy—comparing the results collected via automation and manual simulation setup. We included identical task conditions in the manual simulation setup and replicated the ergonomics analysis (summarized Table 1—Reach Gap). Therefore, the manual simulation was a duplicate of the automated simulation, only done manually. The manual simulation setup included the designer applying numerous manikin and scene manipulations and navigating within Siemens Jack’s graphical user interface (GUI) via series of point-and-click keyboard and mouse inputs. These activities involved fine-tuning the manikin and environment models, including neck, head, joint angles, and whole-body posture adjustments, and arranging and positioning the CAD cockpit model. At the start of each simulation, the reference manikin posture (described in Sect. 3.2) was assigned via Jack’s Occupant Packaging toolkit. Therefore, when assigning position and posture to manikins manually, the potential error was eliminated by assuring each manikin was positioned precisely at the corresponding seat location by keeping the neutral reference posture standard throughout the data collection. One can see from Table 2 that reach gap values calculated by the automation framework are comparable to that of manual DHM simulations. The differences between each simulation method for Target 1, Target 2, and Target 3 were 2.45 cm, 0.66 cm, and 0.51 cm. While reach gap measures for Target 2 and Target 3 were around 0.5 cm ballpark point, Target 1 resulted in a higher difference. This difference was not because of an error in distance calculations but because movements performed for Target 1 were slightly different due to IK schemes. During manual simulation setup, the Reach IK scheme with the Manipulate option was used instead of the ReachHold function coded within the automation framework via Jackscript.

358

M. S. Gawand and H. O. Demirel

Table 2 Comparison of reach gap distances between manual and automation method (cm) Population

Target 1 (T1)

Target 2 (T2)

Target 3 (T3)

Manual

Manual

Manual

Automation

Automation

Automation

Female (1%)

1.51

4.97

19.43

18.08

24.27

24.57

Female (5%)

1.84

5.08

19.39

18.05

26.35

24.97

Female (50%)

2.77

5.89

18.64

18.41

27.29

26.90

Male (95%)

11.49

12.79

22.75

22.98

34.98

34.77

Male (99%)

13.50

14.65

23.98

24.11

36.73

36.44

Average diff. =

2.45

0.66

0.51

4.2 Percent Loss in Luminance Measures This section provides an additional ergonomics assessment competence to computational ergonomics analysis by extending DHM vision analysis toolkits’ capabilities. The ergonomics evaluations in this section utilize the luminance and percent luminance loss equations discussed in Sect. 3.3. The ergonomics assessment focused on understanding how different smoke accumulation conditions coupled with anthropometry affect visibility. In addition to U.S. (ANSUR II) database used in the Reach Gap study, we also injected regional anthropometric data—Chinese, Japanese, Korean, and Canadian populations with five population percentile cuts (1st, 5th, 50th, 95th, and 99th). The results indicated a substantial loss in luminance when reaching all targets (T1, T2, and T3) in medium and heavy smoke scenarios. In contrast, for the light smoke scenario, the readings are significantly low. This finding can be attributed that the smoke accumulation has not yet covered the entire instrumental panel. One can also see from the mean percent loss in luminance values in Table 3 that the target locations do not significantly affect the luminance readings, but smoke conditions do. Although visibility-related claims require a more comprehensive investigation and should not be attributed as the sole indicator of visibility loss, the results show that the methodology could indicate conditions that may create difficulties for pilots to see the controls. Table 3 Percent loss in luminance within different smoke conditions Population

Light smoke T3

T1

T2

T3

T1

T2

T3

Mean

4.88

7.62

4.29

14.36

15.64

16.89

23.61

20.65

23.00

Std. dev.

1.17

1.29

1.16

1.40

1.12

1.34

2.59

2.69

2.32

Min

2.88

3.49

2.36

12.00

12.96

14.02

17.69

15.14

18.65

Median

5.08

7.70

4.30

14.35

15.87

16.10

23.75

21.22

23.02

Max

6.73

10.04

6.02

16.63

17.90

19.42

27.40

25.12

26.42

T1

T2

Medium smoke

Heavy smoke

Task Simulation Automation via Digital Human …

359

5 Discussion This study addressed a pressing need in computational ergonomics, especially when designing with DHM during early concept development of products and processes that are complex and costly to prototype. The lack of automation tools in current DHM platforms has been an ongoing issue and regarded as one of the bottlenecks that slow the adoption of DHM-based design methodologies. Generating highvolumes of what-if scenarios quickly and accurately via point-and-click controls or using GUI options is time-consuming, erroneous, and often infeasible for studies that require screening of a large set of design parameters. Thus, injecting automation to DHM-based simulation and HFE assessment studies saves time. Likewise, automating posture, anthropometry, task, and environment setup provides standardization when running HFE analysis. Overall, the DHM-based design approach presented in this paper facilitates the embodiment of fire and smoke emergencies within a computational framework, which brings in the advantage of making the emergency assessments part of the early design ergonomics decision-making. The automation framework also minimizes the designer bias by providing a more systematic approach by automating posture and joint adjustments. For instance, preserving a neutral posture and positioning a large set of manikins (various geographical populations and anthropometric percentiles) via automation eliminated the need to fine-tune the manikin position manually at the start of each simulation. Moreover, the automation framework used during calculating reach gap measures substituted the manual joint angle adjustments (slider controls via keyboard and mouse-based on software GUI) with the IK scheme. The reach gap measures indicate that average reach distance differences when compared manual with automation simulation setup for reach targets across manikins are between 0.5 and 2.5 cm. Since the proposed automation framework’s primary focus is early design activities, one can conclude that the automation approach provided reach distance measures without sacrificing accuracy. The automation framework also brought novel simulation (e.g., CFD smoke model) and ergonomics assessment (e.g., smoke accumulation visibility analysis) capabilities through technology integration (e.g., Blender and MATLAB). Such physics-based capabilities currently do not exist in standalone DHM platforms. Even integrated DHM software (e.g., Siemens NX and Siemens Jack) still require advanced physics-based ergonomics toolkits to explore a broader range of human-machine or human-environment issues. For example, percent loss in luminance assessments presented in this cockpit packaging study brought a unique range of decision-making abilities in the context of SFF emergencies. Data collected via a large set of binocular field-of-view simulations, with and without smoke accumulation scenarios across different manikins and reach scenarios, provided novel feedback. In the context of early design, this information can facilitate more systematic decision making. For example, answering questions regarding SFF events early in the conceptualization phase of cockpit packaging brings the advantages of smoothing out many design problems before committing to final designs. As such, this study demonstrates how

360

M. S. Gawand and H. O. Demirel

designers can utilize the proposed automation framework to extend the capabilities of DHM toolkits to answer questions otherwise not possible in early design without physical prototyping.

6 Future Work It is important to note that the framework introduced in this research can be applied to assess a broader range of normal and emergency work conditions. However, a broader coverage requires the development of high-fidelity DHM toolkits. Although this paper provided an alternative approach to extend the current capabilities of DHM, there are limitations to our study. Future work will look into how the proof-ofconcept framework depicted in this paper can be enriched to answer human-centered design questions that require a more holistic scope. Some of the other future work will include developing high-fidelity assessment tools, conducting validation studies, building models to consider cognitive aspects of the work, and measuring time and cost savings. For example, in the context of fire and smoke emergencies, questions such as how the smoke accumulation inside the cockpit affects pilots’ visibility of the instrument panel go beyond the traditional content of DHM-based ergonomics analysis due to the lack of physics-based computational packages in current platforms. However, these questions, and perhaps even more complex inquiries, about human performance becomes more prominent in modern product development. This study only considered reaching gap and percent loss in luminance. These considerations alone are not sufficient to fully assess the cockpit packaging questions. Other factors such as eye view distance from the target, maximum clear visibility, visual acuity, and chemical composition of the smoke should be incorporated to increase the fidelity of DHM. One of the other limitations in this paper is the lack of cognitive considerations in DHM analysis. The framework focuses on automating manikin tasks and postures based only on physiological parameters (e.g., joint angles). Ideally, a holistic design framework in the context of human-centered design should include the cognitive aspects of work (e.g., mental workload). The lack of cognitive toolkit development in the DHM domain diminishes the DHM-based design approach’s potential to answer broader design questions with higher-fidelity. The cognitive aspects of task execution in emergencies, such as the SFF scenario depicted in this paper, are more than just a problem that pushes pilots’ physical limits. Next-generation DHM design should consider mental workload, situational awareness, and sustained attention. An integrated (cognition and physiology) framework is needed to increase the fidelity of the early design HFE assessments. Future work should also include validation studies with human subject data collection. Since the DHM-based design does not include actual users in the loop, incorporating user input will improve our understanding of human-machine interaction in emergencies. Moreover, dynamic aspects of work should also be considered when generating ergonomics assessments. The automation

Task Simulation Automation via Digital Human …

361

framework proposed in this study uses static posture assumptions and single static images (binocular field-of-view) when providing ergonomics assessments. Overall, the development of high-fidelity DHM simulations and ergonomics assessment tools will change the way designers conceptualize tasks and environments in the design of complex systems. The automation framework proposed in this study is just a step forward in the promise of a holistic DHM-based design platform. Like the fire and smoke case study worked out in this research, other scenarios that demand costly prototypes and dangerous experiments will likely take advantage of the DHM approach.

References 1. Federal Aviation Administration (FAA) Fire safety background page—problem statement. https://www.fire.tc.faa.gov/Research/Background. Last accessed 25 Jan 2021 2. Shaw J (1999) A review of smoke and potential in-flight fire events in 1999. In: Advances in aviation safety conference & exposition. SAE, Warrendale, pp 1–12 3. Cherry RGW, Hill R (2017) Research into fire, smoke or fumes occurrences on transport airplanes. Federal Aviation Administration Transport Airplane Directorate 4. VisionSafe emergency vision assurance system—recent smoke in the cockpit events. https:// www.visionsafe.com/recent-smoke-in-the-cockpit-events/. Last accessed 25 Jan 2021 5. Gawand MS (2019) Automating digital human modeling for task simulation and ergonomic evaluation to consider emergency ergonomics early in design. M.S. Thesis. Oregon State University, Corvallis 6. Ahmed S, Demirel HO (2020) A framework to assess human performance in normal and emergency situations. ASCE-ASME J Risk and Uncert in Engrg Sys Part B Mech Engrg 6(1) 7. Demirel HO, Duffy VG (2016) Building quality into design process through digital human modelling. Int J Digit Human 1(2):153–168 8. Gawand MS, Demirel HO (2020) A design framework to automate task simulation and ergonomic analysis in digital human modeling. In: Duffy V (eds) Digital human modeling and applications in health, safety, ergonomics and risk management. Posture, motion and health. International conference on human-computer interaction (HCII). Lecture notes in computer science, vol 12198. Springer, Cham, pp 50–66 9. Royal Aeronautical Society (2018) Smoke, fire and fumes in transport aircraft. past history, current risks and recommended mitigations. Royal Aeronautical 10. Gawand MS, Demirel HO (2020) Extending the capabilities of digital human modeling: a design framework to assess emergencies early in design. In: International mechanical engineering congress & exposition (IMECE). ASME, New York (in-print) 11. Demirel HO (2015) Modular human-in-the-loop framework based on human factors. Ph.D. dissertation. Purdue University, West Lafayette 12. Ahmed S, Demirel HO (2019) A comparison between virtual reality and digital human modeling for proactive ergonomic design. In: Duffy V (eds) Digital human modeling and applications in health, safety, ergonomics and risk management. Human body and motion. International conference on human-computer interaction (HCII). Lecture notes in computer science, vol 11581. Springer, Cham 13. Demirel HO, Duffy VG (2007) Applications of digital human modeling in industry. In: Duffy VG (eds) International conference on digital human modeling. Lecture notes in computer science, vol 4561. Springer, Berlin 14. Zhang X, Kuo A, Chaffin D (2005) Digital human modeling for computer-aided ergonomics. In: Marras WS, Karwowski W (eds) The occupational ergonomics handbook. CRC Press, Boca Raton, FL, pp 1–20

362

M. S. Gawand and H. O. Demirel

15. Fleck JT, Butler FE, Vogel SL (1975) An improved three dimensional computer simulation of motor vehicle crash victims (technical report). Buffalo, NY 16. Feyen R, Liu Y, Chaffin D, Jimmerson G, Joseph B (2000) Computer-aided ergonomics: a case study of incorporating ergonomics analyses into workplace design. Appl Ergon 31(3):291–300 17. Blanchonette P (2010) Jack human modelling tool: a review (technical report). Air Operations Division Defense Science and Technology Organization (DSTO), Victoria, Australia 18. Abdel-Malek K, Arora J, Bhatt R, Farrell K, Murphy C, Kregel K (2019) Santos: an integrated human modeling and simulation platform. In: Scataglini S, Paul G (eds) DHM and Posturography. Elsevier, pp 63–77 19. Ahmed S, Gawand MS, Irshad L, Demirel HO (2018) Exploring the design space using a surrogate model approach with digital human modeling simulations. In: International design engineering technical conferences and computers and information in engineering conference. ASME, New York 20. Burian BK (2004) Emergency and abnormal checklist design factors influencing flight crew response: a case study. In: Wilson H (ed) Proceedings of the international conference on humancomputer interaction in aeronautics, vol 1. EURISCO International 21. Airbus worldwide instructor news—flight operations recommendation—adhering to the SMOKE philosophy. https://www.airbus-win.com/wp-content/uploads/2019/06/managingsmoke-and-fumes-in-flight.pdf. Last accessed 27 Jan 2020 22. Federal Aviation Administration (FAA) (1994) Smoke detection, penetration, and evacuation tests and related flight manual emergency procedures AC 25-9A. In: Advisory circular guideline. FAA, Washington 23. Porter JM, Gyi DE (1998) Exploring the optimum posture for driver comfort. Int J Veh Des 19(3):255–266 24. Yuki A, Takeyoshi T, Hidekazu S (2005) Calculation method for visibility of emergency sign in fire taking into account of smoke adherence. Fire Saf Sci 8:1093–1105

Facility Layout Design Optimization of Wing Assembly of Unmanned Aerial Vehicle Based on Particle Swarm Optimization Hai-Zhe Jin, Zi-Jian Cao, Xin-Yi Chi, and Xue-Xin Fan

Abstract The complex structure, a large number of parts and diverse assembly relations of UAV (Unmanned Aerial Vehicle) wing affect its assembly efficiency, which is an urgent scientific problem to be solved. In this paper, based on the original data of facility layout in wing assembly workshop, Systematic Layout Planning method was used to determine the comprehensive relationship between the work units. Then, a multi-objective optimization mathematical model was established from the perspective of minimizing the total cost of logistics operations and maximizing the degree of close relationship with non-logistics between working units. Finally, the particle swarm optimization algorithm is used to solve the optimization model of facility layout, and the layout scheme that can improve the efficiency of unmanned aerial vehicle wing assembly workshop is found. The research results not only provide technical support for the layout characteristics of UAV wing assembly workshop, but also provide specific ideas and methods for other similar production enterprises. Keywords Facilities layout and design · Systematic layout planning · Muti-objective optimization · Particle swarm optimization · Unmanned aerial vehicle (UAV) wing assembly

1 Introduction and Background 1.1 A Subsection Sample In the twenty-first century, China’s logistics and transportation industry is developing rapidly, more and more transportation enterprises choose unmanned aerial vehicles (UAVs) as a form of inter-city freight transportation. However, due to technical limitations, the number of Chinese enterprises is insufficient. There is still a gap between H.-Z. Jin (B) · X.-Y. Chi · X.-X. Fan Northeastern University, Shenyang 110169, China e-mail: [email protected] Z.-J. Cao Tsinghua University, Beijing 100085, China © The Author(s), under exclusive license to Springer Nature Switzerland AG 2023 V. G. Duffy et al. (eds.), Human-Automation Interaction, Automation, Collaboration, & E-Services 11, https://doi.org/10.1007/978-3-031-10784-9_22

363

364

H.-Z. Jin et al.

demand and supply for dry-branch UAVs with loads of more than a few hundred kilograms [1]. Compared with other means of transport, UAV has complex structure and a large number of components, and diverse assembly relationships in particular [2]. In the whole process of UAV assembly operations, the backwardness of production logistics has become the bottleneck for enterprises to further improve assembly technology and shorten assembly cycle. The rationality of production logistics has an important relationship with the layout design of production system. A good system layout design can shorten the assembly cycle of UAV, reduce the logistics cost in the assembly process, and make more effective use of factory space. The objective of workshop layout design is to make the most effective configuration and arrangement of machines, equipment, transportation channels and sites, so as to achieve the optimization of system layout design [3]. In the 1930s, Taylor, the father of management science, first formally proposed the layout and design of workshop facilities. The research in this period adopted qualitative methods, and the results were greatly affected by the subjectivity of researchers [4]. In 1973, American scientist Richard Muther put forward a system layout design method after summing up a lot of factory layout design experience [5]. This method overcomes the shortcomings of many previous research methods which only rely on subjective qualitative analysis and lack of objective quantitative analysis. When the number of objects involved in the layout is large, both discrete and continuous workshop layout problems are proved to be NP-hard problems. It is difficult to obtain the optimal solution above the medium scale in a limited time using traditional system layout design methods [6]. With the rapid development of computer technology, intelligent optimization algorithms bring new ideas to solve the problems of workshop layout design. It has the following advantages: intelligent algorithm adopts fast parallel processing, and can get multiple alternative solutions at the same time, designers can pick out the appropriate results according to their own needs. The traditional heuristic algorithm is sensitive to the initial arrangement and usually only obtains the local optimal solution, but the intelligent algorithm can seek the global optimal solution [7]. Balakrishnan used a hybrid genetic algorithm to solve the problem of dynamic factory layout design [7]. Ming and Kothari et al. used this algorithm to solve the problem of unequal area planning and single-row facility layout [8, 9]. Hasan used particle swarm optimization algorithm to solve single-row facility layout problem and dynamic facility layout problem [10]. Baykasoglu et al. used simulated annealing algorithm to solve the dynamic layout planning problem [11]. The above research is mainly aimed at the facility layout in traditional manufacturing industry, which takes the minimization of logistics cost as the single optimization objective. It is less targeted at the layout relationship of UAV production with complex body structure, a large number of components and diverse assembly relations. Based on the above research problems, this study takes UAV wing assembly workshop as an example. On the basis of fully considering the complex assembly relationship, the optimization objective is to minimize the total logistics cost and the highest degree of non-logistics relationship. At the same time, the facility layout design optimization based on particle swarm optimiza is presented in this work.

Facility Layout Design Optimization of Wing Assembly …

365

Fig. 1 Wing assembly process

2 Layout Analysis of UAV Wing Assembly Facility 2.1 The Assembly Process of Wing Assembly Facility The final product of the wing assembly workshop is the wing. According to the functional structure, it can be divided into four components including wing box, flap, aileron and wing leading edge. The assembly process of each component is different and it can be further decomposed into several parts, which has the characteristics of a large number of parts and complex structure. The wing assembly process is presented in Fig. 1.

2.2 Division of Wing Assembly Operation Units Whether the division of operation units is reasonable or not will directly affect the logistics cost, assembly efficiency and assembly operation quality of the assembly facility after layout. The wing assembly facility is divided into 5 stations, which are wing box assembly station, flap assembly station, aileron assembly station, wing leading edge assembly station and wing assembly station. Each station has a corresponding assembly frame, equipped with skin processing equipment and rib processing equipment. Under the existing production scale, there is not much demand for skins and ribs, some station processing equipment is often in the idle state. In order to reduce the waste of personnel and equipment, the new facility layout plan eliminates the rib processing equipment and skin processing equipment equipped with each station. Instead, special rib and skin processing stations are set up in the

366

H.-Z. Jin et al.

Fig. 2 Layout of wing assembly

workshop to provide the rib and skin required in the assembly process for other assembly stations. Therefore, the wing assembly operation can be divided into seven operation units. The layout of the existing facilities in the wing assembly facility is presented in Fig. 2.

2.3 Relationship Analysis Among Operation Units Material flow analysis among each operation unit The material flow between operation units was counted in the wing assembly facility, and the logistics intensity among operation units was divided according to the classification standard of logistics intensity put forward by Muther. The results show that wing box assembly unit and aileron assembly unit, wing box assembly unit and flap assembly unit, skin processing unit and wing assembly unit are all of ultrahigh logistics intensity. Therefore, on the premise of ensuring the safe distance, the distance of these operation units should be as close as possible. At the same time, the logistics intensity between the skin processing unit and the flap assembly unit, the rib processing unit and the aileron assembly unit, the skin processing unit and the leading edge assembly unit, the aileron assembly unit and the wing assembly unit is also too large, and the distance between them should also be close. Analysis of non-logistics relationship between operation units In addition to the operation unit, non-logistics factors also affect the optimization of workshop layout design. After consulting the relevant literature and communicating with the person in charge of the workshop, the following six typical influencing

Facility Layout Design Optimization of Wing Assembly …

367

factors of non-logistics relationship between operation units were determined: continuity of assembly process, the degree of close contact between personnel, convenience of management and supervision, sharing the same frame or fixture, frequency of service, and safety, pollution and vibration problems. The non-logistic relationships between units of work are also classified as A, E, I, O, U, X, which mean “absolutely important”, “extremely important”, “important”, “normal”, "unimportant", and “do not approach”. Table 1 is the final determined non-logistics relationship table among the operation units in the assembly shop. In Table 1, the non-logistic relationships between the wing box assembly unit and the aileron assembly unit, the wing box assembly unit and the leading edge assembly unit are “absolutely important”, so the pairs of units should be as close together as possible. In addition, the relationship between the leading edge assembly unit and the wing assembly unit is “do not approach” and the two units are designed to be located as far apart as possible. Comprehensive correlation analysis of operation units The concept of “comprehensive interrelation” is put forward in the method of systematic layout design to comprehensively evaluate the logistic and non-logistic relations between operating units. The final optimization result is more instructive when the weight of logistics interrelationship and non-logistics interrelationship between operating units in the wing assembly shop is set at 2:1. Table 2 shows the integrated interrelationship of the operation units in the wing assembly workshop based on the integrated interrelationship of the original operation units. Table 1 Non-logistic relations between operation units in assembly workshop

368

H.-Z. Jin et al.

Table 2 Comprehensive relationship between operation units in assembly workshop

In Table 2, there is a close comprehensive interrelation between the flap assembly unit and the rib processing unit, the aileron assembly unit and the rib processing unit, the rib processing unit and the wing box assembly unit, and the skin processing unit and the wing box assembly unit. The distance between these operating units should be as close as possible. The skinning unit and wing assembly unit are too close to each other, which may affect the normal operation of the two units, so they need to be far away from each other. Problems in facilities layout At the beginning of the facilities layout, the logistics relationship between operations was not fully considered, which resulted in the high logistics cost within the workshop. Non-logistic relationships between operation units were not considered, resulting in the possibility of mutual influence between operation units. Comparing the current situation of the workshop layout shown in Table 2 with that shown in Fig. 1, it can be found that there are two prominent problems in the workshop layout. First, there is a close comprehensive correlation between the rib processing unit and the flap assembly unit, but the distance between the two operating units in the existing layout is relatively far. Second, under the existing layout, the skinning processing unit and the wing assembly unit are relatively close to each other, which may interfere with each other. Through the analysis of the above problems, two major objectives of facility layout improvement are determined: the minimum total logistics cost and the highest degree of non-logistics relationship, and a mathematical model is built to improve the facility layout based on this main optimization objective.

Facility Layout Design Optimization of Wing Assembly …

369

3 Facility Layout Optimization of Wing Assembly Workshop 3.1 Facility Layout Problems The layout problem of wing assembly workshop refers to the proper arrangement of various production units whose areas have been determined in the workshop with a certain area and shape, so that the system operates with the highest efficiency. The layout design of the workshop belongs to the combinatorial optimization problem, that is, under certain constraint conditions, the “best” arrangement or grouping of discrete events is determined by mathematical methods. The “best” means that the combination of discrete variables obtained can achieve the predetermined planning objectives. Aiming at the layout design problem of the wing assembly workshop studied in this paper, combined with the actual operation situation of the workshop, it is finally determined that the optimization objective is to reduce the material handling cost and make the highest degree of close relationship between non-logistics. The constraint conditions need to be satisfied are: the layout design scheme cannot exceed the existing area of the workshop; a certain safe distance should be ensured between operating units; units of work must not overlap in space.

3.2 Model Assumptions The purpose of improving facility layout in the workshop is to improve assembly efficiency and shorten assembly cycle. Therefore, when modeling layout problems, the objective negative correlation between the practicability and operation of the model should be fully taken into account. In order to take into account the practicality and operability of the model, the following assumptions for model establishment are finally determined: The shape of the workshop and the seven operation units are rectangles, and their lengths and widths are shown as Table 3; The area of the workshop is large enough to house seven units; Logistic and non-logistic relationships have been established between the seven operations units; The operation unit is parallel to the length and width of the factory side; The logistics path between operation units is parallel to the factory side. Establish the mathematical model The purpose of rearranging the assembly workshop is to reduce the material handling cost between operating units and to make the distance between closely related production units as close as possible. In order to achieve the above two objectives, a twoobjective optimization model of minimum cumulative material handling weight and

370

H.-Z. Jin et al.

Table 3 Length and width of workshop No.

Operating unit name

Length (m)

Width (m)

0

Assembly workshop

30.0

25.0

1

Skin processing unit

4.1

3.8

2

Rib processing unit

4.0

3.6

3

Flap assembly unit

5.2

4.8

4

Aileron assembly unit

5.0

4.8

5

Wing box assembly unit

6.1

5.8

6

Front edge assembly unit

5.1

4.9

7

Wing assembly unit

8.0

7.6

maximum non-logistics relationship was established. The objective function can be expressed as: min Z =

n n  

f i j di j , i = 1, 2, . . . , 7, j = 1, 2, . . . , 7

(1)

i=1 j=1

max N =

n  n 

ri j bi j , i = 1, 2, . . . , 7, j = 1, 2, . . . , 7

i=1 j=1

(2)

In order to make the calculation more concise, the normalization factor is often used to convert the multi-objective model into a single-objective model to be solved [12]. The final objective function established is: min = μ1 ω1

n  n 

f i j di j − μ2 ω2

i=1 j=1

n  n 

ri j bi j

(3)

i=1 j=1

  dxi j = xi − x j 

(4)

  d yi j =  yi − y j 

(5)

    di j = xi − x j  +  yi − y j 

(6)

1 n

μ1 = n i=1

j=1

μ2 = n

1 n

i=1

f i j dmax

j=1 ri j

(7) (8)

Facility Layout Design Optimization of Wing Assembly …

371

where i and j are the numbers of the two operation units, (xi , yi ) is the center horizontal and vertical coordinates of operation unit i; dxi j is the distance between the center coordinates of operation unit i and j along the x-axis direction; d yi j is the distance between the center coordinates of the operation unit i and j in the y-axis direction; di j is the Manhattan distance between operation units i and j; Z is the cost of material handling between operating units; N is the sum of the quantized values of non-logistic correlation between operation units; f i j is the material flow between operation units i and j within an assembly cycle (the specific value is shown in Table 6); ri j is the quantified value of the non-logistics relationship between the two operation units i and j (the specific value is shown in Table 7); bi j is correlation factor, represents the work units i and j proximity, bi j values by the sum of Manhattan distance between operation units decision(the specific value is shown in Table 8); d max is the sum of the length and width of the region; μ1 and μ2 are normalization factors; ω1 and ω2 are weight factors to measure the relative importance of logistics relationship and non-logistics relationship, which satisfy ω1 + ω2 = 1. The constraint condition is as follows: Ai j × Bi j = 0

   1 li + l j + dxi j − xi − x j , 0 2

    1   wi + w j + d yi j − yi − y j , 0 Bi j = max 2

(9)



Ai j = max

(10) (11)

li + e − xi ≤ 0 2

(12)

wi + e − yi ≤ 0 2

(13)

where li is the length of operation unit i, wi is the wide of operation unit i; e is the safety distance, according to the actual production situation of the workshop, e = 6. Since the logistics path between operation units is parallel to the factory, Manhattan distance is adopted instead of Euclidian distance. Equation (6) represents the Manhattan distance between two operation units, i and j. Equations (9–11) guarantee no overlap of operation units in space. Equations (12) and (13) are to ensure a prescribed safety distance between operating units, which is 6 m.

372

H.-Z. Jin et al.

3.3 Parameter Design of Particle Swarm Optimization Algorithm Particle dimension and population size There are seven operation units in the wing assembly workshop. Each operation unit has a central abscissa and ordinate. Therefore, the particle representing the spatial solution vector in this application is a 14-dimensional vector. The first 7 dimensions represent the central abscissa of the 7 operation units, and the second 7 dimensions represent the central ordinate of the 7 operation units. The position Pk and velocity vk of the k particle are also 14-dimensional vectors, which can be expressed as: Pk = (xk1 , xk2 , . . . , xk7 , yk1 , yk2 , . . . , yk6 , yk7 )

(14)

  vk = vkx1 , vkx2 , . . . , vkx7 , vky1 , vky2 , . . . , vky7

(15)

The population size refers to the number of particles in each generation population, and the size of population size has obvious influence on the running time of the algorithm. The larger the population size, the longer the running time of the algorithm, but the higher the accuracy and stability of the model. The group size of this application is 30. Fitness function In the process of evolutionary search, the particle swarm optimization algorithm only calculates the fitness of each particle through the fitness function, and compares it with the fitness of its own historical optimal position and the fitness of the global optimal position. Therefore, the complexity of the fitness function will directly affect the complexity of the particle swarm optimization algorithm. The fitness function of the particle swarm optimization algorithm is generally taken as the objective function. However, in order to ensure that no negative value of the fitness function occurs in the iteration process, the objective function is processed as follows: F=

μ1 ω1

n i=1

n j=1

1 n n f i j di j − μ2 ω2 i=1 j=1 ri j bi j + Pe(k)

(16)

The penalty function is as follows: Pe(k) =

1 [s1 (h k1 + h k2 ) + s2 (qk1 + qk2 )] 2

(17)

where h k1 and h k2 are the values on the left side of the equation in Eqs. (10) and (11) when the particle k is Pk = (xk1 , xk2 , . . . , xk7 , yk1 , yk2 , . . . , yk6 , yk7 ), and they are used to punish the case that the operating units overlap in space; qk1 and qk2 are the values of particle k on the left side of the equation in Eqs. (12) and (13), which

Facility Layout Design Optimization of Wing Assembly …

373

are used to punish the case that there is no safe distance between the operating units. If Pe(k) = 0, it means that particle k satisfies all constraints; s1 and s2 are penalty factors, s1 = s2 = 100. Learning factors The learning factors c1 (cognitive coefficient) and c2 (social coefficient) in the particle swarm optimization algorithm can adjust the proportion of individual cognitive part and social part in the change of particle’s flight speed, where c1 represents the proportion of particle learning from its own experience and c2 represents the proportion of particle learning from the global optimal particle. For the problem of facility planning, when the cognitive coefficient c1 and social coefficient c2 are set as 2, the cognitive part and social part of particle swarm can be coordinated [13]. Constraints The particle swarm optimization algorithm is not an optimization algorithm without constraints, but constraints need to be considered when it is applied to the modeling solution of practical problems. Therefore, the penalty function Pe(k) is introduced to punish the phenomenon that does not meet the boundary constraints and spacing constraints. If Pe(k) is 0, it means that particle k satisfies all constraints. Coefficient of inertia The study of Shi and Eberhart shows that it is reasonable to set the inertia coefficient ω as a function that decreases linearly with time rather than as a constant value [14]. The expression of inertia coefficient is as follows: ω = ωmax −

ωmax − ωmin ×t Tmax

(18)

where ωmax is the initial inertia coefficient, ωmax = 0.9; ωmin is the final coefficient of inertia, ωmin = 0.4; Tmax is maximum iterations, Tmax = 1500; t is the current iteration number. Other parameters In addition to the above parameters, the parameters of the particle swarm algorithm also include the number of iterations, error conditions and maximum particle velocity. Set the number of iterations Tmax = 1500 and error conditions miner = 0.0001. The maximum velocity of the particle (vmax ) determines the maximum distance that the particle can move in the solution space in a cycle. Ratnaweera’s research shows that vmax = 0.5 is more appropriate [15].

374

H.-Z. Jin et al.

Table 4 Center coordinates of operation unit No.

Operating unit name

Center abscissa

Center ordinate

1

Skin processing unit

4.1

3.8

2

Rib processing unit

4.0

3.6

3

Flap assembly unit

5.2

4.8

4

Aileron assembly unit

5.0

4.8

5

Wing box assembly unit

6.1

5.8

6

Front edge assembly unit

5.1

4.9

7

Wing assembly unit

8.0

7.6

3.4 Simulation Experiment According to the above designed parameters, the particle swarm algorithm is used to simulate the model. The center coordinates of each work unit are shown in Table 4. Substituting the center coordinates of each operation unit of the plan before and after the improvement into the mathematical model, it can be found that compared with the plan before the improvement, the cost of material handling in the improved plan is significantly reduced, and there is no logistics between the operating units. The degree of non-logistics relationship between operation units is also improved, as shown in Table 5. The advantage of using particle swarm algorithm to solve the facility layout problem lies in that the final layout result obtained by using the system layout design method lays more emphasis on the relative position between operating units; while using particle swarm optimization algorithm to solve such problems, the final result is the specific coordinate value of each operation unit, so it has a greater reference value to improve the layout of facilities in the facilities. At the same time, when there are many objects participating in the facility layout, the system layout design method is difficult to obtain the optimal solution above the medium scale in a limited time; while the particle swarm algorithm has a faster convergence speed, so it is more practical. Table 5 Before and after optimization of workshop layout Operating unit name

Before

After

Logistics intensity between operation units Z

7264.8808

6706.5550

The degree of non-logistics relationship between operation units N

14.2

16

The degree of comprehensive relationship between operation units G

0.0892

0.0754

Facility Layout Design Optimization of Wing Assembly …

375

4 Conclusion Based on the original data of the facility layout of the UAV wing assembly workshop, this paper applies the system layout design method and particle swarm algorithm to reduce the material handling cost between operating units and make the distance between closely related production units as close as possible. Through the quantitative analysis of the comprehensive relationship between operating units, the particle swarm optimization algorithm was used to solve the optimization model of facility layout to find the layout scheme that could improve the operating efficiency of the wing assembly workshop. On the basis of the analysis combined with the actual situation, the corresponding improvement scheme was formulated. The research results not only provide technical support for the layout of the wing assembly workshop, but also provide ideas and methods for the layout of other manufacturers. In this research model, the model is established and solved for the optimization goal of the minimum total logistics cost and the highest non-logistics relationship in the UAV wing assembly workshop. In the future research, the number of optimization targets can be further increased, thereby further increasing the optimization effect of workshop layout design.

Appendix See Tables 6, 7 and 8. Table 6 Flow of material between operating units f ij i

j 1

1 2 3 4 5 6 7

2

3

0

10

4 8

5 80

6 19

30

36

120

0

0

0

0

0

43

0

0

21

4

7 0

0 92

376

H.-Z. Jin et al.

Table 7 The quantified value of the non-logistics relationship between operating units r ij i

j 1

1 2 3

2

3

4

5

6

7

0

3

0

0

2

3

0

3

0

0

0

0

3

0

1

4

0

1

4

0

4 5

−1

6 7

Table 8 Corresponding relations between di j and bi j d ij

bij

0 < d ij ≤ d max /6

1.0

d max /6 < d ij ≤ d max /3

0.8

d max /3 < d ij ≤ d max /2

0.6

d max /2 < d ij ≤ d max /2

0.4

2d max /3 < d ij ≤ d max /2

0.2

d max /6 < d ij ≤ d max

0.0

References 1. Pei S, Shen T, Ning Z et al (2019) KMG: Study on UAV path planning strategy by considering reverse logistics. Syst Eng Theory Pract 39(12):3111–3119 2. Tao J, Tian X, Liu X et al (2020) A multiple alternative processes-based cost-tolerance optimal model for aircraft assembly. Int J Adv Manuf Technol 107(2):667–677 3. Lin Q, Wang D, Iadanza E (2019) Facility layout planning with SHELL and fuzzy AHP method based on human reliability for operating theatre. J Healthc Eng 2019:1–12 4. Tian Y (2017) Study on workshop layout optimization of a company. Hebei University of Science and Technology, Shijiazhuang, pp 35–39 5. Guiherme B, Giovani S, Flavio F (2019) Layout planning in healthcare facilities: a systematic review. Health Environ Res Des J 12(3):31–44 (2019) 6. Guan C, Zhang Z, Liu S et al (2019) Multi-objective particle swarm optimization for multiworkshop facility layout problem. J Manuf Syst 53:32–48 7. Zhu T, Balakrishnan J, Cheng C (2018) Recent advances in dynamic facility layout research. INFOR: Inf Syst Oper Res 56(4):428–456 8. Wang MJ, Hu MY (2005) A solution to the unequal area facilities layout problem by genetic algorithm. Comput Ind 56(2):207–220 9. Kothari R, Ghosh D (2014) An efficient genetic algorithm for single row facility layout. Optimization Letters 8(2):679–690 10. Hasan H, Leila E (2013) A hybrid particle swarm optimization for dynamic facility layout problem. Int J Prod Res 51(14):4325–4335 11. Adil B, Nabil N, Gindy Z (2001) A simulated annealing algorithm for the dynamic layout problem. Comput Oper Res 28:1403–1426

Facility Layout Design Optimization of Wing Assembly …

377

12. Zarei J, Mohammad A, Khorasani A et al (2020) A sustainable multi-objective framework for designing and planning the supply chain of natural gas components. J Clean Prod 259:120649 13. Bangyal W, Ahmad J, Rauf H (2020) An overview of mutation strategies in particle swarm optimization. Int J Appl Metaheuristic Comput 11(4):16–37 14. Shi Y, Eberhart RC (1998) A modified particle swarm optimizer. In: Proceedings of IEEE international conference on evolutionary computation. IEEE, Anchorage, pp 69–73 15. Ratnaweera A, Halgamuge SK, Watson HC (2004) Self-organizing hierarchical particle swarm optimizer with time-varing acceleration coefficients. IEEE Trans Evol Comput 8(3):240–255

Forklift Operator Discomfort and Vision Assessment Through Computer-Aided Ergonomics Analysis Suhas G. Aekanth, Thorsten Kuebler, and Vincent G. Duffy

Abstract A Forklift is an essential and convenient tool for transporting goods and materials within an industry. However, forklift operators are easily prone to musculoskeletal injuries due to their awkward postures and visibility obstructions at work. This paper simulates an industrial set up using RAMSIS to assess the discomfort levels and vision range of a forklift operator driving backward. The analysis is first performed on a driver looking at the scene behind the forklift. The subsequent study is conducted after installing a side-view mirror and later a rear-view camera on the forklift. For each design change, the comfort level and visibility of the operator are determined. Statistical analysis in the form of a Student’s t-Test is also performed on the forklift operator’s discomfort assessment results to determine the best alternative. Keywords RAMSIS · Forklift · Discomfort · Vision · Mirror · Camera

1 Introduction Good health is one of the significant factors that are central to human well-being. It also contributes to economic progress, as healthy populations live longer and are more productive [1]. In a workplace environment, employees are often engaged in some form of physical activity. Some job roles require constant manual labor. A workplace with good ergonomic design can prevent musculoskeletal discomfort, improve productivity, and work efficiency, reduce production costs, and optimize human well-being [2].

S. G. Aekanth (B) · V. G. Duffy Purdue University, West Lafayette, IN 47907, USA e-mail: [email protected] V. G. Duffy e-mail: [email protected] T. Kuebler Human Solutions of North America, Morrisville, NC 27560, USA e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Switzerland AG 2023 V. G. Duffy et al. (eds.), Human-Automation Interaction, Automation, Collaboration, & E-Services 11, https://doi.org/10.1007/978-3-031-10784-9_23

379

380

S. G. Aekanth et al.

As industries and job sites began to expand, more sophisticated tools became necessary to transport heavier goods, equipment, and material within the workplace. The advent of forklifts fulfilled this requirement. These vehicles are commonplace in the industrial environment, at construction sites, large commercial and business centers. Most foundries have multiple forklifts moving loaded pallets from one workstation to the next and darting in and out of inventory stockpiles [3]. However, improper use of forklifts can be harmful to driver health. Sitting for long periods, twisting into awkward positions, and spending all day riding in a machine with no suspension are just a few reasons why forklift operators find themselves with musculoskeletal and repetitive use injuries [4]. The US Bureau of Labor Statistics reported 9050 occupational injuries and illnesses involving forklifts for 2017 [5]. About a 7% increase from the previous year. Musculoskeletal injuries like sprains, tears, strains, soreness, and pain constitute nearly 52% of the incidence rate for 10,000 workers [6], signifying a severe threat to human health. For safe working conditions, it is essential to design an ergonomically sound workplace. Such a workplace can make employees feel comfortable at work, contributing to increased productivity. The main idea here is to effectively understand how humans interact with the work environment and incorporate those findings in the design. RAMSIS is one of the several tools which accomplishes this task. The Digital Human Modelling software is a 3D CAD manikin tool that simulates a real-life environment for ergonomic analysis. Visualization through digital human models helps the decision-maker in an organization to understand potential outcomes based on variations in design [7].

2 Problem Statement Forklift drivers often find themselves in situations that are potential precursors of injury. Apart from regular times, there are many instances where the operator needs to drive backward. A more significant load on the lift of the vehicle blocks the front vision for the operators. In such cases, the operators should move reverse, looking over the shoulder to ensure that the path is clear. The driver’s vision line can also be interrupted by parts of the vehicle itself when they turn back. Positioning themselves in uncomfortable postures puts stress on different elements of their body. Musculoskeletal disorders are the most reported causes of absence from work. They account for over 52% of all work-related illnesses and more than 2% of the European Union’s gross national product [8]. Also, driving backward can put other workers in the vicinity at risk of being injured by the forklift. The present work focuses on assessing the discomfort in different body elements of a forklift operator while driving backward. The analysis makes use of the ComputerAided Engineering software—RAMSIS. The study simulates a forklift geometry scene similar to an industrial setup. The investigation is performed for different modifications within the vehicle design. The forklift operator’s vision range in each case is also determined.

Forklift Operator Discomfort and Vision Assessment …

381

Fig. 1 Double click the ‘launch RAMSIS NextGen’ file to start the RAMSIS Application

Fig. 2 The image shows a forklift session loaded in the RAMSIS workspace. The geometry scene contains four different manikins identified as being dressed in blue trousers

3 Procedure 3.1 Starting the RAMSIS Software and Loading the Geometry Scene RAMSIS application is launched by double-clicking the ‘launch RAMSIS NextGen’ file from the RAMSIS zip folder (see Fig. 1). RAMSIS loads an empty workspace. The forklift session is loaded into the RAMSIS workspace by navigating to file > load session, selecting the folder ‘session’ within the ‘_forklift_scene’ folder, then pressing OK. The session also contains four different manikins, as shown in Fig. 2. It is also helpful to change the rendering style by navigating to menu > view > render style > shaded to better display the workspace.

3.2 Manikin Definition and Activation Each of the four manikins has different object properties visualized through options available in the structure tree—new project > test sample > driver > select manikin of choice > right mouse button > object properties. The window provides information about ‘arthrometry,’ ‘visual field,’ and ‘additional options’ of the manikin (see Fig. 3). Characteristics like arthrometry and visual field of each manikin play a crucial role during analysis.

382

S. G. Aekanth et al.

Fig. 3 The image shows the object properties of a medium male driver manikin. The visual field of the manikin is visible from the tab on the right of the workspace

Fig. 4 Forklift scene showing an active medium male driver manikin. The structure tree on the left of the workspace shows that the remaining manikins are inactive

When the forklift session is loaded, all manikins are in the active state. For this analysis, the medium male driver manikin is chosen (see Fig. 4). The remaining manikins are deactivated by double-clicking against their name in the structure tree.

3.3 Simulating an Industrial Environment and Analysis The scene in focus for the analysis is that of a forklift operator driving backward. in the toolbar places the active manikin Clicking the posture calculation button behind the steering wheel to simulate such an environment. The manikin now shifts to the driver seat (see Fig. 5).

Forklift Operator Discomfort and Vision Assessment …

383

Fig. 5 The medium male driver manikin is positioned behind the forklift’s steering wheel after clicking the posture calculation button present in the toolbar

An additional manikin assumed to be an industrial worker is stationed behind the forklift. Navigating to toolbar > test sample > create results in a new manikin. A female manikin with characteristics, as shown in Fig. 6, is created.

Fig. 6 The image shows the female manikin’s characteristics, created as a part of the analysis

384

S. G. Aekanth et al.

Navigating to toolbar > translate and manually dragging it to the desired coordinates positions the female manikin behind the forklift (see Fig. 7). The female manikin is then oriented to look in a direction away from the forklift. The right hand of the manikin is elevated, simulating a worker engrossed in work. This combination of actions is achieved by activating the inverse kinematics option in the toolbar and dragging the kinematic chains to the desired position (see Fig. 8). The geometry scene is now ready for analysis. The analysis is performed in three different cases. In the first case, the study considers the default forklift. Subsequent stages involve design modifications of the forklift, which is later discussed.

Fig. 7 The female manikin is positioned behind the forklift using the translate option and dragging it to coordinates specified in the image

Fig. 8 The image shows the female manikin’s position and orientation altered by adjusting the kinematic chains using inverse kinematics

Forklift Operator Discomfort and Vision Assessment …

385

Case-1: Forklift Operator Turning Backwards In this scenario, the driver is turning back to observe the scene behind the forklift before driving backward (see Fig. 9). It involves some twisting and turning motion of his body to achieve a clear vision of the female manikin standing behind the forklift. This new position is achieved by activating the inverse kinematics button and adjusting the driver manikin’s kinematic chains. Clicking toolbar > internal view shows the view through the medium male driver manikin’s eyes (see Fig. 10). The ROPS (Rollover Protective Structure) of the forklift slightly obstructs the driver’s view of the female manikin.

Fig. 9 Medium male driver manikin looking back to check the scene behind the forklift before driving backward

Fig. 10 The image shows the view through the eyes of the medium male driver manikin. The female manikin standing behind the forklift is visible, but the ROPS slightly obstructs the vision

386

S. G. Aekanth et al.

Fig. 11 The discomfort assessment test is performed on the medium male driver manikin by clicking the driver and navigating to the menu > analysis > comfort feeling

For this position of the forklift driver, discomfort assessment is performed. It is done by clicking on the driver manikin and navigating to menu > analysis > comfort feeling (see Fig. 11). The readings are tabulated. A side-view mirror is now installed in the forklift to enhance the driver manikin’s vision and comfort. Case–2: Installing a Side-View Mirror on the Forklift In this case, a side-view mirror is attached to the forklift. The mirror is installed on the front ROPS of the vehicle. It is done in two steps. First, ‘Point 1’ is created on the ROPS through menu > geometry > point. The ‘point type’ is specified by choosing ‘Create on Object’ and selecting the ROPS surface (see Fig. 12). The mirror is placed on ‘Point 1’ through menu > ergonomics > mirror definition. It is named ‘Side-View Mirror.’ The origin of the mirror is specified by selecting ‘Point 1’ from the structure tree. The mirror dimensions are set for 100 * 100 mm (W * H) (see Fig. 13). Clicking ‘add default mirror joint’ adds a degree of freedom to the mirror, which enables free movement of the mirror. The values are adjusted accordingly for a precise viewing angle. After setting up the mirror, the driver is oriented to look into it using inverse kinematics (see Fig. 14). Menu > analysis > vision > side-view mirror creates a view pyramid (see Fig. 15). From Fig. 15, one can say that the mirror’s view pyramid encloses the female manikin. For this position of the forklift driver, discomfort assessment is performed using the steps discussed in case 1, and the results are tabulated. The side-view mirror is replaced by installing a rear-view camera at the forklift’s back to examine changes in the forklift driver’s comfort and visibility.

Forklift Operator Discomfort and Vision Assessment …

387

Fig. 12 The image shows ‘Point 1’ being created on the ROPS of the Forklift. The point type selected is ‘Create on Object’

Fig. 13 The mirror parameters, such as width and height, are set for 100 * 100 mm (W * H), respectively, in the ‘Mirror Definition’ tab

Case–3: Installing a Rear-View Camera on the Forklift In this scene, provisions are made in the forklift to incorporate a rear-view camera. The camera is installed on the rear face of the vehicle’s roof, like the side-view mirror’s two-step installation. First, ‘Point 2’ is created through menu > geometry > point. The ‘Point Type’ is specified by choosing ‘Create on Object’ and selecting the middle portion of the vehicle’s roof’s rear face (see Fig. 16). The camera is then placed on ‘Point 2’ through ergonomics > camera definition.. It is named ‘Rear-View Camera.’ The origin of the camera is specified by selecting

388

S. G. Aekanth et al.

Fig. 14 The image on the left depicts the alteration of the driver manikin’s joints using inverse kinematics. The image on the right shows the effect of the implementation of inverse kinematics

Fig. 15 The image shows a view pyramid generated using the side-view mirror by navigating to the menu > analysis > vision > side-view mirror. The view pyramid encloses the female manikin

Fig. 16 The image shows ‘Point 2’ being created on the rear face of the roof of the forklift. The point type selected is ‘Create on Object.’

Forklift Operator Discomfort and Vision Assessment …

389

‘Point 2’ from the structure tree. The opening angle for horizontal and vertical angles is set for 80° and 50°, respectively (see Fig. 17). A plane is created just in front of the operator, simulating a camera display. The location chosen is the rear face of the vehicle’s lift. Menu > geometry > plane creates the required geometry. The ‘plane type’ is defined by choosing ‘on the surface’ and selecting the middle portion of the lift’s rear face (see Fig. 18). The use of inverse kinematics orients the vision of the driver towards the display.

Fig. 17 The image shows camera parameters like opening angle horizontal and vertical set to 80° and 50°, respectively, in the camera definition tab

Fig. 18 The image shows a plane being created on the truck’s lift’s rear face, which will serve as the rear-view camera display

390

S. G. Aekanth et al.

Fig. 19 The image shows a view pyramid generated using the rear-view camera by navigating to the menu > analysis > vision > rear-view camera. The view pyramid encloses the female manikin

Menu > analysis > vision > rear-view camera creates vision pyramid (see Fig. 19). From Fig. 19, one can say that the camera’s view pyramid encloses the female manikin. For this position of the forklift driver, discomfort assessment is performed using the steps discussed in case 1, and the results are tabulated. Additionally, cases 1, 2, and 3 are repeated for a small female manikin in the driver role. Appendix A shows the discomfort assessment results and the field vision of the analysis.

4 Results and Discussion 4.1 Results Discomfort Assessment Figure 20 shows the medium male driver manikin’s discomfort assessment test results for all the three cases discussed above. The index values for different body elements of the manikin are represented in Table 1. A discomfort assessment result ranks various body elements from 1 to 8, where 1 signifies the highest comfort, and 8 signifies the lowest comfort. For example, the health index in the assessment corresponds to the pressure load on the spinal column. Therefore, lower health value is ideal. Figure 21 shows a graphical representation of discomfort assessment for the medium male driver manikin. It is evident from Fig. 21 that when the manikin turns back, it experiences the highest discomfort level. This phenomenon is due to the high values felt by each element of the body. In the longer run, continually turning

Forklift Operator Discomfort and Vision Assessment …

391

Fig. 20 Clockwise from left to right: Results of discomfort assessment when the driver manikin turns back; when driver manikin looks into the side-view mirror; and looking into a rear-view camera display

back can affect the health of drivers operating such forklifts affecting their productivity. It will result in the operator being away from work due to occupational illness. However, there exists one index of the analysis, which marginally favors case 1, amongst others. The driver manikin’s health index is slightly lower when looking back, indicating that the spine’s pressure load is comparatively less. However, this variation is negligible. Discomfort assessment for cases 2 and 3 show improved results. There is a minute variation between the index of different body elements. However, the values are almost consistent. Further analysis between the values obtained from cases 2 and 3 is discussed later in the section. Vision Assessment Figure 22 shows a visual comparison of the field vision between cases 1, 2 and 3 for the driver. When the driver is looking backward, parts of the forklift obstruct his view, and the female manikin is visible at an awkward angle. The driver may find it challenging to ensure the safe movement of the vehicle.

392

S. G. Aekanth et al.

Table 1 Discomfort Assessment results for medium male driver manikin for cases 1, 2, and 3 with their respective mean and standard deviation Discomfort assessment

Case 1

Case 2

Case 3

Neck

8.00

2.90

2.73

Shoulder

4.99

2.15

2.00

Back

3.23

1.76

2.21

Buttocks

1.94

1.75

1.23

Left leg

2.56

2.28

2.30

Right leg

3.24

2.57

2.63

Left arm

3.62

1.89

3.07

Right arm

3.14

2.96

2.40

Discomfort

5.47

4.07

4.00

Fatigue

4.31

3.15

3.15

Health

4.82

4.93

5.11

Mean

4.12

2.76

2.90

Standard deviation

1.59

0.95

0.96

10 Index Values

8

Discomfort Assessment for Medium Male Driver Manikin

6 4 2 0

Body Elements Case 1 Case 2 Case 3

Fig. 21 A bar graph comparing the discomfort assessment results performed on a medium male driver manikin for cases 1, 2 and 3

The field view through the side-view mirror shows improvement. The result is much better when compared to the driver looking back. However, the viewing range is limited. He will also be required to switch between the left and right view mirrors for safe navigation. Compared to cases 1 and 2, the camera vision is advanced. It provides the forklift operator with a wide viewing range. There is no obstruction in the view path. Since there is one central display, the driver’s focus will also remain undisturbed.

Forklift Operator Discomfort and Vision Assessment …

393

Fig. 22 Clockwise from left to right: Field vision when the medium male driver manikin turns back; when looking into the side-view mirror and when looking through a rear-view camera

Rear-View Camera Versus Side-View Mirror From the above analysis, it can be assumed that using either a rear-view camera or side-view mirror results in less discomfort to the operator than turning the neck to look back. The best alternative among these two cases is determined using statistical analysis by performing a Student’s t-Test on the dataset of cases 2 and 3. Table 1 shows that six out of the ten index values in case 3 are slightly larger than case 2. The question that arises is whether there is a significant change when the rear-view camera replaces the side-view mirror. For this reason, the nature of the t-Test is designed to be one-tailed. The data set is considered paired data since the two indices correspond to the same body element. The test is performed in MS Excel software. It is done by navigating to formula ribbon > insert function, browsing for T.TEST in the ‘select a function’ blank, and entering the values of cases 2 and 3 within the array blocks 1 and 2, respectively. Figure 23 shows that the formula result of Student’s t-Test is approximately 0.05. The value indicates a 95% possibility of significant change between the 2 cases. The

394

S. G. Aekanth et al.

Fig. 23 The image shows the results of Student’s t-Test for cases 2 and 3 in MS Excel. Tails-1 indicates the test is one-tailed, and type-1 indicates paired data. The formula result value or p-value is 0.05213029

higher index of discomfort using the rear-view camera may be due to the camera display placement. The present camera display position is considerably higher than the operator’s eye line requiring the manikin to look upwards. However, having a monitor screen that meets the operator’s eye line may result in better comfort. This modification can be made possible by suspending additional structures from the roof of the forklift. The results of visual and discomfort assessment are intriguing. The forklift operator’s field vision after using the rear-view camera is better than using a side-view mirror. However, the operator experiences more comfort when using a side-view mirror to observe the scene behind him than by looking up into a rear-view camera’s display. From the standpoint of industry, the trade-off here is the cost associated with the accessories installed. Using a side-view mirror can be an economical option compared to a camera installation. However, a camera can provide a wide-angle view to the operator. Additionally, an ergonomic fitting of a camera display could alleviate operator discomfort. Appendix 1 shows the results of Student’s t-Test when a small female driver manikin is looking into a side-view mirror and through a rear-view camera. The p-value from the analysis was found to be 0.0001, indicating a significant change in comfort between the two cases (see Appendix 1). Once again, the difference may be attributed to camera display placement. For the analysis, the female and the male manikins use the same camera display. However, the shorter height of the female manikin results in the camera monitor being further away from her eye line. Lowering the monitor may result in lower discomfort values. The field vision for both the manikins is similar, and the camera display shows the best results (see Appendix 2).

Forklift Operator Discomfort and Vision Assessment …

395

4.2 Discussion The overall project experience has been an excellent learning curve. The use of RAMSIS software helped in understanding the implications of uncomfortable postures on the human body. Having interned in several manufacturing firms, I have observed forklift operators drive in narrow lanes with awkward positions, making their task even more challenging. Designing ergonomically sound vehicles can help improve their navigation and reduce musculoskeletal injuries. The RAMSIS analysis has helped comprehend the design modifications within the forklift and their effect on the operators. One of the significant challenges I experienced during this study was positioning and adjusting the side-view mirror. Initially, I had to identify an ideal location within the forklift for installing the mirror. Setting up the mirror required calculating the location coordinates and adding degrees of freedom to the mirror to get the best view. It needed me to simultaneously switch between the mirror’s internal view and object properties to arrive at the best possible result. I would suggest that students explore mirror definition within the software to develop precision in handling mirror adjustments. Prior background in physics like knowledge of torsion and degrees of freedom can be helpful.

5 Future Work Human digital modeling through RAMSIS realistically simulates vehicle occupants and analyzes the ergonomics of interiors, allowing designers to ensure a high level of product maturity [9]. The software provides the user with an understanding of how systems are designed, considering the extent of human abilities. During the analysis, all the manikins are assumed to be healthy and non-disabled. However, individuals can be limited in their abilities in a real-life environment due to physical constraints and pre-existing medical conditions. Some of them include but are not limited to vertigo, postural hypotension, foot problems, and eye diseases. Workers with physical disabilities may have different feelings of comfort and discomfort. They may experience varying levels of stress on various body elements as compared to non-disabled co-workers. Such health conditions can affect human performance. This data is currently unavailable in the CAD environment. It would be useful to include such data in the future as it helps in advanced analysis.

Appendix 1 See Table 2 and Fig. 24.

396

S. G. Aekanth et al.

Table 2 Discomfort Assessment results for small female driver manikin for cases 1, 2 and 3 with their respective mean and standard deviation Discomfort assessment

Case 1

Case 2

Case 3

Neck

8.00

3.45

4.01

Shoulder

6.70

2.05

2.06

Back

1.47

2.45

2.85

Buttocks

0.72

1.96

2.11

Left leg

1.59

2.61

3.27

Right leg

1.74

2.6

3.21

Left arm

1.93

2.66

2.93

Right arm

1.98

2.87

3.17

Discomfort

2.85

4.56

5.24

Fatigue

2.30

3.43

3.99

Health

4.95

5.05

5.22

Mean

3.11

3.062727273

3.46

Standard deviation

2.369973916

0.98546528

1.070868806

Fig. 24 The image shows the results of Student’s t-Test for cases 2 and 3 in MS Excel. Tails-1 indicates the test is one-tailed, and type-1 indicates paired data. The formula result value or p-value is approximately 0.0001

Appendix 2 See Fig. 25.

Forklift Operator Discomfort and Vision Assessment …

397

Fig. 25 Clockwise from left to right: field vision when the small female driver manikin turns back; when looking into the side-view mirror and looking through a rear-view camera

References 1. World Health Organization: Health and Development, https://www.who.int/hdp/en/#:~:text= Better%20health%20is%20central%20to,health%20services%20for%20its%20people. Last accessed 19 Oct 2020 2. Chim JMY (2018) 6Ws in ergonomics workplace design. In: Congress of the international ergonomics association. Springer, Cham, pp 1282–1286 3. Garesche J (2018) Forgotten safety regulations: forklift safety. Modern Casting, 45. Gale General, https://link.gale.com/apps/doc/A554907237/ITOF?u=purdue_main&sid=ITOF&xid= 62e0a94f. Last accessed 19 Oct 2020 4. Toyota Material Handling Northern California Blog, https://www.tmhnc.com/blog/how-forkliftdrivers-can-prevent-injuries. Last accessed 19 Oct 2020 5. United States Bureau of Labor Statistics: Fact Sheet Occupational Injuries, Illnesses, and Fatalities Involving Forklifts, June 2019, https://www.bls.gov/iif/oshwc/cfoi/forklifts-2017-chart2data.htm. Last accessed 20 Oct 2020 6. United States Bureau of Labor Statistics: Injuries, Illnesses and Fatalities, https://www.bls.gov/ iif/soii-chart-data-2017.htm. Last accessed 20 Oct 2020

398

S. G. Aekanth et al.

7. Duffy VG (2012) Human digital modeling in design. In: Salvendy G (ed) Handbook of human factors and ergonomics, 4th edn. John Wiley & Sons, New Jersey, pp 1016–1030 8. Labus N, Gajšek B (2018) Use of ergonomic principles in manual order picking systems. Logistics Sustain Transp 9(1):11–22 9. Human Solutions Homepage, https://www.human-solutions.com/en/products/ramsis-general/ index.html. Last accessed 21 Oct 2020

Promoting Safety and Injury Prevention of Electric Transportation WooJune Jung

Abstract Electric transportation is one of the emerging modes of transportation around the globe. Due to its compact size, lower emission of pollutant, and affordable price, many people prefer them as a short-range transportation device. However, as more people use the e-mobility, more accidents occur, and there is a high risk of injury without proper regulation and injury prevention methods. To find which research has been conducted on this issue, articles have been searched in various databases (Web of Science, Google Scholar, Scopus, etc.). Trend analysis of the search was conducted by observing the number of articles written in each year and the trend was increasing. Engagement measure was calculated from Vicinitas by analyzing engagement level on Twitter over ten days. Then, the influential papers were chosen through co-citation analysis using CiteSpace and VOSviewer. Leading table was used to analyze the data from BibExcel, extracted from the articles from Google Scholar using Harzing’s Publish or Perish 7. The contents from the chosen references were analyzed using VOSviewer and MAXQDA. China and United States were the leading countries in research of electric transportation injury prevention. The most common cause of accident was losing balance and falling off the mobility device. Furthermore, most of the patients analyzed were not wearing helmets or proper protective equipment. Some prevention methods, such as registering the mobility device and promoting safety education to the riders were suggested. However, to lower the risk of potential hazards, further research needs to be conducted to more accurately predict the behavior of the drivers or factors that can lead to accident. Keywords Electric transportation · E-bike · E-scooter · E-mobility · Injury · Prevention · Safety · MAXQDA · VOSviewer · CiteSpace

W. Jung (B) Purdue University, West Lafayette, IN 47906, USA e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Switzerland AG 2023 V. G. Duffy et al. (eds.), Human-Automation Interaction, Automation, Collaboration, & E-Services 11, https://doi.org/10.1007/978-3-031-10784-9_24

399

400

W. Jung

1 Introduction Various modes of transportation are becoming an essential part of our lives. Traditional modes of transportation have developed from active transportation, such as walking or riding a bicycle, to automobiles using internal combustion engines. However, transportation nowadays is experiencing another shift from traditional modes to electric transportation. This shift from traditional to electric transportation is due to numerous reasons, strict environmental regulation and development of energy storage technology can be examples of the reasons. Some advantages of electric transportation are that they are compact, affordable and energy-saving. Electric mobility such as electric bikes, mopeds and scooters are growing modes of transportation in many countries. According to Zhang et al., production of electric bikes/mopeds have increased from 58,000 in 1998 to 32 million in 2016. Thus, there were 250 million electric mobility registered in China in 2016 [14]. As more electric transportation, which is energy-efficient and convenient, are on the road, safety about this mode of transportation must be considered thoroughly. According to Goetsch, “Motor vehicle accidents are the leading cause of death in the US each year,” and Brauer also mentioned, World Health Organization (WHO) stated that 1.2 million deaths occurred on the road around the world [2, 6]. Therefore, the accident should be prevented or the injuries due to accident must be prevented at least. In addition, as the number of electric transportation users increase, the potential for hazard increase without proper safety measure. Proper regulations and traffic laws applies to the electric vehicles; however, other types of electric mobility, such as electric bike or electric scooters, are not being regulated strictly while the number of mobilities increase rapidly. Therefore, studies in various types of electric transportation will be overviewed.

2 Purpose of Study The purpose of this study is to perform systematic literature review on articles about electric transportation injury prevention. This review will prove how injury prevention of electric transportation is an important topic to research on and to find what more can be done to improve current prevention method. For the data collection and analysis, various tools such as Harzing’s Publish or Perish 7, Web of Science, VOSViewer, MAXQDA 2020, VOSviewer have been used.

Promoting Safety and Injury Prevention of Electric Transportation

401

Table 1 Search terms and number of articles from each database Terms

Database

# of articles

Electric transportation injury

Scopus

149

Web of Science

48

Google Scholar

126,000

Springerlink

6031

Scopus

424

Web of Science

122

Google Scholar

123,000

Electric vehicle injury

Electric transportation injury prevention

Springerlink

9352

Scopus

28

Web of Science

7

Google Scholar

66,000

Springerlink

2605

3 Procedure 3.1 Data Collection Multiple databases were used to gather articles related to electric transportation injury prevention. “Electric transportation injury” was the first term to be searched since it is the most relevant term to my topic. Google Scholar showed the highest number of results for all the terms that I searched for; nonetheless, Web of Science (WoS) and Scopus only returned with 48 and 149 results, respectively. Thus, I broadened my search to more general term “Electric vehicle injury.” Then, the results from Scopus and WoS were 424 and 122, respectively. Since Google Scholar showed numerous results, so the keyword was narrowed down to “Electric transportation injury prevention,” which can return with more directly related articles. The search returned with 66,000 results, which was enough to conduct analysis on. Table 1 summarizes the data collection result explained above.

3.2 Trend Analysis Trend analysis was conducted using results from Scopus and Google Scholar through Harzing’s Publish or Perish 7. Figure 1 is the trend diagram of articles per each year given by Scopus. The diagram depicts that there was not much of increase until 2002; however, the number of articles has been rapidly increasing since then. The graph at the end is dropped, because the articles that will be published in 2021 have not been considered in the analysis.

402

W. Jung

Fig. 1 Trend analysis of article per year from Scopus search of “Electric vehicle injury”

The data from Google Scholar was analyzed with Bibexcel to extract the data and convert them into pivot chart. Figure 2 is the pivot chart generated with the data from Google Scholar. The chart demonstrates that steady increase in number of articles begins from 1993. In a recent ten years from 2010 to 2019, the number of articles has increased from 37 to 71, which is almost double the amount. One interesting trend found is that there were two steep increase in number of articles in 2012 and 2016, which is a year or two after a steep increase in electric vehicle injury. Thus, we can assume that after a research on injury itself is conducted, the prevention method for such injuries may have been conducted.

80

Number of articles per year

70 60 50 40 30 20 0

2021 2019 2017 2015 2013 2011 2009 2007 2005 2003 2001 1999 1997 1995 1993 1991 1989 1987 1985 1982 1979 1977 1975 1972 1968 1926 1909

10

Fig. 2 Pivot chart about number of articles per year from Google Scholar search of “Electric transportation injury prevention”

Promoting Safety and Injury Prevention of Electric Transportation

403

Fig. 3 Cumulative engagement (left) and posts (right) over time acquired from Vicinitas

3.3 Engagement Measure Engagement measure was calculated through Vicinitas. Vicinitas is a website that can gather the posts from Twitter and analyze the engagement level and influence level. Engagement level is calculated by the interaction with the post such as retweets, replies etc. Influence level is calculated by the number of followers of the user when the user sends the post. There were 2053 posts gathered with “electric transportation” as a keyword. Engagement and influence was 6600 and 58.7 million, respectively. Thus, the engagement is only three times of the posts, but the influence they have is about 30,000 times of those. Figure 3 is showing the cumulative engagement and posts are steadily increasing, which means users are actively discussing and reposting tweets about electric transportation recently.

3.4 Emergence Indicator Emergence indicator is also calculated with the data acquired from Google Scholar search through Harzing’s Publish or Perish 7. The search term was “electric transportation injury prevention” and results from 2010 to 2019 were used to calculate emergence in this topic. Emergence indicators consist of growth, network and persistence. Growth is calculated by dividing the number of articles in 2019 by that of 2010. Network is the number of authors published in that period and persistence is the number of articles published during that period. Table 2 summarizes the emergence indicators and their values about the results from 2010 to 2019. Growth is 1.92, which means the interest in this topic has almost doubled over ten years. Network and Persistence also demonstrates the consistent interest in this topic over a decade.

404 Table 2 Emergence indicators based on Google Scholar search on “electric transportation injury prevention”

W. Jung Emergence indicators

Emergence values

Growth (Articles in 2019/Articles in 2010) 1.92 Network

501 Authors

Persistence

524 Articles

4 Result 4.1 Co-citation Analysis Co-citation analysis was conducted to sort out the influential articles that were cited by other authors also. This analysis will help us to narrow down the articles that should be reviewed about this topic. Tools that were used for co-citation analysis were CiteSpace, VOSviewer and pivot table generated through Bibexcel. Figure 4 is the co-citation cluster created by CiteSpace. The data used to form this cluster is a result of “electric vehicle injury” from WoS. There are four authors being shown in the cluster: Du W., Siman-Tov M., Langford B. C., and Popoutsi S. This means that articles from these authors were cited by others frequently. Citation burst was done to see the co-citation analysis in detail since it can show the strength and the time period that the article was referenced. Figure 5 is the result of the citation burst through CiteSpace. It only returned a single result, an article from Du et al. [4]. This paper was referenced between 2017 and 2019 the most, and since the paper by Du W. et al. is a top 1 reference from the data acquired, it was added to the list of articles to be reviewed for this systematic review. The words in bright red color are keyword clusters that will be used in the analysis later. Next tool I used for co-citation analysis is VOSviewer. The data used for cocitation analysis was same data from the previous analysis. Result from VOSviewer is shown in Fig. 6. Du W., Siman-Tov M., Langford B. C., and Popoutsi S also shows up in this diagram, and Fig. 7 is the list of papers included in co-citation analysis having minimum occurrence of 6 times. Only 13 articles met the threshold, and we can see Du W.’s article has the highest link strength and there also is another article by Du W. as the fifth highest link strength. Data from Google Scholar through Harzing’s Publish or Perish 7 was also used to identify leading authors. Since Google Scholar returned more result than WoS, the detail was added to the search term, so “electric transportation injury prevention” was used. Table 3 shows that Guo Y., Xu C., and Li Y. are the leading authors for having 7 articles about electric transportation injury prevention. Thus, the most cited articles from each author were added for the content analysis. With the co-citation analysis conducted with the data from Google Scholar and WoS, list of references that will be further reviewed are organized in Table 4. Article written by Li Y. was not included, because there were multiple authors with different first name beginning with Y.

Promoting Safety and Injury Prevention of Electric Transportation

405

Fig. 4 Co-citation cluster created with CiteSpace using “electric vehicle injury” data from WoS

Fig. 5 Citation burst result from Fig. 4

Fig. 6 Co-citation analysis from VOSviewer using “electric vehicle injury” data from WoS

4.2 Content Analysis After narrowing down the articles to review on, content analysis on the articles and this topic was conducted. Tools that were used for the content analysis are VOSviewer, Vicinitas and MAXQDA. Vicinitas was used in the engagement measures section also, but the word cloud generated from Vicinitas will be reviewed in this section. Word cloud is the visualization method to show the frequency of keywords by varying

406

W. Jung

Fig. 7 List of reference included in co-citation analysis from VOSviewer

Table 3 Pivot table for number of articles per author about “electric transportation injury prevention” from Google Scholar

Author

Sum of Articles

Guo Y

7

Xu C

7

Li Y

7

Wang Y

6

Wang X

6

Baker SP

6

Zhang Y

6

Miller TR

6

Wang H

6

Li X

5

Yang H

5

Chen Z

5

Floyd HL

5

Yang X

5

Li Z

5

Wang L

5

Wang Z

5

Grand Total

97

their sizes. Figure 9 is the word cloud generated by Vicinitas with a search term, “electric transportation” (Fig. 8). Some noticeable keywords other than search terms are future, vehicle, shift etc. Therefore, the public thinks of electric transportation as a future and the shift from traditional modes of transportation to electric transportation is happening currently. Another word cloud was generated using MAXQDA. This word cloud is generated

Promoting Safety and Injury Prevention of Electric Transportation

407

Table 4 List of articles chosen from co-ciation analysis Authors

Title and publication info

Year

Du, Wei, Jie Yang, Brent Powis, Xiaoying Understanding on-road practices of 2013 Zheng, Joan Ozanne-Smith, Lynne Bilston, electric bike riders: an observational study and Ming Wu in a developed city of China. Accident Analysis Prevention 59:319–326 Siman-Tov, Maya, Irina Radomislensky, Israel Trauma Group, and Kobi Peleg

The casualties from electric bike and motorized scooter road accidents. Traffic Injury Prevention 18(3):318–323

Guo, Yanyong, Zhibin Li, Yao Wu, and Chengcheng Xu

Evaluating factors affecting electric bike 2018 users’ registration of license plate in china using Bayesian approach. Transportation Research Part F: Traffic Psychology and Behaviour 59(2018):212–221

Du, Wei, Jie Yang, Brent Powis, Xiaoying Zheng, Joan Ozanne-Smith, Lynne Bilston, Jinglin He, Ting Ma, Xiaofei Wang, and Ming Wu

Epidemiological profile of hospitalised injuries among electric bicycle riders admitted to a rural hospital in Suzhou: a cross-sectional study. Injury Prevention 20(2):128–133

Wang, Chen, Chengcheng Xu, Jinxin Xia, and Zhendong Qian

Modeling faults among e-bike-related fatal 2017 crashes in China. Traffic Injury Prevention 18(2):175–181

2017

2014

Fig. 8 Word cloud generated by Vicinitas with data from “electric transportation”

with the articles and books that we considered to review on after the co-citation analysis. Figure 10 is the word cloud generated by MAXQDA with minimum frequency of 30 times. The reference materials mostly about electric transportation and its injury prevention, so the word cloud consists of many words related to that. However, words such as scooter and e-bike show that research on e-scooters and e-bikes are being done actively. The word “China” is also in the word cloud, which means more research are being done in China about electric transportation. “Battery” and “motor” are major component of electric transportation, so the improvement of the components can be one crucial area for injury prevention. Other than the word cloud, content cluster analysis was done with VOSviewer. The diagram below is generated with the search of “Electric transportation injury

408

W. Jung

Fig. 9 Word cloud generated using MAXQDA about articles and books chosen as references with minimum frequency of 30

Fig. 10 Content analysis of “electric transportation injury prevention” data from Google Scholar created using VOSviewer

prevention” from Google Scholar through Harzing’s Publish or Perish 7. Figure 11 shows the network diagram created with minimum occurrence of 20 times. There were 51 words that met the threshold, but after removing unnecessary and redundant words, 29 words remained. “Injury” is the most used keyword; however, other than the words included in the search term, “system” occurred 136 times and “safety” occurred 88 times. Two countries that were included in this diagram were China and United States, so we can assume that more research is being conducted in these countries than others. Table 5 is a leading table from WoS with a search term “electric vehicle injury”, and it supports that China and United States are the leading countries in this research topic. Electric and chemical hazard due to battery may have been

Promoting Safety and Injury Prevention of Electric Transportation

409

Table 5 Leading table of top 10 countries with research articles about “electric vehicle injury” from WoS Countries/regions

Records

% of 122

Peoples R China

40

32.787

USA

36

29.508

Australia

9

7.377

Israel

9

7.377

Germany

8

6.557

Canada

7

5.738

Japan

6

4.918

Sweden

5

4.098

South Korea

4

3.279

Switzerland

4

3.279

researched thoroughly since “electric injury” and “chemical” also appears in the diagram.

5 Discussion 5.1 Electric Transportation Injury Trend Number of electric transportations is increasing worldwide, so multiple cities in various nations are conducting research on injury trends of electric transportation accident. For example, in China, where electric bikes and mopeds are one of the major modes of transportation, research were conducted in cities such as Nanjing, Taixing, Suzhou, and Zhejiang. Electric mobility does take a big portion of traffic accident since 32.7% of traffic accidents observed over 3 years in Zhejiang was related to e-bike according to Zhou et al. [15]. Common cause of injury observed from those studies are not wearing helmet and not following the traffic regulation properly. According to Du et al., only 9% of the riders wore helmet, and that can possess a great risk of injury [3]. Another research conducted by Du et al. in Suzhou, China, stated that 57.2% of traffic injuries were related to e-bike riders and the odds of traumatic brain injuries increased significantly for night-time crashes [4]. Thus, the increase in e-bike injury in China is an emerging problem to be solved for safer transportation. Other than China, number of electric transportations is increasing in United States also. E-scooter sharing began to spread around the US since late 2017, and multiple researches were conducted to analyze the injury trend and find safety measures. According to Rix et al., injuries per million vehicle miles traveled (MVMT) were calculated with the data acquired from Austin, Texas in 2018. E-scooters had 180

410

W. Jung

injuries/MVMT, while other motor vehicles had 0.9 injuries/MVMT, and that is 200 times more injuries per miles traveled [11]. Thus, it is crucial to analyze the injury trend and establish the safety regulations. Most common cause of injury in US was falling from the vehicle (84.7%) and head & face were most common part to be injured (45.5%) according to English et al. [5]. Furthermore, only 1.6% of the patients analyzed by English et al. [5] wore helmet. Paper by Bloom et al. [1] also supports the forementioned trend since out of 248 cases, only 3% of patients wore helmet and 50% got into accident by loss of balance. However, there were only one accident occurred in bike lane, and a large portion of patients with severe injuries were under the influence of alcohol or marijuana [1]. According to Puzio et al., the observation and analysis conducted in Indianapolis, Indiana, none of the patients used any protective gear and 33% were under the influence of alcohol; thus, protective gear and driving under influence can be a big risk [10]. Besides two major countries, Israel and Singapore has also conducted research on electric transportation injury since electric transportation is a global trend. According to Siman-Tov et al., 92% of patients were riders and 33% of total casualties were children of age under 14. Thus, Siman-Tov questioned if electric transportation is socially and economically beneficial [12]. In Singapore, from all the injuries caused by personal mobility devices, e-bike and e-scooter riders got into the most severe injury according to King et al. [8]. Thus, research is being conducted globally, but the common cause of injury and factors that increase severity are showing similar trend worldwide.

5.2 Injury Prevention Methods After finding the analysis of common injury patterns, some injury prevention methods were suggested by various articles also. Electric mobility is currently in the grey area of motor vehicles and human-powered vehicles. Electric transportation is much faster than human-powered vehicles, so it possesses higher risk of severe injury. However, it is not officially being registered and regulated strictly. Therefore, article by Guo et al. states that e-bike should be registered for monitoring illegal behaviors on the road [7]. Along with the registration, electric transportation riders should have proper safety education similar other motor vehicle drivers. The most important injury prevention method is to wear with proper safety equipment. Multiple articles mentioned that most of patients analyzed did not wear helmet and they suffer from heat or face injuries [1, 4, 5, 15]. Other than the efforts of individual riders, traffic system can be improved to reduce the occurrence of injury. Wang et al. mentions that bike lane physically separated from other lanes, video surveillance system for e-bike riders, and strict regulation on drunk e-bike riders can be systematic solutions to prevent injury [13].

Promoting Safety and Injury Prevention of Electric Transportation

411

6 Future Work Thorough analysis of has shown that electric transportation is emerging modes of transportation globally, and potential hazard on riders and pedestrians are increasing proportionally. Number of prevention methods, such as promoting registration of electric mobility, restricting the use of electric mobility under the influence of alcohol or other chemical, and wearing helmet have been suggested by the references mentioned above. However, to discover the factors that causes the accident drivers’ behavior need to be analyzed in detail also. Award by NSF.gov, ‘BIGDATA: IA: Predictive Analytics of Driver’s Engagement for Injury Prevention’, is an award for the research projects that can contribute to prevent transportation accident by predicting driver distraction from secondary activities in vehicle. Monselise et al. conducted a research to develop a model to predict an accident with given road condition and behavior of driver. 7707 trips were collected and analyzed with four different machine learning and deep learning models, and a gradient boost model showed the most accurate and interpretable result for accident prevention. This study stated that the most important variables in predicting were pre-incident maneuvers and secondary task duration; therefore, keeping an eye on the road and focusing on driving are the most important factors to prevent accident [9]. According to Zou et al., the safety feature for rapidly advancing connected and autonomous vehicle will be crucial. It should be able to not only perceive the onroad data, but also predict the behavior of other drivers especially for conventional vehicles or active transportation users (pedestrians or cyclists). The development of reliable accident prevention system needs to be developed before the fully automated vehicles are widely spread in the market [16]. As the computing methods and sensors advance, more data on road and drivers can be collected and analyzed to generate more accurate model. With more accurate model, new factors that were not considered previously can be found and decrease the risk of potential hazard.

References 1. Bloom MB, Noorzad A, Lin C, Little M, Lee EY, Margulies DR, Torbati SS (2021) Standing electric scooter injuries: impact on a community. Am J Surg 221(1):227–232. https://doi.org/ 10.1016/j.amjsurg.2020.07.020 2. Brauer RL (2016) Essay. In: Safety and health for engineers, 219–16w. Wiley, Hoboken, NJ 3. Du W, Yang J, Powis B, Zheng X, Ozanne-Smith J, Bilston L, Wu M (2013) Understanding on-road practices of electric bike riders: an observational study in a developed city of China. Accid Anal Prev 59:319–326. https://doi.org/10.1016/j.aap.2013.06.011 4. Du W, Yang J, Powis B, Zheng X, Ozanne-Smith J, Bilston L, He JL, Ma T, Wang X, Wu M (2014) Epidemiological profile of hospitalised injuries among electric bicycle riders admitted to a rural hospital in Suzhou: a cross-sectional study. Injury Prevention 20(2):128–133. https:// doi.org/10.1136/injuryprev-2012-040618 5. English KC, Allen JR, Rix K, Zane DF, Ziebell CM, Brown CV, Brown LH (2020) The characteristics of dockless electric rental scooter-related injuries in a large U.S. city. Traffic Inj Prev 21(7):476–481. https://doi.org/10.1080/15389588.2020.1804059

412

W. Jung

6. Goetsch DL (2019) Essay. In: Occupational safety and health for technologists, engineers, and managers, pp 18–28. Pearson, New York, NY 7. Guo Y, Li Z, Wu Y, Xu C (2018) Evaluating factors affecting electric bike users’ registration of license plate in china using Bayesian approach. Transp Res Part F Traffic Psychol Behav 59: 212–221. https://doi.org/10.1016/j.trf.2018.09.008 8. King CS, Liu M, Patel S, Goo TT, Lim WW, Toh HC (2020) Injury patterns associated with personal mobility devices and electric bicycles: an analysis from an acute general hospital in Singapore. Singapore Med J 61(2):96–101. https://doi.org/10.11622/smedj.2019084 9. Monselise M, Liang OS, Yang CC (2019) Identifying important risk factors associated with vehicle injuries using driving behavior data and predictive analytics. In: 2019 IEEE International conference on healthcare informatics (ICHI). https://doi.org/10.1109/ichi.2019.8904860 10. Puzio TJ, Murphy PB, Gazzetta J, Dineen HA, Savage SA, Streib EW, Zarzaur BL (2020) The electric scooter: a surging new mode of transportation that comes with risk to riders. Traffic Inj Prev 21(2):175–178. https://doi.org/10.1080/15389588.2019.1709176 11. Rix K, Demchur NJ, Zane DF, Brown LH (2021) Injury rates per mile of travel for electric scooters versus motor vehicles. Am J Emerg Med 40:166–168. https://doi.org/10.1016/j.ajem. 2020.10.048 12. Siman-Tov M, Radomislensky I, Israel Trauma Group, Peleg K (2017) The Casualties from electric bike and motorized scooter road accidents. Traffic Inj Prev 18(3):318–323. https://doi. org/10.1080/15389588.2016.1246723 13. Wang C, Xu C, Xia J, Qian Z (2017) Modeling faults among e-bike-related fatal crashes in China. Traffic Inj Prev 18(2):175–181. https://doi.org/10.1080/15389588.2016.1228922 14. Zhang X, Yang Y, Yang J, Hu J, Li Y, Wu M, Stallones L, Xiang H (2018) Road traffic injuries among riders of electric bike/electric moped in southern China. Traffic Inj Prev 19(4):417–422. https://doi.org/10.1080/15389588.2018.1423681 15. Zhou SA, Ho AF, Ong ME, Liu N, Pek PP, Wang YQ, Jin T et al (2017) Electric bicycle-related injuries presenting to a provincial hospital in China. Medicine 96(26). https://doi.org/10.1097/ md.0000000000007395 16. Zou X, Vu HL, Huang H (2020) Fifty years of accident analysis & prevention: a bibliometric and scientometric overview. Accid Anal Prev 144:105568. https://doi.org/10.1016/j.aap.2020. 105568

Smart Cities and Connected Vehicles

Designing for Me! What Older Dwellers’ Want to Improve Mobility in an Age-Friendly City Pei-Lee Teh, Ver Nice Low, Deepa Alex, Qasim Ayub, and Shaun Wen Huey Lee

Abstract More than half of the world population lives in cities. Since the late 2000s, the World Health Organization (WHO) has been promoting the creation of agefriendly cities to address the grand challenges posed by rapid growth of aging population. Most of our current insights on age-friendly cities are from developed countries, with limited studies originating from developing countries. This study aims to explore older dwellers’ perceptions and preference of transportation and mobility in relation to the age-friendly city concept. Using a purposive sampling, 32 older dwellers aged 50 years and above living in Sunway City, a township in Malaysia were interviewed. Participants’ viewpoints are identified after a field experiment. Qualitative results unfolded three categories namely transportation system, land pattern use and urban design as well as health status. Results demonstrate that older dwellers valued the outdoor spaces, physical infrastructure and social inclusion. Our findings provide an actionable insight for businesses, property developers, construction companies and P.-L. Teh (B) · V. N. Low School of Business, Gerontechnology Laboratory, Monash University Malaysia, 47500 Bandar Sunway, Selangor Darul Ehsan, Malaysia e-mail: [email protected] V. N. Low e-mail: [email protected] D. Alex Jeffrey Cheah School of Medicine and Health Sciences, Monash University Malaysia, 47500 Bandar Sunway, Selangor Darul Ehsan, Malaysia e-mail: [email protected] Q. Ayub Genomics Facility, School of Science, Monash University Malaysia, 47500 Bandar Sunway, Selangor Darul Ehsan, Malaysia e-mail: [email protected] S. W. H. Lee School of Pharmacy, Gerontechnology Laboratory, Monash University Malaysia, 47500 Bandar Sunway, Selangor Darul Ehsan, Malaysia e-mail: [email protected] School of Pharmacy, Taylor’s University Lakeside Campus, Subang Jaya, Selangor Darul Ehsan, Malaysia © The Author(s), under exclusive license to Springer Nature Switzerland AG 2023 V. G. Duffy et al. (eds.), Human-Automation Interaction, Automation, Collaboration, & E-Services 11, https://doi.org/10.1007/978-3-031-10784-9_25

415

416

P.-L. Teh et al.

policy makers to frame and tackle some of the biggest challenges to develop an agefriendly city in the developing countries. Implications for organizational research are also discussed. Keywords Age-friendly city · Developing country · Mobility · Older people · Qualitative

1 Introduction and Background Society is increasingly becoming older, due to improved health services and better quality healthcare. It is estimated that there were 703 million people aged 65 years and older in 2019 and this figure is projected to double to 1.5 billion by 2050 [1]. With advancing age, it becomes increasingly important that older adults maintain mobility. Mobility impairment has been shown to be a strong predictor for hospitalisation and mortality, and is inversely associated with the quality of life [2, 3]. As such, the living spaces around older adults will need to be adapted to meet their needs as they age. The importance of this agenda is widely recognised, and the United Nations (UN) has set this to be an important key policy parameter where it has called for broad commitments for cities to be age-inclusive [4, 5]. The recent UN New Urban Agenda has laid out the various standards and principles needed in the planning, construction and development of urban spaces, and specifically calls for urban transportation to be made safe, efficient, sustainable and accessible for all [4]. As such, an understanding of the relationship between population ageing characteristics and the changes in living spaces around them are needed to ensure that a supportive environment can be provided to support healthy ageing. Older adults will experience mobility issues due to a decline in their health, vision or hearing abilities. This will lead to restricted life space mobility and they will be more reliant on their immediate surroundings to maintain independent mobility [6–8]. Several studies have sought to understand this phenomenon better. Older adults are also more vulnerable to the challenges of the built environment due to different mobility needs. In addition, older adults tend to experience a decline in cognitive and mental abilities, and a smaller social network which affect their ability to cope with environmental demands [9, 10] Most of the current literature has focused primarily on high income countries and the younger population, with limited studies from low-middle income countries [9, 11–14]. One of the key gaps that has been highlighted is the lack of research on how mobility, the built environment and ageing affects access to transportation. Indeed, as highlighted by various studies, an understanding of the requirements of older adults is needed, especially during the design stage of a city since any critical flaw can be addressed early and avoid costly design changes later [15]. More importantly, there is a need to better understand the complexities of older people’s everyday assemblages in greater depth. In this study, we examined how we could ensure the “Age Friendly

Designing for Me! What Older Dwellers’ …

417

City” concept, which looks at how urban spaces can be reconfigured to support healthy and active ageing.

2 Literature Review The research field of human factors and ergonomics (HFE) has developed broadly since its inception 70 years ago and has spawn a stream of specializations such as “design for health, safety, and comfort” and “design for individual differences” [16]. According to Karwowski [17], HFE is a discipline that focuses on the nature of human–artifact interactions, viewed from an integrated perspective of the science, engineering, design, technology, and management of human compatible systems, including both natural and artificial products, processes, and physical environments. Research relevant to the field of HFE encompasses basic research such as understanding the effects of ageing on reaction time and applied research such as the understanding of multimodal cues improve visual search performance in air traffic control [18]. Examples of contemporary HFE include the focus of Technical Groups of the Human Factors and Ergonomics Society, as listed in Table 1 [19, 20]. Table 1 Technical groups of human factors and ergonomics society (selected groups only) No Technical group

Description/areas of concerns

1

Aging

Human factors applications appropriate to meeting the emerging needs of older people and special populations in a wide variety of life settings

2

Environmental design

Relationship between human behavior and the designed environment. Common areas of research and interest include ergonomic and macroergonomic aspects of design within home, office, and industrial settings. An overall objective of this group is to foster and encourage the integration of ergonomics principles into the design of environments

3

Individual differences

A wide range of personality and individual difference variables that are believed to mediate performance

4

Safety

Development and application of human factors technology as it relates to safety in all settings and attendant populations. These include, but are not limited to, aviation, transportation, industry, military, office, public building, recreation, and home environment

5

Surface transportation Human factors related to the international surface transportation field. Surface transportation encompasses numerous mechanisms for conveying humans and resources: passenger, commercial, and military vehicles, on- and off-road; mass transit; maritime transportation; rail transit, including vessel traffic services (VTSs); pedestrian and bicycle traffic; and highway and infrastructure systems, including intelligent transportation systems (ITSs)

418

P.-L. Teh et al.

The topic of design for human disability and aging is often regarded as a special topic with common misconception that this group of population is small [21]. However, the ageing of population is a global challenge that is widespread in both developed and developing countries [22]. In the context of HFE, there are age-related differences between younger and older adults which require specific design considerations [23]. Nevertheless, there are several unanswered questions regarding the impact of global population ageing for system design within the domains of transportation and health care [18]. To this end, the study on age-friendly design for older adults is critical. Figure 1 shows the trend analysis of publications on older adult, mobility and sustainable transportation over 15 years. The review targeted all articles published between 2006 and 2020 with keywords “older adult”, “mobility” and “sustainable transportation”. The search was undertaken in Springer AuthorMapper [24]. Based on the abovementioned search terms, a total of 2879 publications were found. At the outset, there were only eight articles (0.28%) published in 2006. There was about 3% increase of publications from year 2016 (256, 8.89%) to 2017 (341, 11.84%). It was observed that the highest number of the articles (640, 22.23%) was published in the year 2020, followed by 483 (16.78%) publications in year 2019. Overall, the evolution of publications on older adult, mobility and sustainable transportation shows a rising trend. In addition, the publications trend on “older adult”, “mobility” and “age-friendly city” was computed from year 2006 to 2020 using Springer AuthorMapper [24]. Referring to Fig. 2, a total of 405 indexed articles was generated through the keywords “older adult”, “mobility” and “age-friendly city.” Notably, there was no publication between 2006 and 2007. Two articles were published in 2008 and 2009, respectively. The number of publications was increased from year 2015 (21 articles, 5.19%) to year 2016 (43 articles, 10.62%). The highest number of articles (94, 23.21%) was

Fig. 1 Evolution of publications on older adult, mobility and sustainable transportation

Designing for Me! What Older Dwellers’ …

419

Fig. 2 Evolution of Publications on Older Adult, Mobility and Age-friendly City

published in the year 2020. In sum, the evolution of publications on older adult, mobility and age-friendly city depicts an upward trajectory. The trend analysis of publications between 2006 and 2020 provides fresh insights into the recent research patterns and focus areas. Namely, studies focusing on ageing, mobility, sustainable transportation and age-friendly city has witnessed a significant rise in 2016. This finding is not surprising, given that 2016 marked the first year of the implementation of the United Nation Sustainable Development Goals (SDGs) [25]. This global agenda which comprises a set of 17 goals to end poverty, protect our planet and ensure sustainable development for all, has guided scholars to tackle some of these grand challenges through their research. The SGDs are propelling multidisciplinary scholars from business, engineering, human factors and ergonomics to achieve these targets through collaborative and coordinated effort. Of the 17 SGDs, one of them (SDG 11 Sustainable Cities and Communities), is devoted specifically to sustainable development, making cities, transport systems and mobility inclusive and sustainable for all, with special attention to the needs of women, children, people with disabilities and older adults. Given that developing countries are ageing rapidly, the challenges faced by older adults are more severe in developing countries compared with developed countries. Of the developing countries, Malaysia is of particular interest in this study because it has seen in rapid urban development and achieving SDGs in Malaysia is a high priority. Against this backdrop, this study draws on human factors literature to understand the older dwellers’ perceptions and reactions in relation to the concept of age-friendly city in Malaysia through focus groups and interviews.

420

P.-L. Teh et al.

3 Materials and Methods 3.1 Sampling We used a purposive sampling to recruit participants in this study. Participants were eligible if they were city dwellers living in Sunway City, a medium sized township with a population of 200,000 residents [26]. This site was chosen since it has a transect of population typical of those who live in Malaysia. The site includes a diversity of built environment, and infrastructure which provided us with a range of mobility challenges and opportunities. Our study participants were aged 50 years and above, as we intended to include individuals with mobility issues related to transition in social circumstances, health and income [27]. Participants were recruited via advertisements, talks with residential associations and during community engagement activities. All potential participants were contacted, and purpose of the study explained. Written informed consent was obtained prior to their inclusion in this study. The Monash University Human Research Ethics Committee approved the study (Review Reference: 2020-19,083-41,584).

3.2 Field Experiment All participants were provided with a smartphone loaded with the “Sunway Free Shuttle Bus” application and pedometer (Fitbit Alta HR) for the session. The field experiment was held outdoors, covering three locations, namely Monash University Malaysia, Sunway Medical Centre and Sunway Pyramid Mall (see Fig. 3). Participants began by taking a bus to the mall, which included a short walk from the station to the mall and then back. Participants were then requested to complete a face-to-face focus group interviews and a survey questionnaire. Each participant was compensated (approximately US$ 16 per person) for their time and effort.

3.3 Focus Group Discussions and Interviews The purpose of the focus groups and interviews was to understand the participating older dwellers’ concerns, perceptions, reactions, observations, and thoughts in connection with the different transport systems and services offered in Sunway City. A detailed set of open-ended questions was used to guide these focus groups and interviews (see Appendix). Participants were asked to share their travel experience within Sunway City, and their perception of the age-friendly transportation design in Sunway City (e.g., What do you think about the priority seating for older people in Sunway City? Can you give some examples?). We also asked participants to identify concerns they had throughout the field trip (e.g., What aspects do you like and

Designing for Me! What Older Dwellers’ …

421

Fig. 3 Travel Route from Monash University Malaysia to Sunway Lagoon Theme Park. The green line shows the route travelled by the BRT while the blue dots indicate the distance travelled by walking. (Source Google Maps)

dislike about Sunway Free Shuttle Bus?). At the end of each focus group/interview, we requested participants to share other details that the participants felt were relevant. All focus groups/interviews were audio-recorded and transcribed verbatim. Each focus group discussion and individual interviews lasted approximately 30–60 min each. All qualitative data were analysed using a grounded theory approach [28, 29]. Drawing from the grounded theory, the analysis began with open coding, followed by axial coding and selective coding. Two researchers were responsible for the coding process. One researcher checked and coded the initial analysis of all transcripts, and another researcher independently reviewed and coded the transcripts. To address discrepancies in the interpretation of data, comparisons and iterative discussions were held between both researchers until agreement was reached.

4 Results A total of nine focus groups and ten individual face-to-face interviews were held in the Gerontechnology Laboratory between October 2019 and January 2020. The study included a total of 32 participants, which were mostly females (n = 21) with a median age of 64 years (range: 50–84 years). Most of the participants were married (n = 22) and had at least a degree in tertiary education (n = 31). Only ten participants were still working at the time of the interview. We provide an overview of participant

422

P.-L. Teh et al.

Table 2 Participant demographic characteristics Frequency (n = 32)

Percentage (%)

Male

21

65.6

Female

11

34.4

Married

22

68.8

Divorced

2

6.3

Never married

4

12.5

Widowed

4

12.5

Postgraduate

5

15.6

Degree/Professional

9

28.1

Diploma/Pre-U

8

25

Secondary

9

28.1

Primary and below

1

3.1

Retired

19

59.4

Working full time

6

18.8

Working part time

4

12.5

Others

3

9.4

Chinese

24

75

Indian

4

12.5

Malay

2

6.3

Others

2

6.3

Variables

Example

Median age, years

64 (range 50–84)

Gender, n Marital status

Employment status

Ethnicity

demographic characteristics recruited from Sunway City, specifically age, gender, marital status, education, employment status, and ethnicity in Table 2. The analyses from the coded data were complex and suggest the interconnectivity and interaction between multiple factors. For example, mobility restrictions were not typically a result from a single cause, but an interaction between various personal and environmental factors. Many of these older adults tend to travel less outside their neighbourhood, as they approach the age of retirement. This coupled with poorer health makes their immediate environment important in order to ensure healthy ageing. These factors can be broadly summarised into the following main categories: (1) Transportation system; (2) Land pattern use and urban design; and (3) Health status.

4.1 Transportation System Several traffic related characteristics have been associated with better mobility and higher willingness to travel among older adults. In the current study, the availability of

Designing for Me! What Older Dwellers’ …

423

street walking paths and trails were found to be of importance to the study participants as it served as a promoter of mobility to older adults. Yea[h], I think it’s very important to live in a walkable community or a public transport so community and I’m super impressed with the connectivity of Sunway City. PID_005, S8_G08 It’s very easy, it’s very convenient. I think [you have] the walkway there and [the walkway is] connected to the free shuttle bus. So, that’s very good, very good even for senior citizens as well as youngsters. I [as] a senior personal[ly] feel very comfortable. PID_016, S7_G07

In addition, the presence of nearby transit stops which allowed for easy access and connectivity to areas outside the neighbourhood was a factor which increased mobility among older adults. [..because I can [get from the] bus stop here to [train station] KTM, there is bus that go to KTM. Then there are buses going to Puchong, then there are buses going to Klang, then there are buses going to KL. [So there is not] only one bus, there are a lot of buses, so the transportation here is good. PID_060, S19_G19 […. ever since they have this BRT service (new bus service) open service open about few years back, I find it very convenient especially to the Sunway Pyramid area (shopping high street). [Traffic around the area can be] very bad traffic not to mention the parking… [I] sometimes can’t even get any parking for half an hour or inside the parking area itself. So, if possible I will actually take LRT up to the BRT stop and then from the BRT stop to Sunway Pyramid. PID_006, S5_G05

4.2 Land Usage and Urban Design Proximity to particular destinations has been widely assessed as a promoter of mobility among older adults. Presence of destinations may increase mobility by providing locations for recreational walking, or by providing access to needed services, such as grocery stores. Other key features which will promote mobility are the presence of age-friendly designs incorporated into the landscape including disability friendly features. Because one thing is that there are steps to go up to the appropriate places to sit and as you walk in one thing I notice that there are space for the handicapped to park their wheelchair, … PID_084, S15_G15

However, several older adults have identified several physical infrastructure problems as well as the behavior of vehicle drivers who often parked on pathways thereby reducing accessibility. Another barrier mentioned was the lack of parking spaces especially among those who are used to driving, which reduced their mobility. …motorbike all they put it in the corridor there cannot walk. Especially it rains worse isn’t it? You have to walk outside of the corridor. The corridor at least [there is] shelter but if they park inside the motorbike then you have to walk outside… PID_069, S18_G18

424

P.-L. Teh et al.

Parking is a nightmare. You see a lot of people actually double park so create a lot of congestion. So the option is to park in [the mall] Sunway Pyramid, and this again is another nightmare. So they are really no choice… Weekdays also Sunway Pyramid is very jam. It is very hard to find parking. I go on weekdays also hard to find parking. PID_032, S9_G09

Some aspects of pavements were problematic for particular types of mobility. For example, tactile surfaces as well as poorly planned stairs, ramp design including their gradient and faulty lifts were mentioned as problematic particularly for those older people using walking aids. And then there is no proper steps la, especially for old people that is quite a high curb. PID_027, S10_G10 [I feel] the step should be broaden, it is too narrow, very difficult. Because of the balancing. PID_060, S19_G19 But I still prefer no step. I mean for me la. I rather walk the other one, the one without the steps. Ah yeah the ramp. But the only thing is the ramp if they got rail it will be better, then at least the whole time you can hold onto the rail. Because when you have ramp our knee cap is not so taxing. PID_060, S19_G19

In relation to systems, bus routing, reliability and frequency of service were commonly experienced problems, particularly for many older people, as using this mode of transportation was a key part of their mobility, due to its lower cost. A key aspect that emerged was in relation to the security of the area which influenced the decision for older adults to choose to live in a particular area. For most older adults, security was ranked as a key requirement for a liveable city. Very good. No traffic, no obstacle, no children running around, you know that kind of thing. It’s good, it’s good. And I don’t have to worry about traffic weaving in and out or motor cycle or bicycle also whatever, you know. It’s good. PID_084, S15_G15 [The area has] CCTV all over, so security wise it is… Very nice. You realize or not there is all CCTV camera not just too far from each other, you don’t realize that? PID_035, S9_G09 … [I feel] safe [as there] isn’t [any] snatch thieves [around the area]… so quite safe I think so. PID_023, S17_G17 You have all the security, you have all the police bantuan, you feel safe also when you come down here. PID_008, S4_G04

Several older adults also took into consideration how well the city planning was, in their decision to choose a liveable city. In particular, the urban landscaping was one of the key factors that older adults took into consideration, as they felt it promoted their mobility and could help lead to a better, healthier, lifestyle. Yeah I think it’s in terms of conception and also location and in terms of the way the city has been constructed taking into account of you know floral and you know taking into accounts into a people wants to live in a city where it’s less polluted, more greenery, more ability to walk, you know breathe fresh air. PID_019, S14_G14

Some older adults highlighted that one aspect that was often overlooked in urban planning was the lighting around the area or lack thereof, which they felt became a hindrance for them to move about, especially at night.

Designing for Me! What Older Dwellers’ …

425

This city is very good in concept. It’s well planned, wide streets, but it is based on data that is out of date. PID_004, S2_G02 [I} think at night is very dark, the area. The stations [and getting up to the walkways], that area is very dark. PID_027, S10_G10

Several participants also suggested the need for better and clearer signages to be erected around the city, in order to improve mobility, as they felt these were insufficient or were poorly designed and positioned on the road. … signage is ok. But, very important don’t block by the trees. Do not let the trees block the signage. PID_027, S10_G10 To me, [the signages are] a bit small especially near the…[especially when I get] near the inter junction there they have the signage but the wordings are a bit small. [I would] have appreciate if the wordings is a bit bigger… PID_016, S7_G07

4.3 Health Status The older adults also described how they managed to navigate around with their declining health status. Several participants described how they had to actively adapt to mobility, and took a more positive attitude to adapt to the existing built environment to support their mobility. I like the canopy walk… because [when I] walk [it] is good for health. Every morning I walk. Like half an hour… PID_083, S13_G13

Yet for some others, the existing built environment felt like it hindered them to travel, and restricted their mobility. For people who are aged it’s quite difficult to walk …. [Look at me now, I am] panting … PID_017, S7_G07 but the distance part is the thing that for old folks, the distance alone will deter them already. When they see a, imagine you look from one end the minute you exit the BRT you look to the other, wow the walkway is really long, yes. PID_084, S15_G15 … we tried to use the lift and we were told the lift was not working. We tried to use the escalator and it is under maintenance … we have to walk the staircase. [and imagine if it was an] older adult or those who are disable. PID_027, S10_G10

5 Discussion Mobility is often measured using quantitative techniques that assess physical function or activities of daily living [30], which often do not include the more subjective aspects of mobility that are fundamental to understanding the needs of older adults. For our study, we found that there was a mixture of complexity of issues related to mobility, well-being as well as built environment, with particular reference to

426

P.-L. Teh et al.

older adults living in an urban environment. Despite the well-planned services and infrastructure in Sunway City, we found that one very important element which can undermine or enable mobility is people’s behaviour. For example, we noted that some unacceptable behaviours such as parking across sidewalks or double parking can impact another’s mobility, highlighting the importance of mindfulness. The findings also highlight the need for a holistic approach towards developing urban spaces in order to enhance mobility among older adults. This needs to be done taking into consideration not only the quality of the place, but also how these interact with older adults as well as better public infrastructures that need to be in place to support mobility. Other issues that were highlighted in the study include problems experienced by those with specific conditions, such as those with vision impairments or physical disability. This group identified particular factors and locations which they felt were problematic to their mobility not identified by others without these conditions. Studies to date [31] have shown that there are some key benefits in co-designing cities to ensure it is sustainable and promotes healthy ageing. Age and socioeconomic status have been identified in prior studies as one of the issues which affect mobility among older adults [32–34]. In this study, we did not find such an association between gender or those who are 65 and above versus 64 and below. A previous study conducted in Malaysia found that age and gender were significant factors in influencing choice of transportation [6, 35]. Our study departs from the common scholarly findings of this association, possibly due to the small sample of participants recruited. Indeed, in this study, the authors noted that men typically made more journeys than women, due to their role as the head of their household [6]. In our study, we noted how government policies, such as investment in public transportation impacted the lives of older adults [36]. They spoke of how the car ownership policy and reduced investment in bus services and pavement repair was increasing car dependence, making it difficult for older adults to move about [37].

6 Conclusion This study has several important practical implications. First, the qualitative nature of the study provides us with a contextual picture of the barriers and facilitators towards mobility in an ageing population. For example, we identified the need for some physical infrastructure, such as the location of benches or the quality of pavements which would not have been identified using quantitative methods. Given that issues are not readily identifiable from the datasets, they are not likely to be wellconsidered in decision-making relating to age-friendly spaces. Results of this study can also be used by future studies which aim to improve mobility during their design to promote a more complete and comprehensive solution towards mobility. Most studies [38] only aim to improve physical functioning or exercise levels, and rarely focus on improving aspects such as better connectivity to a desired location. Second, the study investigators comprised a cross-disciplinary group, including gerontology,

Designing for Me! What Older Dwellers’ …

427

pharmacy, business and public health. This helped facilitate the interpretation to be more interdisciplinary. This study is not without its limitations. First, this study took a qualitative approach focusing on the investigation of how urban spaces can be reconfigured to support healthy and active ageing, which is of high relevance for supporting age-friendly cities implementation. Our perspective should be complemented by further investigations using documentation and case study approaches. Second, the small sample size (n = 32) may affect the generalizability of the findings. While our study highlighted the growing breadth and depth of the urban aging and mobility literature, we call for more empirical research using larger data to extend our understanding of this real-world issue. Acknowledgements We wish to acknowledge the financial support of Monash University Malaysia Sustainable Community Grant Scheme (Project code: SCG-2018-02-SCI). We would also like to thank the participants for their enthusiastic support of this research project. The names of individuals have been changed to protect their identities. No other alterations have been made in the data presented. Special thanks are extended to Sunway for their support to recruit participants, Kristel Tan and Muhammad Zarul Hanifah Md Zoqratt for their valuable research assistance.

Appendix Interview questions 1. I would like to ask you some questions about your travel experiences within Sunway City: (a) Can you describe how was your travelling experience within Sunway City? (b) If needed, please ask regarding the: (i) Environment. (ii) Linkage between buildings. (iii) Challenges/Difficulty. (c) How did you solve those difficulties? (e.g., find alternative) (d) Think about your travel experience just now. Can you describe what you liked about traveling by public transportation system (i.e. using a smartphone app, walking to BRT station, waiting at the shuttle bus station etc.) within Sunway City? (e) What did you dislike about traveling by public transportation system (i.e. using a smartphone app, walking to BRT station, waiting at the shuttle bus station etc.) within Sunway City? 2. I would like to ask you some questions about Sunway City: (a) What aspects do you like or dislike about the environment in Sunway City (e.g., cleanliness, regulations, noise levels, green spaces and walkways,

428

P.-L. Teh et al.

(b) (c)

(d)

(e)

(f) (g) (h) (i) (j) (k) (l) (m) (n)

outdoor seating, pavements, roads, traffic, safety, services, buildings and public toilets)? Can you give some examples? What do you think about the transportation affordability in Sunway City? Can you give some examples? What do you think about the transportation reliability and frequency in Sunway City? Including services at night and at weekends? Can you give some examples? What do you think about the connectivity of key destinations such as hospitals, health centers, public parks, shopping centers and banks in Sunway City? Can you give some examples? What do you think about the accessibility, cleanliness and communication of signage of public transportation in Sunway City? Can you give some examples? What do you think about the specialized transportation services for people with disabilities in Sunway City? Can you give some examples? Probe the following if not given: What do you think about the priority seating for older people in Sunway City? Can you give some examples? What do you think of the public transportation’s drivers in Sunway City? What do you think of the safety and comfort of public transportation in Sunway City? Can you give some examples? What aspects do you like or dislike about the transport stops and stations in Sunway City? Can you give some examples? What do you do think of the communication of travel information to older people in Sunway City? What aspects do you like or dislike about the taxis in Sunway City? What aspects do you like or dislike about the roads in Sunway City? Can you give some examples? What aspects do you like or dislike about the parking in Sunway City? Can you give some examples?

3. I would like to ask you some questions about your opinion about Bus Rapid Transit (BRT) within Sunway City: (a) What is your first impression of the Bus Rapid Transit (BRT)? (b) Do you think BRT is easy for people to use? Why? In particular, think about what it would be for an older adult or those who have mobility issues (c) Do you feel that BRT will be comfortable for people to use? Why? (d) Do you feel that BRT is convenient? How? (e) What aspects do you like and dislike about BRT? (f) What aspects of BRT need improvement? (g) Do you feel that BRT can help to improve mobility within Sunway City? How? 4. I would like to ask you some questions about your opinion about Sunway Free Shuttle Bus within Sunway City: (a) What is your first impression of the Sunway Free Shuttle Bus?

Designing for Me! What Older Dwellers’ …

429

(b) Do you think Sunway Free Shuttle Bus is easy for people to use? Why? (c) Do you feel that Sunway Free Shuttle Bus will be comfortable for people to use? Why? (d) Do you feel that Sunway Free Shuttle Bus is convenient? How? (e) What aspects do you like and dislike about Sunway Free Shuttle Bus? (f) What aspects of Sunway Free Shuttle Bus need improvement? (g) Do you feel that Sunway Free Shuttle Bus can help to improve mobility within Sunway City? How? 5. I would like to ask you some questions about your opinion about elevated covered canopy walk within Sunway City: (a) (b) (c) (d) (e) (f)

What is your first impression of the elevated covered canopy walk? Do you think elevated covered canopy walk is easy for people to use? Why? Do you feel that elevated covered canopy walk is convenient? How? What aspects do you like and dislike about elevated covered canopy walk? What aspects of elevated covered canopy walk need improvement? Do you feel that elevated covered canopy walk can help to improve mobility within Sunway City? How?

References 1. United Nations, https://www.un.org/en/development/desa/population/publications/pdf/ageing/ WorldPopulationAgeing2019-Highlights.pdf. Last accessed 19 Dec 2020 2. Hirvensalo M, Rantanen T, Heikkinen E (2000) Mobility difficulties and physical activity as predictors of mortality and loss of independence in the community-living older population. J Am Geriatr Soc 48(5):493–498 3. Shafrin J, Sullivan J, Goldman DP, Gill TM (2017) The association between observed mobility and quality of life in the near elderly. PLoS ONE 12(8):e0182920 4. Parnell S (2016) Defining a global urban development agenda. World Dev 78:529–529 5. World Health Organization (2007) Global age-friendly cities: a guide. World Health Organization, Geneva 6. Ang BH, Lee SWH, Oxley J, Yap KK., Song KP, Kamaruzzaman SB, Chin AV, Tan KM, Khor HM, Chen WS (2019) Self-regulatory driving and riding practices amongst older adults in Malaysia. Transp Res Part F Traffic Psychol Behav 62:782–795 7. Hawkesworth S, Silverwood RJ, Armstrong B, Pliakas T, Nanchalal K, Jefferis BJ, Sartini C, Amuzu AA, Wannamethee SG, Ramsay SE, Casas J-P, Morris RW, Whincup PH, Lock K (2017) Investigating associations between the built environment and physical activity among older people in 20 UK towns. J Epidemiol Commun Health 72(2):121–131 8. Kerr J, Rosenberg D, Frank L (2012) The role of the built environment in healthy aging: community design, physical activity, and health among older adults. J Plan Lit 27(1):43–60 9. Rosso AL, Auchincloss AH, Michael YL (2011) The urban built environment and mobility in older adults: a comprehensive review. J Aging Res 2011(2011):816106 10. Yen IH, Michael YL, Perdue L (2009) Neighborhood environment in studies of health of older adults. Am J Prev Med 37(5):455–463 11. Firth CL, Stephens ZP, Cantinotti M, Fuller D, Kestens Y, Winters M (2021) Successes and failures of built environment interventions: using concept mapping to assess stakeholder perspectives in four Canadian cities. Soc Sci Med 268:113383 (1982)

430

P.-L. Teh et al.

12. Jones A, Goodman A, Roberts H, Steinbach R, Green J (2013) Entitlement to concessionary public transport and wellbeing: A qualitative study of young people and older citizens in London, UK. Soc Sci Med 1982(91):202–209 13. Mahmood A, Chaudhury H, Michael YL, Campo M, Hay K, Sarte A (2012) A photovoice documentation of the role of neighborhood physical and social environments in older adults’ physical activity in two metropolitan areas in North America. Soc Sci Med 74(8):1180–1192 (1982) 14. Sharmin S, Kamruzzaman M (2017) Association between the built environment and children’s independent mobility: a meta-analytic review. J Transp Geogr 61:104–117 15. Van Hoof J, Kazak JK, Perek-Białas JM, Peek STM (2018) The challenges of urban ageing: making cities age-friendly in Europe. Int J Environ Res Public Health 15(11):1–17 16. Salvendy G (2012) Handbook of human factors and ergonomics, 4th edn. John Wiley & Sons, New Jersey 17. Karwowski W (2005) Ergonomics and human factors: the paradigms for science, engineering, design, technology, and management of human-compatible systems. Ergonomics 48(5):436– 463 18. Czaja SJ, Nair SN (2012) Human factors engineering and systems design. In: Salvendy G (ed) Handbook of human factors and ergonomics, 4th edn. John Wiley & Sons, New Jersey, pp 38–56 19. Karwowski W (2012) The discipline of human factors and ergonomics. In: Salvendy G (ed) Handbook of human factors and ergonomics, 4th edn. John Wiley & Sons, New Jersey, pp 3–37 20. Human factors and ergonomics society, www.hfes.org. Last accessed 19 Dec 2020 21. Vanderheiden GC, Jordan JB (2012) Design for people with functional limitations. In: Salvendy G (ed) Handbook of human factors and ergonomics, 4th edn. John Wiley & Sons, New Jersey, pp 1409–1441 22. Baldwin CL, Lewis BA, Greenwood PM (2019) Designing transportation systems for older adults. CRC Press, Boca Raton 23. Boot WR, Nichols TA, Rogers WA, Fisk AD (2012) Design for aging. In: Salvendy G (ed) Handbook of human factors and ergonomics, 4th edn. John Wiley & Sons, New Jersey, pp 1442–1471 24. Springer AuthorMapper, https://www.authormapper.com/. Last accessed 20 Dec 2020 25. United Nations, https://www.un.org/development/desa/disabilities/envision2030.html. Last accessed 20 Dec 2020 26. Sunway Group, https://ir2.chartnexus.com/sunway/doc/sustainability-reports/sr2018.pdf. Last accessed 20 Dec 2020 27. Robertson GK (2014) Transitions in later life: a review of the challenges and opportunities for policy development. Working Older People 18(4):186–196 28. Corbin JM, Strauss A (1990) Grounded theory research: procedures, canons, and evaluative criteria. Qual Sociol 13(1):3–21 29. Glaser B, Strauss AL (2017) The discovery of grounded theory: strategies for qualitative research. Routledge, New York 30. Soubra R, Chkeir A, Novella J-L (2019) A systematic review of thirty-one assessment tests to evaluate mobility in older adults. Biomed Res Int 2019:17 31. Cinderby S, Cambridge H, Attuyer K, Bevan M, Croucher K, Gilroy R, Swallow D (2018) Codesigning urban living solutions to improve older people’s mobility and well-being. J Urban Health 95(3):409–422 32. Ang BH, Oxley JA, Chen WS, Yap KK, Song KP, Lee SWH (2019) To reduce or to cease: a systematic review and meta-analysis of quantitative studies on self-regulation of driving. J Safety Res 70:243–251 33. Eronen J, von Bonsdorff M, Rantakokko M, Portegijs E, Viljanen A, Rantanen T (2016) Socioeconomic status and life-space mobility in old age. J Aging Phys Act 24(4):617–623 34. Rantakokko M, Iwarsson S, Portegijs E, Viljanen A, Rantanen T (2015) Associations between environmental characteristics and life-space mobility in community-dwelling older people. J Aging Health 27(4):606–621

Designing for Me! What Older Dwellers’ …

431

35. Ang BH, Jennifer O, Chen WS, Lee SWH (2019) Factors and challenges of driving reduction and cessation: a systematic review and meta-synthesis of qualitative studies on self-regulation. J Safety Res 69:101–108 36. National Automotive Policy 2020: Ministry of International Trade and Industries, Kuala Lumpur 37. Mohd Jawi Z, Lamin F, Abdul Manap AR, Abu Kassim KA, Abas F, Wong SV (2012) Review of the national automotive policy on car maintenance issues: Malaysia’s automotive ecosystem explained. Malaysian Institute of Road Safety Research Review Report, Kuala Lumpur 38. Lee JLC, Ho RTH (2020) Exercise spaces in parks for older adults: a qualitative investigation. J Aging Phys Act 1–9

A Literature Review of Technological Trends in Urban Logistics: Concepts and Challenges Bruno Machado, Carina Pimentel , Amaro Sousa , Ana Luísa Ramos , José Vasconcelos Ferreira , and Leonor Teixeira

Abstract With the recent increasing of e-commerce worldwide, high number of urban deliveries are made every day, typically performed by companies that use fleets powered by fossil fuels. Thus, urban logistics is facing problems to control its negative impacts to the quality of life in urban areas, such as pollutions, noise, traffic congestion and accidents. These difficulties enhance the importance of adopting new technologies and concepts in urban logistics context to increase its performance. This chapter aims to identify some concepts and technological trends that are being adopted and how they impact the performance of urban logistics. To achieve this goal, a literature review is performed to understand and detail what are these technologies and concepts. Lastly, as a result of this chapter, a framework is presented with the

B. Machado Department of Economics, Management, Industrial Engineering and Tourism (DEGEIT), University of Aveiro, 3810-193 Aveiro, Portugal e-mail: [email protected] C. Pimentel · A. L. Ramos · J. V. Ferreira Governance, Competitiveness and Public Policies (GOVCOPP), Department of Economics, Management, Industrial Engineering and Tourism (DEGEIT), University of Aveiro, 3810-193 Aveiro, Portugal e-mail: [email protected] A. L. Ramos e-mail: [email protected] J. V. Ferreira e-mail: [email protected] A. Sousa Instituto de Telecomunicações (IT), Department of Electronics, Telecommunications and Informatics (DETI), University of Aveiro, 3810-193 Aveiro, Portugal e-mail: [email protected] L. Teixeira (B) Institute of Electronics and Informatics Engineering of Aveiro (IEETA), Department of Economics, Management, Industrial Engineering and Tourism (DEGEIT), University of Aveiro, 3810-193 Aveiro, Portugal e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Switzerland AG 2023 V. G. Duffy et al. (eds.), Human-Automation Interaction, Automation, Collaboration, & E-Services 11, https://doi.org/10.1007/978-3-031-10784-9_26

433

434

B. Machado et al.

technologies and concepts and the areas of their contributions, to achieve better urban logistics. Keywords Urban logistics · Sustainability · Technological trends · Framework

1 Introduction The continuously increasing share of population living in urban areas and the rise of online internet sales (e-commerce) reinforce the concern and importance of adopting solutions that help urban areas to improve their quality of life. Freight and transport have been considered as a significant disturbance to the quality of the urban spaces, especially near areas of historical and artistic value [1]. According to [1] in 1950 only 30% of the global population was living on urban areas, but it is expected that in 2050 this value increases to 66% and in 2100 it reaches 85%, representing an increase of 8 billion of people living is such areas. According to reports by eMarketer (May 2019), annual B2C e-commerce sales reached 3.5 trillion dollars in 2019, 20% more than the previous year, and it is expected to reach 6.5 trillion dollars in 2023. It is important to note that the increase in the transport of goods to the house of consumers instead of conventional retail stores will result in a significant increase in logistics transport vehicles and, consequently, in an increase in urban traffic. This increase in traffic contributes significantly to the increase of traffic congestion, noise and environmental pollution, road accidents and the emission of greenhouse gases [2]. Additionally, new services and startups have emerged to respond to the growing market segment where end consumers or companies that buy online intend to receive their goods in less than an hour or two, already representing about 2.5% of all transport within large cities [1]. Also, according to these authors, the increase in this type of instant transport leads to a significant increase in the number of vehicles operating in city centers. Thus, although the main focus of urban logistics is on the main carriers operating in the market, it is necessary to think of solutions for urban logistics transversal in the supply chain, making it integrated and sustainable [3]. Consequently, in order to potentially decrease the negative effects on congestion, safety, and environment, nowadays, the urban logistics is introducing creative, innovative, and out-of-the-box concepts and ideas at all levels of system design, operations planning, and real-time execution [4]. From a technological perspective, [4] discuss digital connectivity, big data, and automation, automotive technology, and unmanned aerial vehicles as advances in technology. This paper aims to perform a literature review about the concepts and technological solutions that are being adopted on urban logistics field. The research questions considered are the following: RQ1: What are the main concepts and technological trends supporting urban logistics?

A Literature Review of Technological Trends in Urban Logistics…

435

RQ2: What are the relationships between these technological trends and concepts and urban logistics’ dimensions, namely, regulation and policies, sustainability, operational excellence, collaboration and digitalization. The remaining of this paper is structured as follows: Sect. 2 provides an overview of the concept and challenges of urban logistics field; Sect. 3 details the methodology adopted in this work; Sect. 4 describes the results with Sect. 4.1 being detailing the concepts and technologies trends found in the literature that are supporting urban logistics, and Sect. 4.2 proposing a framework to explain how these technologies and concepts are influencing the urban logistics topic. The work finishes with a Sect. 5 of conclusions of the research.

2 Urban Logistics: Overview, Concept and Challenges Urban logistics, also referred to as urban (freight) distribution, last mile logistics, city logistics, or city distribution [4] is considered by Dolati Neghabadi et al. [5] one of the most argued concerns in most cities around the world regarding recent phenomena such as urbanization or the rise of citizen’s expected welfare level. A large number of driving changes contributed significantly to the steady growth of city logistics. Savelsbergh and Van Woensel [4] include population growth and urbanization, the growing importance of e-commerce, the desire for speed in supply chains, the rise of the sharing economy, and the increased attention to climate change and sustainability. Lagorio et al. [6] points the continuously increasing share of the population living in urban areas, the concerns for pollution and safety in cities and the issues of traffic and congestion. Bosona [7] highlights the increase of Internet infrastructure and growth of e-commerce, during the past two decades. Urban logistics is being introduced as a popular term whose solutions aim, on the one hand, to reduce all of the aforementioned negative impacts and, on the other hand, to offer better and faster deliveries [6]. However, even though urban logistics has been investigated for several years, there is not an universal definition to it [6]. According to [4] all existing definitions have in common that urban logistics is about finding efficient and effective ways to transport goods in urban areas while considering the negative effects on congestion, safety, and environment. Accordingly to [6], urban logistics subject is still evolving because of the continuous changes in citizens habits and the unceasing technological evolution enabling new delivery scenarios. Indeed, there exist many challenges specific to the urban logistics that makes difficult the implementation of solutions. One of the challenges for the urban logistics is the creation, implementation and operations management of networks to provide a good service at a low cost with better coordination of the flows of goods, higher consolidation of the freight volumes, and multi-organization cooperation [4]. On the other hand, focusing on freight transport, increasingly fragmented demand due to the spread of e-commerce and the synchronization and harmonization of the different flows of goods are pointed by Lagorio et al. [6].

436

B. Machado et al.

3 Methodology The Literature Review (LR) is selected as the research method for this study due to the nature of the research questions, aiming to understand the trending concepts and technologies supporting urban logistics and how they influence it. LR was performed based on the approach presented in [8] in which the main stages are: (1) identify a topic of interest and spend time identifying keywords, (2) using keywords to conduct a search of relevant literature, (3) review all references sourced and retrieve a copy of relevant references, (4) read all relevant sourced literature and identify new references through citations and, lastly, (5) organise all material in preparation for analysis and integration in the review. Although the main searching procedure presented above is a 5-step process easy to understand, the steps 1, 2 and 3 have to be further detailed. Step 1: Identify a topic of interest and spend time identifying keywords. The topic of interest for investigation was urban logistics. The keywords selected for the query were based on the article [6] and they are: “urban logistics”, “city logistics”, “urban delivery”, “last mile delivery” and “urban freight”. Since the research objective is to identify the technologies and concepts that support urban logistics and its automation, the term “concept” and the truncation “technology*” were added as keyword. Step 2: Using keywords to conduct a search of relevant literature. After selecting the keywords for the study, some criteria had to considered to be conduct the search of relevant literature. The database Scopus was used to perform the search. As the first filter, only journal articles were considered to do the search. Secondly only Social Sciences, Engineering, Business, Managing, Accounting, Environmental Science, Decision Sciences, Energy, Mathematics, Economics, Econometrics and Finance areas were considered, excluding areas like Medicine, Neuroscience and others. In addition to these two filters, only studies after 2016 were considered. This decision was based on the existence of two LR [4, 6] published on that year and due to the high number of studies on this topic that have been trending, with a rapidly increasement in 2016. Finally, only studies written in English were considered for this study. As a result of this search, a total of 154 articles were found. Step 3: Review all references sourced and retrieve a copy of relevant references. To identify a sample of relevant articles, the title, abstract and the article itself was read to ensure that it is related to the objective of this research. At the first stage, each title was read and the article would not be considered only if the title mismatches the research objective. After that, each abstract was also read and the same logic of title criteria was applied. If the title and abstract analysis were not sufficient to reach a conclusion, the article was considered and a full paper was analyzed. During this process, the main mindset for this articles’ selection was based on the research questions, always looking for articles that could provide trending concepts and technologies supporting the urban logistics.

A Literature Review of Technological Trends in Urban Logistics…

437

4 Results As a result of the literature review twenty-five articles, with technological trends, were selected and analyzed in detail, and a summary of the main contributions of this set of articles to this research is explored and analyzed in this section. Thus, in the Sect. 4.1 the technological trends that are supporting urban logistics, answering to the research question 1 “What are the main topics and technological trends supporting urban logistics”, will be identified and examined. Next, in the Sect. 4.2, a framework with the relationship between the technologies and the topic urban logistics will be presented, allowing to answer the research question 2, i.e. “What are the relationships between these technological trends and concepts and urban logistics’ dimensions, namely, regulation and policies, sustainability, operational excellence, collaboration and digitalization”.

4.1 Technological Trends in Urban Logistics This subsection details the technologies and trends that are supporting urban logistics that were found on the literature review performed. An overview, with the main technologies/concepts, their definition, the respective papers and their goals, is summarized on Table 1, and then discussed in more detail. Unmanned Aerial Vehicles (also known as drones). Using drones for last mile delivery is gaining popularity, since many large companies such Amazon, FedEx, DHL and UPS are currently investigating the effective use of drones for last mile delivery [12]. This popularity is due to the potential to decrease delivery costs and elimination of congestions costs leading to less miss-deliveries, since the delay from the dispatch to the delivery is very short when compared to truck based deliveries [13]. The research reported by Aurambout et al. [13] focuses on the European market and the economic viability of implementing drone solution for last mile delivery. The main goal of their paper is to provide a reality check of this drone delivery concept and investigate the potential optimal location of the distribution centers to accommodate the landing and take-off of the drones. The conclusion of the study points to the viability of the drone delivery based on distribution centers to perform last mile delivery in many European urban areas, confirming the drone delivery as a trending technology for the next years. On the other hand, Boysen et al. [12]’ study presents an alternative approach to decrease the costs of the network of distribution centers to receive and launch the drones. The authors propose a prototype of a truck-based drone delivery solution, where trucks serve as both a mobile depot, in which the shipments to be delivered are transported, and as a mobile launching platform for one or multiple drones based on the top of the truck. The collaboration of these two types of vehicles is truly important, since the delivery truck moves between different customer locations, performing

438

B. Machado et al.

Table 1 Technological trends found on literature that are supporting urban logistics Technology or concept Description

Goal

Article

Unmanned aerial vehicles (Drones)

Adoption of drones’ technology to perform last mile delivery

– Minimize the operational cost and urban traffic

[9–12]

– Study the market and economic viability of these solutions in Europe

[13]

Different stakeholders share their resources to perform last mile deliveries

– Determine the sustainability potential of crowd logistics

[14–16]

– Provide insights about crowd logistic business models

[17]

– Determine potentials of sharing parking spaces

[18]

– Determine potentials of integration of freight and passenger flows

[19]

– Provide recommendations from an extensive empirical survey with experts

[20]

– Study the costs and sustainability impact

[21]

– Provide customer insights about pick-up parcel lockers

[22, 23]

– Minimize the external and operational costs

[24, 25]

– Optimizes the changing locations to minimize the number of pick-ups

[26]

– Presents the factors that influences the acceptance of autonomous robots

[27]

– Study the efficiency impact of this concept

[28]

Sharing economy

Cargo bikes

Pick-up points

Autonomous delivery robots (ADR)

Platooning van

Utilization of cargo-bikes to perform last mile delivery

Secured location where customer can pick-up their orders instead of being delivered at home

Autonomous robots that perform the last mile delivery from trucks to city centers

Platoons of connected vans – Models and simulates this performing last mile platoon solution to delivery where the firs one is decrease the number of driven and the others are vehicles operating driverless, following the first van instructions

[29]

(continued)

A Literature Review of Technological Trends in Urban Logistics…

439

Table 1 (continued) Technology or concept Description

Goal

Article

New energy logistics vehicles (NELV)

Usage of battery electric vehicles (BEV) to perform last mile delivery

– Study the adequacy and performance of BEV in urban logistics

[30, 31]

Connected cities

Utilization of an open – Enhance the ecological system engaging and and societal potential of interconnecting the actors to interconnectivity solution perform the las-mile delivery

[32]

Cloud-based order fulfilment

Utilization of a new clod-based process to plan orders fulfilment

[33]

– mprove efficiency of orders planning

conventional home deliveries, and the drone simultaneously serves additional near customers, one at a time, returning to the truck after each delivery. The work of [10] studied the Travelling Salesman Problem with Drones (TSPD) where a delivery could be performed by a truck or a drone, but the drone had to be launched and rejoin later the same truck at another location. The objective was minimizing the operational cost of the system, including the transportation cost and the waiting penalties when a vehicle has to wait for the other. Similarly, [9] has substantial savings are possible adopting emerging technology (drone) when collaborating with conventional trucks. Kitjacharoenchai et al. [11] performed a research on the Multiple Travelling Salesman Problem with Drones (mTSPD). On their study, both trucks and drones can perform deliveries. However, some details are different from the previous studies, resulting on different approaches which are: orders being delivered only by conventional trucks, conventional trucks performing deliveries, simultaneously with drones departing from trucks to deliver additional customer returning to an available truck (not necessarily the same), drones performing deliveries directly from the depot and returning to an available truck or the initial depot. The research goal is to model and seek an optimal delivery route in an urban location with the objective to minimize the total cost of deliveries, which consists in the cost of truck travels, the cost of drone travels and the cost of simultaneous truck and drone travels. Results have shown that using multiple drones and trucks provides shorter delivery times than conventional truck deliveries. Some benefits of adopting this technology, based on the considered articles are; (i) faster than trucks; (ii) reduction of delivery costs; (iii) shortest delivery time; (iv) elimination of congestion time; and (v) environmentally friendly solution (reduction of air emissions). Regarding the potential issues adopting this solution, can be pointed out: (i) cargo weight restricted to the weight that drones can carry; (ii) shorter travel range; (iii) drones can only transport one shipment; (iv) drone safety and noise during deliveries; (v) mandatory existence of distribution centers close to the customers location; and (vi) local government policies.

440

B. Machado et al.

Sharing Economy. The growing interest in shared passenger and freight transportations practices indicates that an important opportunity could be reached in combining both. Crowd Logistics (CL), alternative termed crowdshipping originates from the term crowdsourcing which covers both the word “crowd” or a mass of people and “outsourcing” or the shift of processes, functions and duties to third parties [34]. CL is a promising concept as it encourages passengers to use their spare carrying capacity on cars, bikes, buses and planes to carry packages to other people. CL uses the excess capacity on premeditated trips that already take place to make deliveries, leading to maximization of logistics efficiency and reduction of emissions and traffic congestions [35]. The idea is to encapsulate the physical objects in packets and containers. These containers are then routed as efficient as possible, absorbing spare capacity in transport systems and ensuring that they get to their destination in time [14]. Additionally, [17] research was performed to evaluate the nature and characteristics of CL business models and propose a four step process that practitioners need to follow to implement a sustainable crowd logistics service. (1) (2) (3) (4)

Be innovative and try to provide a new added value service for stakeholders; Expect a negative return on investment in the long term; Be informed about country-specific regulations and restrictions; Build up the network as soon as possible.

As an example, [16] proposes a mobile application called CALMeD SURF (Crowdsourcing Approach for Last Mile Delivery) as a practical approach to implement crowd-logistics in an urban area. This application is accessible for two types of users: those who want to deliver a parcel, and those who wish to serve as occasional deliverers in an urban area. The users register in the system, and CALMeD SURF locates them in the city in real-time. Thus, when there is a delivery request, the app uses the geo-localized temporal deliverers, to compute an optimized path for delivering the parcel to its final destination. It is important to highlight that, when calculating an optimal path, multiple objectives are used such as sustainable means, economic issues, temporal constraints, etc. Also, it is possible that the path may be constructed as a chain of collaborative deliverers, passing the parcel to different deliverers along the way. The main objective of this approach is to minimize new emissions originated by path that deviate the deliverer from his/her daily routes. Their results show that this is a feasible approach and it is a feasible solution for last mile delivery. The study of [15] goes further on this topic and present the integration of CL with item sharing. Item sharing is the term for a relationship among a sharing community where members can rent items from one another. This concept is particularly useful for items that are needed on rare or just temporal occasions and the benefits are that multiple members can sequentially use the same item instead of each buying one such item, individually [36]. The research [15] intends to integrate the CL and item sharing into a single platform that has access to information on supplies and requests of items and on announced trips of crowdshippers for an upcoming planning period. This platform will be based on collecting information over a certain period of time rather than on immediately responding to each single incoming request,

A Literature Review of Technological Trends in Urban Logistics…

441

resulting in the advantage of the opportunities for fulfilling more demand. Responses in real-time are not needed but a fast, scalable and high-quality decision making is needed for operating the platform. The main objective of the research is showing the potentials of this joined solution, concluding that this integration of concepts is, generally, profitable but it depends on the crowdshippers’ flexibility to deviate from their original route and the compensation paid to them. Other type of sharing economy concept is investigated by Melo et al. [18]. Their paper analyses if a shared parking solution leads to a better environmental, energy and traffic performance. The solution consists of sharing parking spaces previously used exclusively by city logistic vehicles with other users, for example, parents dropping their kids to school. Since these two flows are, typically, not coincident in time, the same reserved spaces can be used by both. Their results reveal that if the municipality would implement the shared usage of the current exclusive places for urban logistics operations, private freights and public transports would experience a decrease in delays and improvements in their speeds, resulting in improvements on environmental, energy and traffic performance. Lastly, Ozturk and Patrick [19] proposes an integration of urban freight transport and urban rail flow, using the same infrastructures to perform last mile delivery and passenger transportation. The solution was based on gear trains only with goods with the trains of passenger transportation, departing on the same trip. The research goal is to develop a decision support framework for the optimal transportation of freight by urban rail at an operational level. Some advantages of sharing economy referred in the analyzed articles are: (i) reduction of delivery cost; (ii) environmentally friendly solution (reduction of air emissions); and (iii) reduction of traffic congestion. Regarding the potential issues adopting this solution, can be pointed out: (i) hard to monitor the quality and service level; (ii) hard to predict the adherence from the crowd (in case of crowd logistics) to plan delivery services; and (iii) hard to guarantee cargo safety. Cargo-bikes. Cargo-bikes are recently being used to perform last mile deliveries. Typically, a two-wheeled vehicle, can be as fast or even faster than the conventional vans and trucks performing deliveries within a city. This is because they are less affected by traffic congestion, and because they can often take faster routes where trucks and vans cannot go, such as pedestrian streets or bicycle paths [37]. The study [20] provides some recommendations supporting cargo-bikes use at local level, highlighting that the regulations and policies of municipalities play an important role for the use of cargo-bikes. A practical example of cargo-bikes utilization is presented on the study of [21] where they suggest the utilization of cargo-bikes to perform last mile deliveries. They study the synchronization of cargo-bikes with vans to perform last mile delivery within an urban area. The study concludes that emissions can be reduced through the substitution of vans by cargo bikes. Some advantages of utilization of cargo-bikes are: (i) environmentally friendly solution (reduction of air emissions); (ii) faster than vans performing last mile deliveries; (iii) elimination of congestion time; and (iv) low cost of use.

442

B. Machado et al.

Some disadvantages are related to (i) hard to guarantee cargo safety; (ii) need of decent cycle infrastructure; and (iii) limited load capacity. Pick-up Points. To deal with the growing volumes of delivered and returned parcels, increasing customer expectations and toughening market competition, retailers and logistics service providers are exploring and implementing innovative tools such as self-service technologies. In last mile deliveries context this technologies are parcel lockers (also named as locker boxes, automated lockers, self-service delivery lockers or intelligent lockers) used as a self-service collection and return of goods purchased online [22]. The interest by parcel locker networks is increasing and they already represent a significant share of last mile deliveries where the customer plays an active role during the distribution process [38]. The paper [22] has studied the customer value and perspective of the adoption of parcel lockers to pick up their products purchased online. They performed a focus group interview with 26 participants that have been purchasing on-line. To ensure that all participants had the same level of experience dealing with parcel lockers, all have collected and returned a parcel using a parcel locker. With insights from the interview they were able to understand how customer look at parcel lockers on last mile delivery. Yuen et al. [23] perform a similar study identifying that convenience, privacy, security and reliability are important factors to enhance the perceived value of smart lockers by the customers. A practical example of this solution for last mile delivery is presented by Schwerdfeger and Boysen [26] through the study of the potential of mobile parcel lockers compared to stationary parcel lockers. According to the authors, mobile parcel lockers have the advantage of flexibility, changing their locations during the day to where the customer concentration is higher, either autonomously or moved by a human driver. Results have shown that mobile parcel lockers can achieve the same service level of stationary lockers with only ¼ of the lockers number. Orenstein et al. [25] study the utilization of flexible parcel lockers to identify the potentials of this solution. It is called “flexible” because, on their experiments, each customer can be supplied from different parcel lockers with the same effort rate. The goal is to formulate a problem with flexible parcel lockers and determine the number of vehicles, their routes and assigning parcels to vehicles. Results strengthen the conclusion that exploiting the flexibility of parcels lockers makes the delivery process more efficient. Arnold et al. [24] study two different scenarios and compare them with the current situation. The first scenario is the utilization of pick-up points where customer collect their parcels and second scenario is the utilization of cargo-bikes (a concept mentioned above) to perform the last mile delivery to customer houses. Conclusions are that the adoption of pick-up points reduce the operational costs of companies while the implementation of cargo-bike distribution system decrease the congestion and emissions. Some advantages of this solution are: (i) flexible pick-up time; (ii) no missed deliveries; and (iii) shorter delivery routes by logistics service providers.

A Literature Review of Technological Trends in Urban Logistics…

443

Regarding the disadvantages it is possible to list: (i) increasing number of private cars trips to collect the parcels; and (ii) customer dependency to collect the parcels; and (iii) software/hardware potential errors or flaws. Autonomous Delivery Robots (ADV). As a response to the current challenges of city logistics, autonomous deliveries are gaining popularity to perform the last mile delivery. Autonomous delivery vehicles are compact robots applied to parcel deliveries moving along the sidewalks till their customer destination [28]. Kapser and Abdelrahman [27] have studied the factors that determine the acceptance of ADV as a delivery alternative to the conventional ways of last mile delivery. To do that, a survey methodology was employed by using validated scales. The results of their survey show that the price sensitivity was the most impactful factor on the acceptance of ADV from the side of the customers. Even though autonomous vehicles for last mile deliveries is a recent concept, the study of [28] presents a model of truck-based robot solution to schedule the truck route and minimize the truck fleet. On their model, vans and robots are full with parcels to be delivered. Each van leaves the initial depot with robots and follows a route delivering parcels directly to customer locations. During the route there are drop-off points where the robots can leave the van to perform parcels deliveries to customers that are outside of the route, and then returning to the original van. Their results show that the truck fleet can considerably be reduced if autonomous robots support the delivery process. Some advantages of this solution are: (i) easy integrated with an app to track; (ii) environment friendly solution (reduction of air emissions); (ii) need of a person at home to receive the parcel. Some disadvantages are: (i) limited to pedestrian speed; (ii) technological interface with customer; (iii) limited load capacity; and (iv) autonomous vehicles can only transport one shipment at a time. Platooning Van. A solution recently emerging for last mile delivery is the platooning van. Only one paper was found proposing this solution. Lupi et al. [29] propose a transport system using automatic van platooning to perform deliveries to a city center. According to the authors, van platooning occurs when a series of vans follow automatically behind a leading van. This leading van has a driver and does not transport cargo and the other vans are driverless and contain cargo to be delivered. On their study, they propose a transport system where the van platoon moves from an urban distribution center to a dedicated location close to the city center, so-called “split-up-location”, where the platoon is broken and each van of the platoon (apart from the first-one), independently from the others, carries out the last part of its delivery route moving without any driver. After completing the deliveries, all vans return to the same split up location and gather again in a platoon. Here a driven van is added to the platoon and new platoon return to the urban distribution center. They created a model to optimize the deliveries from urban distribution centers to “split-up location” and minimize the number of last mile deliveries. The results show that the total travel time of delivery trips and the total km travelled are much lower in the proposed transport system, than in the conventional transport systems.

444

B. Machado et al.

Advantages of this solution are: (i) reduction on staff costs; (ii) higher speed than conventional autonomous vans; and (iii) energy saving, since the aerodynamic resistance is lower. Regarding the disadvantages, can be pointed out: (i) air emissions are the same of conventional systems; and (ii) high investments needed. New Energy Logistic Vehicles (NELVs). Vehicles that are moved by other energies that are alternatives to the fossil fuels have been gaining attention recently [31]. The paper of [31] highlights the factors that influence the market penetration of these new energy logistics vehicles. The factors are: (1) Policy promotion: Recent incentive polices have promoted the growth of NELVs. (2) Improve of the technology level: For example, domestic pure electric technology is gradually approaching the international advanced level. (3) Public awareness of environmental protection: With the increasing awareness of environmental protection, the public has increasingly realized the importance of the adoption of “green technologies”. (4) Market awareness: In terms of market recognition, the right-of-way, cost and social responsibility promote companies to choose NELVs. An example of NELV application is the study of [30] that intends to know how battery electrical vehicles (BEV) contribute to sustainable urban logistics. The research work evaluated the adequacy of BEV in urban logistics in Lisbon, based on a real-world application. Their results show that the adoption of BEV on urban logistics context allows maintaining the same operation patterns, regarding the number of kilometres travelled per day. When comparing the energy consumption, the adoption of BEV allows a reduction of 76% of the consumed energy. Advantages of this concept are: (i) significant reduction of the air emissions; and (ii) reduction of noise within the cities. Some disadvantages are related to: (i) high investments needed to changeover the fleet to BEV (or other types of green energy); and (ii) highly dependent of the technological advances. Connected Cities. This concept was found on just one paper during the review. The study of [32] proposes the last mile delivery process based on the key concept of interconnectivity, which is an open system where a multiple of diverse actors can utilize the interconnected urban logistics facilities and usable spaces of the physical internet. These facilities are hubs, warehouses, distribution centers, etc. Also, another key pillar for this interconnected system is the encapsulation of goods in standard modular, smart and reusable containers to be used across the system. The paper model and simulates this concept considering as objective the minimization of the delivery costs, the ecological footprint and the increasing of societal efficiency. Their results have shown that this interconnectivity concept can positively impact these performance indicators for urban logistics.

A Literature Review of Technological Trends in Urban Logistics…

445

Some advantages of this concept are: (i) sharing available capacities on the city; (ii) reduction of cartoon packaging; (iii) flexibility on deliveries; and (iv) shorter routes and delivery times. As disadvantages, we can point out: (i) investments on the development of containers; and (ii) air emissions are the same of conventional systems. Cloud-based Order Fulfillment. Leung et al. [33] developed a cloud-based order fulfillment of the orders to the logistics providers and retailers on the field of urban logistics. According to the authors, this proposed cloud-based order fulfillment process helps retailers and logistics providers when they receive orders from their customers, since they are able to effectively plan for the upcoming internal processing operations of received orders. The proposed cloud-based process consists on consolidating the pending e-orders, using a cloud-database, and then it creates an optimal internal order processing plan, instead of processing the orders one-by-one right after they are received. According to the study, this intelligent process allows warehouse postponement strategy to be adopted, increasing retailers and logistic providers’ flexibility and capacity to satisfy their customers. Reduction of processing times is also an important advantage of this concept. Some advantages of this concept are: (i) lower processing times of the orders; and (ii) higher flexibility and capacity to satisfy the customer expectations. Some disadvantages are related to: (i) process re-engineering by logistics services providers is needed; and (ii) investments on the cloud database.

4.2 Contributions of Technologies and Innovative Concepts on Urban Logistics Grounded on the literature review performed and considering the results summarized in Table 1, a framework, which allows to understand the contribution of each technology or concept in urban logistics and in which dimensions it has an impact, is proposed (see Fig. 1). From the nine technologies and concepts discussed on this study, it is possible to identify contributions on five urban logistics’ dimensions. These dimensions are Regulation and Policies, Sustainability, Operational Excellence, Collaboration and Digitalization. Impacts on Regulation and Policies were found on paper [20] studying the contributions made in the law and policies adopted by the local authorities and municipalities that intend to cargo-bikes technology. Sustainability is one of the most impacted dimensions by the eight of the nine technologies studied. This dimension is related to quality of life and ecological footprint, and can be found impacts on the analyzed papers [9–12, 14–16, 18, 21, 26, 28–30, 32]. Operational Excellence dimension is also very contributed from the technologies and concepts presented, since the papers [10, 11, 17, 19, 21, 24, 25, 28, 29, 31] and [33] refer the impact on performance of the urban logistics process, measuring indicators such as operational cost, delivery times, service level etc. Collaboration is the key relationship of stakeholders of urban

446

B. Machado et al.

Fig. 1 Framework of technologies and concepts contributions to urban logistics

logistics that can give advantages in some manner, as mentioned by the studies [17, 22], and [23]. Lastly, and in accordance with the findings described in [33]. Digitalization is represented by process improvements that transform the bureaucratic or manual work in some digital, easier and smarted digital way. Thus, this framework allows us to understand the main contributions of the technological trends for urban logistics and relationships between those technological trends and the urban logistics dimensions that may need further research. Undoubtedly, the main dimensions impacted are Sustainability and Collaboration among the stakeholders, since eight of nine of the technologies and concepts found in the literature have directly mentioned the impact on each one of them. Also, some dashed arrows are represented on the framework that connect technological trends to dimensions of urban logistics. These arrows represent relationships that, even if there were no papers found on the LR studying these impacts, the adoption of respective technological trend has a very high potential to impact the pointed dimension.

A Literature Review of Technological Trends in Urban Logistics…

447

5 Conclusions This chapter presents a research on urban logistics field that leads to a framework of technologies and concepts supporting urban logistics dimensions, such as Regulation and Policies, Sustainability, Costs, Collaboration and Digitalization. Using a methodological literature review, trending technologies and concepts adopted worldwide were presented and detailed. The goal of this chapter was to understand what were these technologies and concepts and in what dimensions they interact with urban logistics. The result of this chapter was the proposed framework that helps to understand the relationships and the benefits of adopting these technologies reviewed in urban logistics context, providing valuable theoretical contribution for this topic. Also, this work provides practical contribution helping practitioners, investors, public or private companies, or regulation authorities to understand what are the main technologies that are available and that they may adopt and implement according to their goals. Although the literature review was methodically performed, the main limitation of this work is the fact that some articles may not be on the scope and some trending technology may be missing on the study. For future work, it is recommended to perform a systematic literature review on this research topic to extend the study and catch all the potential technologies and innovative concepts being adopted worldwide and hopefully confirm the relationships represented on the framework as dashed arrows. Acknowledgements This work is co-financed by the European Regional Development Fund (FEDER) through COMPETE 2020 (Operational Program for Competitiveness and Internationalization) through the project SOLFI- Urban logistics optimization system with integrated freight and passenger flows (POCI-01-0247-FEDER-039870). The work was also supported by the research unit on Governance, Competitiveness and Public Policy (UIDB/04058/2020) and by Institute of Electronics and Informatics Engineering of Aveiro (UIDB/00127/2020) funded by national funds through FCT—Fundação para a Ciência e a Tecnologia.

References 1. Dablanc L, Morganti E, Arvidsson N et al (2017) The rise of on-demand ‘Instant Deliveries’ in European cities. Supply Chain Forum An Int J 18:203–217. https://doi.org/10.1080/16258312. 2017.1375375 2. Demir E, Huang Y, Scholts S, Van Woensel T (2015) A selected review on the negative externalities of the freight transportation: modeling and pricing. Transp Res Part E Logist Transp Rev 77:95–114. https://doi.org/10.1016/j.tre.2015.02.020 3. Piecyk M, Browne M, Whiteing A, McKinnon A (2015) Green logistics: improving the environmental sustainability of logistics 4. Savelsbergh M, Van Woensel T (2016) City logistics: challenges and opportunities. Transp Sci 50:579–590. https://doi.org/10.1287/trsc.2016.0675 5. Dolati Neghabadi P, Evrard Samuel K, Espinouse ML (2019) Systematic literature review on city logistics: overview, classification and analysis. Int J Prod Res 57:865–887

448

B. Machado et al.

6. Lagorio A, Pinto R, Golini R (2016) Research in urban logistics: a systematic literature review. Int J Phys Distrib Logist Manag 46:908–931 7. Bosona T (2020) Urban freight last mile logistics—challenges and opportunities to improve sustainability: a literature review. Sustainability 12:8769. https://doi.org/10.3390/su12218769 8. Timmins F, McCabe C (2005) How to conduct an effective literature search. Nurs Stand 20:41– 47. https://doi.org/10.7748/ns2005.11.20.11.41.c4010 9. Agatz N, Bouman P, Schmidt M (2018) Optimization approaches for the traveling salesman problem with drone. Transp Sci 52:965–981. https://doi.org/10.1287/trsc.2017.0791 10. Ha QM, Deville Y, Pham QD, Hà MH (2018) On the min-cost traveling salesman problem with drone. Transp Res Part C Emerg Technol 86:597–621. https://doi.org/10.1016/j.trc.2017. 11.015 11. Kitjacharoenchai P, Ventresca M, Moshref-Javadi M et al (2019) Multiple traveling salesman problem with drones: mathematical model and heuristic approach. Comput Ind Eng 129:14–30. https://doi.org/10.1016/j.cie.2019.01.020 12. Boysen N, Briskorn D, Fedtke S, Schwerdfeger S (2018) Drone delivery from trucks: drone scheduling for given truck routes. Networks 72:506–527. https://doi.org/10.1002/net.21847 13. Aurambout JP, Gkoumas K, Ciuffo B (2019) Last mile delivery by drones: an estimation of viable market potential and access to citizens across European cities. Eur Transp Res Rev 11. https://doi.org/10.1186/s12544-019-0368-2 14. Buldeo Rai H, Verlinde S, Merckx J, Macharis C (2017) Crowd logistics: an opportunity for more sustainable urban freight transport? Eur Transp Res Rev 9. https://doi.org/10.1007/s12 544-017-0256-6 15. Behrend M, Meisel F (2018) The integration of item-sharing and crowdshipping: can collaborative consumption be pushed by delivering through the crowd? Transp Res Part B Methodol 111:227–243. https://doi.org/10.1016/j.trb.2018.02.017 16. Giret A, Carrascosa C, Julian V et al (2018) A crowdsourcing approach for sustainable last mile delivery. Sustain 10. https://doi.org/10.3390/su10124563 17. Frehe V, Mehmann J, Teuteberg F (2017) Understanding and assessing crowd logistics business models—using everyday people for last mile delivery. J Bus Ind Mark 32:75–97. https://doi. org/10.1108/JBIM-10-2015-0182 18. Melo S, Macedo J, Baptista P (2019) Capacity-sharing in logistics solutions: a new pathway towards sustainability. Transp Policy 73:143–151. https://doi.org/10.1016/j.tranpol. 2018.07.003 19. Ozturk O, Patrick J (2018) An optimization model for freight transport using urban rail transit. Eur J Oper Res 267:1110–1121. https://doi.org/10.1016/j.ejor.2017.12.010 20. Rudolph C, Gruber J (2017) Cargo cycles in commercial transport: potentials, constraints, and recommendations. Elsevier Ltd 21. Anderluh A, Hemmelmayr VC, Nolz PC (2017) Synchronizing vans and cargo bikes in a city distribution network. Cent Eur J Oper Res 25:345–376. https://doi.org/10.1007/s10100-0160441-z 22. Vakulenko Y, Hellström D, Hjort K (2018) What’s in the parcel locker? Exploring customer value in e-commerce last mile delivery. J Bus Res 88:421–427. https://doi.org/10.1016/j.jbu sres.2017.11.033 23. Yuen KF, Wang X, Ma F, Wong YD (2019) The determinants of customers’ intention to use smart lockers for last-mile deliveries. J Retail Consum Serv 49:316–326. https://doi.org/10. 1016/j.jretconser.2019.03.022 24. Arnold F, Cardenas I, Sörensen K, Dewulf W (2018) Simulation of B2C e-commerce distribution in Antwerp using cargo bikes and delivery points. Eur Transp Res Rev 10. https://doi. org/10.1007/s12544-017-0272-6 25. Orenstein I, Raviv T, Sadan E (2019) Flexible parcel delivery to automated parcel lockers: models, solution methods and analysis. EURO J Transp Logist 8:683–711. https://doi.org/10. 1007/s13676-019-00144-7 26. Schwerdfeger S, Boysen N (2020) Optimizing the changing locations of mobile parcel lockers in last-mile distribution. Eur J Oper Res 285:1077–1094. https://doi.org/10.1016/j.ejor.2020. 02.033

A Literature Review of Technological Trends in Urban Logistics…

449

27. Kapser S, Abdelrahman M (2020) Acceptance of autonomous delivery vehicles for last-mile delivery in Germany—extending UTAUT2 with risk perceptions. Transp Res Part C Emerg Technol 111:210–225. https://doi.org/10.1016/j.trc.2019.12.016 28. Boysen N, Schwerdfeger S, Weidinger F (2018) Scheduling last-mile deliveries with truckbased autonomous robots. Eur J Oper Res 271:1085–1099. https://doi.org/10.1016/j.ejor.2018. 05.058 29. Lupi M, Pratelli A, Farina A (2020) Modelling and simulation of a new urban freight distribution system based on automatic van platooning and fixed split up locations. IET Intell Transp Syst 14:1034–1047. https://doi.org/10.1049/iet-its.2019.0681 30. Duarte G, Rolim C, Baptista P (2016) How battery electric vehicles can contribute to sustainable urban logistics: a real-world application in Lisbon, Portugal. Sustain Energy Technol Assessments 15:71–78. https://doi.org/10.1016/j.seta.2016.03.006 31. Jiang X, Guo X (2020) Evaluation of performance and technological characteristics of battery electric logistics vehicles: China as a case study. Energies 13. https://doi.org/10.3390/en1310 2455 32. Ben Mohamed I, Klibi W, Labarthe O et al (2017) Modelling and solution approaches for the interconnected city logistics. Int J Prod Res 55:2664–2684. https://doi.org/10.1080/00207543. 2016.1267412 33. Leung KH, Choy KL, Siu PKY et al (2018) A B2C e-commerce intelligent system for reengineering the e-order fulfilment process. Expert Syst Appl 91:386–401. https://doi.org/10. 1016/j.eswa.2017.09.026 34. Mehmann J, Frehe V, Teuteberg F (2015) Crowd logistics—a literature review and maturity model. In: Proceedings of the Hamburg international conference of logistics. Hamburg, Germany 35. Arslan A, Agatz N, Kroon LG, Zuidwijk RA (2016) Crowdsourced delivery: a pickup and delivery problem with ad-hoc drivers. SSRN Electron J 1–29. https://doi.org/10.2139/ssrn.272 6731 36. Bardhi F, Eckhardt GM (2012) Access-based consumption: the case of car sharing. J Consum Res 39:881–898. https://doi.org/10.1086/666376 37. De Decker K (2012) Cargo cyclists replace truck drivers on European city streets 38. Morganti E, Dablanc L, Fortin F (2014) Final deliveries for online shopping: the deployment of pickup point networks in urban and suburban areas. Res Transp Bus Manag 11:23–31. https:// doi.org/10.1016/j.rtbm.2014.03.002

A Cost-Effective and Quality-Ensured Framework for Crowdsourced Indoor Localization Lulu Gao and Shin’ichi Konomi

Abstract With the increasing user demands for the ubiquitous availability of location-based services, and the acknowledgement of their substantial business prospects, researchers have extensively studied indoor localization techniques that do not rely on the Global Positioning System (GPS) or other localization technologies that do not work well in indoor environments. Thanks to the rapid advancement of the machine learning technologies, many of the indoor localization schemes utilize machine learning and pattern recognition techniques based on the received signal strength indicator to estimate indoor locations of mobile devices. One of the key challenges for deploying such machine learning-based indoor localization systems is the labor-intensive and time-consuming tasks of annotating radio maps with the corresponding position information in advance. One could exploit crowdsourcing to solve the problem of radio signature collection, while there are various uncertainties about the location annotations contributed by the crowd, which can affect the performance of the localization model. We propose a crowdsourcing-based indoor localization framework, ALCIL, which utilizes the active learning technique to collect the informative data to improve the performance of the model under a certain cost. We then employ global and local optimization strategies considering the multiple attributes of locations to improve the accuracy of location prediction in different dimensions. In addition, we propose a sample selection method based on a stream-based active learning so as to improve the quality of radio maps and enhance the performance of the indoor localization model without penalizing the location-annotation process. The effectiveness of the proposed framework is verified through the experiments in the context of practical multi-story buildings. Our experiment shows that the proposed method can localize users’ mobile devices accurately at the given fixed budget. Keywords Indoor localization · Crowdsourcing · Active learning · Cost effective L. Gao (B) Graduate School of Information Science and Electrical Engineering, Kyushu University, 744, Motooka, Nishi-ku, Fukuoka 819-0395, Japan e-mail: [email protected] S. Konomi Faculty of Arts and Science, Kyushu University, 744, Motooka, Nishi-ku, Fukuoka 819-0395, Japan © The Author(s), under exclusive license to Springer Nature Switzerland AG 2023 V. G. Duffy et al. (eds.), Human-Automation Interaction, Automation, Collaboration, & E-Services 11, https://doi.org/10.1007/978-3-031-10784-9_27

451

452

L. Gao and S. Konomi

1 Introduction With the rapid development of wireless technology and pervasive computing, location-based services (LBSs) and systems such as social network, tracking, navigation, recommendation and social distancing considered to be an effective nonpharmaceutical measure to reduce the contagious disease transmission, especially the ongoing COVID-19 pandemic, have shown the tremendous value [1, 2]. The essence of LBSs is to locate the user and then provide useful information at the appropriate time and right location. Therefore, the performance of the LBSs is greatly influenced by the accuracy of localization measures [3]. Localization is a mechanism for determining the spatial relationship based on physical position or logical position of different entities [4]. Depending on the target environment, it can be divided into outdoor localization and indoor localization. Global navigation satellite system (GNSS), such as Global Positioning System (GPS), Galileo Satellite Navigation (Galileo), BeiDou Navigation Satellite System (BDS) and other satellite systems can locate us precisely and reliably in an outdoor open environment, which are widely exploited in our everyday lives [5]. However, GNSS does not perform well in urban canyons, underground environments and indoor environments in which we spend most of the time because of the lack of a unified infrastructure and the weak signal strength of satellites due to the absence of line of sight, the attenuation of satellite signals as they cross through physical objects, especially walls, and noise interference, resulting in inaccurate localization of us or devices [6, 7]. What’s more, indoor localization has played an important role in tracking and navigation, especially in a large building like a shopping mall or underground parking lot. Thus, to fill the gap, indoor localization technology has been extensively researched in decades. Indoor localization is the process of obtaining the location of user or device in an indoor setting or environment, which has been well developed with the joint effort of researcher and engineer in the past few decades. For different scenarios, researchers investigate lots of technologies to build indoor localization systems: radio frequency identification (RFID), Bluetooth, Zigbee, ultra-wideband (UWB), wireless local area network (WLAN), infrared ray (IR), ultrasound, magnetic field and visible light. Among aforementioned technologies, as an infrastructure-free technology, WLAN (or Wi-Fi) is widely used because of the ubiquity in deployment and accessibility on the device [8, 9]. Many indoor localization methods based on Wi-Fi using different techniques that mainly cluster into trilateration, including Time-of-Arrival (TOA), Time Difference-of-Arrival (TDOA), Angle of Arrival (AOA), and fingerprinting have been proposed. In the trilateration method, the distance to three points which could obtain the relative location of user via basic geometry and trigonometry is estimated according to the received signal strength indicator (RSSI) and the path-loss propagation model [10]. In the fingerprinting approach, a number of fingerprints from a various position of different grid within the target area are collected to obtain the training dataset to train a machine-learning algorithm which could predict the coordinates online given the measured fingerprints at the location of the user. Fingerprinting

A Cost-Effective and Quality-Ensured Framework …

453

technique apply machine learning algorithms to find the spatial pattern behind the sensed received signal strength (RSS) data in the target area to reduce the effects of RSS fluctuation in trilateration caused by the complicated indoor environment, plentiful multipath fading and various non-line-of-sight (NLOS) conditions that result in inaccurate propagation mode even though calibrating considerable samples, which is widely researched for accurately locate the user now [11]. The RSSs from detected access points (APs) in different position can be recognized the fingerprints in fingerprinting indoor localization system which include offline training phase and online location prediction phase. Since machine learning algorithms require plentiful training data to achieve great performance, fingerprinting method also demands a large number of fingerprints in offline training phase to improve the accuracy of location prediction. Numerous fingerprints should be obtained from different position of interested region to construct the fingerprinting dataset with annotation by researchers to construct detailed radio map in training phase. Thence, in the next online predicting phase, machine-learning methods trained by the collected dataset could properly return the estimated user’s position after the location query with stored radio map. However, collection and annotation of fingerprints is time-consuming and labor-intensive because of the RSS variance caused by environmental changes between two phases and lack of experts [12, 13]. Crowdsourcing is a potential solution to solve the site survey problem because everyone could become the contributor. With the development of sensor technology and widespread popularity of wireless mobile terminal devices, such as laptops, smartphone and so forth, mobile devices integrate more and more sensors, including Wi-Fi and camera, leading more and more powerful abilities of computing and sensing, which make it possible to encourage ordinary mobile devices users to contribute their effort and large quantities and scalabilities of fingerprint data could be achieved, thus so-called crowdsourcing based indoor localization, reducing the burden of site survey. As the low deployment cost and efficient approach to construct radio map, many different crowdsourcing based indoor localization system have been proposed [13–15]. ABCABCABC. Crowdsourcing-based indoor localization techniques does reduce the burden on researchers, while some new variances like unsure annotation because of the lack of expert knowledge about positioning or Geographic Information System (GIS) is introduced. In fact, without the assistance of GNSS, specific measurements are demanded to calculate the location, which requires enough specialized knowledge and equipment. In crowdsourcing scenario, we do not know the information about the task performer that may work in different fields with different capacities and unable to obtain his/her precise location in detail, including height, latitude and longitude. Therefore, many researchers now bypass the process of data annotation, trying to find other data patterns from different perspective to assist in positioning users or auto generate the annotation by some axillary sensors. While the annotation is indispensable to achieve better performance of fingerprinting approach in indoor scenario. In this paper, for crowdsourcing based indoor localization, a framework, named ALCIL, Active Learning based Crowdsourcing Indoor Localization, which could reduce the efforts of site survey and ensure the performance of the positioning

454

L. Gao and S. Konomi

methods has been proposed. ALCIL can reduce the number of fingerprints data that constitute the radio map without affecting the performance of location prediction, using active learning. Active learning is a modern method in machine learning, aiming to reduce the sample size, complexity by selecting the informative data according to informativeness measures and increase the accuracy of data tasks with minimal costs. Moreover, ALCIL can ensure the accuracy of the fingerprint label via participants’ relative annotation of all data and experts’ reverification of reduced instances with high informativeness. Location can be labelled by absolute position with precise height, latitude and longitude and logical position associated with indoor environment which is available for everyone even without relevant expertise which researchers can calculate the absolute position from it. To achieve ALCIL, participants only need to describe the current relative position, that is uncomplicated, and complex and specialized work with informative data can be done by experts, reducing the annotation variances and cost. To demonstrate the effectiveness of ALCIL, we have conduct extensive experiments over the dataset collected in West Zone of Kyushu University to evaluate the proposed methods. We have developed an application to collect data and annotate data with relative information and physical values of different attributes of location. Experiments indicate that ALCIL can obtain the better dataset with high quality and accurate performance of indoor localization with the constraint of cost. The remainder of this paper is organized as follows. Related work about crowdsourcing indoor localization and active learning is reviewed in Sect. 2. Section 3 introduce the theoretical methodology and the architecture of proposed ALCIL system. Section 4 present the experimental methodology and results in two datasets. Finally, conclusion and future work are presented in Sect. 5.

2 Related Work The most relevant literature to this paper is discussed in this section, which could be divided into fingerprinting-based indoor localization, crowdsourcing-based indoor localization and active learning technique.

2.1 Crowdsourced Fingerprinting Indoor Localization Ambient radio signals can be obtained easily to identify the different locations, serving as fingerprint, and produce an accurate estimation of location in indoor environment [16]. A plethora of indoor localization methods that adopt the radiobased fingerprint to predict user’s location has been proposed. The core idea is to build up a fine-grained radio map consist of the fingerprint of each interested location, which the position can be achieved using matching algorithms. RADAR is the first attempt to apply the fingerprinting-based technique with KNN matching algorithm

A Cost-Effective and Quality-Ensured Framework …

455

in indoor localization, which utilize the WIFI fingerprint [17, 18]. Though the wide GSM, TV signals and FM radio signals are researched as fingerprints for positioning, WIFI fingerprints can be recognized as the most representative fingerprinting-based in indoor localization due to the rapid development of wireless communication and the extensive deployment of WLAN infrastructure [19–22]. WIFI fingerprinting-based indoor localization can measure the RSSs from detected APs at target area to construct the detailed radio map on training phase, which will be used to determine the location via deterministic and probabilistic algorithms with different similarity metrics like Euclidean distance, Kullback–Leibler divergence and Jensen-Shannon divergence [23–27]. The RSS of target location can be calculated in deterministic approaches, whose accuracy is greatly affected by noise and variation which are tackled in probabilistic method [28]. All these approaches will distinguish the location within the previously surveyed fingerprints database, whose accuracy and coverage are critical attributes to achieve better performance, which means labor-intensive and time-consuming [29]. Crowdsourcing is the most suitable approach to collect large-scale fingerprints because everyone can be the potential contributor using the terminal devices with plenty of sensors in-built, including smartphones, PDA, and human mobility. The term crowdsourcing was first presented in 2006 by Jeff Howe [30]. Crowdsourcing was traditionally used to be a distributed problem-solving and production model, but now, it can be seen a promising approach to address some of the growing challenges associated with data collection and data processing, as demonstrated by Amazon Turk, Netflix, and the ESP game, which expand human computation [31, 32]. As a low-cost and efficient method to collaborate the intelligence of different people, crowdsourcing has been widely employed in the acquisition of WIFI fingerprints [33, 34]. We can’t achieve the complete training data just with data collection because annotation is the important part to train the prediction model. However, due to anyone can be a contributor, including people who do not have the expertise about location technique, and the label may be inaccurate. Although there are some solutions without site survey processing to realize indoor localization, label is inevitable to return accurate location prediction in indoor scenario [35–37].

2.2 Active Learning Active learning is a modern method in machine learning, aiming to reduce the sample size, complexity, and increase the accuracy of the data tasks as much as possible with fewer data. The key hypothesis of active learning is that the learning mechanisms will be more intelligent if the learning algorithm can actively choose the most significant unlabeled data. An active learner will query only a small number of valuable unlabeled instances to be labeled by an oracle or annotator to automatically enlarge the labeled dataset, in an intelligent manner [38].

456

L. Gao and S. Konomi

There are three main scenarios that have been studied of active learning, membership query synthesis, stream-based selective sampling, and pool-based sampling [38–40]. • In query synthesis, any unlabeled instance can be queried by an active learner, including the model-generated although it may have no practical meaning and cannot be labeled by a human annotator. While the other scenarios do not have this problem that cannot be labeled, because the learner must query the instances of what it thinks important from the actual input pool. • In stream-based selective sampling, the unlabeled instances will be query sequentially by the learner [41]. And the learner will decide whether the instance be annotated or not. • In pool-based sampling, many unlabeled instances are assumed to be available. In this kind of scenario, the learner should rank the entire unlabeled instances according to an informativeness measure, that is the pool of unlabeled instances, and then, query the most informative one [41]. The main difference between poolbased sampling and stream-based selective sampling is that the former should evaluate all unlabeled data before select query, while the latter just query the instance in sequence [42]. Selective sampling is the most relevant scenario to the crowdsourcing based indoor localization, including stream-based active learning and pool-based active learning. In the stream-based scenario, the decision of annotating the current piece of data can be determined by the informativeness measures which is also applicable to data ranking for data labeling in pool-based active learning after we collect a certain amount of data. Actually, the measure of informativeness evaluation is vital in all active scenarios and can be parted into uncertainty selection, query by committee, expected objective change, and data-centered method [39, 43, 44]. • Uncertainty selection, which would query the most uncertain instance on the pre-diction of the current model. • Query by committee (QBC), which query the most disagreeing instance of the committees’ prediction. Each committee member is a different model based on the current training set. • Expected objective change, which query the instance that could make the maximizing impact on the objective. For example, maximizing model change, maximizing the generalization error reduction, maximizing the output variance reduction. • Data-centered method, which query the most representative of the most informative instance. It can be seen that active learning and crowdsourcing are the critical technologies for optimizing data collection and processing, and many researchers have conducted a lot of studies on the ways to integrate them [45]. Lease suggest that crowdsourcing, with active learning, may provide new insights for better focusing annotation effort on the examples that will be most informative to the learner to accelerate model training, as well as reduce cost of annotation versus traditional annotators [46]. Costa, et al.

A Cost-Effective and Quality-Ensured Framework …

457

propose two methods of combining crowdsourcing and active learning. The two methods were tested with Jester data set, a text humor classification benchmark, and the result shows promising improvements [47]. ALSense is a novel distributed active learning framework under crowdsourcing scenario can be used to indoor localization, which consider the cost of data annotation, and its main goal is minimize the prediction error of crowdsourcing based tasks within a fix annotation cost. Crowdsourcers calculate the informativeness of current data based on their own simple model trained by the initial dataset and upload the data with higher informativeness. After the server collects a certain amount of data, pool-based sampling is utilized to determine the instances that need to be labeled from the collected data and requests the participant to annotate it [40]. Although ALSense can control the cost of annotation, it is necessary to ensure the label’s accuracy, which is difficult to achieve in practice due to different crowdsourcing workers with diverse expertise, and just only target for the classification tasks. Hence, we propose ALCIL to minimize the predictions of indoor localization tasks, including classification and regression, subject to collection and annotation cost constraints under different crowdsourcing workers.

3 Methodology In this section, we analyze the annotation cost of WIFI fingerprints and present the detailed framework of ALCIL. Besides, the optimization strategies are explained for indoor localization with multi-labels.

3.1 Problem Construction and Overview To achieve WIFI fingerprinting-based indoor localization, it is necessary that one or more people carries terminal device with WLAN access and RSSs measurement from different APs to collect the fingerprints on various location of the target indoor region [48]. The fingerprints in different positions should be labeled with some identification, like the building number, floor number, room number, and physical coordinate to form the training dataset, called radio map, which is time-consuming and labor-intensive and requires some expertise or specific equipment. Given a set of fingerprints of X, {x1 , x2 , . . . , xn }, where xk is the RSSs vector of the k-th sample we collect and the value of it should belong to the value space Rx . A set of labels of Y, {y1 , y2 , . . . , yn }, where yk is the labels of the k-th RSSs sample xk , which may include more than one attribute, like height, latitude, and longitude, and the value of it should in the label space Ry . (X, Y ) is trained in offline phase to discover the reflection of f : X → Y to obtain the location in online phase. Actually, f can be each one of hypothesis spacesF and the prediction error or interference n error E of f can be equated by E = i=0 L(yi , f (xi )) that we can measure the

458

L. Gao and S. Konomi

performance of f and it should be minimized, where L is a specific measurement. No matter what the selection and the measurement of f , we all need accurate label y. However, the cost of obtaining Y is higher than collecting X , hence improving the quality of Y at a certain cost to make f perform better is essential. Under the restriction of cost, M, which we can obtain from the incentive method, only selected instances set X  of X can be annotated, and the label of X  we can get is Y  . n     min xi ∈X  ,yi ∈Y  E = L yi , f xi   i=0 s.t. Y   ≤ M

3.2 Architecture Figure 1 shows the proposed indoor localization system architecture, which can be divide into two parts: the mobile device and the positioning server. On the mobile device side, there is a local active learner which can detect the informativeness of collected fingerprints for the localization method based on utility measure, then decides whether to upload a data item with relative annotation, like the building number, floor number, room number and the distance to some obvious indoor objects. Moreover, the local active learner can be updated by the server in a certain period. The global active learner can select informative samples from the uploaded data summited by crowdsourcers based on informativeness measure to remind us to re-calibrate the fingerprints, including the absolute height, latitude and longitude. Therefore, the uploaded data item with accurate annotations is added to the database of labelled fingerprints to train a better model. The indoor localization model and global learner are updated when a new re-calibrated fingerprint appears in the labelled database to make the system perform better. For this system, a few calibrated fingerprints and the informativeness measure need to be settled down according to the localization approaches. A small initial precise data with label should be collected is to accelerate the deployment because to provide the localization service and incentive the crowdsourcer to participate, the performance of indoor positioning model should achieve a certain level. As we mentioned above, there are many different informativeness measures, which can be selected based on our target. The purpose of this system is to obtain a precise model that predict the position stably according to the data we calibrated, therefore, we need to continuously reduce the variance of the model derived. QBCis a model-driven  selection approach, which involves maintaining a committee C = θ 1 , θ 2 , . . . , θ c of models trained on the labelled dataset and select the most disagree one to constrain the hypothesis space, reducing the model variance. To detect the level of disagreement, various distance forms are utilized, among which Kullback–Leibler (KL) divergence can measure the difference between different probability distributions [49, 50]. KL divergence is characterized by,

A Cost-Effective and Quality-Ensured Framework …

459

Fig. 1 The architecture of ALCIL

x K∗ L = argmax x

C c=1

D(Pθ c ||Pc ) , C

where D(Pθ c ||Pc ) =



Pθ c (yi |x)log

i

Pθ c (yi |x) . Pc (yi |x)

Here θ c represents a particular model in the committee, and C represents the C

P c (y |x)

committee as a whole, hence PC (yi |x) = c=1 Cθ i is the agreement that yi is the correct label. Thus, this disagreement measure considers the most informative instance to be the one with the largest average difference between the label distributions of any one committee member and the consensus. As the variant of KL divergence with some useful improvement, including that it is symmetric between

460

L. Gao and S. Konomi

two distributions and it always has a finite value [51], Jensen-Shannon (JS) divergence also used to measure the disagreement. Therefore, QBC is appropriate as the informativeness measure in indoor localization system to improve the prediction accuracy.

3.3 Optimization Strategies As mentioned above, location can be expressed in different terms, including the relative address and absolute address, each of them has different attributes. Relative location is the description of how a place is related to other places, like the floor number is the relative height to the ground, the building number is the relative location from other buildings of a region, directly showing the connection with others. Geographic coordinates of longitude and latitude help us pinpoint the absolute location which is a kind of relative address relative to the equator (latitude) and prime meridian (longitude) and help us understand each other with the fixed standard. Depending on the different purposes, the importance of various attributes is also different. For the significant application of measuring the distance between people, which is widely applied to detect the social distance under COVID-19, building number and floor number are more straightforward than a string of digits because we can directly obtain the conclusion we desired when we have no knowledge about geography. However, accurate coordinates do help us calculate the precise information we need, like the social distance, and if we focus on certain attributes, we cannot achieve it. So, the importance of various attributes is different at diverse phases. The prediction accuracy of building number and floor number is more important at first, as the project carried on, when there is no significant improvement about it, we should focus on the coordinates’ prediction. For indoor localization with multiple labels, we design two optimization strategies in different stages, namely the local optimization strategy and the global optimization strategy. The former one focuses on the optimization of a subset of all features, and the other one optimizes all attributes at the same time. When the current strategy cannot have a significant impact on the location prediction, the accuracy reaches the critical point and it is a waste to take more efforts to continuously improve, and it is time to change the strategies or attributes we focused on. Expected error reduction aims to measure how much its generalization error is likely to be reduced to indicate the changes of the model, which we can obtain from the change of the expected future error of a model trained using current labeled data L with a new point (x, y) added on the test dataset T . It can be characterized by E E R0/1 =

 i

Pθ (yi |x)

T  t=1

1 − Pθ +(x,yi )



 yˆ |xt ,

A Cost-Effective and Quality-Ensured Framework …

461

with 0/1-loss and E E Rlog =



⎛ Pθ (yi |x)⎝−

i

T   t=1

j

Pθ +(x,yi )



⎞    y j |xt log P +(x,y j ) y j |xt ⎠, θ

with log-loss, where θ is the current model and θ +(x,yi ) refers to the new model after it has been retrained with (x, yi ) added to L [39, 52]. When the E E R of the current model after new data added is lower than a threshold ε, it is time to move to the other optimizations or a subset of attributes. As project carried on, location is main purpose, should improve the building, floor accuracy and location accuracy at the same time and then focus on the improvement of location accuracy. For the location problem, we design two strategies to the multi-attributes’ optimization problem, namely the local optimization and global optimization.

4 Experiments In this section, we evaluate the performance of ALCIL on a dataset we collected in the West Zone of Kyushu University. The dataset is randomly divided into training data and test data, and a small collection of training data is selected at randomness.

4.1 Data Collection To collect the necessary data for our experiments, we developed a data collection application on android smartphone, integrated the WIFI fingerprints sensing and data annotation via text or image, where the labels can be relation position or absolute position. It records the (Service Set Identifier (SSID), MAC address, RSSI) for each detected AP. Several participants with the knowledge of localization contribute more than 4000 fingerprints with accurate labels of building ID, floor ID and geographic coordinate, using the same Android phone, LG nexus 5, over different days. Among them, building ID and floor ID are categorical labels and latitude and longitude are quantitative values (Fig. 2).

4.2 Localization Methods Various machine learning methods are widely researched in many different fields, among which random forest (RF) has been deeply studied as the critical feature

462

L. Gao and S. Konomi

Fig. 2 Data collection system

of the convenient behavior both in terms of accuracy and efficiency in prediction independent tasks [53]. RF is an ensemble learning approach behaved  Non many 1 decision trees, and the result is calculated as, f (x) = n=1 N f n (x), where f n is the n-th decision tree and N is the total number which is not always the same, depending on the specific tasks and dataset [54]. Whether it is a classification mission or regression analysis, RF has achieved tremendous success, which is the main reason for being chosen because there are quantitative values of latitude and longitude and categorical labels of building and floor ID [55]. Some separate RF models are performed for the regression of continuous attributes and the classification of discrete labels, respectively, to return the different descriptions of location.

4.3 Results and Discussion In the experiments, we evaluate how the proposed system performs with the increase of annotation acquisition cost M. Because there are multiple labels of the location, to clearly shown the efficiency of it, we can consider it as multiple independent problems, including two classification and two regression tasks using collected dataset.

A Cost-Effective and Quality-Ensured Framework …

463

Fig. 3 Classification accuracy of proposed framework on building ID prediction and floor ID prediction

There are two methods involved in this experiment, the proposed ALCIL and RSCIL, random selection-based crowdsourced indoor localization, where the crowdsourcers will select the data to upload to the server at randomness and the server randomly selects the data to re-calibrate from the uploaded dataset. Splitting the entire samples, we collected, into training data and test data and select 200 samples from the former one to achieve an initial model can bootstrap the system. As the system runs, assuming we select λ samples from  data set each time, the cost of recalibration for each data is fixed at η and then, each cycle, all the accurate annotation we obtained will spend ηλ which should lower than M. Here we set  to 50, λ to 10, and we perform 60 cycles to construct the complete labeled dataset. It means that the number of uploaded should greater than 50 to be used as the data pool of server. From the performance of building ID prediction and floor ID prediction shown in Fig. 3, ALCIL obtain a higher accuracy under the same cost of annotation. Because there are three buildings with different floors involved in this dataset, the initial performance of floor prediction is all not well-accepted, while after a large amount of data training, ALCIL can also perform well. The better performance of ALCIL also occurs in regression tasks of the prediction of latitude and longitude. Crowdsourced indoor localization with active learning introduced, ALCIL, can select informativeness to let the model perform better, including the classification model and regression model, with a certain cost (Fig. 4).

5 Conclusion and Future Work In this paper, we propose a cost-effective and quality-ensured framework for crowdsourced indoor localization, which collects the informative data to further improve the performance of the model at a certain cost. In ALCIL, the researcher needs to obtain a collection of WIFI fingerprints with precise labels as the initial data to train a primary model, which can be comprehensive when some users are motivated by

464

L. Gao and S. Konomi

Fig. 4 Regression mean absolute error of proposed framework on longitude prediction and latitude prediction

incentive measures to contribute more samples in actual deployment. While considering the participants’ ability, the annotation can be the relative location. Then the informativeness of the sample is detected based on QBC to determine whether this sample uploads to the server. After the server obtains a certain number of samples, considering the multiple attributes of location, the data that are valuable for the current target will be selected to recalibrate. Avoiding the annotation of all data can reduce the cost of obtaining the complete dataset. The effectiveness of this framework is demonstrated by the collected data. As part of our future work, we will design an appropriate incentive mechanism to motivate more users to collect the WIFI fingerprints within the target area, and the device heterogeneity will be considered. Acknowledgements This study is supported by CSC scholarship and JSPS KAKENHI Grant Number JP20H00622.

References 1. Kelso JK, Milne GJ, Kelly H (2009) Simulation suggests that rapid activation of social distancing can arrest epidemic development due to a novel strain of influenza. BMC Public Health 9:117 2. Nguyen CT, Saputra YM, Huynh NV, Nguyen N-T, Khoa TV, Tuan BM, Nguyen DN, Hoang DT, Vu X, Dutkiewicz E, Chatzinotas S, Ottersten B et al (2020) A comprehensive survey of enabling and emerging technologies for social distancing—part I: fundamentals and enabling technologies. IEEE Access 8:153479–153507. https://doi.org/10.1109/ACCESS. 2020.3018140 3. Farid Z, Nordin R, Ismail M (2013) Recent advances in wireless indoor localization techniques and system. J Comput Netw Commun 2013(185138):12. https://doi.org/10.1155/2013/185138 4. Xi R, Li Y-J, Hou M-S (2016) Survey on Indoor Localization. Comput Sci 43(4):1–6, 32 5. Alhomayani F, Mahoor MH (2020) Deep learning methods for fingerprint-based indoor positioning: a review. J Location Based Serv 14(3):129–200. https://doi.org/10.1080/17489725. 2020.1817582 6. Davidson P, Piché R (2017) A survey of selected indoor positioning methods for smartphones. In: IEEE communications surveys & tutorials, vol 19, no 2, pp 1347–1370, Secondquarter 2017. https://doi.org/10.1109/COMST.2016.2637663

A Cost-Effective and Quality-Ensured Framework …

465

7. Brena RF, García-Vázquez JP, Galván-Tejada CE, Muñoz-Rodriguez D, Vargas-Rosales C, Fangmeyer J (2017) Evolution of indoor positioning technologies: a survey. J Sens 2017(2630413):21. https://doi.org/10.1155/2017/2630413 8. Zafari F, Gkelias A, Leung KK (2019) A survey of indoor localization systems and technologies. In: IEEE communications surveys & tutorials, vol 21, no 3, pp 2568–2599, thirdquarter 2019. https://doi.org/10.1109/COMST.2019.2911558 9. Sospedra JT, Montoliu R, Trilles S, Belmonte Ó, Huerta J (2015) Comprehensive analysis of distance and similarity measures for Wi-Fi fingerprinting indoor positioning systems. Expert Syst Appl 42(23):9263–9278 10. Abishek R, Abishek KR, Hariharan N, Rakesh Vaideeswaran M, Sundara Paripooranan C (2019) Analysis of machine learning algorithms for wi-fi-based indoor positioning system. In: 2019 TEQIP III sponsored international conference on microwave integrated circuits, photonics and wireless networks (IMICPW), Tiruchirappalli, India, pp 218–222. https://doi.org/10.1109/ IMICPW.2019.8933285 11. Jo HJ, Kim S (2018) Indoor smartphone localization based on LOS and NLOS identification. Sensors (Basel, Switzerland) vol 18(11):3987. https://doi.org/10.3390/s18113987 12. Wei J et al (2018) SP-Loc: a crowdsourcing fingerprint based shop-level indoor localization algorithm integrating shop popularity without the indoor map. Int J Distrib Sens Netw. https:// doi.org/10.1177/1550147718815637 13. Zhou B, Li Q, Mao Q, Tu W (2017) A robust crowdsourcing-based indoor localization system. Sensors 17(4):864. https://doi.org/10.3390/s17040864 14. Yang S, Dessai P, Verma M, Gerla M (2013) Freeloc: calibration-free crowdsourced indoor localization. In: INFOCOM, 2013 Proceedings. IEEE, pp 2481–2489 15. Gu F, Niu J, Duan L (2017) Waipo: a fusion-based collaborative indoor localization system on smartphones, vol 25, no 4. IEEE, pp 2267–2280 16. Seco F, Jimenez AR, Prieto C, Roa J, Koutsou K (2009) A survey of mathematical methods for indoor localization. In: 2009 IEEE international symposium on intelligent signal processing, Budapest, Hungary, pp 9–14.https://doi.org/10.1109/WISP.2009.5286582 17. Zhou X, Chen T, Guo D et al (2018) From one to crowd: a survey on crowdsourcing-based wireless indoor localization. Front Comput Sci 12:423–450. https://doi.org/10.1007/s11704017-6520-z 18. Bahl P, Padmanabhan VN (2000) RADAR: an in-building RF-based user location and tracking system. In: Proceedings of the nineteenth annual joint conference of the IEEE computer and communications societies (INFOCOM 2000), Tel Aviv, Israel, 26–30 March 2000; vol 2, pp 775–784 19. Varshavsky A, de Lara E, Hightower J, LaMarca A, Otsason V (2007) GSM indoor localization. Pervasive Mob Comput 3(6):698–720 20. Rabinowitz M, Spilker Jr JJ (2004) A new positioning system using television synchronization signals. In: Eggert RJ (ed) Evaluating the navigation potential of the national television system committee broadcast signal, Ph.D. dissertation, Air Force Institute of Technology, OH, USA 21. Popleteev A, Osmani V, Mayora O (2012) Investigation of indoor localization with ambient FM radio stations. In: Proceedings of PerCom-2012. IEEE, pp 171–179, (acceptance rate 11) 22. Shi S, Sigg S, Zhao W, Ji Y (2014) Monitoring attention using ambient FM radio signals. IEEE Pervasive Comput 13(1):30–36 23. Youssef M, Agrawala A (2014) The horus WLAN location determination system. In: Proceedings of the 3rd international conference on Mobile systems, applications, and services (MobiSys ‘05). Association for Computing Machinery, New York, NY, USA, pp 205–218. https://doi. org/10.1145/1067170.1067193 24. Abdullah OA, Abdel-Qader I (2018) Machine learning algorithm for wireless indoor localization. In: Farhadi H (ed) Machine learning—advanced techniques and emerging applications, IntechOpen, 9. https://doi.org/10.5772/intechopen.74754 25. Milioris D, Kriara L, Papakonstantinou A, Tzagkarakis G (2010) Empirical evaluation of signal strength fingerprint positioning in wireless LANs. In: ACM international conference on modeling, analysis and simulation of wireless and mobile systems

466

L. Gao and S. Konomi

26. Mirowski P, Steck H, Whiting P, Palaniappan R, MacDonald M, Ho TK (2011) KL-divergence kernel regression for non-Gaussian fingerprint based localization. In: Proceedings of the international conference on indoor positioning and indoor navigation 27. Abdullah O, Abdel-Qader I, Bazuin B (2016) A probability neural network-Jensen-Shannon divergence for a fingerprint based localization. In: 2016 Annual conference on information science and systems (CISS), Princeton, NJ, USA, pp 286–291. https://doi.org/10.1109/CISS. 2016.7460516 28. El-Kafrawy K, Youssef M, El-Keyi A (2011) Impact of the human motion on the variance of the received signal strength of wireless links. In: Proceedings of the 22nd personal indoor and mobile radio communications (PIMRC). IEEE, pp 1208–1212 29. Lashkari B, Rezazadeh J, Farahbakhsh R, Sandrasegaran K (2019) Crowdsourcing and sensing for indoor localization in IoT: a review. IEEE Sens J 19(7):2408–2434. https://doi.org/10.1109/ JSEN.2018.2880180 30. Howe J (2009) Crowdsourcing, New York: three rivers press 31. Brabham D (2008) Crowdsourcing as a model for problem solving: an introduction and cases (PDF). Convergence Int J Res New Media Technol 14(1):75–90 32. Zheng F, Tao R, Maier HR, See L, Savic D, Zhang T et al (2018) Crowdsourcing methods for data collection in geophysics: state of the art, issues, and future directions. Rev Geophys 56:698–740. https://doi.org/10.1029/2018RG000616 33. Wu C, Yang Z, Liu Y (2014) Smartphones based crowdsourcing for indoor localization. IEEE Trans Mob Comput 14:444–457 34. Zhou B, Li Q, Mao Q, Tu W, Zhang X, Chen L (2015) ALIMC: Activity landmark-based indoor mapping via crowdsourcing. IEEE Trans Intell Transp Syst 16:2774–2785 35. Wang H, Sen S, Elgohary A, Farid M, Youssef M, Roy Choudhury R (2012) No need to wardrive: unsupervised indoor localization. In: Proceedings of the 10th international conference on Mobile systems, applications, and services (MobiSys ‘12). Association for Computing Machinery, New York, NY, USA, 197–210 (2012). https://doi.org/10.1145/2307636.2307655 36. Abdelnasser H, Mohamed R, Elgohary A, Alzantot MF, Wang H, Sen S, Choudhury RR, Youssef M (2016) Semanticslam: Using environment landmarks for unsupervised indoor localization. vol. 15, no. 7. IEEE, pp. 1770–1782, 2016. 37. Sikeridis D, Rimal BP, Papapanagiotou I, Devetsikiotis M (2018) Unsupervised crowd-assisted learning enabling location-aware facilities. IEEE Internet Things J 5(6):4699–4713 38. Settles B (2009) Active learning literature survey. University of Wisconsin-Madison Department of Computer Sciences 39. Kangkang L, Xiuze Z, Fan L, Wenhua Z, Alterovitz G (2019) Deep probabilistic matrix factorization framework for online collaborative filtering. IEEE Access 7:56117–56128 40. Xu Q, Zheng R (2017) When data acquisition meets data analytics: a distributed active learning framework for optimal budgeted mobile crowdsensing. In: Proceedings of IEEE conference on computer communications (IEEE INFOCOM 2017), Atlanta, GA, pp 1–9. https://doi.org/10. 1109/infocom.2017.8057034 41. Liu D, Liu Y (2019) An active learning algorithm for multi-class classification. Pattern Anal Appl 22(3):1051–1063 42. Settles B (2012) Active learning: synthesis lectures on artificial intelligence and machine learning. Carnegie Mellon University 43. Bernard J, Zeppelzauer M, Lehmann M, Müller M, Sedlmair M (2018) Towards user-centered active learning algorithms. Comput Graph Forum 37(3):121–132 44. Shuji H, Peiying H, Peilin Z, Steven CHH, Miao C (2018) Online active learning withexpert advice. ACM Trans Knowl Disc Data 12(5):1–22. https://doi.org/10.1145/3201604 45. Gilyazev RA, Turdakov DY (2018) Active learning and crowdsourcing: a survey of annotation optimization methods. Program Comput Softw 44(6):476–491 46. Lease M (2011) On quality control and machine learning in crowdsourcing. In: Proceedings of the11th AAAI conference on human computation (AAAIWS’11 2011), pp 97–102 47. Costa J, Silva C, Antunes M, Ribeiro B (2011) On using crowdsourcing and active learning to improve classification performance. In: International conference on intelligent systems design and applications (ISDA 2011), pp 469–474

A Cost-Effective and Quality-Ensured Framework …

467

48. Luo C, Hong H, Chan MC (2014) PiLoc: a self-calibrating participatory indoor localization system. In: IPSN-14 proceedings of the 13th international symposium on information processing in sensor networks, Berlin, Germany, pp 143–153. https://doi.org/10.1109/IPSN. 2014.6846748 49. Seung HS, Opper M, Sompolinsky H (1992) Query by committee. In: Proceedings of the ACM workshop on computational learning theory, pp 287–294 50. McCallum A, Nigam K (1998) Employing EM in pool-based active learning for text classification. In: Proceedings of the international conference on machine learning (ICML). Morgan Kaufmann, pp 359–367 51. Wang Y et al (2017) Spectral clustering based on JS-divergence for uncertain data. In: 2017 IEEE international conference on systems, man, and cybernetics (SMC), Banff, AB, pp 1972– 1975. https://doi.org/10.1109/SMC.2017.8122907 52. Roy N, McCallum A (2001) Toward optimal active learning through sampling estimation of error reduction. In: Proceedings of the international conference on machine learning (ICML). Morgan Kaufmann, pp 441–448 53. Calderoni L, Ferrara M, Franco A, Maio D (2015) Indoor localization in a hospital environment using Random Forest classifiers. Expert Syst Appl 42(1):125–134 54. Varma PS, Anand V (2020) Random forest learning based indoor localization as an IoT service for smart buildings. Wirel Pers Commun. https://doi.org/10.1007/s11277-020-07977-w 55. Guo X, Ansari N, Li L, Li H (2018) Indoor localization by fusing a group of fingerprints based on random forests. IEEE Internet Things J 5(6):4686–4698. https://doi.org/10.1109/JIOT.2018. 2810601

Intelligent Connected Vehicle Information System (CVIS) for Safer and Pleasant Driving Xin Zhou, Jingyue Zheng, and Wei Zhang

Abstract By continuously exchanging information, connected vehicle (CV) and vehicle-to-everything (V2X) technologies can increase the connectivity of the road traffic system. In this article, two case studies were conducted under the environment of the intelligent CV. Study 1 performed focus group interviews and a questionnaire survey to investigate the preferences of Chinese drivers concerning the design of connected vehicle information systems (CVISs). The results show that participants preferred visual warning signals and descriptive verbal speech. Safety-related functions, such as providing rescue services, vehicle malfunction reminders, and information on hazards in the surrounding area, were valued most among participants. Study 2 investigated the design of vibration warning signals for quicker reaction. Simple reaction time (SRT) and the choice reaction time (CRT) were respectively measured under vibration warnings in two experiments. Results show that the response time was the shortest when vibrations were presented at ears, compared to the vibrations at wrists and shins. The S-R compatibility effect during all tasks was not significant. Keywords Connected vehicle · Warning signal · User preference and attitude · Vibration positions · Response effector · Stimulus–response compatibility

1 Introduction The existing road traffic system consists of various types of road users, vehicles, and the complex surrounding environment [1]. Information flow in the system is bidirectional: the road users receive others’ information as well as generate new information for others. However, this communication system based on traffic lights and horn is inefficient and complicated for several reasons. One major limitation is the restricted methods for drivers to express their intentions to other road users. To address the insufficiency and reduced conciseness problems in the road traffic system, researchers have developed and applied connected vehicle information X. Zhou · J. Zheng · W. Zhang (B) Department of Industrial Engineering, Tsinghua University, Beijing, P.R. China e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Switzerland AG 2023 V. G. Duffy et al. (eds.), Human-Automation Interaction, Automation, Collaboration, & E-Services 11, https://doi.org/10.1007/978-3-031-10784-9_28

469

470

X. Zhou et al.

systems (CVISs) in recent years. Transmitting information via connected vehicle (CV) technologies, CVISs are expected to extend the roadway communication bandwidth by utilizing screens, speakers, and even haptic devices to help convey the intention of road users, as well as other important road information, faster and more accurately. CV and vehicle-to-everything (V2X) technologies can enhance vehicle, pedestrian, and road infrastructure connectivity [2]. There is consensus among previous research that CV technologies have the potential to improve traffic efficiency, road safety, and user comfort [3–7]. Both simulator-based laboratory studies and field studies have been conducted to validate the concept, design, and performance of CVISs. The National Highway Traffic Safety Administration (NHTSA) of the U.S. released a proposal to mandate the deployment of vehicle-to-vehicle (V2V) communications technology in all new US light-duty vehicles [8]. Under the environment of connected vehicles, various collision avoidance systems such as forward collision warning (FCW), blind-spot warning (BSW), lane departure warning (LDW), lane change warning (LCW), and intersection movement assist (IMA) will warn drivers of impending crash situations, reduce the number and severity of motor vehicle crashes [9]. In summary, it is challenging to emphasize users’ preferences toward CVIS design to promote safe, effective, and extensive road use. To address this challenge, we adopted a two-step research approach. In the first step, we wanted to gather users’ demand and preference for CVIS. A focus group interview was used to qualitatively explore participants’ opinions regarding the design of CVISs, including the provided functions, preferred warning signals, information presentation methods. Subsequently, the gathered information was used to formulate a questionnaire. The questionnaire survey was conducted in China to quantitatively investigate the preferences of drivers regarding the design of CVISs. In the second step, we wanted to design and test the vibration warning for reaction time reduction. A driving simulator experiment was performed.

2 Case Study 1: Drivers’ Preference for the Design of CVIS: Warning Signal and Function Settings 2.1 Aims As a technical system, CVISs must be appropriately designed to satisfy the preferences of their potential users. Efforts in this regard have been previously reported; such efforts include an exploration of the design of driving fatigue warning systems [10] and in-vehicle anger intervention systems [11]. Currently, while many studies have focused on the technological aspect of CVISs [12–15], there is still a knowledge gap concerning driver preferences and viewpoints on the functions and interface

Intelligent Connected Vehicle Information System (CVIS) …

471

design of CVISs, which not only affects user acceptance, but also its effectiveness during potential use. One fundamental problem of CVIS design is identifying the demands of the user and defining the designed functions. Providing proper information can improve the situational awareness of drivers and avoid critical situations with an effective early warning system [7]. Another primary concern of CVIS design is the human-machine interface (HMI), which defines how to warn drivers on time and how to present the information accurately. Good HMI design, using appropriate warning signals and providing concise information, could help drivers be aware of potentially difficult situations timely and clearly, and promote an effective driver response [16]. Understanding the attitudes and preferences of users in CVISs is required for providing design input, while also promoting a willingness to use such systems.

2.2 Method Focus Group A total of 21 drivers participated in the 2-h focus group interviews, divided into three groups with seven participants in each group (see Table 1). They were recruited via WeChat (A Chinese social media App by Tencent Co.). All participants signed an informed consent form and completed the demographic questionnaire before the focus group interviews. Then, an experiment facilitator stated that the objective of the focus group was to gather the opinion and demands of drivers regarding the design of CVISs. All focus group sessions were video and audio recorded, and participants were guaranteed that their privacy would be protected. The focus group interviews were conducted in a semi-structured format, revolving around several open-ended questions (see Table 2). Open-ended questions were used to engage the participants in the discussion gradually. Then, transition questions were Table 1 Demographic information of participants Focus groups

Questionnaire survey

Gender Male

14 (66.7%)

625 (58.7%)

Female

7 (33.3%)

440 (41.3%)

Age

34.5 (SD = 5.5)

30.92 (SD = 8.46)

Annual driving distance (1,000 km)

41.3 (SD = 34.7)

19.92 (SD = 54.16)

Driving years

8.5 (SD = 3.3)

4.87 (SD = 4.65)

Education level High school or below

19%

16.3%

Bachelor/college certificate

61.9%

51.2%

Master or above

19%

32.5%

472 Table 2 Set of questions for the focus group interviews

X. Zhou et al. (1) Please give us a brief introduction of yourself. (2) Describe your experience of inefficient roadway communication problems. (3)What functions do you expect from CVISs? (4) Which types of warning signals do you prefer? (5) How do you like to present the information? (6) Do you have any other suggestions or comments for the design of CVISs?

used to lead the key issues that focused on the research objectives. Finally, the ending questions tied the sessions together and provided closure [17]. Questionnaire Survey The questionnaire was developed on a professional online survey platform (www. wjx.com) and distributed through WeChat. Answers were collected automatically. Each participant received 15 Chinese Yuan as an incentive if the questionnaire was valid. The survey received 1,065 valid responses. Most of the participants were from economically developed regions in China, such as Beijing, Jiangsu, and Shanghai, which are likely to have more potential CVIS users. The survey questionnaire was based on the outcome of the focus group interviews. The first section collected the participants’ demographic information, including gender, age, location, educational background (“high school or below,” “bachelor’s degree/college certificate,” or “master’s degree or above”), driving experience, estimated annual driving distance, and traffic citations and collisions during the past three years. The second section focused on the design of CVISs, including the evaluation of system functions, warning signal, and information presentation preference toward CVIS. The questionnaire first provides some background information about CVISs and required the participants to imagine the scenario that they were driving a car that can exchange information with other road users and the road infrastructure. Participants were required to rate their agreement on a five-point Likert scale, ranging from 1 (“strongly disagree”) to 5 (“strongly agree”). Data Analysis All audio recordings from each focus group session were transcribed verbatim and analyzed using an Excel file [10]. During the transcription procedure, the experimenter may consult the video recording to complement the audio recordings. First, the original transcripts were broken into complete sentences. Then, these sentences were sorted and classified. The categories were identified before sorting and included “reasons for the on-road information inefficiency problems,” “different warning signal modalities,” “different information presentation tactics,” and “functions of CVISs.” The sentence that could not suit any predefined category would be put into “other suggestions.” Then, the sorting results were merged into one file. Conflicting items were voted for final sorting.

Intelligent Connected Vehicle Information System (CVIS) …

473

In terms of the questionnaire data analysis, one-way ANOVA was conducted to test the preference difference for different modalities of warning and information presentation tactics. Post hoc analysis was used for multiple comparisons with Bonferroni adjustment. The significance level of alpha for all analyses was 0.05. The data analysis was conducted using R studio (version 1.3.959).

2.3 Results and Conclusion Reasons for the On-road Information Inefficiency Problems When required to “describe the experience of inefficient roadway communication problems,” participants in the focus groups consistently considered it as a prevalent phenomenon and a significant cause of traffic accidents. Four types of reasons were given, as well as the corresponding frequencies (see Table 3). CVIS Functions As shown in Table 4, Participants in the focus group interviews mentioned three types of CVIS functions: (1) improving driving safety by exchanging information (37 times, 72.5%); (2) improving efficiency by adjusting traffic flow (9 times, 17.6%); (3) improving driving comfort by offering information service (5 times, 9.8%). The three types of functions are coherent to the possible advantages of CV technologies. All ratings of functions were above 3 (neutral attitude). Particularly, the first three functions are safety-related. Providing rescue services to drivers and passengers in case of danger achieves the highest rating (M = 4.31, SD = 0.79). The second was reminding the vehicle’s abnormal situations (M = 4.29, SD = 0.79). The third was transmitting information about the surrounding danger (M = 4.23, SD = 0.78). Conversely, the three lowest-rated functions were providing the personality traits of the surrounding drivers (M = 3.39, SD = 1.04), providing advice on driving Table 3 Reasons for the on-road information inefficiency problems

Reason

Frequency

Confined signals for roadway communication

7

Driver limitations Unable to judge the distance or speed.

9

Impaired concentration.

6

Blocked sight.

3

Environment limitations Dense and complex road conditions.

2

Unreasonable road facility planning.

2

Poorly readable signs.

1

Others

3

474

X. Zhou et al.

Table 4 Evaluation of CVIS functions Functions

Mean

SD

Traffic safety Provide rescue services to drivers and passengers.

4.31

0.79

Remind the vehicle’s unusual situations.

4.29

0.79

Transmit information about the surrounding danger.

4.23

0.78

Inform the real-time status of the surrounding drivers.

4.05

0.90

Transmit the near actions of the surrounding drivers.

3.97

0.82

Provide necessary information about surrounding vehicles.

3.88

0.87

Provide the personality traits of the surrounding driver.

3.39

1.04

Inform the traffic lights’ status.

3.92

0.82

Dynamically regulate traffic lights according to the traffic flow.

3.82

0.94

Provide a guide on the next driving operations.

3.79

0.82

4.09

0.80

Traffic efficiency

Traffic convenience Provide information services.

operations (M = 3.79, SD = 0.82), and dynamically regulating traffic lights according to the traffic flow (M = 3.82, SD = 0.94). Warning Signals Types and Information Presentation Methods The full evaluation of warning signals is listed in Table 5. Participants from the focus groups mentioned different modalities of warning signals. Most of the warning signals include only one dimension. Only two signals were a combination of different modalities, so further investigation focused on one modality. Generally, the rating of three modalities varied significantly, F(2, 3192) = 248.8, p < 0.001, η2 = 0.14. Pairwise comparisons revealed that participants showed a significantly higher preference for visual warning signals (M = 3.30, SD = 0.65) than auditory warning signals (M = 3.17, SD = 3.17, p < 0.001) and tactile warning signals (M = 2.63, SD = 0.86, p < 0.001). To be specific, in terms of visual warnings (mentioned eleven times), some participants suggested CVISs could present visual icons on the instrument panel or head-up display (HUD) and use a flashing red light. However, many participants in the focus group disagreed with visual stimuli because they considered it could be easily neglected and hardly attract their attention. Most participants in the focus group accepted auditory stimuli (mentioned eleven times), including alarm and verbal warnings. Participants agreed that auditory stimuli could quickly draw the driver’s attention and would not occupy a visual resource. Participants also mentioned that an alarm could convey a feeling of tension. One participant in particular worried that ambient noise, or music being played in the car, may drown an auditory stimulus. With regards to tactile warnings (mentioned twelve times), some participants expressed their preference for this type of signal because it can only be perceived by the driver and is not easily disturbed by external

Intelligent Connected Vehicle Information System (CVIS) …

475

Table 5 Types of warning signals Modality

Details

Mean

SD

Visual

Show icons on HUD.

3.52

0.90

Show icons on the dashboard.

3.51

0.83

The light bar mounted on the steering wheel starts to flash.

3.28

0.95

Red light flashes in the car.

2.91

1.00

Play short, eloquent speech.

3.69

0.87

Gradual siren sounds.

3.27

0.95

Play high-frequency alarm.

2.57

0.95

The steering wheel begins to vibrate.

2.91

1.12

The seat belt is tightened.

2.91

1.11

The seat begins to vibrate.

2.56

1.05

The foot pedal begins to vibrate.

2.51

1.03

Slight current stimulus.

2.23

1.03

Auditory

Tactile

factors. Some participants suggested that vibrators could be embedded in the driver seat, steering wheel, and brake pedal. Regarding information presentation methods, participants suggested presenting the information through the visual tunnel (mentioned eight times) by combining a brief text description with visual icons on the in-vehicle screen. Some participants suggested that Augmented Reality (AR) could be a useful method with the necessary information directly attached to the virtual scenario. In terms of the auditory tunnel (mentioned ten times), some participants suggested using a brief verbal description, which could be embedded in the process of sending verbal warnings. Finally, one participant proposed that predefined tactile patterns could be utilized to convey information.

3 Case Study 2: Vibration Warning Design for Reaction Time Reduction 3.1 Research Aims and Hypotheses Under the environment of connected vehicles, we need to display different information to drivers through different channels. Information that is not urgent and does not require a driver to respond will be displayed through a visual channel. The in-vehicle screen could convey complex, long, and many useful messages to drivers, whereas, visual warnings are not sufficiently noticeable and are easily missed, so it is not appropriate to convey critical messages [18]. Compared to the visual channel, the auditory channel is good to convey high-priority messages. Thus, the information

476

X. Zhou et al.

that requires driver attention but is not urgent, such as “pay attention to the rear left vehicle” will be presented through visual plus auditory channels. Previous research [19, 20] indicates using auditory warnings could catch drivers’ attention effectively and reduce perception-reaction times. However, the sound of the navigation system, load music, conversations with people in the car may also mask important auditory warnings. Therefore, a vibration channel is often been used to convey important and urgent messages which need a driver response quickly and correctly to avoid a collision. Steering vibration [21] and seat vibration [22–24] both effectively reduced the reaction time. In Spence and Ho’s [25] review, the question “Is there a certain region of the body (of the space surrounding the body) where tactile/multisensory warning signals are particularly effective” was highlighted and awaiting further research. “Effective” contains two goals: rapidly and correctly. When perceiving an urgent warning, the driver must not only respond promptly but also select an appropriate action. For example, the forward collision warning systems alert drivers to brake quickly to avoid a rear-end crash. The lane departure warning systems alert the drivers to adjust direction by steering as soon as possible. Furthermore, intersection movement assistant systems alert drivers of an impending crash that requires drivers to observe and decide what to do. According to Ho and Spence’s [26] research, the compatibility between the vibration position and the required behavioral response (stimulus-response compatibility, S-R) may influence the effectiveness of vibrotactile warning. In this study, we will explore in which region of the body, the tactile warning signals are particularly effective. We will use a simple reaction time task and a choice reaction time task to test two hypotheses. H1: In the simple reaction time task, the closer the vibration is presented to the brain, the faster the simple reaction time. H2: In the choice reaction time task, when the vibration stimulus and the response body part are compatible, the vibration warning is more effective.

3.2 Methods Twenty-four participants (mean age of 25.3, age ranged from 20 to 37, SD = 4.54; 12 males and 12 females) took part in this experiment. All participants reported having a normal sense of touch and a capacity of differentiating colors and were right-handed. To investigate the effectiveness of vibration at different body parts and S-R compatibility, we chose three vibration positions: wrist, shin, and ear. A vibration motor module (24 * 25 mm, HW-738, DC5V, PWM) was used to present the vibrotactile warnings. In each trial, the tactor was fastened to the back of the participants’ wrists, ears, and the lateral of shins (about 5 cm above their ankles) with medical tape (see Fig. 1). The response body parts are hand and foot. Among the 3 (vibration position) × 2 (response body part) combinations, only the shin-foot and wrist-hand conditions follow the S-R compatibility, while others do not.

Intelligent Connected Vehicle Information System (CVIS) …

477

Fig. 1 Examples of vibrators attached to body parts [27]. a Vibration at the ear. b Vibration at the wrist. c Vibration at the shin

In task 1, participants would complete 13 hand-response trials (press the button on the steering as fast as possible) and 13 foot-response trials (press the brake pedal as fast as possible) at fixed order per block. Each trial began with the presentation of a vibrotactile stimulus for 100 ms, terminated after the participants’ response. The inter-trial interval was a range from 4 to 8 s randomly (on average, 10 trials/min). The reaction time in task 1, we called simple reaction time (SRT). In task 2, after a 100 ms vibrotactile stimulus, a LED (mounted behind the steering wheel, 60 cm in front of the participants) would flash blue or red randomly. In each block, blue and red appear 13 times respectively. Participants were requested to press the button when the LED was blue and press the brake pedal down when the LED was red. After the correct response, the LED will go off then a new trial begins. The inter-trial interval was the same as task 1. The reaction time in task 2, we called it choice reaction time (CRT). Compared to the SRT, the extra time in the CRT is to identify the LED’s color and determine which response should be made. Thus, we used the CRT minus SRT, getting the extra time, we called it decision time (DT).

3.3 Results and Conclusion The results (see Fig. 2) showed that participants responded most rapidly to vibration presented at ears, both in the simple reaction task and choice reaction task. Participants also responded significantly more rapidly with their hands than with their feet. The S-R compatibility effect during all tasks was not significant. It only exists in the decision time. The decision time under the compatible shin-foot combination was shorter than the vibration presented at ears or wrists. From the results of our study, the following recommendations are formulated to help reduce reaction time and improve road safety. The ear is the most recommended part for conveying a vibration warning, then is the wrist. The wristbands and wireless headsets could be considered to provide vibration warnings. These wearable devices allow the driver to receive vibration warnings directly regardless of their posture. If the driver needs to make a quick and accurate operation by observing the

478

X. Zhou et al.

Fig. 2 Average time under different vibration positions [27]

road conditions after the vibration warning, the S-R compatibility can be properly considered.

References 1. Ba Y, Zhang W (2011) A review of driver mental workload in driver-vehicle-environment system. In: Rau PLP (ed) Internationalization, design and global development. Springer, Berlin Heidelberg, Berlin, Heidelberg, pp 125–134 2. Harding J, Powell G, Yoon R, Fikentscher J, Doyle C, Sade D, Lukuc M, Simons J, Wang J, others (2014) Vehicle-to-vehicle communications: readiness of V2V technology for application. United States. National Highway Traffic Safety Administration 3. Brell T, Philipsen R, Ziefle M (2019) Suspicious minds?—users’ perceptions of autonomous and connected driving. Theor Issues Ergon Sci 20:301–331. https://doi.org/10.1080/1463922X. 2018.1485985 4. Brell T, Philipsen R, Ziefle M (2019) sCARy! risk perceptions in autonomous driving: the influence of experience on perceived benefits and barriers. Risk Anal 39:342–357. https://doi. org/10.1111/risa.13190 5. Olaverri-Monreal C, Jizba T (2016) Human factors in the design of human-machine interaction: an overview emphasizing V2X communication. IEEE Trans. Intell. Veh. 1:302–313. https:// doi.org/10.1109/TIV.2017.2695891 6. Payre W, Diels C (2019) Designing in-vehicle signs for connected vehicle features: does appropriateness guarantee comprehension? Appl Ergon 80:102–110. https://doi.org/10.1016/j.ape rgo.2019.05.006 7. Payre W, Diels C (2020) I want to brake free: the effect of connected vehicle features on driver behaviour, usability and acceptance. Appl Ergon 82:102932. https://doi.org/10.1016/j.apergo. 2019.102932 8. NHTSA: Vehicle-to-vehicle communication technology for light vehicles: preliminary regulatory impact analysis. United States Department of Transportation, Washington, D C: HTSA (2016) 9. NHTSA: Vehicle-to-vehicle communications: readiness of V2V technology for application (2014) 10. Meng F, Li S, Cao L, Peng Q, Li M, Wang C, Zhang W (2016) Designing fatigue warning systems: the perspective of professional drivers. Appl Ergon 53:122–130. https://doi.org/10. 1016/j.apergo.2015.08.003 11. Li S, Zhang T, Liu N, Zhang W, Tao D, Wang Z (2020) Drivers’ attitudes, preference, and acceptance of in-vehicle anger intervention systems and their relationships to demographic and personality characteristics. Int J Ind Ergon 75:102899. https://doi.org/10.1016/j.ergon. 2019.102899 12. Amoozadeh M, Raghuramu A, Chuah C-N, Ghosal D, Zhang HM, Rowe J, Levitt K (2015) Security vulnerabilities of connected vehicle streams and their impact on cooperative driving. Ieee Commun Mag 53:126–132. https://doi.org/10.1109/MCOM.2015.7120028

Intelligent Connected Vehicle Information System (CVIS) …

479

13. Feng Y, Head KL, Khoshmagham S, Zamanipour M (2015) A real-time adaptive signal control in a connected vehicle environment. Transp Res Part C Emerg Technol 55:460–473. https:// doi.org/10.1016/j.trc.2015.01.007 14. Ibanez JAG, Zeadally S, Contreras-Castillo J (2015) Integration challenges of intelligent transportation systems with connected vehicle, cloud computing, and internet of things technologies. IEEE Wirel Commun 22:122–128 15. Khondaker B, Kattan L (2015) Variable speed limit: a microscopic analysis in a connected vehicle environment. Transp Res Part C Emerg Technol 58:146–159. https://doi.org/10.1016/ j.trc.2015.07.014 16. Mirage events & driver haptic steering alerts in a motion-base driving simulator: a method for selecting an optimal HMI. Appl Ergon 65:90–104 (2017). https://doi.org/10.1016/j.apergo. 2017.05.009 17. Newman LC (2002) Macroergonomic methods: interviews and focus groups. Th Annu Meet 5 18. Campbell JL, Richard CM, Brown JL, McCallum M (2007) Crash warning system interfaces: human factors insights and lessons learned 19. Graham R (1999) Use of auditory icons as emergency warnings: evaluation within a vehicle collision avoidance application. Ergonomics 42:1233–1248. https://doi.org/10.1080/001401 399185108 20. Kramer G (1994) An introduction to auditory display. In: Auditory display: sonification, audification & auditory interfaces 21. Suzuki K (2003) An analysis of driver’s steering behaviour during auditory or haptic warnings for the designing of lane departure warning system. JSAE Rev 24:65–70. https://doi.org/10. 1016/S0389-4304(02)00247-3 22. Fitch GM, Hankey JM, Kleiner BM, Dingus TA (2011) Driver comprehension of multiple haptic seat alerts intended for use in an integrated collision avoidance system. Transp Res Part F Traffic Psychol Behav 14:278–290. https://doi.org/10.1016/j.trf.2011.02.001 23. Sayer TB, Sayer JR, DevonshIre JMH (2005) Assessment of a driver interface for lateral drift and curve speed warning systems: mixed results for auditory and haptic warnings. In: Driving assessment 2005: proceedings of the 3rd international driving symposium on human factors in driver assessment, training, and vehicle design. University of Iowa, Rockport, Maine, USA, pp 218–224 24. Stanley LM (2006) Haptic and auditory cues for lane departure warnings. Proc Hum Factors Ergon Soc Annu Meet 50:2405–2408. https://doi.org/10.1177/154193120605002212 25. Spence C, Ho C (2008) Tactile and multisensory spatial warning signals for drivers. IEEE Trans Haptics 1:121–129. https://doi.org/10.1109/TOH.2008.14 26. Ho C, Spence C (2014) Effectively responding to tactile stimulation: do homologous cue and effector locations really matter? Acta Psychol (Amst) 151:32–39. https://doi.org/10.1016/j.act psy.2014.05.014 27. Zheng J, Zhang T, Ma L, Wu Y, Zhang W (2021) Vibration warning design for reaction time reduction under the environment of intelligent connected vehicles. Applied Ergonomics 96103490-S000368702100137X 103490. https://doi.org/10.1016/j.apergo.2021.103490

Travel Behaviour and Mobility in Smart Cities: An Interdisciplinary Review of Mass Transit in a Smart City in Malaysia Santha Vaithilingam, Pei-Lee Teh, Pervaiz K. Ahmed, Chee Pin Tan, and Sui-Jon Ho Abstract Sustainability agendas are inextricably linked to modern urban planning, and while such pursuits abound in today’s cities, their scale varies in scope and in effectiveness. This study provides a use-centric framework to assess the effectiveness of sustainable transportation services based on how readily they can be accessed/used (User Agency), how quickly they help commuters traverse distances (Performance), how well they maintain service levels over larger distances and/or volume (Scalability), and how affordable they are (Cost). Research is conducted on Sunway City, a Malaysian planned township, selected for its rapid developmental trajectory and diverse portfolio of sustainable transportation initiatives (referred to as ‘SITE’ projects). By analysing adoption and usage behaviour from 306 survey and 90 focus group participants, this study endeavours to summarize the effectiveness of major programs/facilities that support both inter- and intra-city travel. Finally, the authors will make recommendations on areas of improvements through use-centric ‘developmental profiles’ that direct potential intervening measures to priority commuter requirements. These intervening measures, in turn, are anchored by the 7i-Innovation

S. Vaithilingam · P. K. Ahmed Institute for Global Strategy and Competitiveness, Sunway University, 47500 Bandar Sunway, Selangor Darul Ehsan, Malaysia e-mail: [email protected] P. K. Ahmed e-mail: [email protected] P.-L. Teh (B) · S.-J. Ho School of Business, Monash University Malaysia, 47500 Bandar Sunway, Selangor Darul Ehsan, Malaysia e-mail: [email protected] P.-L. Teh Gerontechnology Laboratory, Monash University Malaysia, 47500 Bandar Sunway, Selangor Darul Ehsan, Malaysia C. P. Tan School of Engineering, Monash University Malaysia, 47500 Bandar Sunway, Selangor Darul Ehsan, Malaysia e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Switzerland AG 2023 V. G. Duffy et al. (eds.), Human-Automation Interaction, Automation, Collaboration, & E-Services 11, https://doi.org/10.1007/978-3-031-10784-9_29

481

482

S. Vaithilingam et al.

model which outlines the application of seven attributes (i.e. Infra-infostructure, Institutions, Intellectual Capital, Interaction, Incentives, Integrity) to ultimately facilitate the creation of a sustainable, knowledge-based economy, aligning with the prevailing national economic mandate. Keywords Travel behaviour · Smart city · Malaysia

1 Introduction and Background Sustainability became a key thrust in global developmental policies during the midtwentieth century and has since coloured pledges for economic, social, and political cooperation. Over this period, the scope of sustainable development has evolved from a single goal of sustainable use of natural resources toward a more diverse interdisciplinary framework that spans technology and innovation, knowledge-cultivation, environmentalism, and ethics. This global call-to-action has culminated in the 2030 Agenda for Sustainable Development outlined by the United Nations Development Programme (UNDP) in 2015, which provided a common blueprint comprising 17 Sustainable Development Goals (SDGs) to steer the priorities of all member countries. Sustainable transportation, in particular, is aligned to Goal 11 of the SDG framework (‘Sustainable Cities and Communities’). By itself, Goal 11 directly impacts infrastructural affordability and accessibility, health and safety, resource efficiency, economic resilience, and environmental performance across the entire urban ecosystem. Additionally, United Nations [23] notes that these outcomes would also introduce positive spillover effects that contribute to the attainment of six other additional SDGs. However, the many piecemeal initiatives undertaken so far have mostly fallen short of this aspiration. To reorient the transportation sector effectively, it is perhaps better to align solutions to user behaviour, rather than the inverse—thus, conceiving a well-designed assessment of factors that encourage or hinder the use of sustainable transport is an essential first step. As a developing country, Malaysia has implemented numerous travel and transport models over the recent years, at varying degrees of success. This study assesses the contributions made by a planned township in the country—Sunway City—towards sustainable transportation under its ‘Sustainable, Intelligent Transport Ecosystem’ (SITE) developmental programs. These analyses will be substantiated by a combination of focus group interviews and local surveys. Conclusions on the effectiveness of the township’s SITE initiatives will be based on the degree of user adoption and perception by commuters. Further, the recommendations of this study will provide valuable insights for other townships across the country and in other developing countries to implement SITE initiatives.

Travel Behaviour and Mobility in Smart Cities …

483

2 Literature Review Sustainable transport as a research area is an emergent one, brought about by the growing need to address environmental, resource, as well civic welfare concerns in modern society. However, the existing literature on the subject is largely fragmented, with studies being built on narrow hypotheses that focus on specific drivers of sustainable transport use. The majority of research is founded on establishing the reliability and accessibility of the different transport modes (e.g. [14]); psychological factors (e.g. [3, 7] or built environmental factors (e.g. [9, 24]. Other studies, for example, attitude-based studies such as Elias and Shiftan [10] focused on the role of the perceived environmental benefit of using public transport and perceived risk in using private cars. They found environmental concern and awareness do not contribute towards the intention to use such services, though the perceived risk of fatality from traffic accidents does. Eriksson et al. [11] identified fun, lifestyle match and feeling secure as the main drivers of intention to use sustainable transport, and time and costs as significant determinants as well. Meanwhile, Chen [6] used the modified Technology Acceptance Model (modified TAM) and the theory of planned behaviour (TPB) to understand the use of YouBike— a Taiwan based public bike system. The study identified perceived pleasure to use and subjective norms to have the strongest positive association with green loyalty. Other theories used to explain travel behaviour also include the theory of reasoned action (TRA; [12]), the norm activation model (NAM; [20]) and the transtheoretical model (TTM; [2, 18]). These theories assist in understanding the interaction effects of structural and psychosocial factors on behaviours [25]. Spears et al. [21] focused on consumers’ concerns about their safety and their impact on the use of public transportation as a sustainable mode of transport. These studies largely conform to the travel choice theoretical model developed by Golob, Horowitz, and Wachs [13] where psychological characteristics, namely—attitudes and perceptions, and travel choices mutually influence one another. On the other hand, Kandt et al. [15] suggest a multi-directional relationship between environmental attitudes, travel behaviour and long-term mobility choices (residential location and car ownership). Studies that tie in both attitudinal and environmental factors [4] suggests a strong association between the built environment and individuals’ travel behaviour where mixed land use (existence of residential and business premises in the neighbourhood), availability of transit service and good pedestrian infrastructure and aesthetics contributes to the use of non-motorised travel modes. Cao et al. [5] reviewed 38 empirical studies and concluded the significance of the built environment in influencing behaviour after accounting for attitude-induced residential self-selection. However, with the effects of attitudinal features accounted for, the impact of environmental factors drops significantly [2, 5].

484

S. Vaithilingam et al.

In addition to the above factors, other studies have also considered factors such as interaction with multi-stakeholders that is, forming partnerships between the stakeholders in a knowledge-driven society as critical factors [16, 17] for supporting transportation planning [1]. Stakeholder interaction is when key players are continuously engaged to act on relevant issues and enhance sustainable transport usage [1]. For example, the interaction between public transport companies, utility service providers, and community representatives is necessary to create appropriate travel solutions [22]. Furthermore, authors highlighted the importance of institutions (for example, governmental bodies) and municipalities that have a role in managing public transport companies. Institutions oversee and establish social rules and norms that structure social interactions between the stakeholders [17]. However, the formation of such multi-stakeholder relationships coupled with increasing access to information and knowledge gives rise to competition for resources, culminating in a compromise on standards [8]. Crosling et al. highlighted that sound integrity systems must be in place to counteract this and encourage transparency in decision-making. Overall, past studies have examined disparate factors in isolation—there is a lack of holistic understanding of the factors that influence the use of sustainable transport system in the current literature. This study undertakes to provide an integrated framework capturing the key enabling conditions that should be in place to deepen their impact in improving the use of sustainable transport modes.

3 Methodology This study implemented focus group discussions and structured questionnaire surveys to identify key contributors and deterrents to the willingness of individuals to use sustainable transport (ST) modes across a smart city. The heterogenous nature of user perceptions are a result of several factors, including the respondent’s attitude towards the use of transport services, socioeconomic status of the commuters and their perception of the enabling environment to facilitate mobility. Proper examination of these points-of-view is fundamental for assessing and evaluating the use of ST service. Finally, the aforementioned behavioural assessment will be accompanied by a qualitative outline of how transport sustainability fits into the broader, multifaceted goal of urban sustainability.

3.1 Research Location Sunway City was chosen as the subject for this case study due to the rapid development it has undergone over the past decades, which provides a richer profile for interpretation. This urban area is a Malaysian planned community located in the Petaling district of the state of Selangor Darul Ehsan, and the result of a brownfield

Travel Behaviour and Mobility in Smart Cities …

485

reclamation initiative spearheaded by Sunway Holdings Incorporated Berhad (later merged into Sunway Group). Currently one of the nation’s foremost smart cities, Sunway City is the site for numerous, localised ST initiatives, where new motor and non-motor transit alternatives incorporate many sustainable development concepts such as green materials, carbon-efficient logistics, and waste cycling. The township’s incipient years trace back to 1986 as an attempt to rehabilitate an 800-acre tin mine. This period saw the construction of two major sites—the Sunway Lagoon theme park, and the Sunway College campus—both of which would form the economic lynchpin of the township upon their completion in the early 1990s. Before the turn of the century, Sunway City further enforce its niche as a resort town and international education centre through the establishment of the Sunway Resort Hotel & Spa (1996), Sunway Pyramid themed mall (1997), and the Monash University Malaysia campus (1998). With an increasing population numbering at an estimated 200,000 as of end-2018,1 a commuter base spanning approximately 500,000 residents from local and adjacent districts,2 as well as 42 million footfalls annually attributed to the township’s leisure and hospitality destinations, the need for aligning the township’s civil infrastructure capacity to its booming growth is now more urgent than ever. Sunway City is one of the principal townships in the Klang Valley region, also referred to as the Greater Kuala Lumpur conurbation. Inter- and intra-city traffic congestions are frequent and severe across the many high-density municipalities due to reliance on arterial thoroughfares to connect major densified residential and commercial zones. Exacerbated by intense urban migration over the last two decades, the decline in travel quality has put increasing strain on economic productivity and societal well-being of the 7.5 million affected residents. As such, transport and travel sustainability are vital developmental metrics in the country’s post-2020 urban planning policies. Sunway City’s existing ST initiatives can contribute to the agendas outlined by the national development plans of the previous and upcoming decade—namely, the New Economic Model (NEM) and the Shared Prosperity Vision 2030 (SPV), respectively. Both growth charters place heavy emphasis on the development of knowledge economies, high-competence workforce, and industrial innovation to promote greater overall economic inclusivity.

3.2 Qualitative and Quantitative Analysis This study is divided into two phases. The first is a focus group discussion with existing and potential users to identify factors that influence their use of each transport mode, such as regularity, reliability, safety, comfort, cleanliness, fares, and environmental consciousness. Ninety respondents across 20 focus groups were randomly

1 2

https://ir2.chartnexus.com/sunway/doc/sustainability-reports/sr2018.pdf. https://ir2.chartnexus.com/sunway/doc/sustainability-reports/sr2019.pdf.

486

S. Vaithilingam et al.

selected from various Sunway City localities. Each group was collectively interviewed until consensus was reached, and no new insights emerged from the discussion. All the interviews3 were semi-structured, using mostly open-ended questions. The respondents highlighted the key factors that they perceived would facilitate the use of ST. The second phase which involved a structured questionnaire survey was conducted by adult respondents (over 18 years), and the average completion time was 20 min. Of the 500 questionnaires distributed, 306 were usable and returned, which represents a 61% response rate. The quantitative survey was conducted on a different set of respondents from focus group respondents. The respondents expressed a rate of satisfaction on each attribute, according to a scale of evaluation from strongly disagree to strongly agree. An incentive of RM25 (USD6) was provided to each focus group interviewee, while RM5 (USD1) was given to every survey questionnaire respondent. The consolidated findings from both phases are used to gauge the effectiveness of existing SITE services based on specific quality attributes [16, 17] that influence on individual’s intent-of-use. The identified determinants of desired travel behaviour would finally be translated into policy recommendations to enhance user adoption and perception of Sunway City’s SITE facilities in the future.

3.3 SITE and Non-SITE Travel Modes While Sunway Group maintains a broad portfolio of SITE initiatives, this study will focus its recommendations and guidance on the three most-used means of travel (SITE and non-SITE) identified by both survey and focus group respondents, namely: ● Personal vehicles—self-owned automobiles, and supporting parking infrastructure ● ‘Eco-walk’/‘Canopy Walk’—solar-powered canopied walkway ● BRT Sunway—electric bus rapid transit, integrated with the multicity Klang Valley Integrated Transit System. The survey also documents’ bridging facilities’ that intermediate movement between SITE and non-SITE transport networks, which includes a variety of secondary travel modes like walking, shuttle bus services, commuter parking garages, and ride-sharing.

3

The interviews included questions that captured the participants (1) travel experiences within the City and (2) opinion about sustainable transportation system—the Bus Rapid Transit (BRT), Free Shuttle Bus, elevated covered canopy walk, seasonal parking.

Travel Behaviour and Mobility in Smart Cities …

487

4 Findings A survey is conducted on a sample of 306 resident and non-resident commuters to ascertain their top three preferred means of intracity movement from six distinct modes of transportation—walking, personal vehicle, bus rail transit (BRT), ridesharing, shuttle bus, and park-and-ride facilities. The respondents are composed of Sunway City residents (28.2%) and non-residents (72.8%). Their purpose of commute was also captured to provide contextual insight, as an overwhelming 97.1% of respondents was found to either work or study in Sunway City. This provides an indicator for the usage behaviour to inform further analyses. The results indicate that personal, self-operated means of travel is significantly favoured over facilitated travel. Both ‘Walking’ and ‘Personal vehicle’ use are considerably higher than other facilitated modes like mass transit or on-demand ride-sharing, as shown in Fig. 1. There is also an apparent disparity between SITE modes of travel, with the ‘BRT Sunway’ electric bus service outperforming the ‘Sunway City Shuttle Bus’ option twice-over. Interestingly, despite the major cost advantage of the shuttle bus service (being completely free), it is utilized to a similar extent as the most expensive travel option surveyed—ride-sharing. The least-used travel facility in the preliminary survey is the ‘MyRapid Park N’ Ride’ commuter multi-storey carparks. Typically, such a facility is adopted for travel from place-of-residence to transit network stop, rather than from the transit network stop to destination. The low usage of commuter parking, therefore, indicates that Sunway City residents who use mass transit would not need to or be able to use personal vehicles to access these stops, or that Sunway City residents who do use personal vehicles would use them throughout the entire commute process. The following section discusses the top three major preferred methods of travel in greater detail. Walking

89.5%

Personal vehicle

77.0%

BRT Sunway

47.4%

Ride-sharing

24.4%

Sunway City Shuttle Bus MyRapid Park N’ Ride

21.5% 9.6%

SITE travel mode

Fig. 1 Use of transportation modes

Intermediate travel mode

Non-SITE travel mode

488

S. Vaithilingam et al.

4.1 Walking The ubiquity of walking in Sunway City can be attributed to two reasons. First, in the context of end-to-end commute (where an individual walks the entirety of their commute process), the preference for walking arises when the effort to traverse doorto-door distances on foot is outweighed by the time and material costs of alternative means of travel. The low effort threshold indicated by the survey results is indicative that Sunway City’s relatively concentrated urban layout has encouraged most routine travel to be conducted on foot—residential clusters are situated proximately between high-traffic commute destinations (college, university, mall, hospital) not exceeding a kilometre in distance. Second, in the context of mixed-means commute (where an individual engages the use of multiple methods of travel), walking is mostly infrastructure-independent and requires no dedicated ‘bridging’ facility. In contrast, for example, a commuter who uses both personal vehicle and a rail line requires parking depots proximate to their transit hub; and a commuter who uses multiple train lines requires an interchange station between both lines. Hence the high preference for walking indicates that respondents are regularly able to transition to or from other means of commute, and there is adequate pedestrian space spanning the necessary distances. The strong preference for walking is shared by a large majority of resident respondents for regular travel. The respondents’ top three activities most associated with travel-via-walking is identified to be exercise, followed by grocery shopping, and lastly entertainment. This is characteristic of planned, self-contained urban development where lifestyle facilities are intentionally made more proximate to residential units (Fig. 2). It should be noted that the survey results also bear an unusual feature—the dominance of both ‘Walking’ and ‘Personal vehicle’. The significantly high percentages of respondents engaging in both methods suggest that walking and driving are often combined as mixed-means commute. These may include three possible scenarios— inadequate footpaths to cover the whole commute distance, forcing some to drive partway; inadequate vehicle paths to cover the whole commute distance, forcing some to walk partway, or; inadequate vehicle-to-destination ‘bridging’, forcing some to walk the remaining distance to destination. The first scenario is unlikely as one would simply drive the whole way; however, the latter two can be symptomatic of inadequate parking facilities to support regular commuter flow. This conjecture is supported by the significant number of non-residents walking as a means of regular

Resident

80%

Non-resident

37% Daily

Fig. 2 Residency effect on walking

Weekly

33% Monthly

Never

10% 23%

8%

2% 7%

Travel Behaviour and Mobility in Smart Cities …

489

travel (approximately one in three individuals)—likely from distant parking depots to their daily destinations.

4.1.1

Related SITE sustainability initiatives

Due to walking being a longstanding, integral way of intracity travel, Sunway City has put considerable attention in developing sustainable pedestrian facilities to connect the township’s main destinations. The resultant ‘Canopy Walk’ comprises a four-kilometre circuit of elevated footpaths that serially connects major residential, commercial, and education thoroughfares. Solar-powered LEDs are installed along the circuit to further decarbonise the township’s energy dependency, while surveillance cameras and auxiliary personnel are in place to secure the area throughout its operating hours. Respondents are generally satisfied with the utility afforded by the Canopy Walk, but believe specific improvements can be made. Notably, these include the introduction of more entrances and branching paths for time-saving parallel connections, fuller coverage from the elements, slip-resistant rubber flooring, and panic buttons at regular intervals.

4.2 Personal Vehicle The high automotive dependency of respondents is unsurprising, given that 70% surveyed are non-residents. Sunway City, being part of the Klang Valley urban conglomeration, draws traffic from the principal adjoining municipalities of Kuala Lumpur, Subang Jaya, and Petaling Jaya. High-volume commute across these older economic centres is typical. These organically-developed hubs bear the characteristic sprawl of 20th-century urban densification, where the aggregation of commercial and industrial zones pushed residential development outwards to yield a lateral cityscape. This ultimately channels human movement into arterial thoroughfares that connect the long distances between home and work (Fig. 3). When comparing the effect of residency on personal vehicle use, it is clear that regular intercity commute is almost entirely facilitated through driving. This strong

Resident

29%

22%

32%

Non-resident

85% Daily

Weekly

Fig. 3 Residency effect on personal vehicle use

Monthly

Never

17% 9% 4%

2%

490

S. Vaithilingam et al.

behavioural preference is indicative of a systemic inadequacy in intracity transportation lines to effectively ferry commuters between municipalities. This will be discussed in further detail under ‘BRT Sunway’.

4.3 BRT Sunway Mass transit access into Sunway City is facilitated by the Klang Valley Integrated Transit System, comprising a vast network of 10 principal rail lines in addition to the BRT Sunway intracity transit system. The degree of integration between each line is irregular—networks closer to the Kuala Lumpur capital naturally see proportionately higher numbers of interchange hubs. Meanwhile, there are only two lines which link to BRT Sunway intracity rail— namely, the 37-station light rail transit (LRT Kelana Jaya Line), and the 27-station KTM Komuter commuter rail system. Each of these two lines are connected to one of BRT Sunway’s terminals. Even without considering the varying levels of efficiency of these intercity transport services, the linearity of this network’s layout would by itself result in severely extended commute times (Fig. 4). When comparing the effect of residency on BRT Sunway use, only 4% of nonresident respondents regularly use this mass transit service for a regular commute. The resident preference here also sheds light on the adequacy of BRT Sunway as a means of intracity travel within Sunway City—again, the irregularity of usage indicates that residents prefer alternative methods due to lower costs, efforts, and/or coverage gaps.

4.3.1

Related SITE sustainability initiatives

BRT Sunway is the result of a public–private partnership (PPP) between Sunway Berhad and Prasarana Malaysia (the latter being the principal state-owned public transport operator) to create an environmentally sustainable intracity transport system within Sunway City. Comprising a fleet of 15 zero-emission electric buses, BRT Sunway was launched in 2015 and services seven stations in total. Respondents perceive BRT Sunway as a considerably high-quality travel infrastructure in terms of commute performance. However, widespread usage is deterred by the relatively high cost of tickets compared to other mass commute options provided

Resident

19%

17%

Non-resident 4% 6%

49% 48%

Daily

Fig. 4 Residency effect on BRT use

Weekly

Monthly

15% 42%

Never

Travel Behaviour and Mobility in Smart Cities …

491

by the Klang Valley Integrated Transit System. As indicated earlier, limited integration with other transit lines, as well as considerable distance and stops across intracity and intercity circuits makes BRT Sunway less attractive for regular usage. Additionally, in-station payment facilities are noted to be inadequate, ranging from unstaffed booths to an absence of electronic ticketing machines to support non-cash alternatives. While respondents were deterred by the cost of tickets, they provided several suggestions that can be implemented to increase usage of the BRT. This is shown through the excerpts below. I think maybe for BRT you could give like seasonal discounts or stuff like that because the prices are usually fixed throughout right if I’m not wrong so maybe like every once in a while or maybe once a month, maybe a specific day you say take the BRT for 50% off, I think that could actually encourage people to go for it. aside from cheaper fares, what they should do is to give a subsidy, liken to a concession card, entitles, and target those who usually drive, you want to encourage them to leave their cars at home, so you need to subsidise them taking buses.

5 Essential Guidance 5.1 Reconciling Transport Performance and Transport Sustainability The concept of developmental sustainability came to the fore with the rise of Urbanism in the post-1990s. Much like urban sustainability, the evaluation of transport sustainability should be approached from a holistic lens, detailing not just a cross-sectional performance of its various initiatives, but also accounting for how they may be continuously embedded in the community to improve economic, social or environmental health over generations. The performance of SITE initiatives will employ a systematic framework characterised by the elements in the 7i-Innovation model [16, 17] to identify attributes of ST modes. The framework outlines the resources that underpin the developmental initiatives in a country, defined as the following: ● Infra-infostructure The physical infrastructure and digital infostructure that foster connectivity and enables transportation within the community. This represents the foundational condition for resource diffusion (labour, capital, or information) to ultimately enable a knowledge economy.

492

S. Vaithilingam et al.

● Institutions Bodies that oversee and establish social rules and norms that determine social interactions between the stakeholders. As one of four driver conditions, Institutions represent the presence of coherent, well-demarcated, and enforced regulatory frameworks to steer the knowledge economy. These encompass both developmental as well as moral agendas, ensuring optimal resource management throughout the economy. ● Intellectual capital The second of four driver conditions, this alludes to the presence of technological collaborations to encourage development (and safeguarding) of new intellectual properties that would incrementally contribute to the overall productivity of knowledge economy. ● Interaction Representing the third of four driver conditions, Interaction describes the presence of cross-enterprise collaborations, where numerous stakeholders within the knowledge economy can freely and transparently work toward common goals. Improvements in this dimension would allow quicker reiteration and optimization of market systems, leading to better long-term economic performance. ● Incentives Intervening policies that encourage or discourage the participation of parties in a particular initiative, representing the fourth driver condition in the 7i framework. In practice this is manifested through targeted fiscal and non-fiscal market stimuli. ● Integrity systems that oversee and continuously monitor, assess and benchmark the transportation system so that it adheres to the sustainable development of transportation ecosystem. SITE contributes both to the physical as well as digital components of the Sunway City’s knowledge economy infra-infostructural conditions. While mass transit and travel facilities like the SITE-conceived BRT Sunway and Canopy Walk, or non-SITE intercity bus, train, and light-rail transit services provide varying degrees of physical mobility for the labour force throughout the township’s commercial and industrial sites, they also grant consumers greater accessibility to goods and services. However, as indicated in the findings, these methods of travel are regularly utilised by residents only. Non-residents rely significantly more on personal vehicles (or personal vehicles alongside walking) for their travels. While adoption and utilisation levels of SITE facilities can be enhanced through the improvement of each method of travel independently, any further developmental endeavours should be more efficiently prioritised based on the community’s combinatory travel behaviour—that is, what combination of travel modes are typically chosen for a particular purpose of travel. The choice of these combinations is typically motivated by the intention to maximise a set of transport quality attributes [19] relevant to the purpose of travel. For instance, in the context of routine time-sensitive travel like daily work commute, users would favour a combination of travel modes that maximises ‘Reliability’ and

Travel Behaviour and Mobility in Smart Cities …

493

‘Price’ attributes; meanwhile, recreational travel may emphasise more on ‘Price’ and ‘Safety’ attributes. Figures 5 and 6 will map the survey and focus group findings from earlier to Redman’s (2013) taxonomy of travel quality attributes. This provides an additional perspective to our understanding and gives more in-depth insights into the issues and challenges faced by commuters in the use of ST modes which ultimately create a combinatory travel profile to guide future policymakers. Based on the focus group and survey findings, some common themes among the respondents that emerged are identified in Figs. 5 and 6. These can be summarised into four overarching parameters:

Fig. 5 Physical transport quality attributes of Sunway City’s major modes of travel

Fig. 6 Perceived transport quality attributes of Sunway City’s major modes of travel

494

S. Vaithilingam et al.

● User Agency How readily can a commuter begin their journey, without being dependent to predetermined schedules or boarding/alighting points. ● Performance How consistently a specific travel mode can maximize commuter throughput and minimize travel time to all available destinations. ● Scalability How dependent a specific travel mode’s performance is on the volume of concurrent users and/or distance travelled. ● Cost How much monetary expense is required by commuters to use a specific travel mode. To simplify subsequent analyses, each travel mode will be ascribed one main strength and one main weakness, based on the aforementioned four parameters. Walking Strength: User Agency Commuters can nearly always Walk to common intracity destinations without requiring significant dedicated facilities/infrastructure using either the SITE facilities (Canopy Walk) or otherwise. Weakness: Scalability Walking is only feasible for proximate travel, and its Performance diminishes when users need to traverse over larger distances Personal Vehicle Strength: User Agency Commuters can nearly always employ their Personal Vehicles on-demand, without being constrained by external factors that would significantly limit when/where they may drive Weakness: Scalability The Performance of Personal Vehicles declines rapidly as user volumes exceed a threshold, especially during the peak-hour traffic congestions, and recurrent respondent complaints on lack of parking bay availability BRT Sunway Strength: Performance Based on existing levels of utilization, BRT Sunway is considered by respondents to reliably and predictably facilitate travel across its covered areas, at broadly acceptable service levels

Travel Behaviour and Mobility in Smart Cities … Work & education Sports & exercise

4.8%

Entertainment

42.6% 46.9% 69.9%

9.1% 6.7% 65.1%

26.3%

1.9%

Non-routine shopping Service institutions

62.7%

21.1%

3.4%

Social visitation Grocery shopping

495

62.7%

9.1% 6.7%

69.3%

18.7%

2.9% 5.7%

Personal vehicle

63.2%

22.0%

Walking

BRT Sunway

Fig. 7 Purpose of travel

Weakness: Cost The main criticism faced by BRT Sunway is its above-average fares compared to other mass transit lines under the Klang Valley Integrated Transit System, which has significantly diminished user adoption especially for routine travel Having simplified each travel mode into its key strength and weakness, it is now possible to determine which characteristic would most influence adoption behaviour. However, the degree that User Agency, Performance, Scalability and Cost contributes to a commuter’s decision to use a particular travel mode is dependent on the context— service reliability, for instance, may be more important for work-related travel than for social calls. Aligning the sought-after characteristics of travel modes to the Purpose of Travel can be done by reviewing following survey data (Fig. 7). It can be inferred that commuters would use a travel mode more for a particular purpose if the relevance of its strength outweighs that of its weakness; likewise, if a characteristic weakness has a greater influence on adoption than the corresponding strength, a travel mode would intuitively be used less. To determine what constitutes as ‘more’ and ‘less’ use, this analysis will look to how the percentage of utilization of a single travel mode is distributed across the seven major purposes of travel. An above- and below-‘average’ (or more appropriately, median) percentage would indicate relatively ‘more’ and ‘less’ use, respectively. This is summarised in Table 1.4 The final step is to map each the relevant attributes that motivate/discourage a travel mode’s usage for the associated purpose, thus yielding the sought-after characteristics that can be targeted for further enhancement in upcoming SITE initiatives (Table 2).

4

By recalculating the said travel mode percentages based on their difference from the median.

496

S. Vaithilingam et al.

Table 1 Travel Mode Usage Relative to Median Work and education

Personal vehicle

Walking

BRT Sunway

Below-median

At-median

At-median

Sports and exercise

Below-median

Above-median

Below-median

Social visitation

Above-median

Below-median

Above-median

Grocery shopping

Above-median

Above-median

Below-median

Non-routine shopping

Below-median

Below-median

Above-median

Service institutions

Above-median

Below-median

Below-median

Entertainment

At-median

Above-median

Above-median

Table 2 Travel mode key usage drivers/deterrents

Personal vehicle

Walking

BRT Sunway

Work and education

Scalability

*User agency

*Performance

Sports and exercise

Scalability

User agency

Cost

Social visitation

User agency

Scalability

Performance

Grocery shopping

User agency

User agency

Cost

Non-routine shopping

Scalability

Scalability

Performance

Service institutions

User agency

Scalability

Cost

Entertainment

*User agency

User agency

Performance

(*Where adoption rates are ‘at-median’, usage behavior is assumed to be influenced more by corresponding travel mode’s advantage, as if it were ‘above-median’.)

5.2 Essential Guidance This section discusses potential Targeted Developmental Profiles designed to prioritize improvements on Sunway City’s SITE services based on the research findings. As summarised previously, the intention behind a commuter’s journey determines the characteristics they seek and expect from a travel mode—the more aligned the strengths are to their needs, the more incentivised they would be to use a particular mode. The goal here is to ensure that the sustainable travel services offered can be better oriented toward these behavioural drivers. The prescribed intervening measures in the Targeted Developmental Profiles will emphasize on the 7i framework to create a viable, mutually beneficial service economy for the long-term.

Travel Behaviour and Mobility in Smart Cities …

497

This report will discuss three targeted transport development propositions, which categorizes the previous seven surveyed purposes of travel into: ● Routine-Lifestyle travel ● Non-Routine-Lifestyle travel ● Routine-Non-lifestyle travel. 5.2.1

Routine-Lifestyle Travel (‘Sports & exercise’, ‘Grocery shopping’, ‘Service institutions’)

The first targeted transport development propositions will be intended to address the intention for commuting in order to fulfil recurring non-livelihood-related tasks, typically centred on household consumption. The regularity and perceived necessity of such events naturally motivates users to seek out the cheapest (Cost) and most convenient (User Agency) modes to travel (Table 3). This poses several implications on how Sunway City’s SITE facilities can be enhanced. First, policymakers and service operators need to remedy the longstanding complaint on BRT Sunway being too pricy. The obvious route is to remarket the service at lower fares, though the uncertainty in user response may result in an untenable net loss. Alternatively, the fare structure can be distributed across a larger channel ecosystem—instead of just accommodating cash and commuter card payments, BRT Sunway operators should look to integrate a broader variety of e-wallets and digital currencies, where the cost-per-ride to commuters can be partially offset by the numerous wallet operators in the Malaysian market (who in turn commercially benefit from higher transaction frequency/ubiquity). In addition to such third-party e-wallets, Sunway City itself can also synergize the fare channels with its existing commercial, recreation, healthcare, and education ecosystems. At present, these sites are unified under a single loyalty program called ‘Sunway Pals’, which allows consumers to accumulate redeemable points via card and mobile app through footfall and patronage at affiliates. Point-conversion to commuter ticketing and running discount campaigns for campuses and offices (Incentives) and running in-app broadcasts (Infra-infostructural) are part of the 7i framework that policymakers need to investigate to properly develop an integrated smart city. Table 3 Travel mode key usage drivers/deterrents— routine-lifestyle travel

Personal vehicle

Walking

BRT Sunway

Sports and exercise

Scalability

User agency

Cost

Grocery shopping

User agency

User agency

Cost

Service institutions

User agency

Scalability

Cost

498

5.2.2

S. Vaithilingam et al.

Non-routine-Lifestyle Travel (‘Social visitation’, ‘Non-routine shopping’, ‘Entertainment’)

The second developmental profile is aimed at mostly infrequent travel instances typically culminating in larger consumer spending. There is a clear gap in the viability of major travel modes to support high concurrent commuter volumes and/or extended commute distances (Scalability)—this makes sense, as such travel typically converges on a specific time and place (notably, holidays, sales and audience events) (Table 4). For SITE facilities to keep up with these seasonal travel behaviours, it should first address the shortcomings of Walking and Personal Vehicle travel modes noted by survey respondents—namely, the coverage of supporting infrastructure. For the former, an extension of the Canopy Walk footpaths to service residential hotspots (not just commercial hubs, as it does currently) is vital not just for decarbonizing the township, but also in improving safety and quality-of-life for resident commuters. Instead of looking to create a fully-integrated pedestrian network, however, resources may be better allocated by decentralizing the Canopy Walk facility. Individuals are unlikely to traverse large distances on foot but would find much greater utility if footpaths help them access ‘bridging’ points to other motorised travel modes, such as park-and-ride hubs, BRT stations, or bus stops. Additionally, the Canopy Walk’s numerous well-defined safety and sustainability standards (such as manned security checkpoints, solar-powered lighting, and renewable slip-proof flooring) can also be replicated within the township’s privatelyrun campuses, hospitals, and hotels. As with sustainability agendas in general, sound economic collaboration should underpin these initiatives to ensure long-term viability—the sharing of material suppliers and intellectual property, for instance, will ensure private institutions are more willing to meaningfully participate in the broader SITE directive. Given that most of these centres already act as hubs for the BRT Sunway network, these privately-maintained footpaths will also act as a feeder system for shorter-distance, inter-neighbourhood travel within Sunway City, ultimately reducing the reliance on Personal Vehicles for intracity travel. Table 4 Travel mode key usage drivers/deterrents— non-routine-lifestyle travel

Personal vehicle

Walking

BRT Sunway

Social visitation

User agency Scalability

Performance

Non-routine shopping

Scalability

Performance

Entertainment

User agency User agency

Scalability

Performance

Travel Behaviour and Mobility in Smart Cities … Table 5 Travel mode key usage drivers/deterrents— routine-non-lifestyle travel

5.2.3

Work and education

499

Personal vehicle

Walking

BRT Sunway

Scalability

User agency

Performance

Routine-Non-lifestyle Travel (‘Work & education’)

The third developmental profile emphasizes on high-frequency instances of travel, motivated by an individual’s or household’s livelihood. The findings from the study suggests the weakness in most ST initiatives lies in their limited coverage, and by extension, their inability to facilitate intercity travel satisfactorily (Table 5). This major hurdle can only be addressed through large-scale Institutional collaboration with multi-jurisdictional bodies (like Prasarana). Currently, the sprawling intracity lines that feed into BRT Sunway and the Sunway Shuttle Bus service make regular usage untenable due to prolonged travel times—one possible remedy is to designate dedicated intercity that only stop at select interchanges in adjacent municipalities, thereby reducing the number of stops for routes leading into Sunway City. Optimizing these routes will require collaborative efforts from the township and its principal employment hubs as well—identifying the main intercity travel paths taken from doorstep-to-doorstep through a micro census that provide invaluable information for the relevant agencies/institutions to consider. Meanwhile, concerning intracity commute, various smaller-scale improvements on SITE facilities can be enacted through strategic Interaction between the various stakeholders and provision of Incentives to reduce the weaknesses of the BRT Sunway and Walking travel modes (see related guidance points in Routine-Lifestyle Travel and Non-routine-Lifestyle Travel sections). Other Institutional directives that may be introduced to directly disincentive Personal Vehicle reliance can be taken from the mainstream policy playbook such as mandated scheduled lane redirection/closures during peak hours, enforced dedicated mass transit road lanes, and the odd–even plate policy.5

6 Conclusion This study had set out to address the shortcomings of piecemeal developmental programs on sustainable transportation and had intended to provide a behaviourcentric assessment framework to inform the design of more viable policies/initiatives in the future. Overall, the role of context in motivating the use of ST is evident. This paper contributes to existing literature by prescribing a holistic, user-centric assessment of any existing ST infrastructure, though the lens of the 7i-Innovation 5

https://www.thejakartapost.com/news/2018/04/23/what-you-need-to-know-about-jakartas-oddeven-traffic-policy.html.

500

S. Vaithilingam et al.

model which outlines the application of seven attributes (Infra-infostructure, Institutions, Intellectual Capital, Interaction, Incentives, Integrity) to ultimately facilitate the creation of a sustainable, knowledge-based economy. Thus, the most important journey toward a sustainable future will not be one that is simply motivated by the benefits envisioned by international pledges, governments, or institutions, but primarily driven by the individual perception of sustainable transport. Acknowledgements This research has benefited from the generous financial support of the Monash University Malaysia Sustainable Community Grant Scheme (SDG-2018-03-ENG). We thank the research participants for sharing their insights on this topic. Special thanks to Choong Chai Lim, Lee Wan Juin, Sim Jing Yuan, Jason Chan, Kristy Choy and Hwang Li-Ann for their research assistance.

References 1. Barfod MB (2018) Supporting sustainable transport appraisals using stakeholder involvement and MCDA. Transport 33(4):1052–1066. https://doi.org/10.3846/transport.2018.6596 2. Biehl A, Ermagun A, Stathopoulos A (2018) Community mobility MAUP-ing: a socio-spatial investigation of bikeshare demand in Chicago. J Transp Geogr 66:80–90. https://doi.org/10. 1016/j.jtrangeo.2017.11.008 3. Blainey S, Hickford A, Preston J (2012) Barriers to passenger rail use: a review of the evidence. Transp Rev 32(6):675–696. https://doi.org/10.1080/01441647.2012.743489 4. Cao X, Mokhtarian PL, Handy SL (2009) Examining the impacts of resi dential self-selection on travel behaviour: a focus on empirical findings. Transp Rev 29(3):359–395. https://doi.org/ 10.1080/01441640802539195 5. Cao X, Mokhtarian PL, Handy SL (2009) The relationship between the built environment and nonwork travel: a case study of Northern California. Transp Res Part A Policy Pract 43(5):548–559. https://doi.org/10.1016/j.tra.2009.02.001 6. Chen S-Y (2016) Green helpfulness or fun? Influences of green perceived value on the green loyalty of users and non-users of public bikes. Transp Policy 47:149–159. https://doi.org/10. 1016/j.tranpol.2016.01.014 7. Coelho F, Pereira MC, Cruz L, Simões P, Barata E (2017) Affect and the adoption of proenvironmental behaviour: a structural model. J Environ Psychol 54:127–138. https://doi.org/ 10.1016/j.jenvp.2017.10.008 8. Crosling G, Nair M, Vaithilingam S (2015) Creative learning ecosystem and innovative capacity: a perspective from higher education. Stud High Educ 40(7):1147–1163 9. De Angelis M, Prati G, Tusl M, Battistini R, Pietrantoni L (2020) Mobility behaviors of Italian university students and staff: exploring the moderating role of commuting distances. Int J Sustain Transp 0(0):1–11.https://doi.org/10.1080/15568318.2020.1771641 10. Elias W, Shiftan Y (2012) The influence of individual’s risk perception and atti-tudes on travel behavior. Transp Res Part A Policy Pract 46(8):1241–1251. https://doi.org/10.1016/j.tra.2012. 05.013 11. Eriksson L, Friman M, Gärling T (2013) Perceived attributes of bus and car mediating satisfaction with the work commute. Transp Res Part A Policy Pract 47:87–96. https://doi.org/10. 1016/j.tra.2012.10.028 12. Fishbein M, Ajzen I (1975) Belief, attitude, intention and behavior: an introduction to theory and research. Addison-Wesley, Reading, MA 13. Golob TF, Horowitz AD, Wachs M (1979) Attitude-behavior relationships in travel demand modelling. In: Hensher DA, Stopher PR (eds) Behavioral travel demand modelling. Croom Helm, London, pp 739–757

Travel Behaviour and Mobility in Smart Cities …

501

14. Ho C, Mulley C (2014) Metrobuses in Sydney: how high capacity and high frequency services are benefiting the Metropolitan fringe. Res Transp Econ 48:339–348. https://doi.org/10.1016/ j.retrec.2014.09.061 15. Kandt J, Rode P, Hoffmann C, Graff A, Smith D (2015) Gauging interventions for sustainable travel: a comparative study of travel attitudes in Berlin and London. Transp Res Part A Policy Pract 80:35–48. https://doi.org/10.1016/j.tra.2015.07.008 16. Nair M (2007) The “DNA” of the new economy. Econ Bull 8:27–59 17. Nair M (2011) Inclusive innovation and sustainable development: leapfrogging to a high income economy. In: Ramasamy R (ed) ICT strategic review 2011/12: transcending into high value. Putrajaya and Selangor: Ministry of Science, Technology and Innovation (MOSTI) and Persatuan Industri Komputer dan Multimedia (PIKOM), The National ICT Association of Malaysia 18. Prochaska JO, Velicer WF (1997) The transtheoretical model of health behavior change. Am J Health Promot AJHP 12(1):38–48. https://doi.org/10.4278/0890-1171-12.1.38 19. Redman L, Friman M, Gärling T, Hartig T (2013) Quality attributes of public transport that attract car users: a research review. Transp Policy 25:119–127. https://doi.org/10.1016/j.tra npol.2012.11.005 20. Schwartz SH, Howard JA (1981) A normative decision-making model of altruism. In: Rushton JP, Sorrentino RM (eds) Altruism and helping behavior. Lawrence Erlbaum, Hillsdale, NJ, pp 189–211 21. Spears S, Houston D, Boarnet MG (2013) Illuminating the unseen in transit use: a framework for examining the effect of attitudes and perceptions on travel behaviour. Transp Res Part A Policy Pract 58:40–53, 78 22. Susniene D, Jurkauskas A (2008) Stakeholder approach in the management of public transport companies. Transport 23(3). https://trid.trb.org/view/873754 23. United Nations (2016) Mobilizing sustainable transport for development. Retrieved from https://sustainabledevelopment.un.org/index.php?page=view&type=400&nr=2375&menu= 1515 24. van Wee B (2012) How suitable is CBA for the ex-ante evaluation of transport projects and policies? A discussion from the perspective of ethics. Transp Policy 19(1):1–7. https://doi.org/ 10.1016/j.tranpol.2011.07.001 25. Wang T, Chen C (2012) Attitudes, mode switching behavior, and the built environment: a longitudinal study in the Puget Sound Region. Transp Res Part A Policy Pract 46(10):1594– 1607. https://doi.org/10.1016/j.tra.2012.08.001

A Systematic Literature Review of Improvements to Transportation Safety Through Crowdsourced Data Brent Homcha

Abstract The following discussion is a systematic literature review of recent research papers and articles focused on the topic area of crowdsourcing data from consumers, most notably from mobile phones, wearable technologies, and embedded sensors, to improve vehicular transportation safety. This area is being reviewed because despite decreasing motor vehicle fatality rates (Wagner in Fatality rate per 100,000 licensed drivers in the U.S. from 1990 to 2018, 2020a [Wagner I (2020a) Fatality rate per 100,000 licensed drivers in the U.S. from 1990 to 2018. Statista. Last modified 14 December 2020. https://www.statista.com/statistics/191660/fatalityrate-per-100000-licensed-drivers-in-the-us-since-1988/] [1]) and relatively stable motor vehicle injury rates (Wagner in Traffic-related injury rate in the U.S. from 2010 to 2018, 2020b [Wagner I (2020b) Traffic-related injury rate in the U.S. from 2010 to 2018. Statista. Last modified 24 July 2020. https://www.statista.com/statistics/ 191720/traffic-related-injury-rate-per-100000-us-population-since-1988/] [2]) in the United States over the last decade (2010–2020), motor vehicle fatalities remain the leading cause of accidental death in the United States (Goetsch in Occupational safety and health for technologists, engineers, and manager. Pearson, New York, NY, 2019 [3]). Aside from accident prevention technology incorporated into vehicles themselves, crowdsourcing data from users has the potential to further reduce the risk of injury and death, especially as technology improves in the mobile devices, wearables, and sensors that can collect this data. Thus, topic searches were carried out on the Web of Science and Google Scholar databases to find articles of interest to review. Software tools like Harzing’s Publish or Perish, VOSViewer, CiteSpace and MAXQDA were then used to narrow down the results of the database searches to the articles reviewed within this paper. The results show that there are many current uses for mobile and spatial crowdsourced information and that there is an increased desire to leverage the use of more passive collections systems of sensors (no user input required) to collect more data (Lucic et al. in Smart Cities 3:341–361, 2020, [21]). Keywords Crowdsourcing · Safety · Traffic · Vehicles · Accident B. Homcha (B) Purdue University, West Lafayette, IN 47907, USA e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Switzerland AG 2023 V. G. Duffy et al. (eds.), Human-Automation Interaction, Automation, Collaboration, & E-Services 11, https://doi.org/10.1007/978-3-031-10784-9_30

503

504

B. Homcha

1 Introduction The viewing of any local news broadcast in the United States makes it clear that the transportation industry is laden with threats to human health and life. Passenger vehicle accidents dominate the majority of new topics reported with the emphasis on injury, loss of life, or property damage being cast in the spotlight. While the costs associated with vehicular accidents are not negligible, it can be argued that the permanent impact or ending of human life should be of primary concern in these accidents. The World Health Organization estimates that roughly 1.35 million people are killed each year globally due to motor vehicle accidents and that another 20–50 million are seriously injured [3]. In the United States alone, it is expected that the number of traffic-related deaths to increase to over 42,000 individuals in 2020, which represents the largest increase in vehicular death rate since 1926 despite the workfrom-home/stay-home directives of various government and health organizations due to COVID-19 [4]. Additionally, these statistics are inclusive of both work-related incidents as well as non-work-related incidents. In 2019, the Bureau of Labor and Statistics reported that transportation deaths accounted for the largest portion of fatal work accidents [5]. Clearly, with this information at hand, efforts must be made to improve the overall safety of the transportation industry concerning the injury and fatality rates of passenger vehicle accidents.

2 Purpose of Study The sensors installed in both mobile devices and commercially available consumer products utilized extensively worldwide can generate a great quantity of data. In addition to this, wearable technology devices, such as smartwatches or fitness trackers, have been gaining popularity and contain similar types of sensors. The data generated from these sensors is relevant to the movement and positioning of the device and thus the person with the device. When this data is collected from users for evaluation, it allows a large population—a “crowd”—to become the test group. Such crowdsourced data can be used to track different environmental conditions that may have occurred before, during, or as a result of an accident. The purpose of this study is to establish that crowdsourced data is being utilized to improve transportation safety; moreover, this study is intended to show that the use of data collected in this fashion has increased in interest in recent years. The primary areas of concern in this study are in the realm of pedestrian-vehicle, vehicle-vehicle, and vehicle-object accidents, movements, and interactions. As discussed later in this evaluation, incidents and studies involving bicycles and pedestrians, motor vehicles, and other objects are not being included. Similarly, aerospace accidents or the use of crowdsourced data to improve aerospace safety is outside of the scope of this review.

A Systematic Literature Review of Improvements …

505

When crowdsourced transportation data is collected and evaluated, it can be used to understand what conditions led to the occurrence of a traffic accident. The knowledge of this information can then be used to assess how a situation developed to cause an accident and what steps may be taken in the future to prevent other accidents from occurring. Since traffic accidents are a leading cause of injury and death worldwide [1–3, 6], improvements to transportation safety from crowdsourced data stand to drastically improve the quality of life for individuals in both their private and professional lives.

3 Research Methodology 3.1 Data Collection Searches for data relevant to the topic area of crowdsourcing information to improve transportation safety were performed in the Web of Science database [7] as well as Google Scholar via keyword searches. The Google Scholar data searches were initially performed with Harzing’s Publish or Perish software [8] to survey the largest amount of source material; once an article of interest was found in the Publish or Perish search results, it was then retrieved from the Google Scholar database. Mendeley Desktop [9] software was used to manage reference sources once they were identified. All searches were performed with the keywords of “safety”, “crowdsourcing”, and “transportation”. However, it was found that many initial search results were studies concerned with bicycle and air transportation in addition to vehicular transportation in relation to the aforementioned keywords. These two specific areas were outside the scope of this review, so the keyword search was modified to become “safety AND crowdsourcing AND transportation OR traffic OR vehicle OR accident NOT bicycle NOT airline” for the remainder of this review. The Web of Science database was searched with the above keyword phrase and further refined to the “Transportation Science Technology” Web of Science category for 2015–2021. This query resulted in the return of 34,770 articles of interest of which the top 1500 articles sorted by times cited had their full record and cited references exported for further analysis with VOSViewer [10] and CiteSpace [11]. Similarly, Harzing was utilized to search Google Scholar with the aforementioned keyword search query. This search was executed for the period of 2011–2021 to survey any additional sources outside of the realm contained on Web of Science. This search was stopped at 500 records and then exported for further analysis as well.

506

B. Homcha

3.2 Establishment of Emergence and Impact One of the leading goals of this review was not only to survey available literature but also to establish that there is increasing research interest in the topic area of crowdsourcing data to improve transportation safety. The search results from Web of Science were utilized to show the emergence and impact of this topic area. From 2006 through 2019, the annual number of published articles grew at a steadily increasing rate from approximately 1750 articles in 2006 to approximately 6500 articles in 2019, as shown in Fig. 1. It can be seen that the number of publications per year was stagnant at 1750 articles before the emergence of interest in the 2006–2007 timeframe. The increased interest most probably emerged at that time due to the release of the first smartphone in 2007 [12] since that would represent the first widely popular device that contained the technology to enable crowdsourcing this type of data. The apparent decline in research interest that shows up in 2020 was most likely the result of COVID-19 travel restrictions. With corporations shifting to remote work locations to comply with various governmental stay-at-home orders, the vehicular transportation industry was drastically reduced over the 2020 calendar year. It is expected that this reduction will carry over into 2021 but then recover as COVID-19 vaccination becomes more readily available. The academic impact of crowdsourcing in the field of vehicular transportation safety can be seen by further analyzing the Web of Science search results to analyze what was occurring with the number of authors—the researchers—of publications. Table 1 outlines the results of this analysis. It shows that both the number of authors and the number of articles grew by over 70% between the period of 2011–2013 and the period of 2017–2019. This growth validates that there is an increasing academic interest in the topic area.

Fig. 1 Graph showing the number of publications per year in the Web of Science database from 2002 to 2021 for the keyword search arrived at in Sect. 3.1 and shown in Table 1

A Systematic Literature Review of Improvements … Table 1 Search results from the Web of Science database showing that there is emerging research interest in the area of crowdsourcing information to improve transportation safety

Years searched

507 2011–2013

2017–2019

Growth ratio

Number of Articles

10.632

18,874

1.78

Number of Authors

20,849

35,650

1.71

Search Keywords

“Safety AND crowdsourcing AND transportation OR traffic OR vehicle OR accident NOT bicycle NOT airline”

4 Results 4.1 Initial Content Analysis via VOSViewer The first 500 bibliographic records retrieved from the Web of Science database search and Hazing Google Scholar search were uploaded to VOSViewer to create a cooccurrence diagram. This type of diagram was utilized to show the interconnectedness of terms amongst reports to further analyze the results of citation-based analyses discussed later in this report. Figure 2 shows the results of this analysis for the Web of Science data with the minimum number of occurrences for a word being set at fifteen. The results of the Harzing co-occurrence analysis, while not shown, were very similar to the results of the Web of Science analysis. From these reports, a clear connection can be seen that reports of interest would discuss items such as “connected vehicles”, “networks”, “optimization”, “performance” and “management” with respect to vehicle transportation safety.

Fig. 2 Diagram showing the VOSViewer co-occurrence analysis results of the first 500 bibliographic records retrieved from the Web of Science database for “safety AND crowdsourcing AND transportation OR traffic OR vehicle OR accident NOT bicycle NOT airline”. In both the cluster on the left and the table on the right, the top twenty most co-occurring terms are shown

508

B. Homcha

Fig. 3 Diagram showing the VOSViewer co-citation analysis results of the first 500 bibliographic records retrieved from the Web of Science database for the topic of crowdsourcing information in regards to transportation safety

4.2 Citation Related Analyses Co-citation Analysis Similar to the co-occurrence analysis above, VOSViewer was utilized to create a cocitation diagram of the first 500 bibliographic records retrieved from Web of Science as well. This type of mapping is a representation of how many times cited sources appear together. From this analysis, a connection was established amongst twelve reports, as shown in Fig. 3. These reports were then reviewed, and the most relevant ones were selected for inclusion in this report. Citation Burst Analysis Bibliographic information from the Web of Science database search was then uploaded to CiteSpace [9] to create a cluster analysis and determine the existence of any citation bursts. The citation burst would visually show what research papers were being cited the most over specific periods. There were sixty-nine nodes developed during the cluster analysis which enabled the creation of the citation burst shown in Fig. 4 for use in determining the most relevant articles to include in the data review.

4.3 Content Analysis Results from MAXQDA Articles of interest for inclusion in this report were identified using the above citation analyses as well as some manual review of the database search results. The manual review of search results involved the utilization of lexical search within the search results with the words identified in the co-occurrence analysis. As articles were identified for inclusion in this report, they were uploaded to MAXQDA [13] to be analyzed for the most frequently appearing terms. The top 100 most frequent words, after removing any terms that did not add technical value, are shown in Fig. 5 as a word cloud. This word cloud visually represents the underlying themes of the articles analyzed. A select assortment of the keywords from the word cloud created is shown in Table 2 along with the frequency with which each keyword is used. The word cloud of Fig. 5 helps undeniably establish that crowdsourcing information is playing a critical role in the attempt to improve vehicular transportation safety.

A Systematic Literature Review of Improvements …

509

Fig. 4 The CiteSpace citation burst for the Web of Science database search results showing the top twenty-five research articles cited the most times over the given periods

Fig. 5 Word cloud in the shape of a car displaying the top 100 most frequently occurring words used in the references analyzed a minimum frequency of seventy-five times

510 Table 2 Selection of some of the most frequently occurring words found in the references and shown in word cloud shown in Fig. 5

B. Homcha Keywords

Number of occurrences

Vehicle

906

Crowdsourcing

361

Transportation

277

Safety

269

Crash

195

Pedestrian

186

Accident

178

Performance

148

It also helps to show why and how this is being accomplished through the presentation of words that conjure thought. When one’s eye is drawn to a word sequence such as “numb driver”, as can be seen in the word cloud, one can quickly see why there is a desire to formulate a “strategy” to “improve” the “analysis” of “information” to enhance “vehicle safety”.

5 Discussion 5.1 Overall Use in Transportation Lucic et al. outline the three major forms of crowdsourcing data that are utilized in the field of transportation research as mobile, spatial, or passive sensing applications [14]. While these three areas are distinct, they also very much overlap at the point of application as the majority of crowdsourced information seeks to utilize the information presented by each field, like location and speed. Much progress and usage of mobile and spatial crowdsourcing already exist in the form of “personal mobile device” applications, such as Waze, which rely on user input combined with device location data to generate reports [14]. These applications are crucial to the current applications of crowdsourcing information as not all conditions can be easily sensed and reported by automatic means [15]. However, as technology continues to improve, the incorporation of more passive sensing applications in vehicles is only expected to make crowdsourced information more reliable and available [14]. Mobile and spatial applications of crowdsourcing have already helped lead to a better identification of high-risk traffic incident corridors [16] which then helps both infrastructure planners and policy enforcers make changes to better serve and protect the individuals using these areas for transit. Moreover, these forms of crowdsourced data help to inform users of current road conditions to allow for better routing decisions as they travel. Both of these end uses help improve vehicular transportation safety as they inform operators of areas where the risk of an incident is increased. Current areas of passive sensing crowdsourcing information will be discussed in Sect. 5.2 of this report.

A Systematic Literature Review of Improvements …

511

One area of overlap within all three forms of crowdsourced data that should be of key interest to contributors is the level of privacy of their reports contributed. Zhang et al. describe the ethical concerns associated with the exposure of what consumers consider or may consider confidential location data [17]. Since most crowdsourced data has a location identifier associated with it, it should be of paramount concern to developers using this information to ensure that user/contributor privacy is not violated by exposing their location history. The encrypted form of location data reporting described in “A Decentralized Location Privacy-Preserving Spatial Crowdsourcing for Internet of Vehicles” allows for user anonymity while preserving actual location information in a readily deployable fashion [17].

5.2 Traffic Volume and Navigation Research into the use of passively collected crowdsourced information is being utilized in many ways to improve the overall safety and efficiency of vehicular transportation. One notable subdomain of its use is in interconnected networks of vehicle and mobile device sensors to improve issues related to traffic flow, volume, and navigation. In “Real-time road traffic prediction with spatio-temporal correlations”, Min and Wynter confer that there exist many predictions for traffic flow and volume based on historical data, but there is an increasing desire to make real-time and future predictions of this information as well [18]. The incorporation of live speed and location data from multiple sensors enabled them to understand traffic volume and then make accurate predictions of speed and volume. Using this information allowed for the formation of a computationally light model that could be deployed to large urban traffic centers. Such information can be readily shared amongst connected devices and sensors both autonomously and at the discretion of users. The utilization of sensors in multiple vehicles also allows for the establishment of vehicle-vehicle communication networks. As the connectivity and robustness of these sensor networks and computers grow, it allows for their information to be integrated and shared with other vehicles and drivers. Milanés et al. showed that this type of communication can enable the safe creation of a “cooperative adaptive cruise control” system that would interconnect the cruise control settings of multiple vehicles behind one established as the lead vehicle [19]. Such connectivity amongst groups of vehicles allows for quick communications of changes due to traffic conditions within the group, like slowing down due to congestion, or rapid changes in the driving environment, like vehicles using the space between vehicles of the connected vehicle network to make lane changes and advance position through traffic. Additionally, Kesting, Treiber, and Helbing have shown that the “cooperative adaptive cruise control” network of vehicles could be further improved by the additional sharing of acceleration information beyond just sharing vehicle speed and position information; the sharing of this additional data source amongst vehicles resulted in an even more controlled and consistent automated vehicle operation (i.e.—fewer jarring movements for vehicle occupants) [20].

512

B. Homcha

Further evidence is presented by Shladover, Su, and Lu showing that the utilization of crowdsourced information from sensor networks between vehicles and devices can increase the carrying capacity of installed lanes of travel in our current infrastructure [21]. While this increase in carrying volume of vehicles for the roadway does not decrease the number of vehicles present, it improves roadway safety by using the interconnected sensors amongst vehicles to execute localized computations within a vehicle to make adjustments to the vehicle’s operating conditions faster than an operator; it also removes the “human factor” and delayed response from negatively affecting the safety of the occupants of the vehicle. Beyond highway travel networks, researchers are also investigating how the use of crowdsourced data amongst a network of connected vehicles can improve traffic safety and efficiency in intersections. Lee and Park present a model which utilizes the information from the connected sensors of adjacent vehicles to map vehicle movements through intersections in which traffic signals could be eliminated [22]. While significant travel improvements were not noted in non-congested intersections, they were able to see a 33% improvement in overall travel time while decreasing fuel consumption by 44% in crowded intersections [22]. It is worth noting, however, that this research was only performed at a simulation level. The authors advised that improvements may need to be made in their modeling for accident potential in case of communication or sensor failure between connected vehicles while traversing the intersection. Other research has been performed on how crowdsourced information could eliminate some of the need for personal vehicles by having “shared autonomous vehicles” for our transportation needs [23]. Such a transition in vehicular transportation—or possibly more aptly referred to as “vehicle use programs”—would not only result in a decreased end-cost to consumers but also result in a decreased environmental impact. Moreover, it would also result in fewer cars being on roadways which would only improve the statical safety of highway travel, assuming the operation of the shared autonomous vehicles is at least as safe as human operation of vehicles.

6 Future Work While much progress has been made in improving transportation safety from crowdsourced information, future work can delve further into other applications of such data. Other systematic reviews of the research in the general topic field of crowdsourcing reveal that transportation only accounts for a very small proportion—less than 5%—of what is being studied [24]. Another area where this topic could be applied that overlaps with the data collection areas utilized in transportation could be its application to the manufacturing worker and setting. Crowdsourcing data in this area could help manufacturing companies that do not have a large proportion of their workers driving vehicles to improve their safety records as well. “LookUp: Enabling Pedestrian Safety Services via Shoe Sensing” is a National Science Foundation funded study that was carried out on pedestrians equipped with

A Systematic Literature Review of Improvements …

513

sensor-laden shoes to help detect when they entered an active roadway from a suitable walkway. This study showed a 90% success rate at showing and notifying a pedestrian when he/she entered a roadway from a suitable walkway using an aftermarket (not built-in) sensor applied to the shoe [25]. Such an application can be readily applied to a manufacturing setting in which safety shoes are mandatory so that a worker can be notified once they enter an area of increased hazards, such as high voltage areas or locations of in-progress rigging operations. This notification would require the establishment of a smartphone application or similar device to deliver the notification to workers, and future research should investigate alternative “detection” schemes that may come at a lower deployable cost than the inertial sensors utilized for this study. An additional application of crowdsourced information for the manufacturing environment can be found by following the lead of Ye, Yan, and Tang in their review of research on monitoring the safety behavior of construction workers. Many suitable, low-cost (low-cost as compared to the cost of the accidents being prevented) monitoring technologies exist that can be easily implemented to help improve the safety monitoring of both construction workers, as reviewed in this presentation, as well as manufacturing workers. Such a deployment of technologies was found to offer “successful recognition of safe and unsafe behaviors…97% and 92%” (of the time) respectively [26].

7 Conclusion 7.1 Summary With almost seventy-five percent of all traffic incidents being the results of human error [27], it should be clear why effort is being put forth to understand how to best use information reported by crowdsourced sources to try to detect, avoid, or prevent vehicular accidents. It would be cost-prohibitive for infrastructure creators and owners to install the networks of sensors that are otherwise available as a “crowd” of commercially used products to report this transportation information. This paper presents a review of the current research literature on the use of crowdsourced information as it pertains to the improvement of vehicular transportation safety. Data was gathered from Web of Science and Google Scholar database searches to ensure that as much information as possible could be presented for review. Search results were then analyzed for the most relevant contributing papers, and their contents were discussed. There is a clear emerging movement towards the utilization of crowdsourced data to create a “data-driven intelligent transportation system” [27] that minimizes the number of traffic accidents and improves overall transportation efficiency [19], 20, 22. This movement is utilizing information reported through mobile, spatial, and passively sensed [14] means to optimize vehicular transportation and improve the safety of vehicle operators. Future contributions to the field are then discussed as

514

B. Homcha

a reflection on how crowdsourced information like that gathered and presented to improve vehicle safety can be used to potentially improve the safety of manufacturing workers as well. This take on future research work was selected to show how crowdsourced information in the realm of human-computer interaction can play a large role in the future of Safety Engineering across both the transportation front as well as other areas like manufacturing.

7.2 Relevance to IE558 Safety Engineering and Human–Computer Interaction As presented in Chap. 2 of Goetsch’s Occupational Safety and Health for Technologists, Engineers, and Managers and even more so in Chap. 14 of Brauer’s Safety and Health for Engineers, transportation accidents continue to plague both companies and individuals as a major cause of injury and death [1–3, 6, 28]. Research into the use of crowdsourcing information from mobile devices, wearable technology, and other sensors to decrease the frequency and severity of accidents is increasing and, hopefully, will only continue to improve. Moreover, this field is directly related to the field of human-computer interaction because it relies upon the use of “mobile computers” consumers carry in the form of smartphones and wearable technology in order to function. The mini devices worn, carried, and/or otherwise used by people worldwide daily are utilized to collect specific data that can then be used to compute solutions to reduce the severity of accidents or to prevent them altogether. The application of this means of data collection is not limited to the topic of transportation safety improvements; rather, it can be applied to many different fields of interest where users can contribute sensor data for analysis.

References 1. Wagner I (2020a) Fatality rate per 100,000 licensed drivers in the U.S. from 1990 to 2018. Statista. Last modified 14 December 2020. https://www.statista.com/statistics/191660/fatalityrate-per-100000-licensed-drivers-in-the-us-since-1988/ 2. Wagner I (2020b) Traffic-related injury rate in the U.S. from 2010 to 2018. Statista. Last modified 24 July 2020. https://www.statista.com/statistics/191720/traffic-related-injury-rateper-100000-us-population-since-1988/ 3. Road traffic injuries. World Health Organization. Last modified 7 February 2020. https://www. who.int/news-room/fact-sheets/detail/road-traffic-injuries 4. Motor Vehicle Deaths in 2020 Estimated to be Highest in 13 Years, Despite Dramatic Drops in Miles Driven. National Safety Council. Last modified 4 March 2021. https://www.nsc.org/ newsroom/motor-vehicle-deaths-2020-estimated-to-be-highest 5. Census of Fatal Occupational Injuries Summary, 2019. U.S. Bureau of Labor Statistics. Last modified 16 December 2020. https://www.bls.gov/news.release/cfoi.nr0.htm 6. Brauer RL (2016) Chapter 14: transportation. Essay. In: Safety and health for engineers, 3rd edn, pp 375–409. Wiley, New York. ProQuest Ebook Central

A Systematic Literature Review of Improvements …

515

7. Web of Science (n.d.). https://apps-webofknowledge-com.ezproxy.lib.purdue.edu/WOS_Gen eralSearch_input.do?product=WOS&search_mode=GeneralSearch&SID=8EpAjcfAJALx 63aCnzt&preferencesSaved=. Accessed 17 Mar 2021 8. Harzing’s Publish or Perish (n.d.). https://harzing.com/resources/publish-or-perish/windows. Accessed 1 Apr 2021 9. Mendeley (n.d). https://www.mendeley.com/download-desktop-new/. Accessed 28 Mar 2021 10. VOSViewer (n.d). https://www.vosviewer.com/download. Accessed 29 Mar 2021 11. CiteSpace (n.d). https://citespace.podia.com/dashboard. Accessed 6 Apr 2021 12. Arthur C (2012) The history of smartphones: timeline. The Guardian, Guardian News & Media Limited. Last modified 24 January 2012. https://www.theguardian.com/technology/2012/jan/ 24/smartphones-timeline 13. MAXQDA (n.d). https://www.maxqda.com/products/maxqda-analytics-pro. Accessed 28 Mar 2021 14. Lucic MC, Wan X, Ghazzai H, Massoud Y (2020) Leveraging intelligent transportation systems and smart vehicles using crowdsourcing: an overview. Smart Cities 3(2):341–361. https://doi. org/10.3390/smartcities3020018 15. Rantanen P, Sillberg P, Soini J (2017) Towards the utilization of crowdsourcing in traffic condition reporting. In: 2017 40th international convention on information and communication technology, electronics and microelectronics, MIPRO 2017—proceedings, no. 2017, pp 985– 990. https://doi.org/10.23919/MIPRO.2017.7973567 16. Li X, Dadashova B, Yu S, Zhang Z (2020) Rethinking highway safety analysis by leveraging crowdsourced waze data. Sustain (Switzerland) 12(23):1–18. https://doi.org/10.3390/su1223 10127 17. Zhang J, Yang F, Ma Z, Wang Z, Liu X, Ma J (2021) A decentralized location privacy-preserving spatial crowdsourcing for internet of vehicles. IEEE Trans Intell Transp Syst 22(4):2299–2313. https://doi.org/10.1109/TITS.2020.3010288 18. Min W, Wynter L (2011) Real-time road traffic prediction with spatio-temporal correlations. Transp Res Part C: Emerg Technol 19(4):606–616. https://doi.org/10.1016/j.trc.2010.10.002 19. Milanés V, Shladover SE, Spring J, Nowakowski C, Kawazoe H, Nakamura M (2014) Cooperative adaptive cruise control in real traffic situations. IEEE Trans Intell Transp Syst 15(1):296–305. https://doi.org/10.1109/TITS.2013.2278494 20. Kesting A, Treiber M, Helbing D (2010) Enhanced intelligent driver model to access the impact of driving strategies on traffic capacity. Philos Trans R Soc A: Math Phys Eng Sci 368(1928):4585–4605. https://doi.org/10.1098/rsta.2010.0084 21. Shladover SE, Su D, Lu XY (2012) Impacts of cooperative adaptive cruise control on freeway traffic flow. Transp Res Record 2324 (Idm):63–70. https://doi.org/10.3141/2324-08 22. Lee J, Park B (2012) Development and evaluation of a cooperative vehicle intersection control algorithm under the connected vehicles environment (06121907, P. 1: 88). IEEE Trans Intell Transp Syst 13(1):81–90 23. Fagnant DJ, Kockelman KM (2014) The travel and environmental implications of shared autonomous vehicles, using agent-based model scenarios. Transp Res Part C: Emerg Technol 40:1–13. https://doi.org/10.1016/j.trc.2013.12.001 24. Ozcan S, Boye D, Arsenyan J, Trott P (2020) A scientometric exploration of crowdsourcing: research clusters and applications. IEEE Trans Eng Manage 1–15. https://doi.org/10.1109/ TEM.2020.3027973 25. Jain S, Borgiattino C, Ren Y, Gruteser M, Chen Y, Chiasserini CF (2015) LookUp: enabling pedestrian safety services via shoe sensing. In: MobiSys 2015—proceedings of the 13th annual international conference on mobile systems, applications, and services, pp 257–271. https:// doi.org/10.1145/2742647.2742669 26. Ye G, Lu R, Yang J, Tang X (2021) Research trends of information technology application in construction workers’ behavior monitoring. In: Proceedings of the 23rd international symposium on advancement of construction management and real estate. Proceedings of the 23rd international symposium on advancement of construction management and real estate, pp 1256–1268. Springer Singapore. https://doi.org/10.1007/978-981-15-3977-0

516

B. Homcha

27. Zhang J, Wang FY, Wang K, Lin WH, Xu X, Chen C (2011) Data-driven intelligent transportation systems: a survey. IEEE Trans Intell Transp Syst 12(4):1624–1639. https://doi.org/ 10.1109/TITS.2011.2158001 28. Goetsch DL (2019) Chapter 2: accidents and their effects. Essay. In: Occupational safety and health for technologists, engineers, and managers, 9th edn, pp 18–28. Pearson, New York, NY

Index

A Acceleration, 6, 8, 9, 11–13, 511 Accessibility, 104, 164, 423, 428, 452, 482, 483, 492 Accommodate, 92, 105, 236, 349, 437 Accuracy, 140, 173, 183, 207, 208, 285, 332, 357, 359, 372, 451–455, 457, 460–463 Advances, 58, 75, 154, 158, 238, 253, 254, 272, 288, 318, 329, 330, 411, 434, 444, 451, 511 Aeronautical, 317 Aeronautics, 348 Aerospace, 317, 318, 504 Affective, 61, 139, 157–159, 164–168, 243 Aging, 76, 84–86, 99, 104, 105, 238, 415, 417, 418, 427 Airline, 346, 505, 507 Airplane, 119, 125, 346 Algorithms, 20, 35, 38, 82, 198, 215, 222, 230, 271, 274, 286, 301, 348, 363, 364, 372–375, 452–455 Analytics, 168, 411 Anthropometry, 102, 105, 350, 351, 358, 359 Anthropomorphic, 273 Anthropomorphism, 230 Artificial, 60, 169, 182, 297, 417 Assistive, 75, 76, 92, 99–101, 105, 281, 282 Auditory, 82, 132, 173, 284, 334–340, 474–476 Augmented, 225, 273, 329, 333, 334, 336, 340, 475 Author mapper, 159, 160, 163–168, 313–315, 418

Automotive, 4, 6, 75, 78, 123, 150, 175, 242, 268, 434, 489 Automotive displays, 149 Autonomously, 215, 222, 236, 240, 442, 511 Autopilot, 81, 86, 111–113, 118–121, 123–125, 274, 282, 287, 325, 347, 349

B Bibexcel, 119–121, 211, 294, 302–304, 399, 402, 404 Bibliometric, 96, 102, 103, 111–113, 126, 157, 158, 166, 208, 302 Bikes, 131, 140, 141, 400, 407, 409, 410, 438, 440–442, 445, 483 Brake, 7, 58, 200, 475–477 Braking, 6, 7, 12, 21, 178, 201, 202, 285 Bursts, 113, 121, 123, 162, 163, 166, 168, 169, 318, 319, 404, 405, 508, 509 Buses, 136, 140–142, 420, 421, 423, 424, 426–429, 440, 486, 487, 490–492, 498, 499

C Calibrating, 267, 268, 271–274, 276, 351, 453 Calibration, 180, 181, 185, 186, 190, 223, 225, 268–274, 276, 287, 351 Cameras, 58, 63, 246, 247, 285, 379, 386, 387, 389–394, 397, 424, 453, 489 Cars, 3, 4, 6, 11, 15, 33, 58, 59, 61, 63, 64, 66, 68–70, 83, 84, 131, 140, 147–151, 153, 154, 171, 198, 199,

© The Editor(s) (if applicable) and The Author(s), under exclusive license to Springer Nature Switzerland AG 2023 V. G. Duffy et al. (eds.), Human-Automation Interaction, Automation, Collaboration, & E-Services 11, https://doi.org/10.1007/978-3-031-10784-9

517

518 236, 249, 250, 271, 283, 284, 426, 440, 443, 472, 474–476, 483, 491, 509, 512 Citespace, 121–123, 162–164, 166–169, 318, 321, 322, 399, 404, 405, 503, 505, 508, 509 Cities, 164, 363, 407, 409, 415–420, 422–429, 434–436, 438–441, 443–445, 481, 482, 484–493, 496–499 Clusters, 121–123, 150, 162–165, 167, 198, 209, 212–214, 235, 243–247, 272, 299–301, 318, 321, 322, 324, 404, 405, 407, 452, 488, 507, 508 Coauthorship, 114, 318, 322 Cocitation, 113, 158, 159, 161–163, 168, 169, 207, 208, 211, 212, 293, 295, 298, 299, 301, 302, 305, 319, 320, 322, 508 Cockpit, 112, 124, 325, 345–353, 356, 357, 359, 360 Cognition, 20, 102, 105, 106, 152, 317, 318, 360 Comfortable, 59, 86, 151, 349, 351, 380, 423, 428, 429 Congestion, 238, 424, 433–435, 437, 439–442, 485, 494, 511 Construction, 6, 85, 249, 380, 415, 416, 457, 485, 513 Consumers, 82, 86, 147, 154, 155, 157, 158, 168, 173, 223, 333, 434, 483, 492, 497, 498, 503, 504, 511, 512, 514 Cooccurrence, 507, 508 Covid-19, 167, 250, 452, 460, 504, 506 Crashes, 4, 58, 59, 61, 69, 76, 77, 79, 178, 189, 199, 285, 287, 332, 348, 407, 409, 470, 476, 510 Crowdsourced, 454, 463, 503–505, 510–514 Crowdsourcing, 440, 451, 453–457, 503, 505–508, 510–512, 514 Customers, 133, 137, 140, 154, 195, 222, 437–439, 442, 443, 445 Cybersecurity, 85 Cyclist, 58, 148, 223, 411

D Dangers, 237, 260, 326, 473, 474 Database, 58, 79, 95, 103, 113, 114, 117, 118, 159–161, 163, 166, 167, 208, 209, 224, 225, 293–295, 298, 314, 315, 322, 332, 346, 352, 358, 399,

Index 401, 436, 445, 455, 458, 503, 505–509, 513 Defensive, 58, 227, 230 Designers, 3, 14, 15, 112, 125, 126, 147, 157, 158, 167, 168, 271, 275, 286, 294, 347, 349, 353, 357, 359–361, 364, 395 Deterministic, 35, 37, 40, 43, 455 Digital Human Modeling (DHM), 345–350, 353–361, 380 Disabilities, 86, 91, 95–97, 100, 395, 418, 419, 423, 426, 428 Discomfort, 54, 55, 379, 380, 386, 390–396 Distracted, 58, 152, 271 Distractions, 86, 147, 171–182, 185, 186, 189, 190, 276, 411 Distrust, 235–238, 241–245, 247–250, 270, 271, 274 Driverless, 196, 438, 443 Drones, 260, 437–439 Dynamics, 35, 133, 164, 200, 216, 225, 253–255, 257–259, 263, 264, 268, 282, 284, 285, 352, 360, 364 E Ecological, 439, 444, 445 Emergencies, 6, 200, 201, 239, 254, 273, 283, 288, 345–350, 352, 359, 360 Emerging, 140, 166, 167, 208–210, 216, 311, 314, 326, 333, 399, 409, 411, 417, 439, 443, 507, 513 Emissions, 399, 434, 439–445, 490 Emotional, 4, 133, 241 Empirical, 59, 60, 138, 223, 225, 239, 254, 258, 262, 264, 274, 275, 427, 438, 483 Entertainment, 138, 140, 272, 276, 333, 488, 496, 498 Environmentally, 439, 441 Epidemiological, 407 Ergonomic, 96, 105, 106, 132, 143, 148, 155, 157–159, 165, 168, 169, 181, 209, 316–318, 320, 345–351, 353, 356–361, 379, 380, 386, 387, 394, 395, 417, 419 Ergonomically, 92, 102, 380, 395 Errors, 30, 31, 44, 67, 69, 76, 83, 84, 86, 98, 112, 182, 190, 197, 199, 204, 215, 263, 264, 287, 288, 293–301, 303, 304, 306, 307, 312, 313, 317, 320–325, 331–335, 340, 346, 349, 351, 357, 373, 443, 456, 457, 460, 464, 513

Index Evidence, 19, 20, 50, 54, 55, 57, 60, 61, 68, 70, 76, 79, 80, 84, 96, 104, 221, 223, 225, 228, 229, 258, 274, 286, 333, 512 Experimental, 6, 10, 19, 20, 76, 86, 164, 172, 174, 176, 177, 179–181, 183, 185–187, 190, 227, 255, 260, 337, 454 Eyetracking, 334

F Facility, 105, 140, 335, 363–366, 368, 369, 373–375, 444, 473, 481, 486–489, 491, 492, 494, 497–499 Failures, 10, 85, 112, 142, 197, 200, 230, 255–257, 269, 306, 331, 346, 347, 512 Fatigue, 44, 321, 325, 392, 396, 470 Feedback, 198, 216, 255, 258, 272, 336, 339, 340, 359 Fitness, 44, 372, 504 Fumes, 346

G Games, 102, 455 Gaze, 5, 151, 177, 179, 180, 225, 334, 354 Generalizability, 70, 253, 264, 427 Gerontechnology, 421 Gerontology, 426 Gestures, 58, 339, 340 Guided, 21, 83, 126, 177, 239, 419

H Haptic, 198, 334, 336–338, 470 Hazard, 4, 86, 97, 102, 105, 106, 125, 202, 281–286, 288, 323, 325, 346, 348, 399, 400, 408, 411, 469, 513 Healthcare, 103, 416, 497 Height, 97, 102, 105, 387, 394, 453, 454, 457, 458, 460 Helmet, 399, 409, 410, 411 Human centered, 196, 198, 200, 204 Human computer, 514 Human–machine, 4, 60, 85, 142, 230, 267, 268, 272, 273, 346, 348, 359, 360, 471 Human robot, 199 Humans, 4, 9, 19–21, 33, 55, 57–61, 69, 70, 75, 76, 78, 80–86, 92, 111, 112, 124, 125, 131–134, 136, 138, 141–143, 158, 159, 165, 166, 168, 169, 174,

519 176, 195–199, 201, 202, 204, 207–209, 214–216, 221–231, 235–239, 241, 243, 247, 249, 250, 253, 254, 257–259, 261–264, 267–274, 281–289, 293–301, 303–307, 311–313, 316–318, 320–326, 331–333, 335, 337, 345, 346, 348, 349, 359, 360, 379, 380, 395, 410, 417–420, 442, 455, 456, 471, 489, 504, 512–514 Human solutions, 21, 23 Hypotheses, 19, 20, 57, 59–61, 68, 70, 134, 228, 455, 457, 458, 475, 476, 483

I Impairment, 91, 92, 97–102, 105, 106, 172, 238, 416, 426 Impolite, 57, 59–61, 63, 66–70 Incident, 58, 101, 105, 197, 293–295, 305, 307, 322, 346, 347, 411, 504, 510, 513 Indicators, 50, 160, 161, 176, 178, 179, 182, 188, 190, 225, 262, 313, 317, 350, 358, 403, 404, 444, 445, 451, 452 Industrial, 297, 379, 380, 382, 383, 417, 485, 489, 492 Information processing, 75, 81, 83, 181, 275 Infotainment, 82, 150, 151, 154, 172, 181 Injuries, 77, 91, 95–97, 105, 294, 348, 379, 380, 395, 399–411, 503–505, 514 Innovation, 75, 168, 249, 307, 481, 482, 485, 491, 499 Intelligent, 75, 79, 82, 83, 221, 272, 364, 417, 442, 445, 455, 469, 482, 513 Intentions, 57, 59–61, 63, 65–70, 111, 198, 202, 223, 237, 238, 269, 271, 273–275, 469, 470, 483, 492, 496, 497 Intersections, 85, 470, 476, 512 Interventions, 53–55, 76, 85, 98, 103, 200–202, 263, 268, 281, 287, 470

J Journey, 132–134, 137–142, 197, 239, 240, 249, 273, 349, 426, 494, 496, 500

K Kansei, 157–161, 163–168 Kinematics, 352, 354, 384–386, 388, 389

520 L Likelihood, 19, 20, 25, 26, 30, 31, 33–38, 40–43, 45, 53, 54, 80, 153, 216, 223, 262, 287, 346 Luminance, 345, 347, 350, 352, 355–360 M Maneuver, 14, 178, 200, 411 Maneuvering, 79, 81, 201, 268 Manikins, 348, 349, 351, 352, 354–357, 359, 360, 380–386, 388, 390–397 Manufacturing, 75, 112, 334, 346, 364, 395, 512–514 Maps, 36, 53, 83, 114, 139–142, 152, 163, 212, 213, 236, 299, 300, 318–322, 421, 451, 453–455, 457, 493, 495, 512 Maritime, 294, 330, 333, 334, 336, 338, 340, 417 Maxqda, 116–118, 121, 126, 165–169, 207, 208, 214, 293, 294, 301, 302, 321–323, 399, 400, 405–408, 503, 508 Medical, 96, 97, 100–102, 105, 106, 215, 307, 395, 420, 476 Metaanalyses, 21, 50, 55 Mining, 157, 158, 163, 168 Mismatch, 267, 270, 283, 436 Misuse, 223 Mobile, 82, 142, 172, 173, 238, 336, 437, 440, 442, 451, 453, 458, 497, 503, 504, 510, 511, 513, 514 Modelling, 22, 133 Moderators, 228, 229 Monitoring, 4, 10, 15, 54, 55, 84, 86, 183, 216, 325, 340, 349, 350, 410, 513 Mopeds, 400, 409 Mortality, 93, 416 Motivate, 22, 464, 495, 497 Motivated, 196, 463, 492, 499, 500 Motorway, 85 Multitasking, 83, 288, 319 Municipalities, 441, 445, 484, 485, 489, 490, 499 Musculoskeletal, 348, 379, 380, 395 N National Aeronautics and Space Administration (NASA), 6, 11, 115, 179, 317, 331 National Highway Traffic Safety Administration (NHTSA), 4, 11,

Index 172–176, 178, 181, 185, 186, 189, 268, 275, 470 Naturalistic, 58, 70, 79, 225 Navigating, 82, 173, 335, 357, 381, 383, 384, 386, 388, 390, 393 Navigation, 82, 86, 132, 150, 176, 203, 236, 272, 335, 336, 392, 395, 452, 476, 511 Networking, 137, 190 Noise, 137, 260, 285, 338, 427, 433, 434, 439, 444, 452, 455, 474

O Observations, 36, 69, 70, 115, 121, 148, 167, 339, 410, 420 Occupants, 85, 86, 148, 153, 155, 351, 357, 395, 511, 512 Occupational, 316, 324, 325, 380, 391, 514 Operators, 4, 36, 81, 124, 183, 196, 200, 216, 222, 227, 246, 257, 261–264, 268, 270, 271, 286, 287, 299, 301, 302, 314, 329–331, 334–340, 379, 380, 382, 385, 389, 391–395, 490, 497, 510, 512, 513 Optimization, 363, 364, 366–369, 372–375, 447, 451, 457, 460, 461, 507 Organizational, 134, 311, 312, 324, 416 Overtake, 59

P Passengers, 84, 86, 132, 137, 138, 142, 147, 152, 153, 235–240, 247, 249, 274, 346, 417, 438, 440, 441, 447, 473, 474, 504 Patients, 62, 66, 92, 307, 399, 410 Pedestrians, 58, 148, 271, 273–276, 281, 283–286, 289, 411, 417, 441, 443, 470, 483, 488, 489, 498, 504, 510, 512, 513 Perception, 15, 20, 60, 63, 77, 136, 138, 142, 157, 168, 169, 236, 238, 241–245, 247, 248, 250, 285, 338, 415, 419, 420, 476, 482–484, 486, 500 Perceptual, 21–23, 82, 83, 176 Personalized, 263, 340 Personnel, 105, 135, 190, 207, 326, 329, 330, 365, 367, 489 Physiological, 84, 85, 97, 142, 178, 179, 225, 360

Index Pilots, 111, 112, 124–126, 154, 213, 242, 282, 332, 346, 347, 349–353, 356, 358, 360 Pollution, 367, 433–435 Postures, 348–351, 353, 354, 357, 359–361, 379, 380, 382, 383, 395, 477 Precision, 125, 150, 174, 182, 228, 349, 395 Predictions, 80, 151, 168, 222, 228, 253, 262–264, 348, 451, 453–457, 460, 462–464, 511 Probabilistic, 19, 20, 36, 263, 455 Probability, 19, 20, 34, 46–52, 54, 55, 197, 228, 229, 261, 271, 458 Procedures, 7, 11, 19–21, 63, 101, 112, 133, 158, 172, 176–182, 187, 189, 190, 198, 200, 227, 239, 240, 293, 314, 326, 332, 339, 347, 350, 381, 401, 436, 472 Prototypes, 236, 346, 359, 361, 437 Psychological, 20, 269, 483 Psychosocial, 483 Q Questionnaire, 6, 7, 11, 62, 63, 65, 77, 78, 81, 138, 139, 189, 224, 225, 235, 239–241, 243, 253, 254, 420, 469–473, 484, 486 R Ramsis, 148, 150, 151, 153, 379–381, 395 Reach, 57, 134, 199, 287, 345, 347–354, 356–359, 434, 436, 460 Regulate, 61, 173, 325, 474 Regulated, 112, 400, 410 Regulatory, 171, 172, 181, 312, 324, 325 Reliability, 175, 196, 197, 237, 238, 242, 254, 255, 259, 261, 262, 270, 271, 287, 307, 330, 424, 428, 442, 483, 485, 492, 495 Reliable, 76, 118, 126, 177, 187, 197, 236, 244, 249, 287, 294, 411, 510 Riders, 399, 407, 409–411 Ride sharing, 236, 239, 240, 247, 250, 486, 487 Roads, 4, 6, 7, 11, 57–59, 61, 63, 64, 66, 70, 76–81, 85, 86, 148, 151, 152, 171–175, 177, 180, 181, 185, 197, 235, 236, 238, 241, 243, 267, 268, 271, 273–276, 281, 283–286, 288, 305, 400, 407, 410, 411, 417, 425, 428, 434, 469, 470, 472, 473, 477, 478, 499, 510, 511

521 Roadway, 81, 180, 281, 288, 470, 472, 473, 512, 513 Robotaxi, 58 Robots, 60, 61, 224, 230, 254, 438, 443 Routes, 84, 85, 132, 136, 137, 139, 185, 236, 239, 421, 439–443, 445, 497, 499 S Safe, 6, 9, 11, 13, 14, 46, 79, 86, 111, 125, 172, 173, 200, 207, 221, 228–230, 268, 285, 326, 334, 347, 349, 366, 369, 373, 380, 391, 392, 416, 424, 470, 511–513 Safely, 84, 125, 147, 275, 283 Safely pilot, 155 Sampling, 30, 31, 34–38, 40, 43, 183, 184, 188, 243, 415, 420, 456, 457 Satisfaction, 6, 9, 11, 13, 14, 61, 79, 86, 486 Scene, 7, 357, 379–382, 384, 385, 387, 394 Score, 44, 65, 81, 180, 181, 189, 190, 221, 238, 299, 300 Seating, 148, 151, 247, 420, 428 Self driving, 59, 86, 123, 223, 236–238, 271 Semiautonomous, 281 Sensing, 125, 283, 453, 461, 510, 512 Sensory-motor, 50 Severity, 61, 287, 410, 470, 514 Shipping, 330 Ships, 329–331, 334, 336–340 Shuttles, 78, 131, 235–237, 239–243, 245–248, 250, 420, 421, 423, 427–429, 486, 487, 499 Signals, 7, 23, 82, 111, 182–184, 200, 215, 285, 334, 335, 451–455, 469–476, 512 Simulations, 34, 54, 151, 183, 184, 187, 250, 260, 264, 273, 345, 347–350, 352–354, 357, 359, 361, 374, 512 Skill, 4, 76, 83, 84, 102, 124, 136, 142, 174, 199, 285, 330–332, 334, 346 Smart, 105, 334, 442, 444, 484, 485, 497 Smartphones, 82, 132, 140, 142, 236, 272, 336, 420, 427, 453, 455, 461, 506, 513, 514 Society, 4, 75, 78, 250, 268, 318, 320, 333, 416, 417, 483, 484 Solar powered, 100, 486, 489, 498 Speed, 6, 7, 19, 79–83, 153, 178, 183, 200, 202, 203, 271, 272, 275, 350, 353, 373, 374, 435, 441, 443, 444, 473, 510, 511 Sports, 496, 497

522 Steering, 6, 8, 11–13, 150, 151, 172, 173, 178, 198, 200, 273, 382, 383, 475–477 Subjective, 6, 7, 9, 11, 13, 14, 34, 59, 79, 81, 178–180, 187, 189, 213, 224, 225, 270, 335, 364, 425, 483 Surprises, 78, 111–113, 117, 123–126, 243 T Taxonomies, 195, 198–201, 204, 493 Tested, 58, 70, 80, 131, 159, 169, 171, 175, 176, 185, 187, 222, 307, 457 Train, 132, 140, 142, 176, 187, 261, 423, 441, 452, 455, 458, 463, 488, 492 Traveler, 131–134, 139, 141, 223 Travelling, 81, 427, 439 Trucks, 59, 153, 281, 389, 437–439, 441, 443 Trusting, 84, 230, 235, 243, 245, 248, 249, 288 U Uncertainties, 21, 82, 125, 224, 225, 237, 238, 248, 254, 269, 273, 275, 286, 288, 451, 456, 497 Uncomfortable, 59, 77, 380, 395 Unfamiliar, 4, 84, 174 Unsafe, 46, 82, 101, 125, 513 Usability, 78, 79, 133, 138, 172, 174, 175 Usable, 133, 302, 335, 444, 486

Index Usefulness, 19, 20, 78, 79, 223, 287, 345, 347

V Validated, 177, 189, 243, 299, 443 Vans, 438, 441, 443, 444 Vehicle-to-vehicle, 85, 285, 470 Vibration, 334, 336, 367, 469, 470, 475–478 Vibrotactile, 476, 477 Vicinitas, 121, 123, 124, 160, 168, 304–306, 399, 403, 405–407 Visualizing, 165, 166, 198, 348

W Walking, 100, 136, 286, 336, 400, 421, 423, 424, 427, 486–489, 492, 494, 496–499 Warehouse, 335, 444, 445 Warnings, 79, 125, 201–203, 272, 282–286, 288, 325, 350, 469–478 Wearable, 477, 503, 504, 514 Wickens, 82, 336 Workforce, 91, 92, 99, 106, 330, 332, 485 Workspace, 381, 382

Z Zones, 148, 357, 454, 461, 485, 489