Multimodal Approaches to Healthcare Communication Research: Visualizing Interactions for Resilient Healthcare in the UK and Japan 9781350298477, 9781350298507, 9781350298484

Citation preview

A

Multimodal Approaches to Healthcare Communication Research

i

Bloomsbury Studies in Language and Healthcare Series Editor: Jack Pun, City University of Hong Kong Providing a broad, up-to-date, comprehensive, and systematic overview of the latest research into language and healthcare, the Bloomsbury Studies in Language and Healthcare series explores the importance of interdisciplinary research to the fields of language, linguistics, communication, healthcare, and medicine. Written by international researchers from a wide range of disciplines, including healthcare professionals, medical educators, linguists, and clinicians, books within the series offer a variety of perspectives and approaches. Cutting across the boundaries between linguistics and medicine, they investigate how communication affects the quality of care patients receive from health providers in different clinical settings, and how effective communication enhances our understanding of the challenges and expectations in healthcare worldwide.

ii

Multimodal Approaches to Healthcare Communication Research Visualizing Interactions for Resilient Healthcare in the UK and Japan Edited by Keiko Tsuchiya, Frank Coffey, and Kyota Nakamura

iii

BLOOMSBURY ACADEMIC Bloomsbury Publishing Plc 50 Bedford Square, London, WC1B 3DP, UK 1385 Broadway, New York, NY 10018, USA 29 Earlsfort Terrace, Dublin 2, Ireland BLOOMSBURY, BLOOMSBURY ACADEMIC and the Diana logo are trademarks of Bloomsbury Publishing Plc First published in Great Britain 2023 Copyright © Keiko Tsuchiya, Frank Coffey and Kyota Nakamura, 2023 Keiko Tsuchiya, Frank Coffey and Kyota Nakamura have asserted their right under the Copyright, Designs and Patents Act, 1988, to be identified as Editors of this work. Cover image © sudok1/istock All rights reserved. No part of this publication may be reproduced or transmitted in any form or by any means, electronic or mechanical, including photocopying, recording, or any information storage or retrieval system, without prior permission in writing from the publishers. Bloomsbury Publishing Plc does not have any control over, or responsibility for, any third-party websites referred to or in this book. All internet addresses given in this book were correct at the time of going to press. The author and publisher regret any inconvenience caused if addresses have changed or sites have ceased to exist, but can accept no responsibility for any such changes. A catalogue record for this book is available from the British Library. A catalogue record for this book is available from the Library of Congress. ISBN: HB: 978-1-3502-9847-7 ePDF: 978-1-3502-9848-4 eBook: 978-1-3502-9849-1 Series: Bloomsbury Studies in Language and Healthcare Typeset by RefineCatch Limited, Bungay, Suffolk To find out more about our authors and books visit www.bloomsbury.com and sign up for our newsletters.

iv

Contents List of Figures List of Tables List of Contributors Foreword Kazue Nakajima Transcription Conventions Introduction Keiko Tsuchiya

vii ix x xiii xiv 1

Part 1 Multimodal Approaches to Resilient Healthcare in Japan and the UK 1 2

3

4

5

Resilient Performance of a Medical Team in a Critical Care Setting Kyota Nakamura Novel Approaches to Identifying Preventive Factors Against Falls Among Hospital Inpatients Takeru Abe and Kyota Nakamura Creating Interactional Spaces for Making Requests: A Leader’s View and Body Orientation in Emergency Care Simulation Kyota Nakamura, Takuma Sakai, and Keiko Tsuchiya A Multimodal Linguistic Analysis of Gaze and Active Listenership in Emergency Department Team Interactions Sarah Atkins and Małgorzata Chałupnik Predictability and Coordinated Actions in Emergency Care Interactions: Trauma Leaders’ Meta-Episodic Gaze Projection Frank Coffey, Keiko Tsuchiya, and Alison Whitfield

9

21

31

47

65

Part 2 Different Approaches to Healthcare Communication and Culture 6

7

Understanding the Impact of Restricted Interests on the Social Interactions of Adults with Autism Spectrum Disorder Kyoko Aizaki, Chris Walton, and Charlie Lewis Co-Construction of Professional Competence by Japanese and International Care Workers Junko Mori and Chiharu Shima

83 101

v

vi 8

9

Contents Silent Replies as Indicators of Having Common Ground Between a Leader and Members in an Emergency Care Team Akira Taneichi Different Frames in Emergency Care Interactions Between the UK and Japan: Timed Recap and Repetition Chain for Grounding Keiko Tsuchiya, Akira Taneichi, Frank Coffey, and Kyota Nakamura

123

139

Part 3 Visual and Virtual Technologies for Healthcare Simulation 10 The Creation and Validation of a 360-Degree Video-Based Situation Awareness Training and Assessment Tool for Trauma Team Leaders Andrew Mackenzie, Mike Vernon, James Myers, and Robert J. B. Hutton 11 Using Immersive Virtual Environments for Educational Purposes: Applicability of Multimodal Analysis Miharu Fuyuno 12 Capturing a Trauma Leader’s Eye Gaze Through the Development of the Semi-Automatic Gaze-Annotation (SAGA) Program Takeshi Saitoh and Keiko Tsuchiya

159 177

195

Conclusion Keiko Tsuchiya, Frank Coffey, and Kyota Nakamura

209

Index

215

List of Figures 1.1 1.2 1.3 1.4 2.1 2.2 2.3 2.4 2.5 3.1 3.2 3.3 3.4 3.5 4.1 4.2 4.3 4.4 5.1 5.2 5.3 5.4 6.1 6.2 6.3 7.1 7.2 7.3 7.4 7.5 7.6 7.7 9.1 9.2 9.3 9.4 9.5

In-patient flow in our critical care center Modifying the trauma care strategies flexibly Starting allocation to receive trauma casualties Re-allocation based on evaluation Search results from PubMed (1980 to 2020) Levels and evaluations of falls in the systems approach Theoretical distribution of events related to safety Non-linear model to control time-dependent and multilayered factors The word cloud reveals Japanese keywords regarding falls among inpatients The leader’s body orientation (single channel) A top-down view: the leader (L) and the recipient in the player’s zone The leader’s body orientation (multiple channel) A top-down view: the leader (L) in the observer’s zone Leader’s body orientation (making a request to a multi-party group) Still image of the medical team from Extract 1, line 11 Still image of the medical team from Extract 1, line 14 Still image of the medical team from Extract 2, line 8 Still image of the medical team from Extract 2, line 14 The SD’s meta-episodic projection Trauma booklet (SD) The JD’s meta-episodic projection Trauma booklet (JD) M standing up M’s hand gesture M twirls his finger Line 2 Line 7, kore Line 8, soo Line 1, kore dake Line 9, beddo saku Line 10, koko ni Line 30, kotchi no toire The time and length of the leader’s timed recap in the UK data Timed recap The leader’s repetition chain: Trauma Book (patient) The JD’s information sharing with the N2: Trauma Book (procedure) The leader’s informing of all: the call for the Neurosurgery Department

10 12 15 15 22 22 23 24 27 35 36 37 38 41 54 55 58 59 71 73 74 75 90 93 93 107 108 108 110 111 113 114 144 146 149 151 152 vii

viii 10.1 10.2 10.3 10.4 11.1 11.2 11.3 11.4 11.5 11.6 11.7 11.8 12.1 12.2

List of Figures Example screenshot of the video tool Pictorial representation of the “What happens next?” test An example screenshot of the participant task Performance results Calculation of the center of gravity of the face Sample motion-tracking results of face roll movement Box-plots of the face-roll degrees of the top three speakers and the others A user operating the teaching system CG image of the virtual venue created with MAYA Function selection screen Feedback screen Image of the visual field in practice mode The processing of the SAGA The user interface of the SAGA

162 166 167 168 180 181 182 184 184 185 186 186 203 203

List of Tables 2.1 3.1 3.2 4.1 4.2 4.3 5.1 5.2 7.1 9.1 9.2 11.1 12.1 12.2 12.3 13.1

Significant text-factors associated with incident levels in different models The time length of the leader’s view as an observer and a player Single/multichannel communication in leader’s request-making Station completion times Types of markers of active listenership observed in the data (adapted version of Knight and Adolphs’s (2008) coding scheme) Markers of active listenership employed by trainee doctors in response to Linda’s (the nurse’s) handover The leader’s gaze projection and WAI and WAD in the episode of signing-in The leader’s gaze projection and WAI and WAD in the episode of requesting-to-scribe Content adjustments in the notebook Information sharing patterns in the simulation in Japan The huddling and booking frame in the leaders’ information-giving XR technology for medical education Eye-tracking devices Eye-tracking analysis software Multimodal analysis software (video/audio recordings) Multidisciplinary research for resilient healthcare

28 34 34 50 51 52 76 77 116 148 153 188 198 199 201 210

ix

List of Contributors Takeru Abe is Assistant Professor of the Advanced Critical Care and Emergency Centre, working also in the Department of Quality and Safety in Healthcare, at Yokohama City University Medical Centre, Japan. He completed his Ph.D. in Medical Science at the Graduate School of Medical Science, Kyushu University. His research interests include quantitative and mixed methodology, text-mining, and modeling of measurements in quality and safety in healthcare. Kyoko Aizaki is Postdoctoral Research Fellow of the Japan Society for the Promotion of Science / Tokyo Gakugei University, Japan. She holds a Ph.D. in intercultural communication. Her main areas of interest include the communication of Autism Spectrum Disorders (ASD). Aizaki is primarily focused on the effect of socio-cultural context and the language use of people with ASD. Sarah Atkins is Research Fellow of the Aston Institute for Forensic Linguistics, Aston University, UK. Her research investigates language and professional communication. She has conducted research on the linguistic features of simulation with a range of professional groups, including GPs and emergency care providers. Her work has a strong emphasis on applying findings into practice. Małgorzata Chałupnik is Assistant Professor in Linguistics and Professional Communication, and Director of the Centre for Research in Applied Linguistics (CRAL) at the University of Nottingham, UK. Her expertise is in professional communication, discourse analysis and pragmatics. Frank Coffey is Honorary Professor in Urgent and Emergency Care at Nottingham Trent University, UK. He is Head of Service at the Emergency Department and Director of DREEAM (Department of Research and Education in Emergency Medicine, Acute Medicine and Major Trauma), Nottingham University Hospitals NHS Trust. Coffey has a special interest in the field of Simulation Based Education. Miharu Fuyuno is Assistant Professor of the Faculty of Design, Kyushu University, Japan. She received an M.A. in TESOL from the University of Nottingham, England, and a Ph.D. in Linguistics from Seinan Gakuin University, Japan. Her research field includes multimodal corpora, interaction analysis, and application of these in technology-enhanced language learning. Robert J. B. Hutton is a Lecturer in Psychology at Nottingham Trent University, UK, and a Fellow of the Chartered Institute of Ergonomics and Human Factors. His x

List of Contributors

xi

research interests spring from a desire to understand and improve decision making and other forms of cognitive work in complex, dynamic operating environments. He has developed and tested a wide range of cognitive task analysis methods in support of cognitive systems engineering projects in defence, healthcare, and the emergency services. Charlie Lewis is Professor of Family and Developmental Psychology at Lancaster University, UK. His research focuses on both preschoolers’ social cognitive skills and the influence of family factors on the nature and development of these skills. Andrew Mackenzie is Senior Lecturer at Nottingham Trent University, UK. His main research interests lie within the fields of visual psychology and applied cognitive psychology. Working across several domains (e.g., transport, medicine, and sport) he broadly wishes to a) understand the visual processes involved in tasks, b) understand the visual and attentional mechanisms that contribute to the individual differences such as expertise, and c) where possible, develop diagnostic and training tools to assess and develop skill. Junko Mori is Professor of Japanese language and linguistics at the University of Wisconsin-Madison, USA. By using conversation analysis as a central framework, she has examined various types of social interactions involving first- and second-language speakers of Japanese. In recent years, the scope of her research has expanded to address the issue of diversity and inclusion in the classroom, workplaces, and the professional community of language educators. James Myers is a Ph.D. student in the Department of Psychology at the University of Sheffield, UK. His current research focuses on modeling the impact of social isolation on cognitive decline in dementia patients. He has previously worked as a research assistant in the Department of Psychology at Nottingham Trent University, contributing to a variety of research in areas such as psycholinguistics, situation awareness, memory, and virtual reality exposure therapy. Kazue Nakajima, M.D., M.Sc., Ph.D. is Director and Professor of the Department of Clinical Quality Management, Osaka University Hospital, Japan. She has committed herself to patient safety and quality improvement, and medical education for more than twenty years. Her clinical and research interests included non-technical skills, redesign of the system, patient engagement, systems dynamics, and resilient healthcare. Kyota Nakamura, M.D., Ph.D. is Board Certified in Emergency Medicine and Intensive Care. He is Professor of the Department of quality and safety in healthcare, Yokohama City University Medical Center, and the Department of Clinical Quality Management, Osaka University Hospital, Japan. Nakamura has committed himself to education for medical students and healthcare professionals over fifteen years. His interests include resuscitation, airway and ventilator management, the trauma system, disaster medicine, dynamic team performance, and medical simulation training.

xii

List of Contributors

Takeshi Saitoh is Professor of the Department of Artificial Intelligence, Kyushu Institute of Technology, Japan. He received B.Eng., M.Eng., and D.Eng. degrees from Toyohashi University of Technology, Japan. His research field includes image processing and pattern recognition. Takuma Sakai, M.D., is Emergency Care Consultant at Matsuoka Emergency Medical Service, Kagoshima, Japan. He previously worked in the Department of Emergency Medicine at Yokohama City University Medical Centre. His interests include team performance and medical simulation training in emergency care settings. Chiharu Shima is Associate Professor in the Graduate School of Global Communication and Language at Akita International University, Japan. Her research interests include second-language socialization and intercultural communication in institutional settings, in particular, workplaces. Akira Taneichi is Research Fellow of the Graduate School of Urban Social and Cultural Studies at Yokohama City University, Japan. He completed his doctoral thesis on the theme of an interpretative framework for acts of silence in Japanese communication. Keiko Tsuchiya is Associate Professor in the Graduate School of Urban Social and Cultural Studies, Yokohama City University, Japan. Her research interests include multimodal analysis of team interactions in healthcare settings and language education in Japanese contexts, specifically, ELF (English as a Lingua Franca) and CLIL (Content and Language Integrated Learning). Her monograph Listenership Behaviours in Intercultural Encounters: A Time-aligned Multimodal Corpus Analysis was published in 2013. Mike Vernon is Lecturer in Psychology at Nottingham Trent University, UK. He previously worked as a Data Manager and has conducted psycholinguistic-based research investigating how the planning scope of typewriting is mediated by inner speech and the availability of stored motor codes. His additional research interests are in visual attention, transport psychology, and motor control. Chris Walton is Senior Lecturer in Social Psychology in the Department of Psychology at Lancaster University, UK. His research interests focus on the qualitative analysis of the functions of emotions in social interaction and interactions in health and social care settings. Alison Whitfield is Emergency Nurse Practitioner and Clinical Educator in Emergency Medicine at the Emergency Department in Nottingham University Hospital NHS Trust and DREEAM (Department of Research and Education in Emergency Medicine, Acute Medicine and Major Trauma), UK. Her research interests include emergency care simulation and the professional development of simulated patients.

Foreword The Greek philosopher Aristotle noted that the whole is greater than the sum of its parts, which is currently known as synergy. This insight attracts a scientific curiosity about how such a phenomenon emerges. Two different paradigms have been applied to modern science: analytic and synthetic, or reductionistic and holistic approaches. In understanding the team’s performance, the synthetic approach focuses on the interactions of individual team members, while the conventional analytic approach looks into individual performance. In healthcare settings, it has been a long-time interest how team members interact with each other in achieving an excellent performance by the team as a whole. The synthetic approach, in general, needs to handle massive data or non-linear analyses to understand the interactions of components of the system, as exemplified in systems biology as a multidisciplinary science. Similarly, dynamic human interactions in a healthcare team require a multimodal approach because people communicate verbally and non-verbally. The current information technology has enabled us to capture a lot of data including human speech, gazes, motions, and texts in documents to describe everyday clinical work. A multidisciplinary approach is also essential to obtain significant benefits from scientific and technological advancement. This book is an international collaborative work on human interactions and communications with multimodal and multidisciplinary approaches. The book showcases novel findings in how team members achieved their intended goals through verbal and non-verbal interactions in clinical or simulated cases, particularly emergency care situations. The authors from the UK and Japan also elucidated cultural differences in communication or leadership styles in healthcare teams of both countries from interactional perspectives. The researchers’ interests were extended to text analysis of medical records for fall prevention, the co-constructional process of an international care worker’s competencies, and the capture of restricted and repetitive interests in an adult with autism spectrum disorder amid social interactions. The potential of innovative technologies for effective educational tools and the development of a semi-automatic gaze-recognition program was also elaborated. One of the strengths of the research in the book is its incorporation of resilience engineering, which provides a theoretical perspective for understanding how work is achieved in complex adaptive systems. The authors are highly ambitious in wanting to synthesize resilient healthcare systems that can continue to function in the environment with perturbations and constraints based on their methodologies and research findings. This book will serve as a valuable tool that provides methodological approaches to understanding human communications. Further, it will open the door to managing team performance and even taming complex adaptive systems. Kazue Nakajima Osaka University Hospital, Japan xiii

Transcription Conventions Symbol

Explanation

((cough)) ( ) ( may i ) (.) (3.0) [ ]

Nonverbal, para- and non-linguistic actions and events. Unintelligible speech. Assumed, uncertain transcription. Micro pause, estimated, up to approximately 0.2 seconds in duration. Measured pause. Opening brackets are inserted at exactly the point in speaking where the overlap starts, and closing brackets where it ends. Speech latched to previous turn (latching). An abrupt halt or interruption in an utterance. Inhalation. Extended word/sound. Rising/falling/continuing intonation, respectively. Animated and emphatic tone. Marked shift in pitch, up (↑) or down (↓). Double arrows can be used with extreme pitch shifts. A passage of talk quieter than the surrounding talk. A passage of talk faster than the surrounding talk. A passage of talk slower than the surrounding talk. Smiley voice, or suppressed laughter. Relatively high pitch. Entered by the analyst to show a sentence of particular interest.

= hhh er::m ?/./, ! ͬword °word° >word