Augmented Reality in Educational Settings [1 ed.] 9789004408845, 9789004408838

Citation preview

Augmented Reality in Educational Settings

Augmented Reality in Educational Settings Edited by

Theodosia Prodromou

leiden | boston

All chapters in this book have undergone peer review. Library of Congress Cataloging-in-Publication Data Names: Prodromou, Theodosia, editor. Title: Augmented reality in educational settings / edited by Theodosia Prodromou. Description: Leiden ; Boston : Brill/ Sense, 2020. | Includes bibliographical references and index. Identifiers: LCCN 2019040462 (print) | LCCN 2019040463 (ebook) | ISBN 9789004408821 (paperback) | ISBN 9789004408838 (hardback) | ISBN 9789004408845 (ebook) Subjects: LCSH: Virtual reality in education. | Computer-assisted instruction. | Augmented reality. Classification: LCC LB1044.87 .A944 2020 (print) | LCC LB1044.87 (ebook) | DDC 371.33/468--dc23 LC record available at https://lccn.loc.gov/2019040462 LC ebook record available at https://lccn.loc.gov/2019040463

Typeface for the Latin, Greek, and Cyrillic scripts: “Brill”. See and download: brill.com/brill-typeface. isbn 978-90-04-40882-1 (paperback) isbn 978-90-04-40883-8 (hardback) isbn 978-90-04-40884-5 (e-book) Copyright 2020 by Koninklijke Brill NV, Leiden, The Netherlands. Koninklijke Brill NV incorporates the imprints Brill, Brill Hes & De Graaf, Brill Nijhoff, Brill Rodopi, Brill Sense, Hotei Publishing, mentis Verlag, Verlag Ferdinand Schöningh and Wilhelm Fink Verlag. All rights reserved. No part of this publication may be reproduced, translated, stored in a retrieval system, or transmitted in any form or by any means, electronic, mechanical, photocopying, recording or otherwise, without prior written permission from the publisher. Authorization to photocopy items for internal or personal use is granted by Koninklijke Brill NV provided that the appropriate fees are paid directly to The Copyright Clearance Center, 222 Rosewood Drive, Suite 910, Danvers, MA 01923, USA. Fees are subject to change. Brill has made all reasonable efforts to trace all rights holders to any copyrighted material used in this work. In cases where these efforts have not been successful the publisher welcomes communications from copyrights holders, so that the appropriate acknowledgements can be made in future editions, and to settle other permission matters. This book is printed on acid-free paper and produced in a sustainable manner.

Contents List of Figures and Tables ix Notes on Contributors xvi Introduction xxix

PART 1 Early Education 1 Uses of Augmented Reality in Pre-Primary Education 3 Eva Severini, Blanka Kožík Lehotayová and Eva Csandová 2 Uses of Augmented Reality for Development of Natural Literacy in Pre-Primary Education 24 Kateřina Jančaříková and Eva Severini 3 Empowering Teachers to Augment Students’ Reading Experience: The Living Book Project Approach 56 Maria Meletiou-Mavrotheris, Constadina Charalambous, Katerina Mavrou, Christos Dimopoulos, Panayiota Anastasi, Ilona-Elefteryja Lasica, Nayia Stylianidou and Christina Vasou 4 Uses of Augmented Reality in Primary Education 80 Eva Csandová, Renata Tothova and Lilla Korenova 5 Augmented Reality Applications in Early Childhood Education 101 Lilla Korenova, Zsolt Lavicza and Ibolya Veress-Bágyi

PART 2 Advanced Education 6 Mathematics Learning and Augmented Reality in a Virtual School 123 Gilles Aldon and Corinne Raffin

vi

Contents

7 Engaging Students in Covariational Reasoning within an Augmented Reality Environment 147 Osama Swidan, Florian Schacht, Cristina Sabena, Michael Fried, Jihad El-Sana and Ferdinando Arzarello 8 Uses of Augmented Reality in Biology Education 168 Mária Fuchsová, Miriam Adamková and Miroslava Pirháčová Lapšanská 9 Uses of Augmented Reality in Tertiary Education 195 Martina Siposova and Tomas Hlava 10 An Augmented Reality Based Intelligent Diagnosis Platform for Medical Training 217 Utku Köse and Omer Deperlioglu 11 Augmented Reality and Future Mathematics Teachers 236 Martina Babinská, Monika Dillingerová and Lilla Korenova

PART 3 All Ages and outside the School 12 Enlivened Laboratories within STEM Education (EL-STEM): A Case Study of Augmented Reality in Secondary Education 267 Ilona-Elefteryja Lasica, Maria Meletiou-Mavrotheris, Efstathios Mavrotheris, Stavros Pitsikalis, Konstantinos Katzis, Christos Dimopoulos and Christos Tiniakos 13 Augmented Playgrounds: Questioning Simulations to Question Intuitions 295 Chronis Kynigos, Zacharoula Smyrnaiou and Marianthi Grizioti 14 Augmented Reality in Mathematics Education: The Case of GeoGebra AR 325 Melanie Tomaschko and Markus Hohenwarter 15 Automatically Augmented Reality with GeoGebra 347 Francisco Botana, Zoltán Kovács, Álvaro Martínez-Sevilla and Tomás Recio

Contents

16 Augmented Reality in Museums and Cultural Heritage Settings 369 Georgios Papaioannou 17 Applications of Augmented Reality Apps in Teaching Technical Skills Courses 383 Lilla Korenova, Maria Kožuchová, Jiří Dostál and Zsolt Lavicza 18 Devices for Virtual and Augmented Reality 410 Robert Bohdal Index 445

vii

Figures and Tables Figures 1.1 1.2 1.3 1.4 2.1

Thematic categorization/initial evaluation. 15 Thematic conceptualization/ongoing evaluation. 16 Thematic realization/output evaluation. 17 Graphic representation of selective coding results. 19 Visualisation, Department of Biology and Environmental Studies, Charles University, Faculty of Education. 28 2.2 Selected aids used by the author for development of science literacy in primary home schoolers. 28 2.3 Mounted birds (Falco peregrinus, Circus aeruginosus, Pica pica) of Department of Biology and Environmental Studies, Charles University, Faculty of Education, stuffed under supervision of Jan Řezníček, Ph.D. 29 2.4 Natural science collection of Department of Biology and Environmental Studies, Charles University, Faculty of Education. 30 2.5 Scheme of the water cycle (by John Evans and Howard Periman, USGS, http://ga.water.usgs.gov/edu/watercycle.html, Public Domain, https://commons.wikimedia.org/w/index.php?curid=26818355). 32 2.6 The dynamic three-dimensional model of childbirth allows manipulation and thus demonstrates the process more fittingly. The model is in the collection of the Department of Biology and Environmental Studies, Charles University, Faculty of Education. 32 2.7 Thematic categorization/initial evaluation. 43 2.8 Thematic conceptualization/on-the-fly evaluation. 44 2.9 Thematic realization/final evaluation. 45 2.10 Development of scientific literacy in pre-schoolers using augmented reality with respect to the point of view of research subjects. 49 2.11 Position of the teacher and children recorded in time and space presence of development of scientific literacy of children in the augmented-realitysupported teaching based on evidence. 51 4.1 Thematic categorization/initial evaluation. 89 4.2 Thematic conceptualization/ongoing evaluation. 90 4.3 Thematic realization/output evaluation. 91 4.4 Thematic categorization/initial evaluation. 92 4.5 Thematic conceptualization/ongoing evaluation. 93 4.6 Thematic realization/output evaluation. 93

x 4.7 4.8 5.1 5.2 5.3 5.4 5.5 5.6 5.7 5.8 5.9 5.10 5.11 5.12 5.13 6.1 6.2 6.3 6.4 6.5

Figures and Tables

Graphical representation of selective coding. 95 Graphical representation of selective coding. 96 iSandBox. 103 Detail from the DIFER – Speech and sound hearing. 104 Detail from the DIFER – Elemental numeracy skills. 105 AR Dragon on the street. 109 Drawing a horse with aid of sketch AR. 110 The dodecahedron and appearance of the aithêr. 111 Addition app’s card. 112 Multiple shapes at the same time. 112 The Walla Me private message. 113 The tower on the table made by Stack AR. 114 A turtle on my table. 115 Lego product 3D animation. 116 A colored animal who eats his food when it comes to life with the app. 116 A place of collaboration with shared boards. 125 Object augmented with geometrical data. 127 English homework with a teacher. 128 House built by students in the virtual world. 131 (a) The teacher screen showing the available space. (b) A side of this field. 136 6.6 Students extract the coordinates. 137 6.7 Students infer the field’s dimensions. 138 6.8 Back and forth from mathematical world to virtual world. 142 7.1 Illustration of the AR prototype. The inclined plane illustrates the real phenomenon; the Cartesian systems and the table of values illustrate the covariation of distance-time of the dynamic object; the left distance-time Cartesian system and the table of values illustrate discrete covariation; the middle distance-time Cartesian system illustrates chunky continuous covariation; the moving ball illustrates smooth continuous covariation. 148 7.2 The dynamic phenomenon: A cube moving on an inclined plane during a physical experiment. 157 7.3 The real-world experiment and the mathematical model of the dynamic object displayed immediately and in real time. 158 7.4 Coded color method for recognizing the dynamic object. 160 7.5 Cube detection by markers. 162 8.1 Application the Brain iExplore and Anatomy 4D. 177 8.2 AR shaping in educational context. 183 10.1 The architecture of modern clinical decision support systems (from Deperlioglu, 2018). 223

Figures and Tables

xi

10.2 Components of web based clinical decision support systems (from Deperlioglu, 2018). 224 10.3 Structure of the intelligent diagnostic platform. 227 10.4 Components of the intelligent diagnostic platform. 228 10.5 Some AR views from the diagnosis oriented medical training system. 229 11.1 Augmented polyhedrons. 240 11.2 Structure from cube, blueprint. 254 12.1 SAM phases (from Allen, 2012). 271 12.2 EL-STEM project’s approach based on the SAM model. 272 12.3 Different types of laboratories presented by Lasica et al. (2016). 284 13.1 Description of playful experience. 302 13.2 Players trying to avoid the shapes moving at different heights in the game ‘Apples’. 304 13.3 The ‘Wobble Board’ installation. 309 13.4 The field of air-hockey project on the floor and players interact with it with body movements. 315 14.1 Exploring virtual 3D content using GeoGebra AR. 328 14.2 (a) Plane detection. (b) Placed object. (c) Two graphs. 329 14.3 Predefined 3D math object and corresponding mathematical task. 330 14.4 Taking screenshots from different objects. 336 14.5 Penrose triangle. 337 14.6 Constructing the illusion of a closed triangle. 337 14.7 Sierpinski pyramid. 338 14.8 Football. 338 14.9 Klein bottle. 339 14.10 (a) Paraboloid. (b) Changing equations. (c) Two graphs in one view. 340 14.11 Extract from the interactive GeoGebra Book (from Brzezinski, 2018). 341 14.12 Modelling every-day, 3D objects with GeoGebra AR (from Brzezinski, 2018a). 341 14.13 Draft of an interactive GeoGebra AR app. 343 15.1 Klein bottle visualized through the GeoGebra AR app. 349 15.2 Reconstruction, through a GeoGebra layer, of the ruins of a medieval bridge in Granada (from Martínez-Sevilla, 2017). 350 15.3 Dependent maths and reality. Independent maths and user viewpoint (from Martínez-Sevilla, 2017). 352 15.4 Mathematical layer over a renaissance palace facade (Martínez-Sevilla, 2017). 354 15.5 Measuring a distant object with a mobile app (Source: Smart Distance Pro app). 355 15.6 Imagined screen of a GG_RLM app task, computing the height of a building. 357

xii

Figures and Tables

15.7 Imagined screen of GG_RLM app, computing the height of a horseshoe arch. 358 15.8 The automatic derivation tool gives the numerical verification of some property holding among them for the concrete positions of the input data A, B, C, D, such as the alignment of E, F and G. 359 15.9 The automatic discovery algorithm should be able to output the necessary (and sufficient) location for D in order to have the collinearity of D’, D’’, D’’’. 360 15.10 If two lines are drawn from one vertex of a square to the midpoints of the two non-adjacent sides, then they divide the diagonal into three equal segments. 361 15.11 Imaginary detection of reality with an internal representation of a parquet floor being photographed by a smartphone and translated into GeoGebra. 362 15.12 Internal analysis of a conjecture in GeoGebra. 363 15.13 Internal analysis of another conjecture in GeoGebra. 364 15.14 GeoGebra confirmation of statement. 365 17.1 Car Engine AR app (https://play.google.com/store/apps/ details?id=com.magicsw.carenginear). 394 17.2 How does an airport work? AR app (https://play.google.com/store/apps/ details?id=com.books2ar.airportsAR). 395 17.3 Steam Museum AR app (https://play.google.com/store/apps/ details?id=com.thinqdigitalmedia.android.steammuseumar). 395 17.4 Tunnelworks AR app (https://play.google.com/store/apps/ details?id=com.tenalps.tbm). 396 17.5 Spacecraft 3D AR app (https://play.google.com/store/apps/ details?id=gov.nasa.jpl.spacecraft3D). 397 17.6 Katalog IKEA app (https://play.google.com/store/apps/ details?id=com.ikea.catalogue.android). 398 17.7 House in AR app (https://play.google.com/store/apps/ details?id=com.etezo.HouseConcept). 399 17.8 ARki: A-R Architecture app (https://play.google.com/store/apps/ details?id=com.darfdesign.arki). 400 17.9 Bike 3D Configurator app (https://play.google.com/store/apps/ details?id=com.Elementals.Bike3DConfigurator). 401 17.10 LEGO 3D catalogue AR app (https://play.google.com/store/apps/ details?id=com.lego.catalogue.global). 401 17.11 Augmented repair AR app (https://play.google.com/store/apps/ details?id=de.reflekt.enterprise.awedemo). 402 17.12 Assemblr – Create 3D Models AR app (https://play.google.com/store/apps/ details?id=com.octagonstudio.assemblr). 403 17.13 ThomasAR World AR app (https://play.google.com/store/apps/ details?id=com.redfrog.thomasarworld). 404

Figures and Tables

xiii

17.14 HBR AR app (https://play.google.com/store/apps/details?id= com.ptc.hbrar). 405 17.15 ARuler – AR Ruler app (https://play.google.com/store/apps/ details?id=com.grymala.aruler). 406 17.16 Prime Ruler AR app (https://play.google.com/store/apps/ details?id=com.grymala.photoruler). 406 18.1 Scheme of the LCD screen. 414 18.2 Passive vs. active matrix. 415 18.3 Scheme of the OLED screen. 416 18.4 Brewster stereoscope (CC BY-SA 4.0 – Alessandro Nassiri, Museo Nazionale della Scienza e della Tecnologia Leonardo da Vinci, Milano, https://commons.wikimedia.org/wiki/File:IGB_006055_Visore_stereoscopico_ portatile_Museo_scienza_e_tecnologia_Milano.jpg). 418 18.5 The parallax barrier principle. 419 18.6 Display with a time aperture. 420 18.7 The lenticular display principle. 421 18.8 Example of a multi-layered 3D display with LC panels. 423 18.9 The principle of the volumetric display with a rotating surface (photo courtesy of Voxon Photonics). 424 18.10 The electro-hologram principle. 425 18.11 An example of virtual image projected on semi-transparent film. 427 18.12 Google Glass (left, CC-BY-SA-3.0, Tim Reckmann, https://commons.wikimedia.org/wiki/File:Google_Glass_Main.jpg) and Microsoft HoloLens (right, CC-BY-SA-4.0, Ramadhanakbr, https://commons.wikimedia.org/wiki/File:Ramahololens.jpg). 427 18.13 Oculus Rift (CC-BY-SA-4.0, Samwalton9, https://commons.wikimedia.org/wiki/ File:Oculus_CV1_Back.jpg). 429 18.14 The optical HMD operation principle. 430 18.15 Planar, spherical and free-form optical combiner. 431 18.16 Optical combiner with holographic optical elements and waveguide plate. 431 18.17 Headset Gear VR for Samsung Galaxy smartphones (CC BY 2.0, Maurizio Pesce, https://commons.wikimedia.org/wiki/File:Samsung_Gear_VR_ (15247457825).jpg). 432 18.18 The principle of an acoustic-wave touch panel. 434 18.19 Example of an array of subpixels of the CCD sensor. 434 18.20 Microsoft Kinect, the old (public domain, Evan-Amos, https://commons.wikimedia.org/wiki/Category:Kinect#/media/ File:Xbox-360-Kinect-Standalone.png) and the new version (public domain, Evan-Amos, https://commons.wikimedia.org/wiki/Category:Kinect#/media/ File:Xbox-One-Kinect.jpg). 435

xiv

Figures and Tables

18.21 18.22 18.23 18.24 18.25

The principle of the accelerometer. 436 The effect of the Coriolis force on a moving object. 436 The principle of the gyroscope. 437 The principle of a vibrator using electric motor. 438 The principle of how manipulandum works (left), commercial product Novint Falcon (right) (GNU Free Documentation License, Archimëa, https://commons.wikimedia.org/wiki/File:Novint_Falcon.jpg). 439 18.26 HaptX glove (photo courtesy of HaptX). 439 18.27 The scheme of the speaker. 440 18.28 The scheme of the microphone. 441

Tables 1.1

Identified categories and concepts of research participants’ statements from focus group. 12 1.2 Protocol No. 1 Activity analysis (micro-teaching analysis) obtained through direct and indirect observation study of pedagogical reality and diagnostics based on children’s evidence in the learning/playing group (extract). 15 2.1 Diagnostics presence of development of scientific literacy of children in the augmented reality supported teaching based on evidence (excerpt). 43 2.2 Referential framework presence of development of scientific literacy of children in the augmented reality-supported teaching based on evidence. 47 3.1 Evaluation data by level of Guskey (2002) hierarchy. 72 4.1 Protocol No. 1 EUR phases. 89 4.2 Protocol No. 2 EUR phases. 92 5.1 Skills and suitable applications. 117 8.1 Categories of students’ activities. 179 9.1 Inclusion and exclusion criteria. 198 9.2 Descriptive characteristics of included papers with quantitative data. 198 11.1 Main results, secondary and upper secondary teacher-training programme students. 243 11.2 Polyhedrons’ difficulty, secondary and upper secondary teacher-training programme students. 245 11.3 Main results, primary teacher-training programme students. 247 11.4 Polyhedrons’ classification. 250 11.5 AR applications’ implementation into the secondary school mathematics. 253 12.1 Where to use AR/MR to Enliven Laboratories in STEM. 278 13.1 Strategies you would implement if you replayed the game ‘Apples’. 305 13.2 The type of movement made by them in the game ‘Apples’. 306

Figures and Tables

xv

13.3 The physical magnitudes which change when the player plays the game ‘Apples’. 307 13.4 Movements/gestures among students in the game ‘Apples’. 308 13.5 The most often applied strategies if you replay the ‘Wobble Board’ game. 310 13.6 The kind of movement performed by the player in the ‘Wobble Board’ game. 311 13.7 The kind of ball’s movement. 312 13.8 The physical magnitudes which change when the player plays the ‘Wobble Board’ game. 312 13.9 Most used part of the players’ body during the game. 313 13.10 The most important movements during the game. 313 13.11 Strategies most often applied in ‘Air Hockey’. 316 13.12 Hockey puck’ s movement. 317 13.13 Sizes that change when the player plays the ‘Air Hockey’ game. 317 13.14 Players’ roles in ‘Air Hockey’. 318 13.15 Players’ movements for the communication between team members. 318 13.16 Body members which are used by players. 319 17.1 Main areas of technical education in SVK. 389

Notes on Contributors Miriam Adamková has been working in the field of education for several years, in both private and public sector, teaching children and students of different ages. She holds a Ph.D. from the Faculty of Education at Comenius University in Bratislava. As a teacher and researcher, she is primarily focused on increasing the effectivity of educational process, professional development of in service teachers and preservice teachers, development of skills and cross curricular teaching and learning. She worked as a researcher at the Institute of Pedagogical Sciences and Studies at Comenius University (Centre of Preschool and Primary Education at the Faculty of Education). Nowadays she works as a teacher in public kindergarten. Gilles Aldon after having taught in high secondary school and trained teachers at the Research Institute on Mathematics Teaching of Lyon (IREM de Lyon), Gilles Aldon obtained his doctorate in 2011 and is currently working at the French Institute of Education at the École Normale Supérieure de Lyon. His main research topic is devoted to the use of technology in the teaching and learning of mathematics. In particular, the issues of changes in teaching and learning, the contribution of technology to the experimental part of mathematics and to problem-solving processes are at the core of its research. Panayiota Anastasi is a Ph.D. Candidate at European University Cyprus. She received a B.A. in “Primary Education” (2009) and an M.A. in “Language Pedagogy” from the Department of Education of the University of Cyprus. Her research interests focus on Language Teaching, Literacy, Multiliteracies and Literature instruction. Her doctoral research is focused on the teaching of Literature through the use of mobile devices and augmented reality apps. During the past nine years, she has been working as a primary school teacher in public schools of Cyprus. She has also been involved in EU funded research projects. Ferdinando Arzarello is an Emeritus Professor of Mathematics Education at Turin University. He is a member of the Executive Committee of ICMI (International Commission on Mathematical Instruction) as immediate past president of ICMI, and member of the Academia delle Scienze di Torino. He is the author of more than 150 publications about Mathematics Educations in outstanding international

Notes on Contributors

xvii

journals and books and has supervised several Ph.D. Dissertations in Mathematics Education in Italy and abroad. He is involved in different National and International projects about the teaching/learning of Mathematics, and has been invited to give plenary lectures in all the Continents about this topic. Martina Babinská is a teacher and a researcher at the Faculty of Mathematics, Physics and Informatics of Comenius University in Bratislava. In 2015, She attained a Ph.D. degree in the theory of teaching of mathematics. The main part of her research is devoted to self-study, intrinsic motivation and the approach of students and future teachers to mathematics. (Personal webpage: http://enjoymaths.education/en/personal/babinska-martina) Robert Bohdal received a M.Sc. degree in computer graphics from the Faculty of Mathematics and Physics from Comenius University in Bratislava, Slovakia. She also holds a Ph.D. in geometry and topology (Comenius University in Bratislava, Slovakia, 2007). He mainly deals with graphics devices used in computer graphics, scattered data interpolation and picture deformation recovery methods. He currently teaches procedural modeling, computer graphics and modeling of curves and surfaces using splines at Comenius University in Bratislava, FMFI, Slovakia. As a developer, he occasionally contributes to various open-source projects. Francisco Botana is a Professor of Applied Mathematics at the University of Vigo, Pontevedra in Spain. He is the author of a large number of scientific papers and communications at international journals and conferences. The topics of his research are about Automated reasoning in geometry, Dynamic geometry environments, Internet accessible mathematical computation. He actively participates in the scientific community about these topics, as part of the organizing or advisory committee of many international conferences and through his involvement (reviewing, member of the editorial board) in editorial tasks. Further details at http://fbotana.webs.uvigo.es Constadina Charalambous is an Assistant Professor of Language Education & Literacy at European University Cyprus. She holds a Ph.D. in Sociolinguistics & Education (King’s College, 2009), M.A. in Language, Ethnicity & Education (King’s College, 2005, Distinction) and B.A. in Greek Language & Literature (Aristotle University of Thessaloniki). Her main research interests evolve around language educa-

xviii

Notes on Contributors

tion, bilingualism, language policy and the potential contribution of language learning to peace education and social cohesion. She has coordinated a 3-year research project in collaboration with King’s College London, and is currently involved in two Erasmus+ projects. She has recently published a co-authored monograph with CUP. Eva Csandová completed her Ph.D. at Comenius University in Bratislava. Her research focusses on issues related to pre-school education and teacher training. She is particularly interested in researching the relationships between mentor teachers and novice teachers and through qualitative methodology she analyses the cooperation system, the support system and the mutual influence in the teaching community. She also focuses on issues of the early childhood education. Omer Deperlioglu received his B.Sc. in Electric and Electronic (1988) Gazi University, MSc in Computer Science (1996) from Afyon Kocatepe University, Ph.D. in Computer Science (2001) from Gazi University in Turkey. Now he is Associate Professor of Computer Programming in Department of Science, Vocational School of Afyon, Afyon Kocatepe University of Afyon, Turkey. His current research interests include different aspects of Artificial Intelligence applied in Power Electronics, Biomedical, and Signal Processing. Monika Dillingerová holds a B.Sc. in Mathematics at Comenius University in Bratislava. From 1989 to 1994, she taught secondary mathematics. From 1995 until today she is an Assistant professor in mathematics education at Slovak Technical University (Faculty of Mathematics, Physics and Informatics) and Comenius University in Bratislava (Department of Education). Her research interests focus on the teaching of mathematics, integration of digital technologies when teaching mathematics and learning mathematics with digital technologies. Christos Dimopoulos completed his Ph.D. in Control Engineering at University of Sheffield. He has 16 years of experience in the academic sector. He is currently an Associate Professor of Computer Science & Engineering, and co-Director of the Centre of Excellence in Risk and Decision Sciences at European University Cyprus. His research focuses in the areas of Decision Sciences and Educational Technology. He has significant experience as Scientific Coordinator or Research Col-

Notes on Contributors

xix

laborator in a large number of projects. His research accomplishments also include the publication of a considerable number of journal articles and book chapters. Jiří Dostál (Ph.D.) works at the Department of Crafts, Engineering, Technology and Computing at Palacký University in Olomouc. In the scientific and expert area, he focuses on the theory and practice of craft and technology education. He has achieved a certificate of merit by the dean of the Faculty of Education of Palacký University in Olomouc for his results in the area of science and research. In 2015, he was elected the deputy-chairman of the Czech Pedagogical Society (Olomouc branch). He is a member of the Czech Educational Research Association. Jihad El-Sana is a Professor at the Department of Computer Science, Ben Gurion University of the Negev. His research interests include, image processing, computer graphics, augmented reality, and pattern recognition. He has published around 100 papers in refereed journal and conference proceedings. El-Sana received his B.Sc. and M.Sc. in Computer Science from BGU. In 1995, he won a Fulbright Scholarship for doctoral studies in the US and 1999 he earned a Ph.D. in Computer Science from the State University of New York, Stony Brook and joined Ben-Guion University in Israel. El-Sana heads the Visual Media Lab, which hosts research projects in Computer Graphics, Image Processing, Augmented Reality, Computational Geometry, and Document Image Analysis. El-Sana awarded the Catacosinos Fellowship for Excellence in Computer Science and the Ersken Fellowship in 2013. Michael Fried is an Associate Professor at Ben Gurion University of the Negev and chair of the department of Science and Technology Education. His research interests are eclectic and include history of mathematics, mathematics education, semiotics, and history and philosophy of education. He holds a M.Sc. from SUNY Stony Brook in applied mathematics and a Ph.D. from the Cohn Institute for the History and Philosophy of Science at Tel Aviv University. He is the author (with Sabetai Unguru) of Apollonius of Perga’s Conics: Text Subtext Context (Brill, 2001) and Edmond Halley’s Reconstruction of the Lost Book of Apollonius’s Conics (Springer, 2011), and is co-editor (with Tommy Dreyfus), of Mathematics and Mathematics Education: Searching for Common Ground (Springer, 2014).

xx

Notes on Contributors

Mária Fuchsová is a Lecturer at Comenius University in Bratislava since 2010. She completed her Ph.D. in biology, anthropology at the Faculty of Natural Sciences of Comenius University in Bratislava. She completed further supplementary pedagogical studies Her research and publication activities focus on the physiological growth and development of children as well as children with disabilities. She is a member of the research team of KEGA and VEGA grants focused mainly on the education of pupils and pupils with disabilities in the field of environmental and health education. Marianthi Grizioti is a developer and a Ph.D. student at National and Kapodistrian University of Athens. Her research is on computer science education and computational thinking with the use of digital programmable games. As a member of the Educational Technology Lab, she took part in the development of the dynamic 3D Turtle Geometry modeler (http://etl.ppp.uoa.gr/malt2) and the game designer for socio-scientific issues (http://etl.ppp.uoa.gr/ChoiCo). Her research interests also include educational software design, interactive learning environments, game-based learning, and child computer interaction. Tomas Hlava completed his Ph.D. in language teaching methodology and now holds a position as a post-doctoral research assistant at the Institution of Philological Studies, Faculty of Education, Comenius University in Bratislava. His research interests focus on language attainment and its connection to issues of language pedagogy and interlanguage development based on cross-linguistic variation. As a lecturer of morpho-syntax, he is interested in the possibilities of visualising the surface syntactic relations through AR. Markus Hohenwarter created the open source dynamic mathematics software GeoGebra in 2002 as part of his master’s project in Austria. After his Ph.D., he spent three years at universities in Florida before he became a full Professor of mathematics education at Johannes Kepler University Linz in Austria in 2010. Kateřina Jančaříková completed her undergraduate studies in Special Biology and Environmental Studies at Charles University in 1993. In 2008, she completed her Ph.D. studies in Education Charles University. Since 2007 she has been working at the Department of Biology and Environmental Studies as Assistant Professor and is the head of the Centre of Environmental Education. Her research involves

Notes on Contributors

xxi

developmental national and international projects (GAČR, TAČR, Long Life learning program EU, Grundtwig etc.). Her research focuses on environmental and pre-primary science education. Konstantinos Katzis received his B.Eng. degree in Computer Systems Engineering and his MSc degree in Radio Systems Engineering from University of Hull (UK) in 2000 and 2001 respectively. In 2005, he completed his Doctor of Philosophy degree in Electronics at the University of York (UK). He currently holds the positions of Associate Professor of Computer Science & Engineering and Deputy Dean of the School of Sciences at European University Cyprus. His research interests include: attractiveness of engineering education and Technology Enhanced Learning. His research accomplishments include a considerable number of refereed articles and book chapters. Lilla Korenova is an Associate Professor of Mathematics and Computer Sciences at the Department of Didactics of Natural Sciences in Primary Education of the Faculty of Education, Comenius University in Bratislava, Slovakia. She served as visiting professor at University in Debrecen (Hungary). She supervised more than 60 research, bachelor, diploma, and doctoral projects, co-authored many papers and books. In the scientific and expert field, she focuses on didactics of mathematics, digital technologies in teaching and learning, mobile learning, e-testing, e-learning and statistical methods in quantitative research. She is an author and lecturer in several programs of teacher education. Utku Köse completed his B.Sc. degree in computer education (Gazi University in Turkey). He received a M.Sc. degree in computer and digital systems from Afyon Kocatepe University in Turkey. In 2017, he completed his Ph.D. in computer engineering from Selcuk University, Turkey. He is an Assistant Professor at Suleyman Demirel University, Turkey. He has more than 100 publications including articles, authored and edited books, proceedings, and reports. His research interests focus on artificial intelligence, machine ethics, artificial intelligence safety, optimization, the chaos theory, distance education, e-learning, computer education, and computer science. Zoltán Kovács is an Assistant Professor at The Private University College of Education of the Diocese of Linz, Institute of Initial Teacher Training (Austria), since 2015. He is a team member of Center of Mathematics Education of Linz at University of Linz,

xxii

Notes on Contributors

Department of Mathematics Education and a Core developer at GeoGebra, since 2010. The topics of his research are about Mathematics Education, Automated Reasoning in Dynamic Geometry, Computer Algebra. Blanka Kožík Lehotayová is a University Lecturer in Pre-Primary and Primary Education at Comenius University in Bratislava. She is also a co-researcher in research projects. In her scientific and research activities, she focuses on pedagogical theory and practice, especially applied by the teacher’s strategies in teaching. She is the author of scientific and scholarly articles published at national and international journals and the co-author of a monograph entitled Theoretical-Research Paradigm of Graphomotorics in Kindergarten (2018). Maria Kožuchová is a Professor of Pre-school and Elementary Pedagogy at the Faculty of Education of Commenius University in Bratislava. She has a rich publication activity. She has participated in several international grants (mainly UNESCO grants) with research teams of major European universities. Chronis Kynigos is a Professor of mathematics and technology education and director of the Educational Technology Lab, academic leader of Polymechanon. He has engaged in design research involving the infusion of constructionist pedagogy in and out of mainstream schooling. His research has proposed alternative curricular structures with the aim to create desnse meaning making discursive environments for learners. He has designed several constructionist digital media, most recently a programmable dynamic 3D Turtle Geometry modeller (http://etl.ppp.uoa.gr/malt2) and a game designer for socio-scientific issues (http://etl.ppp.uoa.gr/ChoiCo). He has led three multinational European projects in Digital media and Mathematics Education, was as partner in a number of others and served in the editorial boards of the IJCML, BJET and DEME journals. He was a founding member of the Greek Association of Mathematics Education (GARME). Ilona-Elefteryja Lasica is currently a Ph.D. Candidate, Research Associate at European University Cyprus. She holds a B.Sc. in “Digital Systems” (2010) and an M.Sc. in “Technology Education and Digital Systems” (Track: e-Learning) (2012) from the Department of Digital Systems University of Piraeus, Greece. Her research interests focus on Technology Enhanced Learning and Training, Innovation in Education (such as Augmented/Mixed Reality, ΙοΤ), Teachers Professional Develop-

Notes on Contributors

xxiii

ment and Lifelong Learning. During the past years she has been involved in many EU funded research projects in Greece and Cyprus. Her research accomplishments include a number of publications in international conferences and academic journals. Zsolt Lavicza has worked on several research projects examining technology and mathematics teaching in classroom environments in Michigan and Cambridge. In addition, Zsolt has greatly contributed to the development of the GeoGebra community and participated in developing research projects on GeoGebra and related technologies worldwide. Currently, Zsolt is a Professor of STEM Education Research Methods at Johannes Kepler University’s Linz School of Education. From JKU he is working on numerous research projects worldwide related to technology integration into schools; leading the doctoral programme in STEM Education; teaching educational research methods worldwide; and coordinates research projects within the International GeoGebra Institute. Álvaro Martínez-Sevilla is an Associate Professor of Mathematics at University of Granada, Spain. He is the author of papers in the field of Semigroup Theory and Criptography. He is now moving his interest to Artificial Intelligence, Mathematics and Art and Mathematical Education, where he is quite active in congresses, workshops and research. He is a member of the Interuniversity Institute on Data Science and Computational Intelligence (DaSCI) and leading member of the research project MonuMAI, about Monuments, Mathematics and Artificial Intelligence. He is also the leader of the Project “Paseos Matemáticos” which applies technology such as Augmented Reality, 3D modeling and Computer Graphics to Mathematical Education and Dissemination. He was awarded several prizes for these projects. He teachers the Master degree in Cultural Heritage at the University of Granada. Efstathios Mavrotheris joined the Open University of Cyprus in 2007 as Head of the Information and Communications Technologies Department. He has been responsible for the overall strategic planning and implementation of the University’s ICT infrastructure. He has led the establishment of an integrated e-Learning environment and has successfully facilitated its utilization by all Programs of Study at the University, for all aspects of teaching and learning. Currently, he is the coordinator of a €7 million ESF Program for the development of educational content for e-Learning, and a €4 million ERDF Program for the development of University ICT infrastructures.

xxiv

Notes on Contributors

Katerina Mavrou is an Assistant Professor in Inclusive Education and Assistive Technology at European University Cyprus. She holds a Ph.D. in Technology and Inclusive Education, (University of Birmingham), and M.Ed. in Special Needs and Development (University of Manchester) and a B.Ed. in Primary Education (University of Cyprus), as well as a professional certificate in Assistive Technology (AT) (CSUN). Her research interests focus on design of inclusive learning environments and the implementation of AT, ICT-AT learning, and ICT-AT for access to education and social inclusion. Her research record includes European commission reports, journal articles, book chapters in edited volumes, and presentations in international conferences. Maria Meletiou-Mavrotheris is a Professor at European University Cyprus and Director of the Research Laboratory in ICT-Enhanced Education (ICTEE). She has a Ph.D. in Mathematics Education (University of Texas at Austin), an M.Sc. in Statistics (UT Austin), an M.Sc. in Engineering (UT Austin), an M.A. in Open and Distance Learning (UK Open University), a B.A. in Mathematics (UT Austin), and Teacher’s Certification in Primary Education. She has engaged extensively in research and has established a respected record through numerous publications in scholarly journals and books, and securing of considerable funding in support of her research activities from agencies in Cyprus and the EU. Georgios Papaioannou is an Associate Professor in Museum Studies at University College London in Qatar. He directs and participates in research projects of cultural/museum works in the Mediterranean and the Arab world, where he has founded and organized museums and museum exhibitions. His research interests lie in applications of new technologies for cultural heritage, archeology and education. He is a Senior Fellow of the Higher Education Academy (UK), the Secretary General of the Hellenic Studies Society of Near East (HSNES), a member of the International Council of Museums (ICOM), and a member of the Pool of Experts of the European Museum Academy. Miroslava Pirháčová Lapšanská graduated from the Faculty of Humanities and Natural Sciences at Prešov University in Prešov, Slovakia. She worked as an Assistant Professor of Preschool and Elementary Education and Psychology at Prešov University and as a Researcher at Comenius University. She currently works as a Primary school teacher in Bratislava. Her scientific research interests are primarily focused on general didactic problems, undergraduate preparation of preservice teachers

Notes on Contributors

xxv

with the focus on development of their creativity and communication skills in educational context. She is the co-author of monograph, scholarly articles and university textbooks. Stavros Pitsikalis is a Ph.D. Candidate at University of the Aegean. He has 17 years of experience in VET education and he is currently attached to the Institute of Educational Policy in the Projects and Actions Department and the Scientific Units of Technical-Vocational Education and Educational Innovation. He has also been attached to the Hellenic Quality Assurance and Accreditation Agency in the IT-Department, the General Secretariat for Lifelong-Learning in the Vocational Training Department and the administration office of several vocational training institutes. He has been involved in many EU-funded/academic projects as a coordinator or research associate and has many publications. Corinne Raffin has been teaching in high school since 1991 and after her graduation in didactics of mathematics, she works at the French Institute of Education at the Ecole Normale Supérieure de Lyon. Her work focuses on the use of technology in the teaching and learning of mathematics. In particular, the contribution of technology in distance learning such as MOOC, SPOC and immersive pedagogy. Tomás Recio is a Professor of Algebra since 1981 at the University of Cantabria in Santander, Spain. He is the author of over one hundred seventy papers and four hundred fifty communications in different international journals and conferences. He has slmost fifty years of university teaching experience in a variety of Algebra, Geometry and Mathematics Education undergraduate and graduate courses. He is currently involved in the Secondary Education Math Teacher Initial Training Master degree at the University of Cantabria. He has been a Ph.D. advisor of over a dozen students. His former students hold now university positions in Algebra, Computer Science, Geometry or Mathematics Education. His research interests focus on: Real Algebraic Geometry, CAD, Robotics, Computer Algebra, Computational Geometry, Automated Reasoning, Dynamic Geometry, Mathematical Education. Further information at www.recio.tk Cristina Sabena is an Associate Professor at the Department of Philosophy and Science Education of the University of Torino, where she teaches Mathematics Education to future primary teachers. Her research focuses on the contribution of gestures

xxvi

Notes on Contributors

and multimodality in the development of mathematics thinking in the classroom, on formative assessment processes in mathematics and on argumentation. She is currently the Secretary of CIEAEM (The International Commission for the Study and Improvement of Mathematics Teaching). Florian Schacht is a Professor of mathematics education. His research interests include the role of digital technology for learning processes in mathematics including language phenomena and concept formation. After his Ph.D. in mathematics education and a period he taught mathematics and music at a secondary school, Florian worked at the TU Dortmund University for his post-doctoral studies. His Ph.D. was awarded in 2011 by the national society for mathematics education (Gesellschaft für Didaktik der Mathematik) as an “outstanding dissertation for the Young Researchers Promotion Award (GDM-Förderpreis).” Eva Severini received his Ph.D. from the Faculty of Education Comenius University in Bratislava. She currently works as a Lecturer in Pre-primary Education of the Faculty of Education at Comenius University, Bratislava. Through the implementation of action research, she examines the issue of didactic reality, especially the concept of active learning, and the interactivity of learning and teaching in the didactic process. Her research accomplishments include a considerable number of refereed articles and the textbook entitled Teaching and Digital Technology (2012). Martina Siposova is a Lecturer in English Language and Literature at Comenius University in Bratislava. Apart from teaching courses on Methodology of English language Teaching, her research interests focus on teaching English as a foreign language to all age groups as well as teacher training. Her research studies and scholarly articles aim at investigating learners’/teachers’ views on the learning and teaching process and teacher cognition. Zacharoula Smyrnaiou is an Assistant Professor of Science Education and Researcher in Educational Technology Lab in Athens. The past 10 years her work has focused (a) on the teaching of science using new information technologies, (b) on the educational design of digital media and accompanying pedagogical scenarios, (c) on implementing research in real classroom settings considering different factors, (d) on supporting teachers and students during the implementation of inno-

Notes on Contributors

xxvii

vative educational activities with digital media or art-based or work-based and (e) on teaching teacher educators. She is member of the committee of Science Curriculum Reform in Greece, chair of the Scientific Supervisory Board of 1st Experimental High School of Athens and president at three-member committees to evaluate science teachers. She has been involved in projects funded by E.U. (CREATIONS H2020-SEAC-2014-1 CSA, DESCI Erasmus+, PLAY4GUIDANCE-Erasmus+, FP7-Metafora, Erasmus Intensive Programme (Innovative practices for teaching science). Nayia Stylianidou has a Ph.D. in Special / Ιnclusive Education (European University Cyprus). Her research interests involve exploring students’ perceptions of disability, social constructions of disability through social networks, the utilisation of technology and Universal Design for Learning in Education. She holds a first degree in History – Archeology, (Aristotle University) and a postgraduate degree in Special Education (University of Nottingham). She works as a Special Scientist and Research Associate at European University Cyprus and she participates in European and other funded research projects. Osama Swidan is a mathematics education researcher at Ben-Gurion University of the Negev. He completed his Ph.D. degree in 2015 and did his post-doctoral research in the Hebrew University of Jerusalem. In 2017 Osama conducted in Turin, collaborating with Professor Ferdinando Arzarello, a research project that aimed to study the design of learning environments that boost inquiry-based learning. Osama’s research area covers learning and teaching mathematics with digital tools. Particularly, he is interested in studying cognitive and emotional aspects through learning mathematical concepts. He won several prestigious scholarships, such Mandel Institute scholarship for outstanding doctoral students in education, the scholarship of the Planning and Budget Committee for Higher Education in Israel for three years, and recently, he was granted the prestigious scholarship “MAOF” for outstanding young researchers. Christos Tiniakos has a Bachelor’s Degree in Primary Education and a Master’s Degree in Special – Inclusive Education, from the European University of Cyprus. He is currently working as a Special Education Teacher at the Makedonitissa’s B Primary School. He has worked at the European University Cyprus as a Research Assistant in EU-funded projects in the field of Special Education and as an Operator at the Committee for Students with Special Educational Needs. Additionally,

xxviii

Notes on Contributors

he has served as a Teacher-Assistant within the project “Actions for Scholar and Social Inclusion,” under the auspices of the Ministry of Education and Culture of Cyprus. Melanie Tomaschko is a Ph.D. student in Instructional Technology at Johannes Kepler University Linz, Austria and involved in the development of GeoGebra’s mathematics applications. Her research interest focuses on mobile learning, in particular the design and implementation of mobile applications that are used as learning tools. Renata Tothova is a Ph.D. Candidate at University of Žilina in Slovakia. Her research interests focus on modeling and simulation and she published journal articles about Object-in-fluid framework in modeling of blood flow in microfluidic channels, Cell Damage Index as Computational Indicator for Blood Cell Activation and Damage and Simulation study of rare cell trajectories and capture rate in periodic obstacle arrays. Christina Vasou has a B.Sc. in Computer Science, and an M.A in Technologies of Learning and Communication. Her research interests revolve around educational technologies, educational robotics, mobile learning, serious games and coding for young students. She is currently a Ph.D. Candidate at the European University Cyprus, focusing on current educational technologies such as mobile devices and robotics and their prospect in STEAM education. She is currently employed as an ICT teacher at American Academy Cyprus. Since 2015, she has also been working as a Research Assistant at the ICTEE Laboratory, participating in multiple in EU funded research projects. Ibolya Veress-Bágyi is a Ph.D. Candidate at Debrecen University in Hungary. Her Ph.D. research focuses on ‘Mobile Devices in Math Education.’ Her research interests focus on mobile learning, math mobile applications, learning with applications, Augmented Reality mobile apps and Augmented Reality math apps. Her future research plans focus on “math curriculum supported by digital content” including the integration of Augmented Reality technology in mathematics classrooms.

Introduction New digital technologies offer many exciting opportunities to educators who are looking to develop better teaching practices. When technologies are still new, however, ideas for their implementation often outstrip research on how to employ such technologies effectively. This book is intended to provide teachers and researchers with a wide range of ideas from researchers working to integrate the new technology of Augmented Reality into educational settings and processes. It is hoped that the research and theory presented here can support both teachers and researchers in future work with this exciting new technology.

1

What Is AR?

Augmented reality (AR) is a technology that overlays virtual objects (augmented components) onto digital representations of the real world. These virtual objects then appear to coexist in the same space as real-world objects. Augmented reality has the potential to transform how we interact with almost every industry today, and it will be equally transformative both from a consumer and an enterprise perspective. It’s already transforming sectors like real estate, healthcare and education. As the chapters of this book will show, AR is already being used with learners of all ages, from pre-school through medical school, and in many educational settings outside schools, as well.

2

What Is the Place for AR in Education?

When AR technology is used in educational settings, it 1. helps students to engage in authentic explorations in the real world 2. facilitates the observation of events that cannot easily be observed with the naked eye by displaying virtual elements alongside real objects. 3. increases students’ motivation and helps them to acquire better investigation skills 4. creates immersive hybrid learning environments that combine digital and physical objects, thereby facilitating the development of processing skills (e.g., critical thinking, problem solving, and communicating through interdependent collaborative exercises.)

xxx 3

Introduction

What Is the Purpose of This Book?

To date, research into the educational effects and implications of AR technology is in its early stages. Therefore, this book will discuss the advantages and challenges of AR technology used in educational settings using empirical data and will suggest ideas for effective pedagogical practices and areas in which to invest future research and development, so that this technology may be employed to its maximum capacity for supporting formal and informal learning as well as workplace training.

4

The Organization of the Book

The book begins with a section on AR in early education – pre-primary, primary, up to early secondary (we include a chapter on material aimed at 9–15 year olds). It then moves on to a section about AR in more advanced education, from high school onwards. The final section discusses a range of theories and applications that are more general, covering all age groups, and also that may belong outside formal educational settings, for example discussing museum use of AR, which provides informal educational opportunities for people of all ages.

5

Early Education

The section about Early Education begins with a chapter by Eva Severini, Blanka Kožík Lehotayová and Eva Csandová that focuses on possible educational uses of augmented reality to promote digital literacy development in primary school preparation. The goals of this chapter are to identify the impact and benefits of augmented reality on the development of children’s digital literacy in kindergarten. The research was conducted in a natural didactic context using direct and indirect observation of children in primary school preparation class during the usage of augmented reality within the educational activities and a focus group with teachers in the learning group. The authors interviewed the participants and analysed the obtained data using an open encoding method to identify teachers’ views and preconceptions about the augmented reality usage on their didactic practice. Their data provide insight in the impact and benefits of augmented reality usage on the digital literacy development of children. Chapter 2, by Kateřina Jančaříková and Eva Severini, focuses on the potential impact of augmented reality on the development of scientific literacy in

Introduction

xxxi

pre-primary students as well as on the teachers’ professional development. The chapter discusses the opportunities that augmented reality brings to the development of scientific literacy and gives an overview of selected applications suitable for preschool children. The authors, focus on two applications, namely, the 4D animal and Quiver Vision that were used in a research project involving two Czech and two Slovak kindergartens as well as preschool children’s parents. The research showed that many teachers were not familiar with the technology of augmented reality, and some teachers expressed worries about how to employ augmented reality in preschool. Additionally, the teachers shared creative ideas on how these technologies can be included in the offered activities. Chapter 3, by Maria Meletiou-Mavrotheris, Constadina Charalambous, Katerina Mavrou, Christos Dimopoulos, Panayiota Anastasi, Ilona-Elefteryja Lasica, Nayia Stylianidou, and Christina Vasou discusses The Living Book – Augmenting Reading for Life (Erasmus+) project. The project aims to: (1) address the under-achievement of European students in reading by developing an innovative approach that combines offline activities that promote reading literacy with online experiences of books’ ‘virtual augmentation’; and (2) strengthen teachers’ competences when adopting the Living Book approach and dealing with diverse groups of learners. The chapter outlines the theoretical premises of Living Book and provides an overview of the theoretical framework underlying the design of the ‘Augmented Teacher’ professional development course and the content and structure of the course. The fourth chapter, by Eva Csandová, Renata Tothova and Lilla Korenova, focuses on the possibilities of educational uses of augmented reality in Slovak primary school. This chapter identifies the impact of the use of augmented reality in education activities, including benefits on the children’s digital literacy development, as well as emerging problems or difficulties for children while using augmented reality applications. The main finding of this chapter was an increase of children’s inner motivation for learning that subsequently and in close liaison increases also the inner motivation of teachers for further professional development and self-education. Chapter 5, by Lilla Korenova, Zsolt Lavicza and Ibolya Veress-Bágyi, discusses the possibilities of using mobile apps with augmented reality in pre-primary education. The authors introduce some of the most popular and widely used applications for kindergarten-age students that use the camera of mobile phones or tablets to explore 3D objects. They also discuss how these applications offer an immersive experience and edutainment and the alpha generation classrooms could be enhanced and transformed by the implementation of AR technologies.

xxxii 6

Introduction

Advanced Education

The sixth chapter by Gilles Aldon and Corinne Raffin, presents a research study conducted in a dual space comprising a secondary school, and a 3D, virtualworld analogue of the school in which students and teachers act through evolving avatars. It describes a research study that examines an augmented reality secondary school, especially its educational uses and conditions necessary for its successful development and adoption. It assesses the pedagogic contributions of an immersive space, and the development of new pedagogical strategies. The authors also discuss how the main part of this evaluation focuses on the impact of this tool on the pupils’ working methods, on their acquisition of competences and knowledge, and on their relationships with mathematical objects through augmented reality embedded in the virtual world. Chapter 7, by Osama Swidan, Florian Schacht, Cristina Sabena, Michael Fried, Jihad El-Sana and Ferdinando Arzarello, considers the application of augmented reality technology for developing covariational reasoning with an aim to show general potentialities of AR technology and give some hint of the theoretical principles behind them. The chapter discusses the first designcycle from a design-based research study that aims at both the development of an AR toolkit within iterative research cycles, and at its scientific investigation. The main claim of the authors is that engaging students in coordinating continuous real phenomenon (e.g., a moving object along inclined plane) with its mathematical representations (e.g., plotting points of graph, ordered pairs in a table of values) through visual-kinesthetic activities, may help the students to experience the multiple levels of sophistication and develop multiple meanings of the covariational reasoning. Insights from the first design cycle are presented and ideas for next design-cycles are also discussed. Chapter 8, by Mária Fuchsová, Miriam Adamková and Miroslava Pirháčová Lapšanská, discusses the use of AR in the context of Biology education as one of the STEM areas. It focuses on investigating pre-service primary teachers’ perceptions when integrating AR technology in Biology classrooms and on the elements, that influence the use of AR in educational processes and teaching practice at the undergraduate level. The authors recommend future studies that would stress the socio-cultural aspect of using AR and several already existing Biologythemed AR applications that seem to be interesting, enhancing and beneficial in the context of Biology education at different levels. Chapter 9, by Martina Siposova and Tomas Hlava, discusses the integration of augmented reality (AR) in teaching and learning for tertiary education. It analyses selected and relevant research studies published in the Social Science Citation Index (SSCI) journal database, Education Resources Information

Introduction

xxxiii

Centre (ERIC) journal database and Open Access Journals on AR in tertiary education. The authors comment that it seems that the range of scholarly articles and studies aimed at examining the AR potential at university level education does not fully cover the variety of academic fields. Chapter 10, by Utku Köse and Omer Deperlioglu introduces an Augmented Reality-based platform for improving efficiency and effectiveness of medical training by providing simulation-based solutions that enable students to interact with real-world objects to perform their tasks on medical diagnosis or operations based tasks that would help them to understand more about some diseases. The authors describe the platform developed that encompasses augmented reality and Artificial Intelligence to support an intelligent diagnosis function, using signal processing or Machine Learning infrastructure based on widely used Artificial Neural Networks. They also discuss the evaluation of the platform’s accuracy on diagnosis and effectiveness for training processes and positive results were achieved according to the obtained findings. The eleventh chapter, by Martina Babinská, Monika Dillingerová and Lilla Korenova, focuses on the implementation of the Augmented reality application named ‘Augmented Polyhedrons – Mirage 2.2’ (APM) into the teachertraining programme. The chapter discusses the results of the conducted research that supports the suitability of the selected application and explains how the researchers create the implementation method of the APM application for teachers’ training programme mathematical courses. The last part of the chapter offers a view of secondary school mathematical problems which could be supported in teaching by AR applications. The authors associate mathematical problems with specific existing applications, and suggest new AR applications that could possibly be developed and integrated in professional development programs.

7

All Ages and outside the School

Chapter 12, by Ilona-Elefteryja Lasica, Maria Meletiou-Mavrotheris, Efstathios Mavrotheris, Stavros Pitsikalis, Konstantinos Katzis, Christos Dimopoulos and Christos Tiniakos, introduces the project named Enlivened Laboratories in Science, Technology, Engineering and Mathematics (EL-STEM), which aims to integrate these technologies into school laboratories, for attracting secondary school students who might not be interested in STEM-related studies/careers, enhancing the interest of those who have already chosen this field of studies/careers, encouraging student STEM engagement and improving student performance in STEMrelated courses. The chapter provides an overview of the EL-STEM project and describes use cases of Augmented Reality in STEM education.

xxxiv

Introduction

Chapter 13, by Chronis Kynigos, Zacharoula Smyrnaiou, and Marianthi Grizioti, discusses pedagogical design principles of augmented reality games, including young peoples’ invocation of meanings and intuitions as they engage in collaborative play. The chapter describes a research study that shows how students engaged in physical movement as part of game play, and then generated scientific language to explain their strategies. It also discusses the potential of designing games for students to purposefully question their intuitions. The fourteenth chapter, by Melanie Tomaschko and Markus Hohenwarter, presents possibilities for exploiting the potential of augmented reality in learning and teaching mathematics. It elaborates on the novel mobile application GeoGebra Augmented Reality (AR) for iPhones and iPads and describes how GeoGebra AR allows exploration of 3D math objects virtually placed in learners’ environments, while they can walk around them and observe them from different perspectives. Moreover, this chapter discusses guided activities that allow students of all ages to discover math in the real world and suggestions for possible future development of the GeoGebra AR app. Chapter 15, by Francisco Botana, Zoltán Kovács, Álvaro Martínez-Sevilla and Tomás Recio, describes a scenario in which automated reasoning and augmented reality are merged for educational purposes. It further explains the idea of Automatically Augmented Reality (AAR) and its potential application to the development of mathematical competencies for real life and for the didactical utilization of Mathematical Walks, a pedagogical activity that uses GeoGebra as an auxiliary tool. Once the visual representation of reality has been turned into a precise GeoGebra input, it discusses the automatic discovery of the existing mathematics lying behind this geometric version of reality by automatically and systematically deploying GeoGebra reasoning tools over the input, the implementation of the Automated Geometer module in GeoGebra and the educational changes, advantages and dangers that could follow the implementation and popularization of an AAR scenario. Chapter 16, by Georgios Papaioannou, reviews augmented reality applications for cultural heritage settings, particularly museums. It presents the history and the evolution of augmented reality programmes and applications in museums and cultural heritage institutions. It further examines examples of good and poor practices and discusses issues related to the use of augmented reality in museum environments and within museum educational programmes. The seventeenth chapter, by Lilla Korenova, Maria Kožuchová, Jiří Dostál and Zsolt Lavicza, highlights the possibilities of using extended and augmented reality applications in technical education for the preparation of future teachers who teach pupils aged 6–14 in the Czech Republic and Slovakia.

Introduction

xxxv

Chapter 18, by Robert Bohdal, discusses the most commonly used VR/AR devices and how they work so that a reader without technical education can understand their functionality and at the same time get an overview of the many devices that are used. We hope that an audience of professionals and researchers working in the field of Education can gain insights to support educators, practitioners, undergraduate and postgraduate students and curriculum developers who concerned with integrating integrate augmented reality in educational settings (Primary Education, Secondary Education, Tertiary Education and training in a workplace) and prepare citizens to be ready to work and meet the new workforce needs of the 21st century.

PART 1 Early Education

∵

CHAPTER 1

Uses of Augmented Reality in Pre-Primary Education Eva Severini, Blanka Kožík Lehotayová and Eva Csandová

Abstract This chapter is focused on possible educational uses of augmented reality as a support for digital literacy development in primary school. Development of digital literacy, a competence necessary for children to achieve learning goals by themselves, should be a natural part of education practice. Our goals were, through qualitative methodology, to identify impacts and benefits of augmented reality usage on the development of children’s digital literacy in kindergarten. In this action research conducted in a natural didactic context, we used direct and indirect observation of children in primary school preparation class as they used augmented reality in educational activities, as well as a focus group with the teachers in the learning group. For the interview data analysis, we used the open encoding method. We identified teachers’ views/preconceptions about augmented reality usage in their didactic practice. We relied on a strategy of validity and reliability, in which the collection of research data and its analysis must go together, in the form of mutual interaction between what is already known and what yet needs to be known. We used the constant comparison method to analyse the classroom observation data.

Keywords augmented reality – digital literacy – pre-primary education – qualitative methodology

1

Theoretical Paradigm of Addressed Issues

In today’s post-modern era, under the influence of digital technologies, our planet becomes increasingly “smaller,” bringing together peoples, states, and their culture, creating a cosmopolitan way of life in educated society, and thus © koninklijke brill nv, leideN, 2020 | DOI: 10.1163/9789004408845_001

4

Severini et al.

cosmopolitan citizens are formed. Digital technology represents an important and indispensable part of the learning process for teachers and learners in this world. It is characterized by the fact that the child/pupil learns globally, and every day gains new experience, knowledge, values and attitudes, including through digital technologies that help his/her development. For this reason, we place the digital competences among the competences that need to be developed in pre-primary education. Competences are constructs that define the intrinsic potential of entities, have a complex composition including cognitive structures, motivation, attitudes, and behaviours. Competences express the state of the individual to solve their own participation as a member of the social community. The complex of cognitive structures, motivations, attitudes, evaluations, and behaviours express the character and concrete content of this complex composition and express the identity of competence that differentiates each other’s competences. It follows that every competence has its cognitive-conceptual “form” based on its character and content. Competence is a compact unit of specific capability. The individual parts of the competence can be characterized only theoretically, as competence is a complex which manifests as an entity in the actions of the individual. Child competences are compact units that are expressed by how learners master the subject, process, content, goals, and so forth of education in relation to a particular school context. We might observe diversity in this complexity; we can have a character of micro-competence or meta-competence. Competences can regroup and create clusters according to actual use. Some competences can be incorporated among other competences. They vary and are dynamically changing and developing (are constructed and reconstructed by the expansion of children’s thinking in the context of their activities) and may intertwine in the cognitive part of the competence – a cognitive part of the structure of one competence can “overlap” a cognitive structure of other competences (Kostrub & Kikušová, 2012). Acquiring and improving competences is seen as a lifelong learning process, not only in family, cultural, and social life, but also at school. The main objectives nowadays include identifying and defining new transferable competences as well as considering whether they can be integrated into curriculums as they can be maintained and taught throughout life. At present, digital competence has its very own place in an individual’s development. Digital competence includes self-confident and critical use of digital technologies for learning and communicating. At its most basic level, this ability includes the use of multimedia technologies for gathering, evaluating, creating, presenting, communicating, and exchanging information. Therefore, the

Uses of Augmented Reality in Pre-Primary Education

5

digital competences development in the teaching process is important and participates in the development of logical and critical thinking, acquisition of high-level knowledge and development of communication and information capacities for both the teaching and the learning individuals (Blaško, 2010). Digital technologies have become part of the lives of individuals and society. In the context of education, they are a set of resources, tools, environments, and practices that are used to teach, communicate, collaborate, present, create, etc. in the learning process. An individual using these digital systems can be referred to as a digital citizen. Digital literacy is, in general, a set of competency-transferring capabilities for the adequate, safe, and productive use of digital technology for teaching and learning. According to Kalaš (2011, p. 130), it is a set of competences relating to: – uses of digital resources for personal needs, learning, presentation, and complex development, – effective solution of problem situations in the digital environment, – decision-making, and uses of appropriate digital technologies to obtain information, and to process, use, and present them, – critical evaluation and analysis of knowledge gained through digital resources, – understanding the socio-cultural consequences (security, privacy, and ethics) that are caused by the usage of digital technologies in the digital world. Individuals should also understand how information society technologies can promote creativity and innovation and should be aware of issues related to the validity and reliability of available information as well as the legal and ethical principles of the interactive use of technologies in information society. They should be able to use tools to create, present, and understand complex information, and access, search, and use Internet-based services. The use of digital technologies requires a critical and reflective attitude towards available information and the responsible, safe use of interactive media. This competence is also developed by the interest in participation in (socially shared) communities and Internet/Intranet networks used for cultural, social, and/or professional purposes. Basically, digital competence can be considered as an escalating two-cycle ability. At the first – basic – level, it is an ability that (together with its attitudes, values, and knowledge) enables individuals to use digital technologies to acquire, focus, sort, evaluate, store, create, protect, and exchange information, and communicate and participate on the Internet. The second stage is characterized by an independent, conscious, and critical use of digital technologies, digital content, and digital media. This degree is characterized by a

6

Severini et al.

critical, rational thinking about digital technologies usage at a higher level and at an identically developed level of communication competences (Pavlovkin & Ďuriš, 2012). New trends in pedagogy with current educational requirements for children in the 21st century no longer rely on the dominant position of the teacher as the primary provider of information who transfers knowledge to a passive audience, a mode that is imposed on teachers who do not have enough room or resources at school for active participatory learning exercises (Koreňová, 2017; Kostrub, 2010; Kostrub & Kikušová, 2012).

2

Research Paradigm of Addressed Issues

Practical exploration – action research in education is a research type in which teachers systematically explore their own didactic practice with the goal of improving it. It is usually done by teachers directly within their didactic practice where they are part of the learning process, often in collaboration with researchers. Action research should be part of the teaching process of each teacher who wants to fulfil the quality of a teaching system and develop teaching competences (Severini & Kostrub, 2018). The implementation of action research is consistent with the statement by Pavlov (2002, p. 67) that: “if a certain conceptual change is to about occur, these profound layers of the teacher’s professional structure must intervene to realize: the existence of one’s own concept (content, boundaries, etc.), the need and the nature of the expected changes, the path of change – real activities that enrich or partially reconstruct elements of the original individual concept.” Action research, as stated by Kemmis and McTaggart (1982), is “target-oriented research conducted by a group of researchers in the learning process.” It is characterized by spiral cycles of problem identification, systematic data collection, reflection, procedures that are managed based on acquired data and redefinition. The goal of action research, as part of the teacher’s activity, is to obtain practical results that can be used immediately in the didactic situation, without excluding their contribution to the development of wider generalizations or, at a higher degree, for the development of new concepts and scientific theories. Promulgating research findings is generally done through exchanges between teachers within professional associations at school, continuing education institutions, and through higher education institutions. Methodology of the action research process is, according to J. Elliott (1997, In: Hernández, 2007, p. 17), represented by cyclical form, in a spiral or a circle. For example, it can be the implementation of action research with the intention of transforming the applied teaching model within the school educational

Uses of Augmented Reality in Pre-Primary Education

7

context (so-called teacher research) introducing a significant change in the teaching of individual teachers. It is a cyclical process of research that consists of several phases: 1. Teaching process diagnosis 2. Intervention Design (Action Plan) 3. Intervention for the evidence collection 4. Analysing and Interpreting Evidence 5. Illustration of change in the teaching process (emerging theoretical model) These phases are included through reflection – action – reflection (as well as action – reflection – review). The decision for this type of research has come from the awareness that the studied phenomena are perceived by a human perspective, which influences the choice of methodology and influences the overall research character. In this study, teachers of pre-primary education were the research participants and their views on the research subject were the subjects of interest, which was also closely linked to other participants (children). So, the types of data that will be gathered and analysed as part of the action research program were opinions, beliefs, thoughts, etc. about uses of augmented reality. The focus of our interest was children and teachers in pre-primary education. In research with the application of augmented reality in teaching, we used tablets/handheld displays. For the project, tablets required a scanning and display device (a camera to scan the worksheet area and a display to present the image), marker-based tracking technology, and rendering technology software that enables the implementation of digital elements in the display environment. In our research, we used commonly used printed AR markers. Tablets have a fairly high potential, and we consider them to be an appropriate device when using augmented reality in a kindergarten environment. They work with several operating systems (Android, iOS, Windows Phone 7, Bada, etc.) that have app stores (Google Play, Apple Store, Market Place, Samsung Apps, etc.) that allow users to download augmented reality apps. In our research, we used QuiverVision, which is relatively simple, and therefore has the potential of teaching in kindergartens. In education, apps can be used in form of worksheets or books that can show children a preview of a particular picture (video, audio) in a 3D form. The research objective for this project was to identify the impact and benefits of augmented reality on the children’s digital literacy development. To pursue this question, we asked the following research questions: 1. What is the best process of children’s digital literacy development using augmented reality in pre-primary education in a comprehensive, meaningful, and deliberate way?

8

Severini et al.

2.

Does children’s digital literacy develop through using augmented reality in pre-primary education, and if so, how?

3

Our Research Tools and Methods

For data acquisition, we chose a focus group as one of our research tools, for a number of methodological reasons – the acquisition of the opinions, convictions, explanations, and knowledge of participants involved in research of the children’s digital literacy development. The focus group, as a discussion group, represents in miniature the thinking of a specific group in society on a specific subject. It is not a natural group, instead it represents a category of participants who have something to say about the specific issue. They are informatively and experientially “rich,” because of their strong relationship to the issue. The focus group (also called group interview, focus interview or in-depth group interview) focuses primarily on communication between the participants and the gradual convergence of their views (Plichtová, 2002). All of the focus group participants in our study were kindergarten teachers – graduates of 1st or 2nd degree in the Pre-primary education study programme. The second method of obtaining data by micro-teaching (in the framework of realized action research) was unstructured observation of children in selected kindergarten classes. Micro-teaching is a “teaching method based on educational theories, scientific method and digital technologies, more accurately, it is actual learning activity in terms of the number of subjects in the process of teaching, time and learning, on the basis of which pedagogical competences of the teacher are developed under simultaneous audiovisual recording for feedback purpose” (M. Zelina, In: Švec, 1998). The goal of unstructured observation (without a pre-prepared scheme) was the course of the teaching process, in which the teacher began to consciously support his/ her own learning about how to develop the children’s digital literacy through augmented reality. The learning group members involved in the research were familiar with each other (direct, immediate communication between them), respecting a certain value system, and a system of rules regulating their behaviour on issues important to the group – adhering to the appropriate climate of acceptance and learning. The observations were taken in kindergarten. There were five learning groups in which 10 observations were performed over a total of 15 hours. The number of children examined in each group was in line with state legislation (in each learning group 23 children; in total, 115 children).

Uses of Augmented Reality in Pre-Primary Education

4

9

Realization of Our Research

4.1 Focus Group The collection of data was carried out in an action research focus group with teachers of pre-primary education at kindergartens in Bratislava. The research study was carried out in 2017, attended by 10 participants. The conditions and the atmosphere of the focus group were adequate and in line with the requirements for this form of study in the sense of the relevant methodological recommendations (Kostrub, 2016; Miovský, 2006; Plichtová, 2002, and others). By using a focus group, it was expected that participants would openly express their opinions and explain and clarify their views. The focus group initiated discussion in which mutual learning and mutual influence should be (and were) present. The result is to be a (mutually induced) change and the presence of a mutual opinion that is more complex than the original opinions of individuals before the focus group realization (initial representation). This is a process of discussion of individual representations and socio-construction of social representations. “Participation in the interview means participation in the creation of meaning,” states Plichtová (2002, p. 195). Interviews were analysed from a social constructivist viewpoint as interactive episodes in which common meaning is formed. “Only in a free, unstructured interview can we understand how the interviewers interpreted the question asked and from what perspective they approached it” (Plichtová, 2002, p. 206). In our study, the researcher was a moderator, an active participant of the focus group and actively participated in the exchange of views. He also took on the role of the “rapporteur” when he carefully monitored and compiled what had been said and stated. He asked for clarification from the speakers, when necessary, asking additional questions. Later, the researcher/moderator analysed the transcripts and notes from the focus group. At the beginning of the focus group, participants were invited to present their ideas in the form of answers to the question: How to optimally support children’s digital literacy development through augmented reality? Augmented reality was a key construct and the focus. Participants were given a reasonable time (determined according to their requirements) to record their thoughts and opinions on sticky notes. These sticky notes were stuck on a flip chart board next to the question. Then the participants were given a break. During the break, they organized the individual statements into categories for discussion in the next section. The operational categories were identified by reading and arranging sticky notes expressing similar or identical opinions. After a pause, a friendly and open discussion was developed for the participants and

10

Severini et al.

the moderator to present and clarify their opinions and disagreements. The participants had the opportunity to interact with each other, to learn together without realizing this fact. Audio recordings were made of the sessions. Participants were asked to provide consent to audio recording in advance of the focus group. Discussion was held as long as the participants provided relevant research material, which was within two hours. The audio recording from the focus group was transcribed and the transcript was subjected to the open coding procedure, which allowed individual categories to be identified. In the content analysis, a large amount of unstructured data was processed in a non-invasive way – a descriptive encoding of the text units. The complete statements (thoughts) of the focus group members were sorted into text units. This captured the most striking interpretations, repetitive or complementary ideas, and convictions, and identified key dimensions of the discussed key concept. The units of analysis in the open encoding were sentences and paragraphs, with a focus on ascertaining the main idea expressed by each unit. This was indicated by the appropriate code.

5

Observation

The realization of observation (within the micro-teaching in learning groups in kindergarten) identified the presence of the quality of augmented reality application in the teaching process. Micro-teaching was performed in the afternoon with a time span of one week. We observed the behaviour of children in the process of learning, and we looked for phenomena that showed evidence of digital literacy development support through augmented reality. A total of 5 observations were made. Each of the groups was a separate entity which was, to a certain extent, independent. Direct observation took place in a kindergarten classroom where some video footage was collected in the course of teaching, but only some of the students allowed observation of the phenomenon that was tracked, and which were subject to indirect observation. Data from videos were transcribed to written form – literal transcription of the learning process (see Protocol No. 1, below).

6

Elaboration of Research Material

The following methods were used for the research data analysis: – Open coding method – the process of parsing, exploring, conceptualizing, and categorizing research data (Strauss & Corbinová, 1999). The text as a

Uses of Augmented Reality in Pre-Primary Education

11

sequence has been parsed into units to which conceptual markings of phenomena have been assigned. In process of terming statements, we applied the coding questions asking “Who? When? Where? What? How? Why? with subsequent comparing – categorization, conceptualization and data encoding” (Kostrub, 2016; Strauss & Corbinová, 1999; Švaříček & Šeďová, 2007). – Constant comparison method (CCM) – continuous theory generation process. In its realization, it was not our intention to verify the universality of observational data. The CCM procedure ensured the saturation of the desired information in terms of its size and saturation in the context. (Tománková & Kostrub, 2018). – Selective Encoding Method – the central category selection process that has been systematically referred to in relation to other categories. The central category was a central phenomenon around which all other categories were integrated – the grounded theory (Strauss & Corbinová, 1999; Švaříček & Šeďová, 2007). In the report of statements/interpretations, we got the main ideas expressed by specific sentences (concepts) that were categorized. We identified the kindergarten teachers’ perspective of the possibilities of supporting the children’s digital literacy development through augmented reality. When analysing the research participants’ statements in the focus group, all their statements were included and examined for answers to the research questions. Conceptual terms have been assigned to individual events, cases, and other occurrences of the phenomenon. The results of the open encoding are presented in Table 1.1. During the research, several research materials were obtained in form of video and still images. In indirect observation, we repeatedly played back videos of learning activities; we were looking for significant elements to support the children’s digital literacy development through augmented reality. Elaboration of the typology was carried out as a phase of qualitative analysis of realized qualitative research (action research). As Woods (1987, p. 160) states, “This is a special form of description that requires purification from a great deal of material, analyses and massive descriptions. These are forms of descriptions (including the reference frameworks), that can by themselves help to understand the spheres of some social activity. “ When analysing the findings obtained through direct and indirect observation, a reference frame was developed. The registration of the research result was evaluated on the basis of the reference framework with the evaluation commentary used to interpret the research results. As stated by M. Zelina (2006), the basic unit for a microanalysis is the teacher’s input and the reaction of the child. Activity analysis (micro-teaching analysis) was a stage in micro-teaching where the teacher’s and children’s behaviour was analysed and identified. A basic approach to analysing teaching processes and

12

Severini et al.

table 1.1  Identifijied categories and concepts of research participants’ statements from focus group

Interpretative Concepts categories Teacher’s role – – – – – – – – – – – – – – – Teacher’s performance – – – – – – – – – –

Teacher as an adviser. Teacher – listener. Teacher as an organizer. Spontaneous teacher. Teacher as a playmate. Teacher as an assistant. Teacher as a consultant. Teacher as an observer. Contemplating teacher. Reflecting teacher. Teacher, who trust. Thinking teacher. Teacher as equal partner. Teacher as a supporter. The teacher diagnoses the child and the group. Teacher specifically (with aim) supports the child in the learning process. The teacher does not specify goals and activities but co-creates (together with others) situations. The teacher creates space in different situations, so the child can solve problems independently. The teacher realizes discussions with children. Teacher communicates and cooperates with parents. The teacher offers a rich diapason of information. Teacher chooses the right strategy. The teacher plans and offers children adequate activities, content and resources. Teacher supports children through another child who is in role of assistant. Teacher discusses with children about their success and failure in the realization of various activities (supports the child‘s healthy self-confidence). (cont.)

Uses of Augmented Reality in Pre-Primary Education

13

table 1.1  Identifijied categories and concepts of research participants’ statements from focus group (cont.)

Interpretative Concepts categories

Condition

– The teacher directs children to evaluate activity to be able to tell the reflection on the activity and to express self-reflection in the final discussion. – Teacher encourages children by asking them questions developing critical thinking. – Teachers applies experiential learning, active learning by practice. – The teacher is supposed to support the child in decision-making and evaluation. – Teacher is supportive of collaborating (cooperating) with children. – The teacher must start every activity with conversation. – The importance of spiralling knowledge of a child. – The child must have a choice of playing activities and other activities. – The teacher should respect the individuality of each child. – The teacher should be able to coordinate the activities that children realize. – The teacher should allow the child to experience their own success and failure. – The teacher should respect the environment in which the child is located. – The teacher must contemplate and identify what the child can do. – Children, along with the teacher, should create rules for playing activities, activities, and so on. – The teacher must make a good quality plan of potential development zones for each child. – The teacher must provide the opportunity for children to experience something, so that children can fixate a new knowledge based on his/her own experience. – The teacher should ask such some questions based on which children acquire new knowledge in an area that is current and interesting to them. (cont.)

14

Severini et al.

table 1.1  Identifijied categories and concepts of research participants’ statements from focus group (cont.)

Interpretative Concepts categories Children’s performance

Added value

– – – – – – – – – – –

The child decides by him/herself. The child has its own solutions. The child leads (directs) the teacher. Child receives information in various ways. The child solves problems independently (on its own). The child acquires new knowledge. The child improves potential and abilities. The child can spontaneously find out how to do something. The child learns him/herself from adults and other children. The child by being in a social group is developing. The child is the organizer of the game or activity, the child is acting alone. – Teacher by his/her individuality realizes teaching (model of hidden didactics).

their effectiveness was used – a second approach, according to M. Zelina (In: Švec, 1998, p. 267), which is based on observation and monitoring of educational processes of teaching. In the framework of micro-analysis, we studied specific micro facts and deduced and considered the character of support for the children’s digital literacy development through augmented reality. Microteaching analysis provides facts, not impressions, not just opinions and attitudes were registered in the actual conduct of entities in the learning process. Based on the microanalysis of educational teaching, as stated by M. Zelina (in Švec, 1998, pp. 266–267), “Is an interactive fact that has the following attributes: interaction is recorded on media (video); the captured interaction is, therefore, fixed and allows returning to it to subject it to analysis, assessment, and interpretation; the captured and fixed interaction fact can be analysed by more accurate methods compared to classical observation, using mainly qualitative methods; chronological record of interactions allows to track the sequences, procedures of the educational process of teaching.” In the analysis method, we considered events as indicators of the phenomena we gave conceptual designation. Systematic reflection of professional situations, study of pedagogical reality and initial diagnostics based on children’s

– The evocation phase – the teacher invited the children to verbally present their knowledge about the means of transport. Each child provided enough space to create concepts related to the concept of means of transport, leading a discussion so that each child’s preconceptions would come into conflict with new information or a reality with the intent to incite a cognitive conflict. The teacher supported internal motivation of children by discussing proposals for theme realization. The suggestions made by children were written by teacher on the board. At the same time, the teacher diagnosed the children’s concepts on the means of transport (categorized them) – thematic categorization (initial assessment) (Figure 1.1). Children created a conceptual map of their own original concepts, stating their (own) opinion. The teacher supported the cooperation of children by creating the conditions for their realization – it is a pre-active phase of teaching.

Didactic situation description in teaching activity

(cont.)

Children, together with the teacher, created the playing rules. The teacher provided opportunities for children to experience new knowledge based on their own experience. The children acted and decided independently. The teacher’s opinion was not decisive for the children’s behaviour or actions. Children spontaneously developed the play; were separate because their actions contained their own solutions.

figure 1.1 Thematic categorization/initial evaluation

The presence of children’s digital literacy development as part of augmented reality teaching

table 1.2  Protocol No. 1 Activity analysis (micro-teaching analysis) obtained through direct and indirect observation study of pedagogical reality and diagnostics based on children’s evidence in the learning/playing group (extract)

Uses of Augmented Reality in Pre-Primary Education

15

– The awareness of meaning phase – children developed the theme in concrete forms. They could spontaneously find out how to do something. The teacher developed children’s digital literacy through an individual approach using tablets and worksheets with a printed marker. The marker, in the form of the picture, was scanned by children themselves. Also, the type of questions that the teacher asked of children helped them to gain new knowledge in an area that was current and interesting to them (thematic conceptualization) (Figure 1.2). The children themselves came to the fact that the tablets had to be kept at a certain distance and direction with respect to the worksheet to see the 3D object displayed above the marker. On the tablet’s display, children saw a virtual object instead of a marker. Specifically, it was an easily recognizable and generable image on the coloured worksheets about transport. They tracked their play goals and, in co-operation with the teacher, proceeded to process and resolve the learning problem. The children carried out the following activities: they proposed, discussed, etc. – thematic implementation/ ongoing evaluation – an interactive learning phase.

Didactic situation description in teaching activity

(cont.)

Children were able to ask provocative questions for constructive solutions because their own transformation requires functional zones and breaking point (socio-cognitive conflict) and optimally secure safe space and time. They had been asking questions, were thinking and practicing the activities by themselves. Children had room for selfrealization and play development. Children separately solved problems, for example, if the 3D object did not appear. Children handled the tablets in many diffferent ways and they could observe the subject from

figure 1.2 Thematic conceptualization/ongoing evaluation

The presence of children’s digital literacy development as part of augmented reality teaching

table 1.2  Protocol No. 1 Activity analysis (micro-teaching analysis) obtained through direct and indirect observation study of pedagogical reality and diagnostics based on children’s evidence in the learning/playing group (extract) (cont)

16 Severini et al.

all sides because space allowed them to be free. The children carried out the following activities: they proposed, discussed, provided alternatives, approved, voted on the resulting product.

The presence of children’s digital literacy development as part of augmented reality teaching

– The reflection phase – in the final phase of the teaching activity, children together with the teacher made a feedback on the established learning problem by creating a partial product (output evaluation) (Figure 1.3). Children’s reflection (self-evaluation and evaluation) has been related to their digital literacy development through augmented reality. Children decided on their own, they only partly needed a teacher. The teacher’s assessment was related to the partial goals set in the learning activity. The teacher respected the individuality of each child and encouraged them in decision-making and evaluation. At the same time, they created enough time and space to carry out the activity. In the activity, emphasis was figure 1.3 Thematic realization/output evaluation placed on mutual communication and exchange of information – a post-active phase of teaching. Children depicted a link between the terms. They created a conceptual map by themselves. In their games, they turned to the teacher to give them specifijic help. The teacher encouraged children by questions developing their critical thinking. Children were able to fijind solutions. The game was partially spontaneous.

Didactic situation description in teaching activity

table 1.2  Protocol No. 1 Activity analysis (micro-teaching analysis) obtained through direct and indirect observation study of pedagogical reality and diagnostics based on children’s evidence in the learning/playing group (extract) (cont)

Uses of Augmented Reality in Pre-Primary Education

17

18

Severini et al.

evidence in the learning/playing group were preceded by an examination (indirect observation). Implementation of ongoing diagnosis based on children’s evidence in the learning/playing group took place during the implementation of the teaching process, and the outcomes’ diagnosis based on children’s evidence in the learning/playing group took place within a pedagogical reflection on the realized teaching activities in the field of micro-teaching which are presented in Protocol No. 1. Constant comparison was realized as a phase of qualitative analysis of realized action research. When applying the constant comparison method, as reported by Kolb (2012, pp. 83–86), “it is not the intention to verify universality, nor to confirm the causes or other input data. This procedure ensures the saturation of the necessary information. The goal of using a constant comparison method is not to verify but to generate a theory. The size and saturation of the information in the text is monitored until the saturation level is reached.” Glaser and Strauss (1967, p. 105, In: Kolb, 2012, p. 83) state that the constant comparison method has the following levels: a comparison of events usable for a certain category; the integration of categories and their characteristics; definition of theory; and designing (writing) theory. The selective coding was performed within the framework of a qualitative examination as the final phase of the qualitative analysis process (the process of constructing and validating categories). In our research, the central category was abstracted as the children’s digital literacy development through augmented reality. It is the significance of the interaction (teacher-children) and its justification that has been the subject of research activities. When a teacher teaches children various play scenarios with augmented reality (causal conditions) to implement different activities (didactic phenomenon) and allows them to decide independently which activities to perform in the playing process (strategy) and what strategies they choose and apply (intervention conditions), the children provide several suggestions for the playing design (design), select strategies with the possibility to apply them, come up with ideas for their use in playing activities (consequence), and carry out self-evaluation and evaluation of the playing realization (evaluation). The essence and meaning of the entire teaching process is to discuss with the intention of reaching a consensus. The content of the graph (Figure 1.4) shows that within selective coding two main levels of intentionally supporting digital literacy development through augmented reality have been identified. The first level is direct support for the children’s digital literacy development through augmented reality, where the teacher supports digital literacy by communicating, designing, implementing jointly (with children) constructed rules, exploring different learning

Uses of Augmented Reality in Pre-Primary Education

19

figure 1.4 Graphic representation of selective coding results

problems, and exposing children to different situations (with intellectual challenges to their actions). By children’s actions we mean independent communication, decision making, designing, planning, exploration, discovery, cognition, and acceptance and prediction of a role, with reference to the personal (reasonable) responsibility of the child. The second level is indirect support where the teacher in the play process promotes the children’s digital literacy development through augmented reality by creating appropriate space, providing appropriate didactic and game material and appropriate conditions for the implementation of direct support activities (integration of direct and indirect support through organizing program activities at kindergarten).

7

Discussion

In our research, we have obtained a repertoire of possible opinions, views found in the community of teachers of kindergartens in Bratislava. These opinions and views are illustrated by their specific professional experience. Freedom, decision-making, acting, impacting, responsibility, self-evaluation, etc. have been applied and used in the teaching process and in the processes

20

Severini et al.

of individual and group learning, it was preferable to play roles and to discuss in cooperative and collaborative learning. The more support for their selfregulation (and less external control) comes from the adult (in school it is a teacher), the more structured, the more comprehensive, and the more targeted is their (children’s) development. Children become less dependent and more autonomous, competent, authentic, and responsible. Children have the learning potential; the teacher has options and didactic opportunities for comprehensive learning. The teacher is a person with a greater and more qualitatively different socio-cultural experience, while the children are full of developmental intentions and potentials, which is didactically considered in the teaching process with a close relationship with the teaching, as well as with the transforming and supporting components. We have identified a range of views and insights that present the answer to our first question, “What is the right process of children’s digital literacy development through augmented reality in pre-primary education, by comprehensive, meaningful and deliberate methods?” The most important research findings are that the digital literacy development through augmented reality depends on other features of the child’s personality as well as on the teacher (i.e., the close link between specific literacies, autonomy, competence, self-awareness, decision-making, and participation, that leads to collaboration) and vice versa. From the teacher’s point of view involved in the research, the effectiveness of deliberate support of the digital literacy development through augmented reality is reflected in the ability of children to behave and act independently even when they are not asked to do so, or when nobody observes and or controls them. It is also important, as pointed out by the research participants, to prevent orders and prohibitions from the teacher’s side. Another research finding is that research participants started to consciously support the children’s digital literacy development through augmented reality during the action research. The children were offered choices (for which they are reasonably accountable and supported in their take-over) and also appropriate suggestions as well as the use of appropriate forms of creative creation – the development of children’s thinking. In this process, the teacher’s creativity, the imagination, the sense of detail and the complexity of the viewpoint on what is possible to do in and with teaching were unconditionally important. The unity in the process of learning and teaching (their interdependence and complementarity) was pointed out as well. In the research findings, we also present the answer to the second research question “Does children’s digital literacy develop through augmented reality in pre-primary education, and if so, how?” The findings are shown in Protocol No. 1 mainly in the Evocation part (see Table 1.2). According to research participants,

Uses of Augmented Reality in Pre-Primary Education

21

the process of deliberate support of children’s digital literacy through augmented reality and problems associated with it are not ultimately influenced by the social and cultural environment but by the internal motivation and correct attitude of the teacher towards children, and this requires redistribution of power, mutual trust, and equivalence. It is important, as stated by the teachers involved in the research, to give a choice of self-decision making, and the offer of own opinions and suggestions which children could discuss among each other and also between other research participants in the playing/learning group. Based on the obtained research data we also point out that it is important in the process of deliberate support of digital literacy development that children are spontaneous and involved in creating a process in which they develop thinking, communication, creativity, physical ability, self-esteem, respect for oneself and others, the ability to adapt, to be able to agree and to cooperate in joint activities within activities/games, and to optimally assert themselves in the children’s learning group. Another research finding is the requisite on the teacher’s part to respect the children’s interests of knowledge, role-playing, performing various activities, and respect their effort to be self-reliant, based on the children’s inner needs of learning, exploring and discovering new unknowns even through augmented reality. At the same time, teachers need to expect that children can make use all of what they gained or acquired in their favour, and, thus, they can face the problems that life brings.

8

Conclusion

The research objective was the collaborative creation of a model oriented to didactic support for the digital literacy development of children in pre-primary education through augmented reality based on the social-constructivist paradigm principles. The main objective of our research was to identify the impact and benefits of usage of augmented reality to the children’s digital literacy development. Many teachers in pre-primary education declare that they have no problem in their children’s learning groups. However, if, for example, they are participating in a focus group and/or realize micro-teaching, they begin to discuss and publicly admit that there are in fact several problems in the learning process. However, they have difficulty defining some of the problems, and even greater difficulty identifying and defining the cause of these problems. They would rather point out that their job is to teach and not theorize and, therefore, they don’t feel a need to support their considerations with a more serious

22

Severini et al.

analysis. The involvement of pre-primary education teachers in the active study of teaching practice has made it possible to discover what, or who is the cause of problems. It is in fact through their active engagement in action research (“practical examination”) that, as stated by M. White, D. Epston (1993, In: Niemeyer, 2004, p. 19), they can, “rewrite the history, which each creates for himself as well as participative overall.” By uncovering the teacher’s thinking within active and free contribution, we can recognize how the teacher thinks, with what and how he/she disposes in his/her professional reflection not only on the examination subject but also about the teaching process subjects and his/her self-efficacy. The fact that the child can be highly motivated and active during the pre-primary education period has been characterized by a high degree of engagement, interest, and mutual discussion. Because of this, children were able to search, group, and process information as answers to the teacher’s problematic questions. They took a critical attitude by reviewing and assessing of carried out activity relevance. Scientists, as well as practitioners, are currently wondering how a specific way of thinking creates space for a specific form of freedom and autonomy based on the relationships of children and adults. It emphasizes personal responsibility and self-sufficiency on the part of the individual (both the child and the adult). On the other hand, newly-imposed freedoms bring for all participants the obligation to inform themselves of the various options, so that they can make the best possible investment in their own future and society’s future based on active social integration.

Acknowledgement The chapter was written the support of the grant KEGA 012UK-04/2018 “The Concept of Constructionism and Augmented Reality in the Field of the Natural and Technical Sciences of the Primary Education (CEPENSAR).”

References Blaško, M. (2010). Niektoré aspekty školskej reformy. Košice: KIP TU. Glaser, B. G., & Strauss, A. L. (1967). The discovery of grounded theory: Strategies for qualitative research. Chicago, IL: Aldine Pub. Co. Hernández, G. (2007). El Aprendizaje Comprensivo y Creativo a Partir de la Investigación-Acción Como Estrategia Didáctica Epistémica en la Educación Básica. Laurus – Revista de Educación, 13(23), 11–35.

Uses of Augmented Reality in Pre-Primary Education

23

Kalaš, I. (2011). Spoznávame potenciál digitálnych technológií v predprimárnom vzdelávaní. Analytická štúdia. Bratislava: Ústav informácií a prognóz školstva. Kemmis, S., & McTaggart, R. (1982). The action research planner. Victoria, Australia: Deakin University Press. Kolb, M. S. (2012). Grounded theory and the constant comparative method: Valid research strategies for educators. Journal of Emerging Trends in Educational Research and Policy Studies, 3(1), 83–86. Koreňová, L. (2017). Symmetry in elementary education with the use of digital technologies and manipulations. In APLIMAT: 16th Conference on Applied Mathematics (pp. 836–845). Bratislava: Spektrum. Kostrub, D. (2010). Učiteľ a dieťa/deti ako subjekty vzájomne utvárajúce verzie sveta. In Materská škola a svet okolo. Banská Bystrica: Spoločnosť pre predškolskú výchovu. Kostrub, D. (2016). Základy kvalitatívnej metodológie. Keď interpretované významy znamenajú viac ako vysoké čísla. 1. vyd. Bratislava: Univerzita Komenského. Kostrub, D., & Kikušová, S. (2012). Perspektívy pregraduálnej prípravy učiteľov predprimárnej a primárnej edukácie z aspektu nadobúdania profesijných kompetencií. In Kompetencje współczesnego nauczyciela (pp. 326–336). Siedlce: Uniwersytet Przyrodniczo-humanistyczny w Siedlcach. Miovský, M. (2006). Kvalitativní přístup a metody v psychologickém výzkumu. Praha: Grada. Niemeyer, D. T. (2004). Construccionismo social: Aplicación del grupo de discusión en praxis de equipo reflexivo en la investigación cientifíca. Revista de Psicología, XIII(1), 9–20. Pavlov, I. (2002). Profesijný rozvoj pedagogického zboru školy: Námety na školocentrický a personocentrický model kontinuálneho vzdelávania učiteľov. Prešov: MPC. Pavlovkin, J., & Ďuriš, M. (2012). Príprava učiteľov pre informačnú spoločnosť. Banská Bystrica: FPV, Katedra techniky a technológií UMB. Plichtová, J. (2002). Metódy sociálnej psychológie z blízka. Bratislava: Média. Severini, E., & Kostrub, D. (2018). Kvalitatívne skúmanie v predprimárnom vzdelávaní. Prešov: Rokus. Strauss, A., & Corbinová, J. (1999). Základy kvalitativního výzkumu. Boskovice: Albert. Švaříček, R., & Šeďová, K. (2007). Kvalitativní výzkum v pedagogických vědách. Praha: Portál. Švec, Š. (1998). Metodológia vied o výchove. Kvalitatívno-scientické a kvalitatívnohumanistné prístupy. Bratislava: Iris. Tománková, M., & Kostrub, D. (2018). Premiéry, reprízy a omyly na križovatkách alebo stratégie výchovy v súčasnej rodine. Prešov: Rokus. Woods, P. (1987). La escuela por dentro: La etnografía en la investigación educativa. Buenos Aires: Editorial Paidós. Zelina, M. (2006). Kvalita školy a mikrovyučovacie analýzy. Bratislava: OG Vydavateľstvo Poľana, s. r. o.

CHAPTER 2

Uses of Augmented Reality for Development of Natural Literacy in Pre-Primary Education Kateřina Jančaříková and Eva Severini

Abstract The chapter focuses on the educational potential of augmented reality for support of scientific literacy in pre-primary education. Development of scientific abstraction in preschool children (aged 3–7) is one of the goals of scientific pre-literacy (Jančaříková, 2015). The chapter presents the opportunities that augmented reality brings to development of scientific literacy. The authors give a brief overview of the area and of selected applications suitable for preschool children. They then focus on two of them, 4D animal and Quiver Vision, which were used in a research project involving two Czech and two Slovak kindergartens as well as 5 preschool children’s parents. Improvement of the learning process and of teaching methods contributes to a child’s complex development in the contemporary socio-cultural context. That is why the goal of the presented research was to identify the impact of the use of augmented reality on the development of children’s scientific literacy as well as on the teachers’ professional development. This action research project was conducted within the frame of science education context and included semi-structured interviews with teachers from a learning/play community of children involved in the research, and preschool children’s parents, as well as direct and indirect observations of children while involved with augmented reality in educational activities. The research showed that many teachers were not familiar with the technology of augmented reality, and some teachers expressed worries about how to employ augmented reality with preschool-age children. Additionally, the teachers shared creative ideas on how these technologies can be included in the offered activities.

Keywords augmented reality – natural literacy – pre-primary education – qualitative methodology – work with models © koninklijke brill nv, leiden, 2020 | doi: 10.1163/9789004408845_002

Development of Natural Literacy in Pre-Primary Education

1

25

Theory

1.1 Introduction Information and communication technology and especially applications with augmented reality are of unprecedented potential and allow teachers to present very complex objects and processes in the classroom, which teachers increasingly make use of. But do pupils understand these models? What can be done to support understanding? How can misconceptions be minimalised? How can scientific abstraction in preschool children be developed in the right way to ensure pupils’ and students’ success in the years to come? If we want to understand augmented reality well and use it in science education optimally, it must be perceived in a wider context. Development of science literacy is an indispensable part of general education. The special goals that activities with preschool children should target are defined as follows: – Development of sensitivity to nature (environmental sensitivity) built on the relationship to specific animals, trees, plants and products of nature or to some landscape and natural environment and experiences in this environment; – Development of vocabulary in the domain of nature and environment, of language skills that allow a child to describe their experience or observation or to understand their teacher, classmates, educational TV programmes, and of communicative competence including the courage to ask about things they do not know; – Development of a system of basic knowledge about the realm of nature (e.g., that fish die if not in water, etc.) that is not necessarily verbalised; – Acquisition of skills and routines enabling development and deepening of knowledge of nature, including inquisitiveness, interest, creativity, observation, bases for science experiment, use of simple tools of measurement, and mastering of work with models; – Mastering of self-management and hygienic routines, i.e., those that enable risk-free scientific activity, for example, eating no berries in nature without an adult’s consent, keeping shoes and garments dry (does not deliberately walk through puddles), washing hands without being told, etc.; – Identification of children and pupils with scientific talent and their support (Jančaříková & Jančařík, 2016). One of the objectives of science literacy is the ability to understand models (Jančaříková, 2015). This objective comes out of the conclusions of PISA (OECD, 2007), which defines work with models – in a broad sense of “model,” including two-dimensional projections and images, computer simulations and others – as one of the problematic areas in science education.

26

Jančaříková and Severini

1.2 Models 1.2.1 Definition of Models Models are defined as provisional objects that enhance pupils’ ideas and conceptions, specify abstract systems of concepts and support in pupils’ connection of acquired knowledge to real life. Work with models is (among other) included in the Demonstration Method (Skalková, 1999). A model in natural sciences is an object that represents a natural object or phenomenon. Didactics of chemistry often use the term representational structures for models and mental models for students’ understanding of models. To understand what models, represent efficiently, a pupil requires a very specific understanding of the relationship between the model and the real object it is meant to represent. It is very difficult to master this specific skill (Machková & Bílek, 2013). Different authors look at it from different points of view; the very naming of this skill is very difficult. Some authors refer to this skill as the ability to understand visual language (or better, “visual languages,” because in various cultural contexts the same scheme can have very different meanings) and develop the theory of visual languages (Chang, Tauber, Yu, & Yu, 1989). Hortin speaks of visual literacy, defined as the ability to understand (“read”) and use (“create”) images and to think and learn in terms of images (Hortin, 1980). For example, when reading a map, understanding the symbolic relationship between a map and the real landscape as well as the symbolic function of signs is essential (Kosslyn, 1989). Slavík (2001) uses the term isomorphism. Skalková uses the term abstraction, which she specifies as the ability to create new images with the help of illustrative visual aids (Skalková, 2007). For this chapter, we decided to use the term natural science abstraction. Whatever this specific ability is called by researchers, it is obvious pupils need specific skills in order to understand a complex model and to connect, in their minds, a model’s properties to the properties of the real object they are learning about in the so-called “recoding” process. 1.2.2 Classification of Models With respect to their purpose, models in natural sciences can be divided into two basic groups: 1. models of objects; and 2. models of relationships and processes including model solutions of problems (Jančaříková & Jančařík, 2016). Both of these groups are often interconnected in school practice and it is not always possible to isolate one from the other. For example, a model of the solar system not only gives information on individual planets but also gives a lot of information about their relationships, possible positions, orbits, etc. On a

Development of Natural Literacy in Pre-Primary Education

27

higher level of abstraction, the model of an atom in not only its physical visualisation, but also a schematic indication of the relationships between the particles that make up an atom. Models of objects can be divided into several basic categories, including: – Visualisations; – 3D models; – Collections of products of nature; and – Living (model) organisms. Visualisations Visualisations can be in colour or black-and-white, can be of different sizes and proportions, can require small to medium levels of abstraction. Visualisations include drawings, paintings, photographs as well as various types of projections – static projections (slides, products of nature placed on an over-head projector or PowerPoint presentations) and dynamic projections (videos and films, including animated films). Laics often believe the best way to visualise a product of nature is a photograph, but this is a mistake. In contrast to a photograph, a scientific illustration enables the author to emphasise the features important for determination of the specific species or features and properties specific for different seasons (see Figure 2.1), and thus visualisation of the given species is more fitting. The theory of illustration is quite advanced. Čáp and Mareš (2001) present a classification of visual materials according to their function as follows: a. decorative – those that are not related to the text; b. representative – those that provide an adequate visual image of the represented object; c. organizing – those that require procedural knowledge (e.g., manuals, instructions, phased images, sketches, flowcharts, etc.); and d. interpreting – those that help understand in areas that go beyond pupils’ experience. All authors warn of creation of misconceptions in case of a poorly selected illustration. That is why it seems appropriate not to plan lessons only with one type of visualisation but to make use of multiple types including threedimensional models. Three-dimensional models Three-dimensional (3D) models are objects that visualise a selected naturalscience entity. A teacher should present to children and pupils models made of different materials (wood, plastic, plush, etc.), of different colours and sizes. Figure 2.2 presents a selection of models used in natural science activities with

28

Jančaříková and Severini

figure 2.1 Visualisation, Department of Biology and Environmental Studies, Charles University, Faculty of Education

figure 2.2 Selected aids used by the author for development of science literacy in primary home schoolers

preschool children. The photograph includes reduced models (the model of solar system, dinosaur and elephant), enlarged models (models of the lifecycle of a ladybird, ant, stag beetle and mosquito), and real-sized models (models

Development of Natural Literacy in Pre-Primary Education

29

of a bat or plant germination). (Figure 2.2 also includes an illustration of the process of hatching and growth of a duck, which is not a 3D model.) Thanks to the development of technologies, we come more and more often across 3D interactive models that pupils can view on computer monitors, using a stereoscopic projection and augmented reality, (which will be the focus of this chapter). Products of nature, including animal and plant collections and preparations In a broader sense, a specific type of 3D models are products of nature. Preparations are either temporary or permanent. Preparations and other products of nature are those that are made within a lesson and disposed of after the lesson, for example, protozoans in an infusion of hay that can be observed alive with a microscope. Permanent preparations are for example animal mounts (see Figure 2.3) and their parts (skeleton, leg with claws etc.) but also items from herbariums and collections, for example, an entomological collection, a collection of shells and seashells (see Figure 2.4) or a collection of rocks and minerals. Preparations can also be microscopic. There are several companies in the Czech Republic that sell whole sets of preparations related to different areas to schools. When preparing and studying a real natural object, pupils use more senses (e.g., touch, smell) and thus gain experience irreplaceable even by the most up-to-date technology. This category also includes mounted animals, preparations and collections of beetles, butterflies and items from herbariums, although demonstration of a mounted animal borders on visualisation and work with a model.1 Natural science collections are a particularity of science education. One of the needed competences of teachers of biology is thus taxonomy and preservation of products of nature. This is taught at Charles University, Faculty of Education also within the frame of in-service teacher training.

figure 2.3 Mounted birds (Falco peregrinus, Circus aeruginosus, Pica pica) of Department of Biology and Environmental Studies, Charles University, Faculty of Education, stuffed under supervision of Jan Řezníček, Ph.D.

30

Jančaříková and Severini

figure 2.4 Natural science collection of Department of Biology and Environmental Studies, Charles University, Faculty of Education

Living organisms When teaching biology, not only models but also living organisms are used, especially to meet the didactic principle of illustrativeness and for comparison with models. Observation of living organisms in their natural habitat helps pupils learn not only about the organism itself but also about its relationships and interactions with its environment. However, it would be wrong to think that introduction of living organisms is sufficient. The didactic goal of using living organisms is not only that pupils get introduced to the given organism but also secondarily work with a model and natural science abstraction. Model organisms are used for direct observation. A model organism is such that is explored by pupils not only to study the organism itself but also to develop their ability to explore and to expand their knowledge of the diversity of living organisms. A model organism is studied in great detail and children and pupils are expected to be able to generalize and transform knowledge and methods of studying and exploration to other organisms and objects. It is striking that the selection of model organisms as such is methodologically underdeveloped at the level of pre-schoolers, as well as at primary and

Development of Natural Literacy in Pre-Primary Education

31

secondary levels. However, there are some model organisms that have been used at schools for hundreds of years, including earthworms and bees, which have been used in textbooks since the time of Austro-Hungarian Empire. Jančaříková (2015) describes the criteria for selection of model organisms and work with them in detail. An unusual model organism was selected by Hanel and Hanelová (2014). These are the true bugs Elasmucha grisea, which are abundant on birches from May to July, are not protected by the law and show very interesting and easily observable parental behaviour. Observation of these beetles gives an opportunity for inquiry-based education. Models of relationships and processes As with models of objects, models of processes and relationships can be divided into several categories, which are: – Static schemes; – Dynamic schemes; – Dynamic models; – Interactive models; and – Proper (model) observation. Static schemes A static scheme is the depiction of a relationship or process that a pupil must learn. Static schemes put high demands on the pupil’s ability to abstract, for the pupil must make conjectures and recode the presented information into the structures in their minds. On the other hand, static schemes make it possible to describe and emphasise the main parts of a process and partly also the conditions under which these processes take place. The Internet offers a wide range of schemes describing processes and relationships (see, e.g., Figure 2.5). Dynamic schemes A dynamic scheme allows pupils to observe the course of some process. In computer created dynamic schemes, pupils can select which parts of the process they want to study and observe the schematic depiction of their progress. For example, the application Body 4D2 allows pupils to study the human heart in Augmented Reality, which means they see its activity, hear its beating and can view it from different angles, magnify any of its parts and observe its activity at different stages. Dynamic models Dynamic models allow pupils to study and explore not only objects but especially processes. Preschool children prefer dynamic interactive models. An

32

Jančaříková and Severini

figure 2.5 Scheme of the water cycle (by John Evans and Howard Periman, USGS, http://ga.water.usgs.gov/edu/watercycle.html, Public Domain, https://commons.wikimedia.org/w/index.php?curid=26818355)

figure 2.6 The dynamic three-dimensional model of childbirth allows manipulation and thus demonstrates the process more fittingly. The model is in the collection of the Department of Biology and Environmental Studies, Charles University, Faculty of Education

example of such a model is a toy kangaroo with a baby that a preschool child (or other child) can put into and take out of the mother’s pouch. Another example is the model of the birth of a baby shown in Figure 2.6. Interactive models Interactive models allow pupils not only to observe the course of some activity/process but also to influence it by changing various parameters. Interactive

Development of Natural Literacy in Pre-Primary Education

33

applications usually make use of mathematical models describing individual phenomena and their mutual relationships. One of the best-known examples is the simulation of sizes of populations of hares and foxes in nature, which is used when teaching ecology.3 1.2.3 A New Era of Models Development in technologies brought a new era of models with new potential but also new demands on the user. Virtual environments and applications with augmented reality allow 21st century biologists to study models of objects they cannot touch (e.g., DNA double helix, cell, atom) and to simulate phenomena that cannot be observed with human eyes (e.g., mitosis). Thanks to virtual environments, environmental scientists are able to formulate better hypotheses about structure and functions which they verify using computer simulations. New technologies are no longer used only by researchers and in tertiary education, they have entered schools and also preschool education (Johnson et al., 1998). However, grasping a model transferred into a virtual environment puts high demands on the user. Recoding the relationship between the real object and its model or simulation correctly requires developed abstract thinking. The more complex the model or the simulation, the more abstract thinking is needed to grasp it correctly. PISA studies (OECD, 2007) show that many (Czech and Slovak) pupils struggle to grasp a model correctly. In correspondence to the principle of small steps, scientific abstraction should be developed systematically. It cannot be acquired quickly. Pupils cannot be expected to understand a complex simulation in augmented reality without sufficient previous training to fully acquire the new subject matter. 1.2.4 Augmented Reality The main sense of using augmented reality is to provide a real environment with digital information and images. Many applications augment reality with playful elements, but their education potential is negligible. With respect to science education, such fun elements are not beneficial and may be counterproductive, because they not support creation of the student’s understanding of scientific concepts (Jančaříková, 2015). However, the educational potential of some applications is immense. An example of how augmented reality can be used in science education is the application for Android, Star Walk 2 – Night Sky Guide, which allows users to identify stars and constellations in the night sky. If the user observes the sky using the camera of their tablet, they can see not only various celestial objects

34

Jančaříková and Severini

along with their names and other relevant interesting information. Star Walk Kids – Explore Space is adapted for children. Several studies were conducted confirming the great potential of augmented reality for development of scientific imagination (e.g., Hamilton, 2011; Kaufmann, 2004). Critical voices warn of related dangers, the most serious of which is cognitive overload (Carpenter, 2010). A whole range of applications whose goal is to develop science literacy was developed for pre-primary children. Some of them are run using special cards, for example, AR Animal, an application for iPhone and Android. If the user focuses the camera on a special card, a 3D visualisation making simple movements appears. At the same time a voice recording with more information about the animal is played. Another example is Animal 4D+, an application for iPhone and Android. If the user focuses the camera on a special card, a 3D visualisation making simple movements appears. Some cards interact, for example, the card with a monkey and banana (the monkey tears the banana, peels it and eats it). Other applications are run if the child colours a picture. For example, the application Quiver provides a wide range of pictures (volcanos, cells, etc.), which, if coloured by a child, are augmented by 3D models. If the child taps on the tablet display, the model behaves as a dynamic model, for example in the case of a volcano, lava starts rising, which is followed by an explosion. This process is accompanied by sound effects. 1.3 Virtual Reality at the Preschool Level It is only logical to commence with intentional development of scientific abstraction as soon as possible, even as early as the pre-primary level. Developmental specifics of this group should not represent an obstacle. Piaget states that at the preoperational stage (2–4 years) children are already able to work with symbols and start to understand that simple visualisations, projections and models represent real things (Piaget, 1973). Many engaged parents and educators have recently asked the following questions: “Is it meaningful to use tools of augmented reality for development of scientific abstraction in preschool children?” Contero and his colleagues (2012) analysed the efficiency of the activity “Animals” in a group of preschool children (aged 4–5). Participants were split into two groups. The same programme was conducted with both groups but in one group the activity was supplemented by a presentation of animals in augmented reality. The results indicate that children in the augmented reality group achieved better results than children from the control group.

Development of Natural Literacy in Pre-Primary Education

35

Han Jo, Hyun, and Hyo-Jeong (2015) describe how a group of Korean experts from various fields (preschool educators, preschool teachers, specialists in augmented reality and robotics) cooperated on the development of an educational activity with augmented reality for preschool children. Subsequently this activity was experimentally tested in a kindergarten. The results imply that applications with augmented reality can be useful in preschool education. Cascales, Pérez-López and Contero studied the relationship of preschool children to applications offering augmented reality. Their study proves that parents are well aware of the benefits augmented reality can bring (Cascales, Pérez-López, & Contero, 2013). However, a meta-study conducted in 2014 in Japan points at a high variability of efficiency of the use of applications with augmented reality. Its authors found 87 research reports focusing on the study of teaching supported with augmented reality (ARLE) in the digital library IEEE Xplore and other publications on educational technology. Efficiency of the use of augmented reality varied considerably. Some studies report negative impact, some average or moderate effects (Santos, Chen, Taketomi, Yamamoto, Miyazaki, & Kato, 2014). This is easy to explain. Similar to other didactical aids, applications with augmented reality are self-sustaining. If their impact on educational process is to be positive, they must be used in a systematic and thoughtful way. Dayang Rohaya and his colleagues (2012) show that communication, discussions, and telling stories are essential while working with applications with augmented reality. Or they use augmented reality to introduce stories in a more attractive form. We tried to consider what aspects of work with augmented reality with preschool children can be made more efficient and came to the conclusion that augmented reality must be used in accordance with didactical principles of science education. As these principles are not generally known, we decided to present a brief survey of them in the following section. 1.4

Didactical Principles for Use of Applications with Augmented Reality at the Preschool Level 1.4.1 The Principle of Partial Educational Guidance (Process as the Goal) This principle, first formulated probably by the first Swiss educator, philosopher and reformer Johann Heinrich Pestalozzi, points at the fact that apart from knowledge objectives, we must always bear in mind the partial objectives. In the context of science education these partial goals are namely: – To develop vocabulary and speech, – To master work methods including rules of safety and the ability to cooperate,

36

Jančaříková and Severini

– To master the ability to work with various types of visualizations, – To master the ability to work with models, – To develop environmental sensitivity, – To adapt appropriate behaviour patterns, including ethical patterns, – To experience joy of discovery (Jančaříková, 2015). All these partial goals must receive attention when using applications with augmented reality. 1.4.2 The Principle of Scientific Quality The principle of scientific quality states that teachers should be using scientific reasoning, procedures and language when communicating with children during activities. Applications with augmented reality can help preschool children learn to differentiate between real and fictional. A teacher should make sure they select those applications with augmented reality from the wide offer that present reality with a high degree of accuracy and should avoid the use of those that distort reality inappropriately. It is equally important that augmented reality should always be discussed as a visualisation or a model. For example, if an application with augmented reality shows a 3D elephant, it is essential to refer to a “model of an elephant,” not to an “elephant.” 1.4.3 The Principle of Adequacy The principle of adequacy states that the goal, process, means and extent of education must be adequate to particular children’s development and their individual abilities and that overloading children is counterproductive and unhealthy (it can result in discontent, demotivation, neuroses and other psychosomatic disorders). Applications with augmented reality should be presented to children as an option that is fun. They should not be used for stressful activities. Learning should be playful and should be a game. 1.4.4 The Principle of Illustrativeness The principle of illustrativness (demonstrative teaching) refers to the need to connect the built scientific ideas and concepts to real objects by manipulating or interacting with them. It draws attention to the fact that substituting natural objects by their visualisations and projections is not sufficient. Applications with augmented reality cannot substitute the process of getting to know the world, they can only supplement it. This means teachers should, whenever possible, expose children to real products of nature and natural objects that they can touch, smell, weigh and view at all different angles, or even throw to water, crumple, bend, or explore in other ways. Only if the object is not available, is too fragile or vulnerable (a living animal) is it permissible to use only its model, visualisation, projection

Development of Natural Literacy in Pre-Primary Education

37

or augmented reality without the object itself. In any case it seems most meaningful to present to children the product of nature as such as well as a wide range of its visualisation. 1.4.5 The Principle of Activity The principle of activity stresses the importance of being actively involved and the danger of being passive in the learning process. Activity triggers positive emotions, being passive is boring and boredom bears negative emotions. If children are forced to passive roles, they can suffer from behavioural disorders and neuroses. In the context of the use of augmented reality, it is essential that it should be the child itself who controls the iPad or Android regardless of the price of the device.

2

Action Research

2.1 Introduction Specification of the subject of this research is defined as follows: – The problem addressed by the here-reported research study is the inconsistency, aimlessness, arbitrariness, indifference, and misunderstanding connected to the use of augmented reality for development of science literacy in preschool education. – Contemporary pedagogical discourse in the area of development of preschool children’s science literacy has recently developed its quality but experience shows that this development has not entered the teaching practice. 2.1.1 Practical Exploration – Action Research Action research as defined by Kemmis and McTaggart (1982) “is a deliberate exploration conducted by a group of researchers (teachers) in the conditions of an educational process.” The concept “teacher-researcher” has become integral part of postmodern literature on educational reform that supports professional teacher education (including teachers at the preschool level), improvement of conditions for their work, professionalization of the teaching process and development of an individual or group of learners. In the context of the current, new conditions in the system of education, teachers take on a new role in the area of monitoring school education programmes (curricula), of self-evaluation and evaluation of quality of the teaching process. Moreover, teachers, as those who are responsible for the quality of the teaching process, gradually acquire another new competence and status related to the changes in the teaching process: the competence to conduct research and the status of a teacher-researcher. Practical research in education is one type of scientific research in which teachers study their own teaching systematically with the objective of improving its

38

Jančaříková and Severini

quality. This research is usually conducted by teachers (often in collaboration with a researcher) within the process of their own teaching practice. Action research reacts to the current needs of the teacher’s teaching practice (sometimes only partial needs that a professional researcher does not focus on as they regard it as less important), helps to propose solutions that are tested in the classroom and, if they work, they are immediately transferred into the teaching practice. Action research should be an integral part of every teacher’s activity if the teacher aims at meeting the demands of high quality teaching and developing his or her own professional competence. The methodology of the process of action research is presented in a cyclic structure, either in a spiral or a circle (Elliott, 1997; In Hernández Gil, 2007). The cyclic process consists of several stages: (1) Diagnostics of the teaching process; (2) Design of intervention (action plan); (3) Intervention and data collection; (4) Analysis and interpretation of data; (5) Illustration of the change in the teaching process (the emerging theoretical model). The given stages are conducted through reflection – action – reflection (as well as action – reflection – revision). The decision to conduct this type of research was made on the basis of being aware that the studied phenomena are scrutinized by a person who influences the choice of methodology and the character of the research study. The subjects of this research were pre-primary teachers and their opinions on the topic of this research, which is closely connected to other subjects (children). 2.2 Methodology 2.2.1 Research Methodology, Methods and Organization To handle the defined problem, the methodology of qualitative research with a corresponding design was selected. Qualitative research represents an opportunity to focus on human activity and thinking in different dimensions that contradict the conception of quantitative research (Wittrock, 1989, p. 114). It allows a relativist perception of (vision of/perspective on) the world, an individual, social life, etc. It comes out of a socio-critical paradigm (a posteriori). The objective of qualitative research is to transform or change. It uses qualitative methodology: socio-critical; action research; participative. It works with interviews, discussion/focus groups, observations as tools, methods, techniques of research. Bogdan and Biklenová (1992; In Švec, 1998) list the following characteristics of qualitative research: the natural environment, which is the primary source of data, is in the foreground; data in the form of words and images are collected; the researcher analyses data inductively. The main purpose of a qualitative researcher is to discover how people make sense of their lives; a qualitative researcher is interested not only in the processes but also in the products of human interaction. The main features of qualitative research are its intensiveness and longitudinal nature. Qualitative research relies on detailed recordings, including audio and video recordings, and attends to

Development of Natural Literacy in Pre-Primary Education

39

almost everything that is going on in the given environment. The concept “values of qualitative research” is understood as specifics of this type of research that are important, principal and full-featured for success and efficiency of the conducted qualitative research in the sense of principles of qualitative methodology. They are (according to Wittrock, 1989): Iterability (i.e., the same standard is used for generation of similar information); Systematicity – securing no interference (some kind of interference on the basis of confirmation) of our (already existing) ideas caused by a wrong selection of subjects of research and through their answers (or socially desirable behaviour); Reliability, which is related to the nature of the questions and the way they are asked – Questions must be meaningful and adequate in order to generate valid facts related to the studied phenomenon; Transparency of research tools, methods and techniques: the collection of research material and elaboration of a text must be transparent and clear to addressees of the research; and must present the research study exactly as it was conducted and must explain how the researcher proceeded in data collection, in their processing, analysis and interpreting. Research objectives: – To identify the impact and benefits of the use of augmented reality for development of scientific literacy in children as well as the impact of the use of augmented reality on teachers’ professional development. Research questions: 1. What should be the process by which a teacher develops pre-schoolers’ scientific literacy in a complex, intentional and conscious way using augmented reality? 2. Is scientific literacy developed using augmented reality in preschool education and if so, how is it achieved? Methods of data collection: – Semi-structured interview and focus group discussions (Gavora, 2007; Plichtová, 2002) – collecting opinions, beliefs, explanations, knowledge of subjects involved in the research. – Unstructured observation (Gavora, 2008) – direct observation (observing interaction of participants in the lesson) and indirect observation (observation of real situations in classrooms). Methods of analysing research data: – Open coding (Strauss & Corbin, 1999) – categorization, conceptualization and coding of data. – Continuous comparison (Creswell, 1998; Kolb, 2012) – process of continuous generation of a theory. – Selective coding (Strauss & Corbin, 1999) – grounded theory. The data were triangulated (Ferrer & Jiménez, 2006; Mayan, 2001) in the form of systemic comparison of different researchers’ opinions concerning the same

40

Jančaříková and Severini

data. It did not have the form of the use of more tools (methods) for creation of different data. It was an analysis of the same data using various tools. Research sample – criteria for selection of participants in the research study: 1. Graduates from universities with a degree in pre-primary education – with respect to the research topic the participants were teachers from kindergartens where one of the declared objectives of education was development of scientific literacy in pre-schoolers. 2. Length of teaching practice ten years or more – research on development of teachers’ professional competences show that the length of teaching practice allows teachers to connect declarative and experiential knowledge. 3. Home/work address: Czech Republic and Slovak Republic – the selection respected the cultural, social, demographic and ethnic diversity of kindergartens from both countries with historically similar systems of education. 4. Degree of interest and willingness to discuss issues related to the topic of this research. 2.2.1.1 Entering the Research Project Before starting the research, its implementation was discussed with the head teachers and teachers in the selected kindergartens. First, we mapped the places to identify all the relevant information about conditions in the kindergartens: material-technical, organizational, and personal. Based on these findings, we adapted the conditions for conducting the research to the conditions of the particular institutions. We provided all the needed material-technical tools that were not available in the particular kindergarten (e.g., tablets, applications with augmented reality, digital cameras, etc.). Two weeks before starting the research, the teachers and parents of children involved in the research study were asked to sign informed consent forms related to the planned research and given questions for the semi-structured interview: 1. Have you come across applications with augmented reality before being addressed by us? What applications? 2. Which of them do you consider suitable for education of pre-primary children? 3. Do you use any applications with augmented reality when teaching children in the kindergarten? What applications? Why? If not, why do you not use any? Lack of knowledge and experience? Your beliefs? 4. What benefits of activities with augmented reality can you see? 5. What drawbacks of activities with augmented reality do you see? 6. Are you familiar with 4D animal or Quiver Vision? If yes, how did you use it? Was it effective? 2.2.1.2 Conducting the Research The data were collected through focus group discussions within action research at two kindergartens with preschool teachers, and five parents of 5 to

Development of Natural Literacy in Pre-Primary Education

41

7-year-old children in two European countries, the Czech and Slovak Republics. The research subjects were asked to present their ideas and opinions in the form of answering a question asked to initiate discussion: How can science literacy be optimally developed in pre-schoolers using augmented reality? The research was initiated in 2017. Observations within the frame of action research were conducted in kindergartens from 2017 to 2018. The observed subjects were the teachers who took part in the discussions as well as children in the learning/play groups. The teachers participating in the educational activity “Insect” were interviewed again. In semi-structured interviews they were asked the following questions: 1. How do you evaluate the sample activity? Is it age appropriate? 2. Will you consider using this activity (with augmented reality) in activities with children in preschool education? 3. Would you be able to prepare and plan a similar activity? 4. Has your opinion on the use of applications with augmented reality in the educational process changed? 2.2.1.3 Research Data Processing Data analysis is a demanding activity, especially if a lot of research material is collected. Its elaboration requires thorough analysis and repeated returns to the terrain. Researchers are required to interpret subjective concepts possessed by subjects of the research and to conceptualize them in the form of mental abstraction in such a way that they do not diverge from the original concepts of the studied subjects. This needs time, maximal attention and responsibility of the researcher. That is why we must ask questions about such issues as freedom, consciousness, wisdom, morality and the significance of human realities on the background of socio-cultural and historical construction, whose understanding is essential for explanation of permanent, valid and socially acceptable knowledge. Permanent (as derivate of mentally elaborated experience), valid (as socially valuable) and socially acceptable knowledge is not only permanently developed by the members of a society, it is at the same time legitimized through qualitative research of human realities on the basis of constructing consent in dialogue (discourse) and intersubjectivity. By accepting that a qualitative researcher is also simultaneously the research tool, has some share in the research, and can become the subject of research, selfstudy (introspection) becomes a valid research tool. 2.2.1.4

Analysis and Interpretation of Data from Semi-Structured Interviews and Focus Group Discussions When analysing individual protocols of statements of the research subjects in semi-structured interviews and focus group discussions, all their statements were included and the perspective of studying their potential for pedagogicaldidactical development of children’s scientific literacy using augmented reality

42

Jančaříková and Severini

was sought; answers to research questions were looked for. Conceptual labels were assigned to individual events, cases and other occurrences of a given phenomenon. By categories we mean classes of concepts that were identified when comparison of concepts seemed to indicate they belonged to a similar phenomenon. Open coding was used to carry out the processes of analysis, exploration, comparison, categorizing, coding, and conceptualization of data. 2.2.1.5 Analysis and Interpretation of Data from Observation Material collected through direct and indirect observations was analysed. Events and occurrences are perceived as indicators of phenomena which get a conceptual label. Systematic reflection of professional situations, exploration of pedagogical reality and initial diagnosis based on evidence from the children in the learning/play group preceded the research activity (indirect observation). On-the-fly diagnostics based on evidence from the children from the learning/ play group were carried out during the teaching process and output diagnostics based on evidence from the children in the learning/play group were conducted within pedagogical reflection on the conducted educational activity. A lot of research material in the form of video recordings and pictures and photographs was also collected in the research. In indirect observation, video recordings of educational activities were watched several times; significant elements supplying evidence on development of science literacy of pre-schoolers using augmented reality were looked for. When analysing findings from direct and indirect observations, a referential framework was abstracted (see Table 2.1). Registration of research results was evaluated on the basis of this referential framework with evaluative comments used for interpretation of results of the research. 2.2.1.6

Entering the Teaching Process and the Teaching Process Using Augmented Reality Teaching activity: Insect Specific goal: – On the basis of hands-on experience and practical activities, to build a positive relationship to natural environment and to recognize insects as a part of nature. Partial goal: – Focus on characteristic properties of insects and to learn about their life in nature. Motivational strategy: – Direct induction of situation using an application using augmented reality. Communicative situation: – Interview, discussion, and analysis of problems that arise.

– Stage of evocation – the teacher asked the children to present their knowledge of insects verbally. Each child was given enough space to construct ideas connected to the concept of insects. The teacher steered the discussion in a way to make the children’s preconceptions contradict the new information or facts with the purpose of bringing about a cognitive conflict. She also stirred the children’s internal motivation by discussing with them propositions on how to carry out the topic of the activity. Children’s ideas and proposals were recorded on the blackboard. At the same time the teacher was diagnosing the children’s conceptions of insects (by categorizing them) – thematic categorization (initial evaluation) (see Figure 2.7). The children were creating a conceptual mind map of their initial concepts, they were expressing their (own) opinions. The teacher supported children’s collaboration by creating conditions for it – this is the pre-active stage of teaching.

Description of a didactical situation in an educational activity

(cont.)

The children verbally express their ideas on how to carry out this activity. They want their teacher to record the proposals. The children make decisions about individual activities that will be carried out within the whole activity. The children set rules about how to carry out the game together with their teacher. The teacher gave the children an opportunity to experience something and to reinforce the new knowledge by hands-on experience. The children acted and made decisions autonomously. The teacher’s opinion was not determining for the children’s activity in the game. The children developed the game spontaneously; they were autonomous because their activity was based on their own solutions.

figure 2.7  Thematic categorization/ initial evaluation

Presence of development of science literacy of children in the augmented reality supported teaching

table 2.1  Diagnostics presence of development of scientifijic literacy of children in the augmented reality supported teaching based on evidence (excerpt)

Development of Natural Literacy in Pre-Primary Education

43

– Stage of becoming aware of meaning – the children developed the topic in specific forms. They were able to discover spontaneously how they should do something. The teacher developed their digital literacy with an individual approach using tablets and worksheets with a black and white printed marker. The marker in the form of a picture was scanned with the tablet by the children on their own. The teacher also asked questions that served as the basis for constructing new knowledge in the area that was topical and interesting for the children (thematic conceptualization) (see Figure 2.8). Children discovered autonomously that the tablet had to be held in a certain distance from and direction to the worksheet to see the 3D object projected above the marker. Then the children saw a virtual object instead of the marker on the display. Specifically, it was an easily identifiable and easily generated picture on the coloured worksheet on the topic Insect. The children followed their game objectives and in cooperation with the teacher

Description of a didactical situation in an educational activity

(cont.)

The children carry out various activities (that they have proposed) while gaining new knowledge. They were able to ask challenging questions, constructive solutions to which they were looking for, because reshaping asks for functional zones and turns (socio-cognitive conflict) and enough secure space and time. They asked questions autonomously, thought about them and carried out activities in which they were involved. The children had space for self-realization and developed the game. They solved the problems independently, e.g., in the case when the 3D and 4D objects did not appear.

figure 2.8  Thematic conceptualization/ on-the-fly evaluation

Presence of development of science literacy of children in the augmented reality supported teaching

table 2.1  Diagnostics presence of development of scientifijic literacy of children in the augmented reality supported teaching based on evidence (excerpt) (cont.)

44 Jančaříková and Severini

They were manipulating with the tablet, as it allowed orientation in space to such an extent that they could view the object from all perspectives. The children carried out the following activities: they proposed, discussed, offfered alternatives, approved, voted on the fijinal product.

they proceeded in their work on the educational problem – thematic realization/preliminary evaluation. The teacher identifijied what each child can manage – this is the interactive stage of teaching. – Stage of reflection – in the final stage of the educational activity the children together with their teacher gave feedback on the stated educational problem by creating a partial product (final evaluation) (see Figure 2.9). Children’s reflection (self-assessment and assessment) was connected to their development of scientific literacy and also of digital literacy through augmented reality. The children made independent decisions about their activity but partly needed their teacher’s assistance. The teacher’s evaluation was related to partial goals defined for the educational activity. The teacher respected each child’s personality and supported them in decision making and assessment. At the same time, she provided enough time and space for the activity to be carried out. In the activity she put emphasis on communication and information exchange – this is the post-active stage of teaching.

The children depicted the links between the given concepts. They created a conceptual mind map autonomously. If needed, they asked their teacher for help. The teacher supported the children by asking questions that developed critical thinking. The children were able to look for their solutions. They presented the gained knowledge autonomously and evaluated the course of the conducted activity.

figure 2.9  Thematic realization/final evaluation

Presence of development of science literacy of children in the augmented reality supported teaching

Description of a didactical situation in an educational activity

table 2.1  Diagnostics presence of development of scientifijic literacy of children in the augmented reality supported teaching based on evidence (excerpt) (cont.)

Development of Natural Literacy in Pre-Primary Education

45

46

Jančaříková and Severini

Experimenting with the topic (what activities related to the topic will be used in experimenting– how the subject matter will be experimented with – how the topic will be explored by the participating teachers and learners): – What is an insect? What does an insect need to live in nature? Why are insects an important part of nature? Situation of experimenting: – Strategy of stimulating questions, problem-solving methods, and practical activity. Developmental primary activity: – Development of linguistic competence, stimulation of optimal behaviour respecting the particular habitat, stimulation of verbalization and circulation of ideas in a discussion with the help of relevant and suitable questions (not general questions) targeting explanation of some mental plan (what we will do, why we will do it, how and where we will do it, with whom or what we will do it, etc., while using the application with augmented reality on the topic “Insect”). To use acquired knowledge in the context of family and society as an incentive for follow-up discoveries. Children are divided into groups and work on various aspects of the topic with the help of their teacher (if needed). Formalization, representation and symbolization of experience: – to solve a problem in connection to a given condition. Indicators for evaluation of observation: – the child should be able to recognize insects in the surrounding environment, – the child should be able to justify the importance of insects in nature, – the child might know, for example, the process of getting honey. Developmental integrational skills: – manipulation with tablet, – getting to know activities when using applications with augmented reality, – cooperation on and collaboration in activities. Activities: – selection, proposing, considering, sharing, manipulation, discovery, discussion, communication, pondering, justifying, listening to, asking, cooperation, collaboration, determination, classification, and assessment. 2.2.1.7 Elaboration of Research Material Elaboration of typology was conducted as a stage of the qualitative analysis of the conducted research. As Woods (1987, p. 160, in Kostrub, 2016, p. 95) states, “it is a special form of description which requires clearing of a large amount

Development of Natural Literacy in Pre-Primary Education

47

table 2.2  Referential framework presence of development of scientifijic literacy of children in the augmented reality-supported teaching based on evidence

Integration

Contextualization

Decontextualization (socio-cognitive turn)

Coordination of points of view

General framework: children are fully self-reliant; they need no help. They develop fully. They act freely on the basis of their own considerations in accordance with the collectively defijined rules. Framework of a specifijic activity: children ask other subjects in their play group to enter into some game, they maintain interaction in the game, terminate the game, evaluate the game independently, and provide feedback on the game to other subjects (children and teachers). General framework: children are becoming self-reliant; they act autonomously but are not fully independent yet. When acting they ask their teachers for specifijic help. Framework of a specifijic activity: children together with teachers set conditions and rules for the game, while playing they ask their teacher for help, they evaluate the game partly independently and give feedback on the game with minimal teacher’s support. General framework: children learn to act independently. They alternately ask for their teachers’ help and act autonomously in which case they ask questions to check whether their actions are correct. Framework of a specifijic activity: children try to act autonomously but they ask teachers to give them feedback on their activity. Their acting is autonomous but their decisions are not. They wait for the teachers’ opinion on what to do and/or how to do it. The teacher proposes conditions of the game to children. General framework: children are partially dependent; their autonomy is minimal. In most of their action they ask for help, confront their activity with their teachers’ opinion while the teachers’ opinion is decisive. Framework of a specifijic activity: children act on the basis of their teacher’s opinion, which is determining for them. (cont.)

48

Jančaříková and Severini

table 2.2  Referential framework presence of development of scientifijic literacy of children in the augmented reality-supported teaching based on evidence. Source: author’s own elaboration (cont.)

Decentralization

The teacher defijines the particular course of the game and at the same time encourages children to help her. Children are not supported in autonomous decision making. General framework: children are not autonomous; they are fully dependent on the teacher’s decisions and they wait for their invitation and verbal as well as non-verbal instruction, presented patterns and already made decisions. They wait for feedback on correctness/ incorrectness of their actions. Framework of a specifijic activity: the teacher defijines conditions of a particular game according to in advance stated rules. The teacher’s opinion is determining for children’s activity. Children act and make decisions dependently.

of materials, analyses and massive description. They are forms of description (also in the sense of referential frameworks), they can help to understand some area of social activity” (see Table 2.2). Continuous comparison was conducted as a stage of qualitative analysis of the conducted research. When applying the method of continuous comparison, as Kolb (2012) warns, “the objective is not to verify universality or to confirm causes or other input data. This process provides saturation of the needed information. The goal of application of the method of continuous comparison is not verification but generation of a theory. What is studied is the size and saturation of some information until the right level of saturation is achieved.” Glaser and Strauss (as cited in Kolb, 2012, p. 83) claim that the method of continuous comparison has the following levels: comparison of events useful for some category; integration of categories and their properties; definition of a theory; and conception (writing) of a theory. Selective coding was conducted within the frame of the conducted research as the final stage of qualitative analysis (the process of construction and validation of categories). The central category abstracted from the research was the category “intentional support of development of scientific literacy using augmented reality.”

Development of Natural Literacy in Pre-Primary Education

49

2.2.1.8 Outcomes of the Conducted Research and Their Interpretation The perspective of perceiving the potential of pedagogical-didactical development of science literacy of children using augmented reality from the point of view of the subjects involved in the semi-structured interviews and focusgroup discussion within the frame of the research, is presented graphically (see Figure 2.10). The practicing teachers (as well as parents whose children were involved in the research) were an essential source of knowledge because it is them who can evaluate teaching activities and the whole teaching process from their point of view as something no longer acceptable, asking for some change or requiring some practical answer in the form of an adequate solution. Reflection as a form of feedback was crucially important to understand what was happening in the research: – For description of activities of subjects, for grouping of ideas and follow-up, more comprehensive ideas on what was experienced and understood. – For analysis of complex experience from the research and for making conclusions from what was gained and especially for analysis of everything that had happened. Despite the interpretational tangle of statements, it must not be forgotten that children are here, they are part of the adult world to which they must adapt;

figure 2.10  Development of scientific literacy in pre-schoolers using augmented reality with respect to the point of view of research subjects

50

Jančaříková and Severini

they must develop qualities typical for the adult population. Parents’ social responsibility and teachers’ professional roles are to support development of the scientific literacy of children as a part of their cultural literacy, also using augmented reality. What proves to be problematic is the discord in perception of importance of this support, of its profile and extent, etc. Another discrepancy is between the parents’ and teachers’ educational approaches by which they encourage, support, maintain, or block children’s autonomous activity when using applications with augmented reality for development of science literacy. It is essential to realize that many discursive and cultural practices ask for children’s autonomy and self-reliance, especially in the current postmodern society in which children live. That is why urgency of the need to change the perspective to children is emphasised. Explanations of children’s activity in the perspective of teachers involved in the research are related to mental representations (as well as socially acceptable representations) and how these actors, the teachers, personally interpret it. The activity of one actor depends on the quality of their interaction with other actors. Analysis of this interaction and its consequences in the perspective of the actors themselves allows interpretation of this interaction and making deductions. The outcomes of the conducted research from the direct and indirect observations of the teaching process using augmented reality for development of science literacy in learning/play groups of children and from semi-structured interviews and focus group discussions of the involved subject are shown graphically (see Figure 2.11). In current pedagogical/didactical practice, there is not enough support of development of scientific literacy of children with the help augmented reality. The given support is quite limited (possibly insufficient with respect to the developmental potential of the educated children) and has the form of carrying out limited and unequivocal demands, requests, instructions, directives, and manipulations, etc. – teachers do not support children explicitly but only implicitly (randomly, unconsciously, intuitively, and accidentally, etc.). That is why it proves to be essential to change how pre-primary teachers perceive the use of augmented reality in teaching when developing science literacy in pre-schoolers. 2.2.1.9 Notes from the Czech Republic The head teachers from both Czech kindergartens were rather sceptical and asked whether it was suitable at all to use augmented reality in preschool. However, attention of both groups of children was seized by the application Animal 4D+. The children spent most time experimenting with which animals will actually eat bananas.

Development of Natural Literacy in Pre-Primary Education

51

figure 2.11  Position of the teacher and children recorded in time and space presence of development of scientific literacy of children in the augmented-realitysupported teaching based on evidence

The teachers concluded that the application was an interesting supplement, making the learning process special. However, they could not imagine using it regularly in their teaching. They expressed their fears that the children would get the feeling animals were toys and would underestimate the real danger if encountering the real animal (e.g., being bitten by a dog). Namely (For the second thing I see it as dangerous that children could get the feeling it was absolutely safe to go near any animals. For example, an elephant looks like a small cute animal that we can hold in hands, a bee like an animal that can be stroked, etc.). They also expressed their worries about the effect of distortion of proportions. When some of the younger children were manipulating the tablet, the teachers were worried about the possible damage to the tablet and also to fellow students: one three-year-old child used the tablet as a weapon for hitting another child’s bottom. Another obstacle mentioned with respect to its use in the kindergarten was the language barrier: as a Czech version of the application does not exist yet, the children cannot learn more about the animal or cannot use the other opportunities the application offers.

52

Jančaříková and Severini

Having been introduced to the application “Insect” the teachers stated that in this case they saw meaningfulness of applications with augmented reality. 2.3 Findings The conducted research helped us get a repertoire of opinions on and attitudes to the selected issue from the community of pre-primary teachers in the Czech Republic and Slovakia, two different countries with similar systems of education. Interpretation and discussion helped us identify a spectrum of opinions on and views of: (1) What should be the process that would lead to a complex and intentional development of scientific literacy using augmented reality in preschool education? (2) Is scientific literacy of children in pre-primary education developed using augmented reality and if so, how? These opinions and points of view are illustrated by professional experience. At all research stages, the research sample consisted of teachers from different cultural, social, demographic and ethnic backgrounds who were interested in the research and were representatives of different kindergartens with respect to their geographical position in both countries. The subjects of our research in both countries spoke of the need for intentional didactical development of scientific literacy with the use of augmented reality similarly. The researched phenomena were present in opinions of all subjects. Another research finding is that the research subjects (preschool teachers) started to support development of scientific literacy in preschool children by using augmented reality deliberately while conducting the research. What was crucially important in this process on the teachers’ part was their creativity, imagination, sense for details and complex perspective on what can be done about teaching and in teaching. It was also confirmed that the learning and teaching processes are interrelated and complementary. The principles accepted and used in teaching were freedom, decision making, activity, guidance, responsibility, and self-assessment. The principles used in individual and collective learning were role-play, and discussion, forms of cooperative and collaborative learning. The priority of intentional didactical support of development of scientific literacy of pre-schoolers by using applications with augmented reality was autonomous and socio-constructivist understanding of the teaching process was opposed to the transmissive (academic) conception of teaching. The transmissive teaching model was identified in the initial stage of the research. Decontextualization was identified between the initial and final stages of the research (see Figure 2.11 and Table 2.2) – a socio-cognitive turn, which points at the inseparability of social and cultural aspects in interpersonal interaction and transaction in the teaching process, was observed in the phenomenon of socio-cognitive conflict (which is important to bring about socially and culturally conditioned conceptual change). The teaching process

Development of Natural Literacy in Pre-Primary Education

53

used various forms of creative mental activity which were derived from social and cultural contexts in the process of their mutual sharing – social, cultural, or natural environment. In contrast, the teaching model built on socioconstructivist theories that was identified in the final stage of our research uses strategies in which the teacher gives more space to children.

3

Conclusion

Applications with augmented reality bring about entirely new opportunities but also put new demands on teachers and pupils. If the teacher selects suitable applications and adheres to didactical principles for their use, applications with augmented reality bear high potential for development of scientific literacy in general and the preschool age is not an obstacle. However, since manipulation with a tablet is difficult for preschool children, it may be more appropriate to use special glasses for augmented reality rather than tablets.

Acknowledgement The chapter was written the support of the grant KEGA 012UK-04/2018 “The Concept of Constructionism and Augmented Reality in the Field of the Natural and Technical Sciences of the Primary Education (CEPENSAR).”

Notes 1 In fact, they are models and are used as models but teachers rarely consider them as such. 2 See https://itunes.apple.com/cz/app/anatomy-4d/id555741707?mt=8 3 See https://forio.com/simulate/e.pruyt/rabbits-and-foxes/overview/

References Čáp, J., & Mareš, J. (2001). Psychologie pro učitele. Prague: Portál. Carpenter, T. K. (2010). 7 ways augmented reality will change your brain. In Games Alfresco: In pursuit of the ultimate augmented reality experience. Retrieved from http://gamesalfresco.com/2010/02/23/7-ways-augmented-reality-will-changeyour-brain/

54

Jančaříková and Severini

Cascales, A., Pérez-López, D., & Contero, M. (2013). Study on parent’s acceptance of the augmented reality use for preschool education. Procedia Computer Science, 25, 420–427. Chang, S.-K., Tauber, M. J., Yu, B., & Yu, J.-S. (1989). A visual language compiler. IEEE Transactions on Software Engineering, 15(5), 506–525. Contero, M., Cascales-Martínez, A., Laguna, I., Pérez-López, D., & Perona, P. (2012). Augmented reality for preschoolers: An experience around natural sciences educational contents. Proceedings of Multidisciplinary Symposium on the Design and Evaluation of Digital Content for Education (SPEDECE 2012), Alicante, Spain. Creswell, J. W. (1998). Qualitative inquiry and research design: Choosing among five traditions. Thousand Oaks, CA: Sage. Ferrer, M. L., & Jiménez, P. C. (2006). Propuesta estructural para la construcción metodológica en investigación cualitativa como dinámica del conocimiento social. Revista Venezolana de Información, Tecnologia y Conocimiento, 3(3), 33–50. Gavora, P. (2007). Sprievodca metodológiou kvalitatívneho výskumu. Bratislava: Vydavateľstvo Univerzity Komenského. Gavora, P. (2008). Úvod do pedagogického výskumu. Bratislava: Vydavateľstvo Univerzity Komenského. Hamilton, K. E. (2011). Augmented reality in education. WikEd. Retrieved from http://wik.ed.uiuc.edu/index.php/Augmented_Reality_in_Educationá Han, J., Jo, M., Hyun, E., & Hyo-Jeong, S. (2015). Examining young children’s perception toward augmented reality-infused dramatic play. Education Technology Research and Development, 63(3), 455–474. Hanel, L., & Hanelová, J. (2014). Shield bugs of the genus Elasmucha – A model insect group for parental care demonstration within field excursions in Central and Northern Europe. Envigogika, 9(1). Hernández Gil, T. (2007). El Aprendizaje Comprensivo y Creativo a Partir de la Investigación-Acción Como Estrategia Didáctica Epistémica en la Educación Básica. Laurus – Revista de Educación. Universidad Pedagógica Experimental Libertador, 13(23), 11–35. Hortin, J. (1980). Visual literacy and visual thinking. Educational Resources Information Center. Jančaříková, K. (2015). Didaktické přístupy k přírodovědnému vzdělávání předškolních dětí a mladších žáků. Praha: Univerzita Karlova v Praze, Pedagogická fakulta. Jančaříková, K., & Jančařík, A. (2016). Work with models in e-learning environments. In J. Novotná & A. Jančařík (Eds.), European Conference on e-learning (pp. 305–314). Praha: Academic Conferences International Limited. Johnson, A. E., Roussou, M., Leigh, J., Vasilakis, C., Barnes, C., & Moher, T. (1998). The NICE project: Learning together in a virtual world. In Proceedings of virtual reality Annual International Symposium (pp. 1–8). Washington, DC: IEEE.

Development of Natural Literacy in Pre-Primary Education

55

Kaufmann, H. (2004). Geometry education with augmented reality. Vídeň: Technická univerzita Vídeň. (dizertační práce) Retrieved February 09, 2012, from http://www.ims.tuwien.ac.at/media/documents/publications/kaufmann_diss.pdf Kemmis, S., & McTaggart, R. (1982). The action research planner. Victoria: Deakin University Press. Kolb, M. S. (2012). Grounded theory and the constant comparative method: Valid research strategies for educators. Journal of Emerging Trends in Educational Research and Policy Studies, 3(1), 83–86. Kosslyn, S. M. (1989). Understanding charts and graphs. Applied Cognitive Psychology, 3, 185–226. https://doi.org/10.1002/acp.2350030302 Machková, V., & Bílek, M. (2013). Didactic analysis of the web acid-base titration simulations applied in pre-graduate chemistry teachers’ education. Journal of Baltic Science Education, 12(6), 829–839. Mayan, J. M. (2001). Una introducción a los métodos cualitativos: Módulo de entrenamiento estudiantes y profesionales. México: UAM-Iztapalpa. OECD. (2007). PISA 2006. Science competencies for tomorrow’s world (Vol. I, Analysis). Paris: OECD. Piaget, J. (1973). The child and reality: Problems of genetic psychology (A. Rosin, Trans.). Oxford: Grossman. Plichtová, J. (2002). Metódy sociálnej psychológie z blízka. Bratislava: Média. Rohaya, D., Rambli, A., Matcha, W., Sulaiman, S., & Nayan, M. Y. (2012). Design and development of an interactive augmented reality edutainment storybook for preschool. IERI Procedia, 2, 802–807. Santos, M. E. C., Chen, A., Taketomi, T., Yamamoto, G., Miyazaki, J., & Kato, H. (2014) Augmented reality learning experiences: Survey of prototype design and evaluation. IEEE Transactions on Learning Technologies, 7(1), 38–56. Skalková, J. (2007). Obecná didaktika. Prague: Grada. Slavík, J. (2001). Mirror curves. In R. Sarhangi & S. Jablan (Eds.), Bridges: Mathematical connections in art, music, and science (pp. 233–246). Winfield, KS: Bridges. Strauss, A., & Corbin, J. (1999). Základy kvalitativního výzkumu. Boskovice: Albert. Švec, Š. (1998). Metodológia vied o výchove. Kvalitatívno-scientické a kvalitatívnohumanistické postupy v edukačnom výskume. Bratislava: Iris. Wittrock, M. C. (1989). La investigación de la enseñanza, II: Métodos cualitativos y de observación. Barcelona: Paidós/MEC. Woods, P. (1987). La escuela por dentro: La etnografía en la investigación educativa. Buenos Aires: Editorial Paidós.

CHAPTER 3

Empowering Teachers to Augment Students’ Reading Experience: The Living Book Project Approach Maria Meletiou-Mavrotheris, Constadina Charalambous, Katerina Mavrou, Christos Dimopoulos, Panayiota Anastasi, Ilona-Elefteryja Lasica, Nayia Stylianidou and Christina Vasou

Abstract This chapter discusses an attempt to empower teachers to ‘augment’ students’ reading experiences as part of the project The Living Book – Augmenting Reading for Life (Erasmus+). The project’s overall aim is to address the under -achievement of European students in reading by developing an innovative approach that combines offline activities promoting reading literacy with online experiences of books’ ‘virtual augmentation.’ More specifically, recognizing the important role of teachers in any educational reform effort, the project aims at strengthening teachers’ profile and competences in adopting the Living Book approach and in dealing with diversified groups of learners, and particularly with pupils from disadvantaged backgrounds, through a series of training sessions. The chapter outlines the theoretical premises of the Living Book and provides an overview of the theoretical framework underlying the design of the ‘Augmented Teacher’ professional development course. It also describes the content and structure of the course, and the process of evaluation currently taking place in the project partner countries.

Keywords reading literacy – augmented reading – augmented reality – professional development – living library

© koninklijke brill nv, leiden, 2020 | doi: 10.1163/9789004408845_003

Empowering Teachers to Augment Students’ Reading Experience

1

57

Introduction

Poor reading performance combined with a general disinterest for reading has long-term consequences for both individuals and society. In modern societies, reading literacy is a primary skill allowing citizens to ‘be’ and live in a complex world, to work at higher levels, and to enjoy their lives more fully (European Commission, 2012). As schools are usually the main official institutions responsible for the development of reading literacy, children who leave school with poor reading performances are not only at great risk of exclusion from the labour market, but are also in effect excluded from further education and from full participation in a knowledge-based society. For example, the 2013 study by the University College of London (UCL) Institute of Education (IOE) ‘Social inequalities in cognitive scores at age 16’ shows that regular access to books between the age of 10 and 16 drives up standards in mathematics (Sullivan & Brown, 2013). Students with poor reading performance at school,1 in fact, seem to also have serious difficulties in understanding written mathematics problems, thus seriously compromising their chances to get into Science, Technology, Engineering, and Mathematics (STEM) education. Recognizing the importance of reading, the EU Education and training 2020 (ET 2020) benchmarks set in 2009, included having fewer than 15% of 15-year-olds being under-skilled in reading by 2020 as one of its main targets. Unfortunately, while some EU countries have since made significant progress towards improving their students’ performance in reading skills, other countries are lagging behind. The percentage of low achievers in reading at the EU level has actually grown in recent years, from 17.8% in 2012 to 19.7% in 2015, wiping out all the progress made since 2009, when it was also 19.5% (European Commission, 2016). According to the PISA (2015) results, about 50% of the EU countries that participated in the study had significantly low performance in basic science, reading and mathematics skills, while only two (2) European countries (Estonia and Finland) were included within the top-10 rated countries globally (European Commission, 2016). In an attempt to respond to this challenge, the Erasmus+/KA2 project The Living Book – Augmenting Reading for Life project (September 2016–August 2019) was designed to address the under-achievement of European students aged 9–15 in reading skills. The project consortium is comprised of nine partner organizations originating from six different EU countries (Cyprus, Estonia, Italy, Romania, Portugal, UK). In most of the partner countries, the proportion of low-achievers in reading is still dramatically high: for example, it is 35.6% for Cyprus, 38.7% for Romania, and 21.0% for Italy. Groups with low performance tend to be students from socio-economically disadvantaged families, with less

58

Meletiou-Mavrotheris et al.

well-educated parents and with immigrant backgrounds. The project’s overall aim is to increase these young people’s motivation to read and to contribute to meeting the 2020 EU target of reducing the number of underachievers in partner countries and in the EU overall to below 15%, while at the same time also boosting a cluster of other key and transversal competencies in students (e.g., digital skills, learning to learn, critical thinking, co-operative skills). Living Book aspires to achieve this through the development of a novel pedagogical approach that combines offline activities promoting reading literacy with online experiences of ‘virtual augmentation’ of a book and with social dynamics leading to the creation of European communities of young ‘Augmented Readers.’ Living Book has been reaching out to teachers and parents to inform them about new technological developments that could be utilized in primary and secondary schools to augment students’ reading experience with rich media content. Through a combination of Open Educational Resources (OER) and involvement in professional learning activities and pilot experimentation, Living Book has been contributing towards strengthening the profile and competences of European teachers from upper primary and lower secondary schools (ages 9–15) in adopting standards-based practices and dealing with diversified groups of learners, and particularly with pupils from disadvantaged backgrounds. A Living Library (http://thelivinglibrary.eu/) supports the project activities by offering students, parents, and teachers with online tools to augment the reading experience with rich media content. A series of OER toolkits (downloadable guides, videos, links to tools) are being integrated in this multilingual platform, and will be used by students and teachers to curate and create content. The Living Library also hosts a social community of ‘Augmented Teachers’ and ‘Augmented Readers’ across Europe. In the next sections, we outline the theoretical premises of the Living Book project, and describe the ‘Augmented Teacher’ professional development course design. We also provide a brief overview of the pilot testing and followup classroom experimentation process currently taking place in each partner county to evaluate the level of success of the professional development program.

2

Theoretical Framework

Through the Living Book project we designed a professional development program which involved AR enhanced learning activities, through combined use of eLearning and physical meetings. The theoretical framework2 of this

Empowering Teachers to Augment Students’ Reading Experience

59

programme was grounded on and structured under six interrelated bodies of research: (i) Augmented reality technologies in education and reading; (ii) Sociocultural theory and Communities of practice; (iii) Constructivism, Constructionism, and Dialogic Learning; (iv) Contemporary Literary Theories and Theories of Literacy; (v) the Technological, Pedagogical, Content Knowledge (TPACK) Conceptual Framework; (vi) Principles of Adult Education. Each of these is next briefly described, to outline their basic theoretical premises as well as their specific application in the design of our professional development courses. 2.1 AR and AR Books Recent technological advances have provided the opportunity to create an entirely new learning environment that can boost students’ reading engagement by significantly increasing the range and sophistication of possible classroom activities. One promising approach lately explored is the use of AR. This concept has gained a growing interest among researchers during the past few years, especially in the field of education (Cheng & Tsai, 2014; Dünser, 2008). In particular, AR books, which combine physical books with the interactive potentials provided by digital media, constitute an interactive, playful and engaging way for enhancing teaching and learning. Most importantly, through the integration of text, audio, 2D illustrations, 3D virtual content and animation, AR books can meet students’ diverse needs and different learning styles. For instance, visual learners’ needs can be addressed through 2D and 3D illustrations, while auditory learners can hear sounds throughout the book and kinaesthetic learners can engage in tactile activities utilising mobile devices (McNair & Green, 2016). In line with this, Rohaya, Rambli, Matcha, Sulaiman, and Nayan (2012) have stated that a static story experience can be transformed into a dynamic and engaging reading experience through the incorporation of AR into a physical book. It is then possible that AR books, being a physical means of interaction, can lead students to intuitive utilisation of paper books through the assistance of AR (Hornecker & Dünser, 2009). Despite the increased interest on AR books as learning tools, the number of available primary research studies on their integration into teaching and learning is still relatively small due to the novelty of these technologies. Research on the actual impact of AR technologies on student learning outcomes is still scarce and its findings inconclusive. However, the conducted studies do overwhelmingly point towards numerous positive attributes that have the potential to enhance both formal and informal student learning. The findings of several studies (e.g., Gu, Sanchez, Kunze, & Inami, 2015; Martín-Gutiérrez & Contero, 2011; Solak & Cakir, 2015; Yilmaz, Kucuk, & Goktas, 2016) indicate that

60

Meletiou-Mavrotheris et al.

the integration of AR books in instruction is highly engaging, motivating, and fun for the majority of students and helps improve their story comprehension performance. Studies suggest that the affordances provided to students through the wide range of modalities that AR books offer, can support all students (including students with reading difficulties, emotional and behavioural difficulties, low motivation, etc.) as ‘integrating text, audio, 2D illustrations, 3D virtual content, and animation allows students to learn according to their preferred learning style’ (Dünser, 2008, p. 40). As indicated by the research literature, AR books can play a vital role in different areas of student motivation and learning. They can help to make the learning process more enjoyable (Núñez, Quirós, Núñez, Carda, & Camahort, 2008), interesting, and effective (Wojciechowski & Cellary, 2013), thus increasing engagement and motivation (Di Serio, Ibáñez, & Kloos, 2013). They can also lead to improved understanding of concepts (Klopfer & Squire, 2008) through utilising visualisation to make abstract concepts more tangible and concrete (Dori & Belcher, 2005) and to aid comprehension (Kaufmann & Schmalstieg, 2003). AR also focuses students’ attention on the learning process (Aziz, Aziz, Yusof, & Paul, 2012), providing more authentic learning (Ternier, Klemke, Kalz, Ulzen, & Specht, 2012; Yuen, 2011) which can be useful for accommodating multiple learning styles, and for helping students develop contextual awareness (Ivanova & Ivanov, 2011), and critical thinking and problem-solving skills (Dunleavy, Dede, & Mitchell, 2009). Although AR seems an exciting and interesting medium with multiple possibilities for learning, there are also several limitations identified in the literature, including the following: students’ frustration if they find it difficult to use this technology or if the applications are not working appropriately; students’ distraction caused by the virtual information when presented to them for the first time; disruption of natural interaction with others when using HDM (head-mounted displays); teachers’ lack of skills in creating learning material incorporating AR. Dunleavy et al. (2009), for example, found that it is challenging to apply AR in learning without first familiarizing students to the uses of AR. Within a design-based research project, the researchers conducted multiple qualitative case studies across two middle schools (6th and 7th grade) and one high school (10th grade) in the north-eastern United States to document the affordances and limitations of AR simulations from the students’ and teachers’ perspectives. Participating teachers and students reported that the technology-mediated narrative and the interactive, situated, collaborative problem-solving affordances of the AR simulation were highly engaging, especially among students who had previously presented behavioural and academic challenges for the teachers. However, while the AR simulation provided

Empowering Teachers to Augment Students’ Reading Experience

61

potentially transformative added value, it simultaneously presented unique technological, managerial, and cognitive challenges to teaching and learning. The authors report that students participating in their study experienced a cognitive overload, as there were instances where they were overwhelmed and confused with the amount of information provided to them and with the tasks they had to complete. In addition, there were instances where students became so engaged within the game environment of the AR simulations that they lost track of the real environment, something which can constitute a threat to students’ physical safety. Lin et al. (2013; cited in Cheng & Tsai, 2014) state that there is a dearth of studies exploring AR-related learning processes in depth, indicating that there is little empirical evidence for best applications of AR. The same applies for AR books: Researchers argue that there is lack of knowledge regarding the effectiveness of AR books as instructional tools, as most of the research related to AR books has mainly focused on the technological developments and not on the ‘how, what, and why’ (Dünser & Hornecker, 2007, p. 179). Hence, although technological innovations such as AR call for their adoption within the field of education, at the same time it is of vital significance to be aware of potential pitfalls such as technological determinism (Wyatt, 2008) which involves the use of technology for the sake of technology, leaving pedagogical approaches aside. As Bronack (2011; cited in Wu, Lee, Chang, & Liang, 2013, p. 43) argues: ‘these technologies themselves are not important for educational researchers. More important is how the technologies support and afford meaningful learning.’ For AR technologies to be more effectively utilised to enhance learning opportunities for all learners, there needs to be a re-conceptualisation of the design and management of learning environments. Careful strategic planning and reflective implementation, grounded in solid research, is necessary. This should focus on the broad preparation and ongoing engagement of all key stakeholders involved in the educational process (prospective and practicing teachers, teacher educators and other college faculty, adult educators, educational leaders, technical managers, etc.). These professionals should be helped to recognize the added value of ground-breaking technologies such as AR for books, and their true potential for improving teaching, learning, and assessment, and should be informed about best practices in their exploitation as instructional tools. The provision of high quality teacher training, in particular, is of paramount importance (Henderson & Yeow, 2012). A number of research studies have asserted that it is much more demanding for teachers to exploit the growing prominence of AR and other digital technologies and their transformative potential in instructional settings than was originally anticipated,

62

Meletiou-Mavrotheris et al.

and that many teachers remain unprepared to effectively employ Information Communication Technology (ICT) tools in their teaching practices (e.g., Blackwell, 2014; Ertmer, Ottenbreit-Leftwich, Sadik, Sendurur, & Sendurur, 2012). Thus, to bring about the necessary changes in teaching cultures that will enable education to reap the full benefits of AR, it is of utmost importance to provide teachers with high quality pre-service and in-service training opportunities that will equip them with the required knowledge and skills to effectively infuse AR and other emerging technologies into teaching and learning. The role of teachers’ preparation for using AR is highlighted in Clarke (2013), Dunleavy et al. (2009), Solak and Cakir (2015), Dellelo (2014), and McNair and Green (2016). Dellelo (2014) explored pre-service teachers’ perceptions regarding the utilisation of the AR platform Aurasma. The results of the study indicated that teachers frequently perceive novice technologies as disruptive and they tend to reject them. This is connected with teachers’ associated fear and lack of confidence in using new technologies (Ertmer & Ottenbreit-Leftwich, 2010; Ertmer et al., 2012; Moeller & Reitzes, 2011). As one of the pre-service students in Dellelo’s study reported, ‘the reason that more teachers are afraid of these kinds of things is because they have no idea what to do with them or proper training on how to create and use them’ (2014, p. 306). In addition to fear and lack of confidence, another challenge also identified through this study was teachers’ lack of time or motivation for learning and applying new technological skills. In this frame of thought, Dellelo supports that ‘For those (teachers) who did not grow up immersed in or feel confident in using technology, it is vital that support, time, and training be provided in order to integrate technology and pedagogy effectively into classrooms’ (p. 307). These findings are in line with those of McNair and Green (2016), as the student-teachers participating in their study reported that they had to deal with teachers’ and administrators’ negativity towards the application of AR technologies in the classroom and reluctance to apply AR. In dealing with these challenges, Solak and Cakir (2015) have proposed that teachers should begin to get accustomed to AR technologies either via in-service training or by visiting schools. In a similar vein, Dunleavy et al. (2009) underlined the need for teachers’ preparation in using AR and the different pedagogical strategies that this task entails. For instance, learning in an AR environment necessitates a move away from a teacher-centred approach to a learner-centred one (Dellelo, 2014), a notion that makes teachers who depend on a lecture-practice style uncomfortable with this new style of learning provided by AR environments (Dunleavy et al., 2009). Departing from this point, teachers emerge as key persons in beginning to understand the how, what and why of AR books, as they need to provide the

Empowering Teachers to Augment Students’ Reading Experience

63

necessary instructional support to students, decreasing in this way the limitations of AR. However, for teachers to be in this position, training that involves applying AR and other innovative technological tools through standards-based pedagogical approaches seems to be a priority. A renovated effort in providing continuing professional development in the uses of AR is needed, as well as providing teachers with new stimulus to tackle groups at risk. The following section presents the theoretical framework for designing such a professional development programme. 2.2 Sociocultural Theory and Communities of Practice The theoretical underpinnings of this project are located within the sociocultural theory in general, and communities of practice in particular (Hoadley, 2012; Vygotsky, 1978; Wenger, 1998). In Living Book, online communities of ‘Augmented Teachers’ and ‘Augmented Readers’ (both across Europe as well as locally for each partner country) are currently being formed. The process of facilitating communities both locally and through the internet has been contributing towards establishing rapport and sharing experiences among participants. The project Living Book Library (http://thelivinglibrary.eu/) works towards this end, since it has been implemented as an interactive multilingual web portal, which includes space for discussions, chat rooms and application sharing between European teachers and students. If successfully implemented, the Living Book online communities are expected to provide a comfortable atmosphere in which community members (students, teachers, and teacher educators) will be encouraged and able to articulate their own ideas, challenge those of others, and negotiate deeper meanings. As the research literature indicates, when ideas are exchanged and subjected to thoughtful critiques, they are often refined and improved (Kimble, Hildreth, & Bourdon, 2008). 2.3 Constructivism, Constructionism, and Dialogic Learning Constructivism (Piaget, 1952) implies the need for instructional environments to be student-centred, student-directed, collaborative, teacher-scaffolded, and authentic. It hosts a variety of pedagogical approaches and terminology. A theoretical construct particularly relevant to Living Book is that of constructionism, a term first proposed by Papert, seeking to combine the constructivist psychology of Piaget and the ideas of progressive education exemplified by Dewey (Dewey, 1902; Papert, 1980; Papert & Harel, 1991; Resnick, 2013). It emphasises that an effective way for the learner to construct knowledge is to build something ‘tangible,’ a meaningful product. In accord with constructionist views of learning, the Living Book project combines customised reading paths with digital creativity for fostering 9- to

64

Meletiou-Mavrotheris et al.

15-year-old students’ reading engagement, in particular for those at high risk of underperformance. The project has been exploring various digital solutions, included also in the training course, in order to enable teachers to ‘augment’ the experience of students while reading a book. Inspired by the Internet of Things, it connects the physical book to the digital world to turn it into a ‘living book.’ The aim is to create an intense lived experience for young learners, and guide them to transform and augment what they are reading, thus applying digital competences, collaborating with peers, developing reading skills and, ultimately, being more engaged in reading. The training course includes also parent education materials, so that teachers can engage parents in this process, strengthening the connection between school and home. 2.4 Contemporary Literary Theories and Theories of Literacy Given the thematic focus of Living Book on pro-reading activities at school and home, there is a strong influence of theoretical approaches focused on the teaching of literature both as an autonomous course as well as a part of language and literacy courses. Living Book also perceives literature as a useful tool for implementing an interdisciplinary approach to education, since literary texts can be used in other fields of the curriculum, like music, art, physical education, even mathematics and science. The teaching of literature, as envisioned by Living Book is highly influenced by contemporary literary theories, including reader-response theory and reception theory (Iser, 1978; Fish, 1980; Jauss, 1982). According to reception theory, the teacher no longer controls the authority of the meaning of the text. Instead, he/she is equal to students and his/her role is to converse, encourage, and regulate the process of the teaching of a literary work. Reading in the classroom is an act of exploration and discovery and the teacher’s role is to acknowledge the uniqueness of the reader and of each different reading, accepting and encouraging the differences. Rosenblatt’s (1978) transactional theory has also been influential, emphasising the transactional process between a reader and a text. According to the transactional theory, a text may be read efferently or aesthetically for different purposes, and parts of the same text may elicit different stances. The teacher is responsible for developing students’ awareness of efferent and aesthetic stances. Students express their interpretations of literary work through individual work or teamwork. Creative writing, dramatization and art projects can be used as a means of expression. In general, teachers act like moderators: they invite responses, give ideas and time to crystallize and encourage students to reflect upon their responses. In addition, they find patterns in students’ interpretations and points of convergence and encourage discussions on the different points of view.

Empowering Teachers to Augment Students’ Reading Experience

65

In addition to contemporary literary theories, Living Book is also premised on theories of literacy which regard literacy as the ability to function efficiently in multiple environments and communicative contexts, by using oral and written language, or other multimodal means, as well as knowledge, and abilities for achieving social goals. Approaching literacy as social and cultural practice, in line with New Literacy Studies, we pay particular attention to different practices and different literacy events that can take place both in the classroom and in students’ homes and communities (Street, 1995). Drawing also on the theory of critical literacy (e.g., Luke, 2012; Street, 2003), we emphasised the development of critical thinking and language awareness together with an awareness of how ideologies are manifested and constituted through language practices (Jones, 2006; Powell, Cantrell, & Adams, 2001). In order to account for contemporary developments such as (i) technological growth, which has given rise to diverse communication technologies and different types of texts (e.g., digital texts, multimedia, etc.), and (ii) the contact between different languages, which has increased as a result of cultural and linguistic diversity, ‘multiliteracies’ has emerged as a new approach to literacy (Cope & Kalantzis, 2009). Adopting this approach, we recognize that oral and written language are no longer the only means of communication, and acknowledge that new types of literacy have appeared or emerged as part of students’ cultural backgrounds. This entails a significant shift in what a ‘literate’ student looks like and a recognition that there is actually not one ‘thing’ that can be defined as literacy. Instead, the multiplicity of literacies is highlighted. For example, media literacy, digital literacy, computer literacy, political literacy, cultural literacy, and visual literacy are some types of literacy that might be important in contemporary societies. Through the Living Book project, it is expected that teachers can engage students in different types of literacy events that cultivate different forms of literacy, combining traditional practices with more innovative ones, including the use of AR. 2.5 Technological, Pedagogical, Content Knowledge (TPACK) Framework The Living Book teacher professional development course design is guided by TPACK, the conceptual framework proposed by Mishra and Koehler (2006) in response to the absence of theory guiding the integration of technology into education. Building on Shulman’s (1986) idea of Pedagogical Content Knowledge, TPACK emphasizes the importance of developing integrated and interdependent understanding of three primary forms of knowledge: technology, pedagogy, and content. The framework is based upon the premise that effective technology integration for pedagogy around specific subject matter requires developing understanding of the dynamic relationship between all three knowledge components. Thus, teacher ICT training cannot be treated

66

Meletiou-Mavrotheris et al.

as context-free, but should be accompanied with emphasis on how technology relates to the pedagogy and content (in our case teaching literacy & literature). The aim is to move teachers beyond technocentric strategies that focus on technology, and to promote their critical reflection on the instructional use of ICT. TPACK has, in recent years, become central to research into technology education and teacher professional development in many different disciplines (e.g., McKenney & Voogt, 2017; Schmidt & Gurbo, 2008; Tondeur, Pareja Roblin, van Braak, Fisser, & Voogt, 2013; Voogt & McKenney, 2016). In the area of literacy education, several studies targeting pre-service and/or in-service teachers undertaken during the past decade have been grounded in the TPACK model (e.g., McKenney & Voogt, 2017). Conducted studies illustrate the usefulness of TPACK as a research framework for facilitating and assessing teachers’ professional growth in the instructional use of ICT for the development of students’ literacy in young learners. As suggested by the literature, better understanding of TPACK among pre-service and in-service teachers can help enhance integration of technology in their teaching practices, and this, in turn, can foster early literacy. Despite the usefulness of the generic TPACK model, some limitations and challenges do exist. In particular, the basic TPACK model’s individual-oriented focus is a drawback, since it fails to take into account the socially mediated contexts in which teachers develop their TPACK. Concurring with Phillips (2013), the Living Book project considers TPACK not as an individually acquired attribute but as an embodied phenomenon shaped by social, organizational, and cultural factors extending beyond individuals. The project has adopted a more systematic approach to examining and extending teachers’ TPACK, by putting emphasis on the socially mediated contexts in which teachers develop their TPACK. The teaching experimentation and research to be conducted in partner schools will help to fill a serious gap in technology-enhanced learning research, pointed out by Beavis, Muspratt, and Thompson (2015): the lack of research that accounts for the realities of school. Research in actual school settings can help overcome researchers’ and policymakers’ tendency to overlook the difficult conditions under which most teachers and students are operating in schools when investigating ‘state-of-the-art’ technologies. 2.6 Principles of Adult Education Finally, the design of our professional development course was based on some principles of adult education. Although there is no specific theory about learning in practice, several studies have been conducted over the past decades to investigate the ways in which adults develop the required knowledge and skills to effectively function in everyday life and in work situations. The main

Empowering Teachers to Augment Students’ Reading Experience

67

conclusions of these studies are the following (taken from van Groenestijn, 2007): (i) Adults are free to learn; there is no compulsory education for adults. (ii) Learning happens in a functional situation; there is a need for learning. (iii) Learning in practice is characterized by learning through authentic materials. (iv) Knowledge acquired in practice is almost always functional and applicable (‘knowledge-as-a-tool’ rather than ‘knowledge-for-knowledge’). (v) Every learning situation is a socio-culturally determined situation: Learning is an interactive and social act in which everybody takes part. (vi) Learning in practice focuses on ‘shared cognition,’ rather than on ‘individual cognition.’ In work settings, employees are often complementary to one another, asking questions, discussing the problems they meet, and jointly seeking solutions. (vii) The way in which learning in practice takes place is often via showing – imitating – participating, and applying: There is no need to create specific instructional settings. (viii) For learning in practice people construct or re-construct their own ‘rules-of-thumb’ and informal ‘rules and laws’ for managing actions, situations, materials and the environment in which they work. Thus, to make training relevant and attractive for adult learners, they should experience it as usable and applicable to their actual lived-in situations. Moreover, adult training should aim at enabling learners to broaden the perspectives by which they develop their competencies for more effective functioning in their real-life situations and for further learning (van Groenestijn, 2007). Drawing upon the relevant literature, the Living Book training uses adultappropriate teaching strategies. Rather than adopting a transmission-ofknowledge instructional model, the teacher training course has been designed to facilitate inquiry and problem-based learning. Educators participating in the course are responsible for their own learning, facilitated by an environment rich in challenges and interactions. Particular emphasis is being put on drawing upon and extending teachers’ workplace experiences. The teacher training will be followed by experimentally implementing the use of literacyrelated AR pedagogical approaches in the classrooms of the participants. We believe that this can help to further determine the actual educational potential of the Augmented Reading pedagogical approach adopted by Living Book.

3

Implementing the Theoretical Framework – ‘Augmented Teacher’ Course Design

Combining the aforementioned theories and principles, the consortium spent the first 18 months of the project (September 2016–February 2018) developing the content of the ‘Augmented Teacher’ course and its accompanying

68

Meletiou-Mavrotheris et al.

resources. The course was jointly designed by a multinational consortium of educators, representatives of teachers’ organizations, experienced distancelearning instructors, authors of technology-supported courses, and technicians, in order to ensure consideration of all different perspectives into its design. The course design was based on the importance of dialogue and collaboration among teachers and researchers, and of inquiry and exploration as a process of knowledge construction. Particular care was taken to build on teachers’ knowledge and workplace experiences. We next provide a brief overview of the ‘Augmented Teacher’ course content and structure. 3.1 Course Content Taking into account best practices in literacy education, adult education, teacher education, and blended learning, the ‘Augmented Teacher’ course aims to enrich European upper elementary and middle school children’s (aged 9–15) experiences in reading through developing their teachers’ knowledge and skills in teaching using the Living Book approach. The course is made of eight modules, covering the following topics: – Module 1: The Living Book approach: why it is important to motivate to reading and to augment books. The learning outcomes (in person) – Module 2: Pedagogical basis to teach to read (in person) – Module 3: Motivating disengaged pupils from groups at risk (in person) – Module 4: Using the Living Library platform and exploring the tools available (video classes) – Module 5: How to apply the Living Book approach in the classroom and how to match it into the curriculum (in person) – Module 6: Assessing the reading skills and the competence of the ‘Augmented Reader’ (in person) – Module 7: Preparing for, conducting, and reflecting on the teaching experimentation (guided-field practice) – Module 8: Practical course tasks, self-assessment of learning outcomes, and self-generation of the Augmented Teacher Certificate (interactive resource). The ‘Augmented Teacher’ course has been designed to promote teaching to read as a transversal skill for all educators regardless of discipline. Teachers taking the course get familiarized with ways in which the Living Book methodology can help foster students’ motivation towards reading and improve their reading skills, while at the same time strengthening the development of a cluster of other key and transversal competencies such as digital skills, learning to learn, critical thinking, cooperative and collaborative skills. Special focus has been given to ways of increasing the level of participation and achievement of

Empowering Teachers to Augment Students’ Reading Experience

69

the most unmotivated learners from disadvantaged backgrounds. Central to the course design is the functional integration of technology with existing core curricular ideas, and specifically, the integration of Augmented Reality and other technology-enhanced tools and resources provided by the Living Library platform. These include the following: – Living Book Guidelines: E-publication providing teachers (and other stakeholders) with methodological guidelines to implement Living Book approach – A Pedagogical Framework and Curriculum Definition: Layout of structure of different modules, phases, methodologies, and learning outcomes for teachers; description of main topics to be treated – Instructional Contents: A line of research-based curricular and instructional materials to be used during the professional development course – Recorded Video Classes – Living Book Lesson Plans for teachers: Plans will provide teachers with examples and ideas on how to integrate the Living Book methodologies into their classroom activities – Collaboration Tools for professional dialogue and support (e.g., forums, wikis, chats). A special emphasis of the Living Book teacher professional development course is on building an online community for the exchange of ideas, content, tools, and didactic approaches among the European educators participating in the training. Throughout the course duration, teachers across EU countries participate in online discussion forums, exchanging experiences, ideas, and educational resources. The course curriculum and key contents have been developed in English and translated into the partners’ national languages (Estonian, Greek, Italian, Portuguese, Romanian). They have been culturally differentiated to accommodate local conditions in each participating country. The professional development course is being delivered using a blendedlearning method as described next. 3.2 Course Structure The course has adopted a blended-learning method, combining physical with online meetings (synchronous and asynchronous), and pre-recorded video classes. It is made up of three parts: – Face-to-face training seminars: At the beginning of the course, teachers in each country gather together to attend a series of face-to-face seminars covering Modules 1, 2, 3, 5, and 6. Course participants get familiarized with the project philosophy, objectives, and resources. More importantly, they get the

70

Meletiou-Mavrotheris et al.

chance to meet and interact with one another, share issues and problems, as well as exploit the course facilitators’ presence to ask questions about things they are unsure of. This initial in-person meeting reinforces teacher online engagement (Kavanaugh, Carroll, Rosson, Zin, & Reese, 2005) since it mitigates the problem of trust and social presence online (Ardichvili, Page, & Wentling, 2003). The seminars consist of a combination of mini-workshops that will include AR-enhanced and hands-on activities in small groups (5–6 persons), presentations by experts, role-play, videos documenting learning activities with children, and discussions. The language of tuition is the national language of each partner country. – ICT-mediated instruction: The second part covers Modules 4 and 8 of the course, and is being delivered online, utilising the project Living Library. Module 4 includes pre-recorded video classes that acquaint participants with the Living Library platform and the tools available. Module 8 is available as an online interactive page that includes a final practical course task, the self-assessment of learning outcomes and self-generation of the Augmented Teacher Certificate (interactive resource). – Guided field practice: At a final stage, teachers undertake a teaching experiment (Module 7 of the course). They customize and expand upon the lesson plans and learning materials provided, and apply them in their own classrooms. Partners act as mentors, providing their support to teachers using online communication tools. Teachers write up their experiences, including a critical analysis of their work and that resulting from students. This helps them to reflect on their practice and apply self-criticism constructively. Once the guided field practice is completed, they report on their experiences to the other teachers, and also provide video-taped teaching episodes and samples of their students’ work for group reflection and evaluation. Teachers exchange ideas and insights as to how to further improve their teaching practices and increase their students’ achievement. A special emphasis of Living Book is on building an online community for the exchange of ideas, content, tools, and didactic approaches among the teachers participating in the training. Throughout the course duration, teachers across countries participate in online discussion forums, exchanging experiences, ideas, and educational resources.

4

‘Augmented Teacher’ Course Evaluation

Following the design of the ‘Augmented Teacher’ professional development program, a pilot testing of the course is currently taking place in each partner country. Around sixty (60) teachers from Cyprus, Estonia, Italy, Romania, and

Empowering Teachers to Augment Students’ Reading Experience

71

Portugal participate in the pilot-testing. The pilot testing started in the Spring 2018 semester with a series of hands-on professional development seminars, and will continue during the current academic year. In order to evaluate the applicability and success of the training modules, teachers participating in the Augmented Teacher course, will subsequently undertake a teaching experiment during the school year (Fall 2018–Spring 2019), where they will activate Living Book didactical paths. Teachers will customize and expand upon provided lesson plans and other instructional materials, and apply them in their own classrooms with the support of the local consortium team members. During the pilot teaching experimentation, they will also organize activities targeting parents (e.g., parent training workshops, family nights, etc.). The experimentation will be enriched with four (4) short-term exchanges of groups of pupils among the schools involved in the project. A total of approximately 800 students and 80 parents are expected to get involved in the pilot experimentation. The success of the Living Book program in increasing teachers’ level of competence in dealing effectively with reading difficulties and in cultivating young students’ motivation to reading, while at the same time building other transversal competences, will be evaluated using Guskey’s (2002) 5-level hierarchical model. According to Guskey (2002), professional development evaluation should move from the simple (reactions of participants), to the more complex (student learning outcomes), with data from each level building on the previous. Based on Guskey’s model, evaluation will occur at five levels, employing a variety of both qualitative and quantitative data collection techniques to gather information from teachers and their learners, as well as from parents: (i) participant reactions, (ii) participant learning, (iii) organization support and change, (iv) participant use of new knowledge and skills, (v) student motivation and learning outcomes. Level 5 addresses ‘the bottom line’ of professional development. It is concerned with the impact of professional development on student achievement, performance, attitudes, and self-efficacy. Table 3.1 summarizes the data collection techniques that are being employed during the pilot testing and follow-up classroom experimentation to gather information about each level. The analysis of the different sources of data obtained/to be obtained during the pilot testing and follow-up classroom experimentation will inform the revision of the Augmented Teacher course, and of the accompanying methodological and pedagogical frameworks, and instructional materials and services included in the Living Library. The revised course will be finalized and will enter the EU Erasmus+ Training Database for access to educators across Europe. It will be offered as an Erasmus+ training course targeting primary and secondary school teachers, school administrators, teacher educators, or other professionals involved in training activities for teachers.

72

Meletiou-Mavrotheris et al.

table 3.1  Evaluation data by level of Guskey (2002) hierarchy

Guskey level

Evaluation data

1. Participants’ reactions

– – – – – –

2. Participants’ learning

– – – – –

3. Organization support and change

4. Participants’ use of new knowledge and skills

5. Student learning outcomes

5

– – – – – – – – – – – – –

Satisfaction surveys Teacher interviews Focus group meetings Postings in discussion forums Diaries and/or blogs Quantitative Indicators (e.g., number of teachers successfully completing each course) E-portfolio Pre- and post-tests Contributions to online discussions Interviews Teacher and school administrators’ interviews Questionnaires Artefacts Questionnaires Student, teacher, and parent interviews E-portfolio Direct classroom observations Videotaping of classroom episodes Student pre- and post-tests Parent pre- and post-surveys Direct classroom observations Videotaping of classroom episodes Student and parent interviews Student work samples

Conclusions

Acknowledging the crucial role of both traditional and digital forms of literacy in contemporary Europe, the ‘Augmented Teacher’ professional development course, designed and delivered through the Living Book project, aims to build European teachers’ capacity to contribute towards raising children’s motivation towards reading, and engaging them in different types of literacy practices. Taking into account current theoretical developments in literacy education,

Empowering Teachers to Augment Students’ Reading Experience

73

adult education, teacher education, and distance learning, the course aims to offer high quality in-service teacher training, ultimately aspiring to enrich European elementary and middle school children’s experiences and skills in reading. Going beyond traditional teacher training practices, the ‘Augmented Teacher’ course aims to equip participants with the most up-to-date theoretical knowledge, techniques, and tools for the implementation of the Living Book approach in their classroom. Drawing on participatory models of training, the course builds teachers’ knowledge and skills through a hands-on, experiential learning approach that combines effective pedagogical practices with contemporary technologies, including the Internet for maximum flexibility. Teachers are provided with many opportunities for interactive and collaborative learning through use of AR and other contemporary multimedia and internet technologies, and they engage in authentic educational activities such as projects, experiments, group work, and discussions. The strategies employed include open-ended investigations, simulations, visualizations, collaboration, and reflection on one’s own and on others’ ideas and experiences. These strategies are based on current principles of learning and are endorsed by professional organizations in literacy education. Through the use of these strategies, we provide a learning environment that can help participants gain better understanding of informal literacy education and inquiry. Moreover, the learning environment serves as a model to the teachers as to the kind of learning situations, technologies, and curricula they should employ in their own classrooms to promote student motivation towards reading. A central conviction underlying the Living Book design is that learning is a social act best supported through collaborative activities (Vygotsky, 1978; McConnell, 2000). While the project employs innovative augmented reality tools and resources to support educationally useful human-computer interactions, its focus is on exploiting technology to support human-human interactions. The ‘Augmented Teacher’ course provides a virtual space for European educators with a broad range of experiences and expertise to come together to reflect upon relevant education theory and practice, exchange ideas and resources, and build collaborations. Course participants are encouraged and expected to engage in joint discussions and to work collaboratively in completing projects and other assignments. The aim is to build an open knowledgebuilding and sharing environment that fosters sustained participation and allows teachers to take an active role and ownership for the creation of their community (Wenger, 1998; Hoadley, 2012). An important consideration of any model of professional development is whether teachers feel it is useful and supportive of their efforts to improve their teaching practice (Whitetaker, Kinzie, Kraft-Sayre, Mashburn, & Pianta, 2007).

74

Meletiou-Mavrotheris et al.

Historically, professional development efforts have largely been ineffective in producing reform-based classroom change (Templin & Bombaugh, 2005). As Robinson (1998) points out, staff development often fails to transfer to the learners’ ‘real-work’ situations, because it might be too remote from ‘real-work’ needs or organizational realities. The consortium has worked hard to avoid this danger by designing a course relevant to teachers’ work context. Meeting the individual workplace goals of a multinational group of teachers characterized by diversity in a number of different parameters (educational level and grade they teach, discipline(s) they teach, national curricula, cultural and professional backgrounds, etc.) is quite challenging but necessary if course participants are to make the difficult leap from professional development to classroom practice. Maximum dissemination of the project outputs and services in different cultural contexts and long-term sustainability will be achieved through the Living Book Library, which supports multilingual interfaces, transnational collaboration of teachers, and accumulation of collective knowledge from end-users. The Library offers access to validated pedagogical models, didactic approaches, and AR-enhanced and culturally adapted resource materials for teachers that will be of use not only to the project participants, but also for independent study by teachers across Europe and beyond. The project outputs provide a minimum cost service, ensuring that the Living Book Library will be used by large numbers of teachers. The project outputs and services are anticipated to be useful not only to teachers, but also to teacher educators, academic experts, national and European education authorities, teacher training institutions, and designers of online professional development programs. The ultimate beneficiaries will be students, who will eventually benefit from improved curricula and teaching practices that will foster their reading habits and motivation, and will better prepare them to meet the challenges of the digital age. In conclusion, beyond the activities of the Living Book project, the combination of different research fields in theories of learning technologies and literacy suggested in this chapter, form a solid theoretical basis on creating teacher professional development. Such background can be useful for the inclusion of AR in educational activities as well as for further teacher development purposes.

Acknowledgements The work presented in this chapter has been supported by the European Commission European Commission – Erasmus+/Key action 2, Strategic

Empowering Teachers to Augment Students’ Reading Experience

75

Partnerships for School Education program The Living Book – Augmenting Reading for Life (Ref. 2016-1-CY01-KA201-017315).

Notes 1 We recognize that there are different types of literacy and hence students with poor reading performance at school may perform differently in other types of literacy (i.e., non-school literacy, such as video games and other virtual environments). Refer to Section 2 of this chapter. 2 This framework has been jointly developed by the pedagogical and technical experts in the consortium to guide the design and delivery of the professional development course and, consequently, of the infrastructure and services for the dedicated Living Library that supports the project activities and outputs. It will be revised based on knowledge gained through pilot delivery of the professional development course and will be made available to the public through the Living Library.

References Ardichvili, A., Page, V., & Wentling, T. (2003). Motivation and barriers to participation in virtual knowledge-sharing communities of practice. Journal of Knowledge Management, 7(1), 64–77. Aziz, K., Aziz, N., Yusof, A., & Paul, A. (2012). Potential for providing augmented reality elements in special education via cloud computing. International Symposium on Robotics and Intelligent Sensors, Procedia Engineering, 41, 333–339. Beavis, C., Muspratt, S., & Thompson, R. (2015). Computer games can get your brain working: Student experience and perceptions of digital games in the classroom. Learning, Media and Technology, 40(1), 21–42. Blackwell, C. (2014). Teacher practices with mobile technology: Integrating tablet computers into the early childhood classroom. Journal of Education Research, 7, 1–25. Cheng, K. H., & Tsai, C. C. (2014). Children and parents’ reading of an augmented reality picture book: Analyses of behavioral patterns and cognitive attainment. Computers & Education, 72, 302–312. Clarke, J. (2013). Augmented reality, multimodal literacy and mobile technology: An experiment in teacher engagement. QScience Proceedings, (12th World Conference on Mobile and Contextual Learning [mLearn] 2013), 28. Cope, B., & Kalantzis, M. (2009). ‘Multiliteracies’: New literacies, new learning. Pedagogies: An International Journal, 4, 164–195.

76

Meletiou-Mavrotheris et al.

Delello, J. (2014). Insights from pre-service teachers using science-based augmented reality. Journal of Computers in Education, 1(4), 295–311. doi:10.1007/s40692-014-0021-y Dewey, J. (1902). The child and the curriculum. Chicago, IL: University of Chicago Press. Di Serio, A., Ibáñez, M., & Kloos, C. (2013). Impact of an augmented reality system on students’ motivation for a visual art course. Computers & Education, 68, 586–596. Dori, Y., & Belcher, J. (2005). How does technology-enabled active learning affect undergraduate students’ understanding of electromagnetism concepts? Journal of the Learning Sciences, 14(2), 243–279. Dunleavy, M., Dede, C., & Mitchell, R. (2009). Affordances and limitations of immersive participatory augmented reality simulations for teaching and learning. Journal of Science Education and Technology, 18(1), 7–22. Dünser, A. (2008). Supporting low ability readers with interactive augmented reality. Annual Review of CyberTherapy and Telemedicine, 6(1), 39–46. Dünser, A., & Hornecker, E. (2007). An observational study of children interacting with an augmented story book. Technologies for E-Learning and Digital Entertainment, 305–315. Ertmer, P. A., & Ottenbreit-Leftwich, A. T. (2010). Teacher technology change: How knowledge, confidence, beliefs, and culture intersect. Journal of Research on Technology in Education, 42(3), 255–284. Ertmer, P. A., Ottenbreit-Leftwich, A. T., Sadik, O., Sendurur, E., & Sendurur, P. (2012). Teacher beliefs and technology integration practices: A critical relationship. Computers & Education, 59(2), 423–435. European Commission, Directorate-General for Education and Culture. (2012). EU high level group of experts on literacy: Final report. Retrieved from http://ec.europa.eu/dgs/education_culture/repository/education/policy/ school/doc/literacy-report_en.pdf European Commission, Directorate-General for Education and Culture. (2016). PISA 2015: EU performance and initial conclusions regarding education policies in Europe. Retrieved from https://ec.europa.eu/education/sites/education/files/pisa-2015-eupolicy-note_en.pdf Fish, S. (1980). Is there a text in this class? The authority of interpretive communities. Cambridge, MA: Harvard University Press. Gu, H., Sanchez, S., Kunze, K., & Inami, M. (2015, September). An augmented e-reader for multimodal literacy. In Adjunct Proceedings of the 2015 ACM International Joint Conference on Pervasive and Ubiquitous Computing and Proceedings of the 2015 ACM International Symposium on Wearable Computers (pp. 353–356). ACM. Guskey, T. R. (2002). Does it make a difference? Evaluating professional development. Educational Leadership, 59(7), 45–51. Henderson, S., & Yeow, J. (2012). IPad in education – A case study of iPad adoption and use in a primary school (pp. 78–87). HICSS. Proceedings of the 2012 45th Hawaii International Conference on system sciences, Grand Wailea, Maui, HI, USA.

Empowering Teachers to Augment Students’ Reading Experience

77

Hoadley, C. (2012). What is a community of practice and how can we support it? In D. H. Jonassen & S. M. Land (Eds.), Theoretical foundations of learning environments (2nd ed., pp. 287–300). New York, NY: Routledge. Hornecker, E., & Dünser, A. (2008). Of pages and paddles: Children’s expectations and mistaken interactions with physical–digital tools. Interacting with Computers, 21(1–2), 95–107. Iser, W. (1978). The act of reading: A theory of aesthetic response. Baltimore, MD: Johns Hopkins University Press. Ivanova, M., & Ivanov, G. (2011). Enhancement of learning and teaching in computer graphics through marker augmented reality technology. International Journal on New Computer Architectures and their applications, 1(1), 176–184. Jauss, H. R. (1982). Toward an aesthetic of reception (T. Bahti, Trans.). Minneapolis, MN: University of Minnesota. Jones, S. (2006). Girls, social class and literacy: What teachers can do to make a difference. Portsmouth, NH: Heinemann. Kaufmann, H., & Schmalstieg, D. (2003). Mathematics and geometry education with collaborative augmented reality. Computers & Graphics, 27, 339–345. Kavanaugh, A., Carroll, J. M., Rosson, M. B., Zin, T. T., & Reese, D. D. (2005). Community networks: Where offline communities meet online [Electronic Version]. Journal of Computer-Mediated Communication, 10. Retrieved from http://jcmc.indiana.edu/ vol10/issue4/kavanaugh.html Kimble, C., Hildreth, P. M., & Bourdon, I. (2008). Communities of practice: Creating learning environments for educators. Charlotte, NC: Information Age Publishing. Klopfer, E., & Squire, K. (2008). Environmental detectives – The development of an augmented reality platform for environmental simulations. Educational Technology Research and Development, 56(2), 203–228. Luke, A. (2012). Critical literacy: Foundational notes. Theory into Practice, 51(1), 4–11. Martín-Gutiérrez, J., & Contero, M. (2011). Improving academic performance and motivation in engineering education with augmented reality. HCI International 2011 – Posters’ Extended Abstracts, 509–513. McConnell, D. (2000). Implementing computer supported cooperative learning. London: Kogan Page. McKenney, S., & Voogt, J. (2017). Expert views on TPACK for early literacy: Priorities for teacher education. Australasian Journal of Educational Technology, 33(5), 1–14. McNair, C. L., & Green, M. (2016). Preservice teachers’ perceptions of augmented reality. Retrieved from http://www.texasreaders.org/uploads/4/4/9/0/44902393/ ch._12-_mcnair___green_2016.pdf Mishra, P., & Koehler, M. J. (2006). Technological pedagogical content knowledge: A framework for teacher knowledge. Teachers College Record, 108(6), 1017–1054. Moeller, B., & Reitzes, T. (2011). Integrating technology with student-centered learning. A report to the Nellie Mae Education Foundation. Education Development Center, Inc.

78

Meletiou-Mavrotheris et al.

Núñez, M., Quirós, R., Núñez, I., Carda, J. B., & Camahort, E. (2008). Collaborative augmented reality for inorganic chemistry education (pp. 271–277). Proceedings of the 5th WSEAS/IASME International Conference on Engineering Education. Papert, S. (1980). Mindstorms: Children, computers, and powerful ideas. Brighton: Harvester Press. Papert, S., & Harel, I. (1991). Situating constructionism. In S. Papert & I. Harel (Eds.), Constructionism (pp. 1–11). Norwood, NJ: Ablex. Phillips, M. (2013). Investigating in-service teachers’ workplace TPACK development. Australian Educational Computing, 28(2), 1–10. Piaget, J. (1952). The origins of intelligence in children. New York, NY: International Universities Press. Powell, R., Cantrell, S. C., & Adams, S. (2001, May). Saving Black mountain: The promise of critical literacy in a multicultural democracy (I. R. Association, Ed.). The Reading Teacher, 54(8), 772–781. Resnick, M. (2013). Lifelong kindergarten. Cultures of creativity. LEGO Foundation. Robinson, B. (1998). A strategic perspective on staff development for open and distance learning. In C. Latchem & F. Lockwood (Eds.), Staff development in open and flexible learning (pp. 33–44). London & New York, NY: Routledge. Rohaya, D., Rambli, A., Matcha, W., Sulaiman, S., & Nayan, M. Y. (2012). Design and development of an interactive augmented reality edutainment storybook for preschool. IERI Procedia, 2, 802–807. Rosenblatt, L. (1978). The reader, the text, the poem: The transactional theory of the literary work. Carbondale, IL: Southern Illinois University Press. Schmidt, D., & Gurbo, M. (2008). TPCK in K-6 literacy education: It’s not that elementary! In Handbook of Technological Pedagogical Content Knowledge (TPCK) for Educators (pp. 61–86). New York, NY: Routledge. Shulman, L. S. (1986). Those who understand: Knowledge growth in teaching. Educational Researcher, 15(2), 4–14. Solak, E., & Cakir, R. (2015). Exploring the effect of materials designed with augmented reality on language learners’ vocabulary learning. Journal of Educators Online, 12(2), 50–72. Street, B. (1995). Social literacies: Critical approaches to literacy in development, ethnography and education. London: Longman. Street, B. (2003). What’s “new” in new literacy studies? Critical approaches to literacy in theory and practice. Current Issues in Comparative Education, 5(2), 77–91. Sullivan, A. S., & Brown, M. (2013). Social inequalities in cognitive scores at age 16: The role of reading. London: Institute of Education, Centre for Longitudinal Studies Working Paper. Templin M. A., & Bombaugh, M. (2005). An innovation in the evaluation of teacher professional development serving reform in science. Journal of Science Teacher Education, 16, 141–158.

Empowering Teachers to Augment Students’ Reading Experience

79

Ternier, S., Klemke, R., Kalz, M., Ulzen, P., & Specht, M. (2012). AR learn: Augmented reality meets augmented virtuality. Journal of Universal Computer Science, 18(15), 2143–2164. Tondeur, J., Pareja Roblin, N., van Braak, J., Fisser, P., & Voogt, J. (2013). Technological pedagogical content knowledge in teacher education: In search of a new curriculum. Educational Studies, 39, 239–243. van Groenestijn, M. (2007). Theory in MiA. In M. van Groenestijn & L. Lindenskov (Eds.), Mathematics in action, communalities across differences: A handbook for teachers in adult education (pp. 20–36). The Netherlands: ALL Foundation. Voogt, J., & McKenney, S. (2016). TPACK in teacher education: Are we preparing teachers to use technology for early literacy? Technology, Pedagogy and Education, 26(1), 69–83. doi:10.1080/1475939X.2016.1174730 Vygotsky, L. S. (1978). Mind in society: The development of higher psychological processes. Cambridge, MA: Harvard University. Wenger, E. (1998). Communities of practice: Leaning, meaning, and identity. Cambridge: Cambridge University Press. Whitetaker, S., Kinzie, M., Kraft-Sayre, M. E., Mashburn, A., & Pianta, R. C. (2007). Use and evaluation of web-based professional development services across participant levels of support. Early Childhood Education Journal, 34(6), 379–386. Wojciechowski, R., & Cellary, W. (2013). Evaluation of learners’ attitude toward learning in ARIES augmented reality environments, Computers & Education, 68, 570–585. Wu, H. K., Lee, S. W. Y., Chang, H. Y., & Liang, J. C. (2013). Current status, opportunities and challenges of augmented reality in education. Computers & Education, 62, 41–49. Wyatt, S. (2008). Technological determinism is dead; Long live technological determinism. In E. J. Hackett, O. Amsterdamska, M. Lynch, & J. Wajcman (Eds.), The handbook of science and technology studies (pp. 165–180). Cambridge, MA: MIT Press. Yilmaz, R. M., Kucuk, S., & Goktas, Y. (2016). Are augmented reality picture books magic or real for preschool children aged five to six? British Journal of Educational Technology, 48(3), 824–841. Yuen, C. S. (2011). Augmented Reality (AR) in education. Presentation in Creating Futures through Technology Conference, Biloxi, MS. Retrieved from http://www.slideshare.net/scyuen/augmented-reality-ar-in-education

CHAPTER 4

Uses of Augmented Reality in Primary Education Eva Csandová, Renata Tothova and Lilla Korenova

Abstract This chapter focuses on the possibilities of educational uses of augmented reality in Slovak primary school. Usage of augmented reality presents new challenges and opportunities both for childrens’ autonomy and teachers’ work and should be a natural part of an educational practice which helps children to achieve learning goals by themselves. Teachers simply evaluate the quality of childrens’ competencies and their autonomy and masterfulness in augmented reality activities. Our goals were to identify the impact of augmented reality usage, including benefits on the childrens’ digital literacy development, as well as emerging problems or difficulties for children while using augmented reality applications. Our action research was conducted in a natural didactic context, and our research methods included direct and indirect observation, and content video analysis based on the grounded theory methodology. We were able to observe and indicate the impact of usage of augmented reality in education activities on children and our main finding was an increase of children’s inner motivation for learning which subsequently increases also the inner motivation of teacher for futher professional development and self education.

Key words augmented reality – digital literacy – primary education – qualitative methodology

1

Theoretical Paradigm

Constructivist teaching approaches and pedagogical research concerned with the application of this theoretical direction to teaching have often been an interest of researchers (e.g., Bodner, 1986, pp. 875–876; Held, Pupala, & Osuská, 1994; Renström, Andersson, & Marton, 1990; Žoldošová & Prokop, 2007). They dealt with the question of how to bestow the basic scientific principles to © koninklijke brill nv, leiden, 2020 | doi: 10.1163/9789004408845_004

Uses of Augmented Reality in Primary Education

81

children in primary school in order to make the childrens’ learning meaningful and to carry out internalization of scientific knowledge. This trend in research has found continued motivation, both abroad and in Slovakia, because children often use only rote memorization as their learning technique without gaining any in-depth understanding of subject’s essence (Tóthová, 2014). The theory of constructivism provides an alternative pedagogy according to which children themselves construct their knowledge and the teacher can only prepare suitable conditions so that this process of construction can take place in the “desired” direction in a subject, and thus, in natural sciences, toward appropriate scientific (natural) interpretation of the world. In constructivist pedagogy, the teacher works primarily as a facilitator of children’s learning, assisting them when they ask for help, and leading them on the path of cognition, without providing the required knowledge, which children have to construct through their own practice and social discourse activity. Von Glasersfeld mentioned some of the most important implications of constructivism for application to the didactic process (von Glasersfeld, 1993, pp. 32–38): – If we assume that children themselves build their own knowledge, we should bear in mind that no child is a “tabula rasa,” but everyone already has their own “naive theories” or so-called pre-concepts, i.e., their own viable ways to deal with their experienced life (social) environment, and the situations in which they are or get into. Therefore, it is crucial for a teacher to grasp some idea of where these children are in their knowledge, what concepts and theories they may have and see how they are able to verbalize and explain these concepts, views. – The way a child answers a question or “problem” is what makes sense to the child at that moment. The teacher should take it seriously, regardless of the precision or “misconception” of the child’s statement in the teacher’s eyes. A teacher’s direct proclamation “that’s bad or incorrect” to the child is very disruptive and dissuasive and knocks down the child in his activity and the effort to learn and to understand something. It is important to appreciate the childrens’ efforts to construct new knowledge. A child’s own concepts deserve the teacher’s respect, even if they are not consistent with the scientific view of the world (which should be the result of their learning). It is necessary that children have the opportunity to use, challenge and transform their own concepts. – It is advisable to ask the children how they came up with their answer. It is a good way for a teacher to create an image about child’s way of thinking and to try to understand why the child has chosen a such solution and why the idea seems to be for him/her reasonable enough. To the teacher, it

82

Csandová et al.

can tell a lot about the child’s thinking, allowing a step back from his scientific outlook on the world, to also appreciate the beauty and elegance of the child’s constructions, even if they are not in accordance with scientific foundations. – If a teacher wants to encourage children to focus on questions that are rather uninteresting from the child’s point of view, they must to create situations in which children will have the opportunity to experience the pleasure of solving the problem. If the teacher simply says that the answer is “right,” it will not be sufficient (the teacher does nothing) for the development of the child’s concepts if the child does not really care about the issue. – The children’s process of reasoning is more important than “correct” answers. Logical processes of thinking should be rewarded, even when based on unacceptable assumptions. – To understand and appreciate the children’s thinking, the teacher must have an extremely flexible mindset, because children sometimes rely on assumptions that are strange for teachers. From the point of view of applying the constructivist theory to didactics, it is important to find appropriate ways of activating the thinking of children in order to construct their own knowledge. The best way to do this is to let children “fight” with problems of their own choosing and help them only if they ask for help; this way teachers can help them to build knowledge. In the best case, the teacher can orient the constructivist processes of the children into a productive direction, but they must never force them to do so. Of course, this way of teaching takes a lot of time, but once children experience the joy of finding a solution once or twice, they will be well prepared to work even on the problems that the teacher proposes. In terms of constructivism, it is important to realize that the education process pursues two objectives. First, to promote thinking that does not include conceptual inconsistencies (contradictions) and leads to inherently consistent results. Secondly, to bring the children into the consensual domain (understanding of consensus-agreement). The “natural sciences” of today are, as von Glasersfeld says, what naturalists today believe, what they agree on and with. But for a constructivist, an “agreement” does not mean that concepts and interconceptual relationships used by people who “agree with each other” are identical; it just means that under certain conditions their concepts seem to be compatible. Another constructivist consequence for didactics is the fact that it is useless to present children a verbal definition of any concept without having the opportunity to have some kind of relevant experience with the term.

Uses of Augmented Reality in Primary Education

83

Assessment of childrens’ learning in constructivist teaching is not easy. The teacher should not assess the child’s ability to reproduce scientifically correct answers that a child has learned (or memorized), but rather whether the child in some way develops his/her ability to think, creates his/her own concepts and successfully finds solutions to the problems encountered. More important than a specific child’s solution is their conceptualization of the problem and their approach to it. By observing the conceptual ways and means the child uses to solve the problem, the teacher can make a good sense of how far this child is on the way toward a functioning conceptual system in the relevant issue or subject.

2

Method

2.1 Research Questions Four research questions were asked: 1. What questions do children ask while using augmented reality? (What questions and instructions do children exchange with other children while handling augmented reality? What questions does a child ask the teacher in regard of augmented reality?) 2. What surprises children while using augmented reality? 3. When, why and to what do children react with enthusiasm while using augmented reality? What makes them joyful? 4. When, why and to what do children react with disappointment while using augmented reality? What are they dissatisfied with? In the study, the participants used ten tablets/handheld displays with camera, tracking technology and the applications Quiver Vision and Inoveduc. We also used printed AR markers that are most common with these applications. 2.2 Participants Research participants were children of 3rd grade of elementary school and their opinions were the subject of interest. 2.3 Research Objectives Our goals were to identify the impact of augmented reality usage, including benefits on the childrens’ digital literacy development, as well as emerging problems or difficulties for children while using augmented reality applications. We wanted to observe the effect of augmented reality on children in elementary school on the assumption that this period has a decisive influence on the development of child’s further competencies (Piaget & Inhelderová, 1993).

84

Csandová et al.

2.4 Research Plan The plan of research was to observe children in grade 3 of an elementary school during the computer science lesson so that we did not interfere with their activities to ensure the typical conditions of their education during our observation. Our research consisted of four stages: 1. In the first stage, we were granted permission to research from the school principal and the teacher of 3rd grade classroom, which we subsequently visited. We briefed them with our research plan and objectives and became acquainted with the research subjects and the environment in which the research was conducted. We tried to gain their trust. 2. The second stage consisted of direct observation during two classes, with video recording of childrens’ activities as well as teachers’ and childrens’ interaction. For this purpose, we used video cameras on a tripod (Benq), tablets (Samsung) and smartphones (iPhone) with the approval of the principal school principal and the research participants. 3. In the third stage, the obtained data from the observation was transcribed. 4. In the last phase of the research we each re-read data several times and underwent an open coding. We’ve created codes that we’ve included in the categories. We have interpreted results of our research analyzes through the analytical story. 2.5 Materials and Procedure As we have already outlined in the theoretical background, our research is based on an interpretative paradigm (Ondrejkovič, 2002). We are convinced that every human experience is socially organized from the structural point of view, which excludes the possibility of the uninterpreted mental content of statements or internal schemes. The presented knowledge is thus not objective but subjectively designed and based on the agreement, transmission and construction of the reality studied between the researcher and the research participants. Both social constructivism and discourse psychology significantly contributed to our understanding and conceptualization, which gave us the opportunity to grasp research problems from a new perspective, based on the acceptance of subjectivity as a phenomenon. Thus, the emphasis is shifting from the facts as such and objective reality to the significance of the facts mentioned, to the subjective importance of reality in the context of a unique human life. In our everyday life, our reality is shaped largely by social interactions, that is, through the transmission of information.

Uses of Augmented Reality in Primary Education

85

From the developmental point of view, this transmission is clearly evident, as evidenced by a number of sociolinguistic studies, which confirm and describe, for example, the influence of the parental communication style on the way how we handle our memories (Reese, 2002). Our idea was that if we wanted to explore childrens’ interaction and learning with augmented reality, it was important to see and observe the performed activities and be in direct contact with all the new specific situations that augmented reality brings. We knew that obtaining a research sample would not be a problem, and that would allow direct observation. In addition, as Gavora (2007) points out, qualitative research is fascinating, both the authenticity and the plasticity of research are ensured. It allows us to work with a small group of people and create a trustworthy confidence. Within our study we chose not the most commonly used method of observation. As Svaříček states, the observation method makes it possible to capture the routine situation that research participants do not refer to in the interviews due to their unawareness or absence of awareness (Švaříček, Šeď ová, Janík, Kaščák, Miková, Nedbálková, Novotný, Sedláček, & Zounek, 2007). It provided us the possibility to observe things and phenomena about which the research participants had scruples and hesitated to speak, because the researcher is, to them, a stranger. As Gavora (2007) points out, when examining reality, we have identified only specific people and the environment of realization, in our case children of the 3rd grade in primary school. We did not specify any observation categories or observation systems and scales. We used unstructured observations. In order to exploit the full potential of constructivism, we chose participatory observation with partially active participation. Our position was marginal, as we were not full-fledged members of the group. We observed the research participants directly in the classroom environment during training and, as observers, we were somewhat involved in the ongoing activities. During our attendance, we generally avoided interacting or interfering with the research participants, however, some participation occured when children asked us questions or called on us for help. During the observation, we had a video camera placed on a tripod in a corner of the room, where it did not interefere with the research participants, and the particpants did not interfere with its recording. During video recording we tried to remain outside the camera’s view, as we did not want to be a distraction factor. There are several types of qualitative research data evaluation and processing, we have chosen to use a grounded theory, and method of “contrast and comparison” (Miovský, 2010) The basic procedure was the open grounded theory coding, based on Anselm Strauss and Julliet Corbin’s approach, which uses three types of coding

86

Csandová et al.

in their analysis with basic steps open, axial and selective encoding. By encoding, we understand the data to be interpreted, conceptualized and reintegrated (Hendl, 2008, p. 246). In addition to the direct observation of the children, we decided also to reflect the collective opinion of the children and teaching process itself by adding a focus group method. For our specific topic, the focus group is considered to be a suitable complementary research tool not only because it allows us to explore a different angle of view of an interested party but also social discourse along with reasoning and the ability to find out what are the participants subjective theories and see how they react in vivo. Research focus groups tend to be cited as one of the most progressive qualitative methods for data acquisition (Miovský, 2010). We have chosen it because it allows insight into the source of the action which are the attitudes and thinking patterns of the people. This is a research tool for data collection on a specific topic from groups or groups that is defined by shared characteristics or interests. We have used it in our research not only as a complementary method, but also for broadening and deepening the understanding of research results. David L. Morgan (1997), one of the main representatives of this method, used the term “focus,” because within this research method a thematic focus is defined. Miovský (2010) states that the “focus” usually represents a common phenomenon, a topic that draws our interest, and the term “focus” group is therefore a suitable one. The design of research projects using the focus group method is usually based on four premises: The first premise is that groups can benefit from synergy in generating ideas (Fern, 1982). Hess (1968) also believes that focus groups produce a wider range of ideas and opinions than individual interviews. The second assumption regarding the role of the moderator is that expertise, personality and procedures are essential to ensure group interaction (Morgan, 2001). Fern (1982) opposes this since the research results have not shown any significant differences between the productivity of groups with a dominant moderator and those without. Krueger and Casey (2000) for example, do not require that the focus group moderator be an expert in the discipline discussed by the focus group. Miovský (2010) indicates that a moderator’s previous experience with this method can be of decisive importance. The third premise refers to the group size, for which the optimal number is generally considered to be 6 to 10 members (Morgan, 2001). With regard to the size of the group, Miovsky (2010) however, points to the absence of a universal standard that would determine the size of the group. This should be based on

Uses of Augmented Reality in Primary Education

87

the experience of the moderator/researcher, the nature of the topic and the technical and organizational possibilities. In any case, we have to say that the higher the number of participants, the less space each of them has to express. In our case, we worked with the whole class (with 22 children). We chose the whole class group (not divided into groups of six students), because the focus group followed immediately after working with the AR, and it was natural that all children would speak of their experience as one group, because they had just experienced similar experiences. We believe that every child who wanted to express or discuss the issue had enough space. We ended the focus group after we saw that children were repeating their testimonies rather than adding new ones. The fourth premise of the optimal focus group proposal is the homogeneity of the group and the fact that the participants do not know each other. Smith (in Fern, 1982) concludes that mutual acquaintances of participants can seriously affect group dynamics and individual responses. Morgan (2001), on the other hand, indicates that the premise that the focus group must be made by people who do not know each other is incorrect. He admits, however, that friends and strangers as focus group members can induce a different dynamic in the group. The composition of the group would, according to him, “ensure that the participants in each group have something to say about the subject of the research and to feel safe when they pronounce their ideas” (Morgan, 2001, p. 52). In our case, we believe, that the fact that all children in the focus group knew each other well (because they were classmates for the third year) should not have a big impact on the mutual influence of their focus group views. The children’s testimonies in the focus group were related to their immediate experience with the AR that they had used immediately before the focus group. We believe that their experience therefore influenced them more than the opinions of classmates they have long known. Given the fact that the bulk of opinions and attitudes are formed within social relationships, it is only natural that in well-guided focus groups, participants more easily discover and formulate their feelings and thought processes, attitudes, and opinions (Hendl, 2008). 2.6 Research Realization Our research was realized within the CEPENSAR project with 3rd grade children on the 17th of April 2018 – during their second class, and the 18th of April 2018 – during their third class. On April 17th, we started with Quiver Vision application activities developed following socio-constructivist EUR method, where “E” stands for evocation, “U” stands for understanding of meaning and “R” stands for reflection.

88

Csandová et al.

The evocation was the introduction to the class and it took about 5 to 7 minutes. We gave some instructions how to handle tablets; we briefed children about objectives of our research and introduced them to Quiver application very briefly, and, finally, every child was given a picture with an AR marker for Quiver Vision. The children were then given time to color the pictures. In next stage – understanding of meaning of new information phase (the acquisition of awareness, consciousness, knowledge of the true meaning of any new information by children themselves) – children worked independently in small groups about 20 minutes with Quiver application and shared their knowledge within the class. Given the fact that this is supposed to be constructivist-based teaching, where children guide the exploration, we were in the role of observers rather than active participants or leaders. In the introduction of this phase, however, the kids were given a few instructions and tasks. They were instructed to find a new Quiver app icon on the tablet and try to test it, that is, click to open it and focus the tablet on the colored picture and see what happens with the picture and application. The results of their investigation group documented through photos and videos. We recommended that children interact with each other between groups. They could share pictures with each other, as each group had different ones. This second phase of the learning process is all about children actively processing the source of new information, they are going through a new experience. Children link all the new information that comes from an external source at this stage (activity with Quiver application offered by a teacher) with information that has been recalled and arranged in the first phase of the learning process or with the information they already had. The final phase was all about children’s reflection which took 20 minutes (10 minutes of regular class time and additional 10 minutes from break, when children decided to stay in classroom and share their thoughts rather than do anything else). In this stage, we asked 2 questions: 1. What was most entertaining for you? 2. What was bothering you or with what did you have most problems while using augmented reality? The unstructured observation objective (without a pre-prepared scheme) was to capture the course of the teaching process, in which the teacher began to knowingly support the method of social constructivist development of the inner motivation of the child through augmented reality. The children in the learning groups involved in the research had direct, spontaneous communication, while still respecting the rules of the class. Observation videos were transcribed. Researchers reviewed the transcripts, searching for indications of the subjective significance that participants had

– The evocation phase The teacher encouraged the children to verbally present their knowledge about the augmented reality. This led to a discussion about our project and provided children with new information and a reality with the intent to create a cognitive conflict. Children together with the teacher set up rules for following activities and then everybody received a worksheet with a black and white printed marker (according to each child’s preference) for coloring and further usage with tablets, which was to fijind a new Quiver app icon on the tablet and try to test it, and work with it and see what happens with the picture and application. They were also encouraged to take pictures, videos, and to document everything in process of learning about Quiver Vision application. This was a pre-active phase of teaching.a

EUR phases description in the teaching activity

table 4.1  Protocol No. 1 EUR phases

(cont.)

Although children of this class (according to their teacher) don’t really enjoy coloring pictures anymore that much, they did it with enthusiasm, because they saw the purpose and goal of it. They didn’t need any encouragement from teacher or researcher.

figure 4.1 Thematic categorization/initial evaluation

The presence of increase of inner motivation to learn as part of an augmented reality teaching

Uses of Augmented Reality in Primary Education

89

– The understanding of meaning phase The children themselves came to the fact that the tablets had to be kept at a certain distance and direction toward the worksheet to see the 3D object displayed above the marker. They also realized very soon, that they had to cooperate in groups: if they wanted to take a picture of themselves, one had to hold the tablet in the certain angle and take a photo while two other children stood in front of tablet – one holding the worksheet and the other posing. On the tablet’s display, children saw a virtual object instead of a marker. Specifijically, it was an easily recognizable and generable image on the colored worksheets about various topics. This was an interactive learning phase.

EUR phases description in the teaching activity

table 4.1  Protocol No. 1 EUR phases (cont.)

(cont.)

Children communicated very well between themselves and shared their knowledge without teachers asking them to do so.

figure 4.2 Thematic conceptualization/ongoing evaluation

The presence of increase of inner motivation to learn as part of an augmented reality teaching

90 Csandová et al.

– The reflection phases In the fijinal phase of the teaching activity, children together with the teacher gave feedback through sharing their views and opinions in a focus group. Children’s reflection (self-evaluation and evaluation) has been related to increase of their inner motivation for learning through augmented reality, so emphasis was placed on mutual communication and exchange of information. This was a post-active phase of teaching.

EUR phases description in the teaching activity

table 4.1  Protocol No. 1 EUR phases (cont.)

Children came up with the idea of using applications with augmented reality in other subjects as well (history class, nature science class and so on)

figure 4.3 Thematic realization/output evaluation

The presence of increase of inner motivation to learn as part of an augmented reality teaching

Uses of Augmented Reality in Primary Education

91

– The evocation phase The teacher introduced the new application software InovEduc to children and provided the space for them to create and verbalize their ideas related to InovEduc, leading a discussion with the intent to incite a cognitive conflict. The teacher also supported the internal motivation of children by discussing proposals for theme realization. The teacher has supported the cooperation of children by creating the conditions for their realization This was a pre-active phase of teaching.

EUR phases description in the teaching activity

table 4.2  Protocol No. 2 EUR phases

(cont.)

Children were eager to try out this new software application. Several of them mentioned area of Eastern Slovakia as a place where their grandparents aor other relatives live.

figure 4.4 Thematic categorization/initial evaluation

The presence of increase of inner motivation to learn as part of an augmented reality teaching

92 Csandová et al.

– The reflection phases In the fijinal phase of the teaching activity, children together with the teacher gave feedback through sharing their views and opinions in the focus group. Children’s reflection has been related to increase of their inner motivation for learning through augmented reality, so emphasis was placed on mutual communication and exchange of information. This was a post-active phase of teaching.

– The understanding of meaning phase Children developed the theme of historical, natural and multicultural awareness in concrete forms. 24 3D objects, models of historical, cultural, technical and natural sights of Eastern Slovakia and Transcarpathian Ukraine with the possibility of interactive viewing, allowed the Teacher to develop children’s digital literacy through an individual approach and discussion. This was an interactive learning phase.

EUR phases description in the teaching activity

table 4.2  Protocol No. 2 EUR phases (cont.)

figure 4.6 Thematic realization/output evaluation

They could spontaneously fijind out how to work with InovEduc and were keen to present what they learned.

figure 4.5 Thematic conceptualization/ongoing evaluation

The presence of increase of inner motivation to learn as part of an augmented reality teaching

Uses of Augmented Reality in Primary Education

93

94

Csandová et al.

personally attributed to the phenomenon, as well as why he/she (the participant) considered it to be a significant. On April 18, we continued with use of augmented reality but this time we used the application ‘InovEduc” – Innovative learning methods for partnership support. The project presents 24 objects with the possibility of interactive viewing using state-of-the-art imaging technologies, including virtual and augmented reality. The project brings a unique combination of interactive 3D models of historical, cultural, technical and natural sights of Eastern Slovakia and Transcarpathian Ukraine and innovative educational approaches to improve historical and multicultural awareness of border regions. On the Slovakia side, it is represented by Pan-European University. After the return from the field, we transferred the audio video recordings of participants’ statements and observed behavior into protocols with line numbers. The basis of data was formed by the statements of children; when it comes to observed behavior, we considered as important facial impressions of joy, susprise, concentration and also manifestation of children’s motor skills. We processed the research material using the open coding technique. Using this operation, the obtained data is conceptualized. The text is divided into units where words, sentences, paragraphs can be re-composed in a new way, while units are assigned with new names. Then the researcher can work further with them (Švaříček et al., 2007). As stated by Miovský (2010, p. 220), “the coding process is de facto a process of identification and systematic labeling of meaning entities according to established criteria.” We chronologically arranged the transcripts of the recordings in print form. Then we read our records repeatedly for better understanding. By repeated reading, we focused on localization phenomena in the text of statements. On our part, the analysis was based on reduction, categorization, and comparison in order to get the most comprehensive picture of our research. During the reading, we focused on the units of meaning to which we assigned codes. Based on Creswell’s recommendation (2008), we used a manual hand-coding, which enabled us to scroll faster through text and compare. We only worked with paper, pens, highlighters. For this research study, we did not use any qualitative analysis software because our obtained data was manageable without it. When we were creating codes, we followed terms with similar or same meaning and those we have divided into several continuously created categories. The resulting preliminary categories, however, had overlapping content, therefore we decided for their further comprehensive and aggregated processing. Visualization helped us identify the most recurring categories, which also appeared to be the most significant in the overall context, and we could move to the subcategory level (Figure 4.7).

Uses of Augmented Reality in Primary Education

95

figure 4.7 Graphical representation of selective coding

After open coding, we focused on the second stage (axial coding) to search for causality relations and interdependence of the various categories. We identified two main areas – the first of which involves intentional action (conduct) of the teacher in order to strengthen the inner motivation and learning effectiveness of the child. For this action, research participants mentioned the following: providing feedback to children, communication (intentional, not random), providing all relevant information and interest in innovation, advancement, career development, self-education, commitment, humanity and empathy, expression of confidence in the competence of children, and the impact of a teacher on children in the sense of social constructivism. The second round of strategies includes action and conduct of child, but also a group of children aimed towards the efficiency of learning, autonomy, and inner motivation to learn: engaging in the process of acquiring knowledge and skills, working in small groups but with the whole team, providing feedback to classmates and teachers, communication with each other and with the teacher (intentional, not random), interest in advancement, self-education, commitment, humanity and empathy. At the final stage of the qualitative analysis, the selective encoding process, we identified as a central category the “growth of inner motivation for

96

Csandová et al.

figure 4.8 Graphical representation of selective coding

learning” that the participants of the research – the children – make conditional on the overall atmosphere of education in the classroom, by introducing elements such as augmented reality, by providing appropriate information, and by developing trust by showing confidence in children’s abilities. In the area of the educational process, participants marked as important the following: “wow effect” (a special state of mind that occurs when we are surprised by something unexpected and great), the ability to try and study new things independently, the opportunity to present their knowledge in group (or as a group), emotional support, and finally practical support as well (Figure 4.8).

3

Results

In our research, we have obtained and identified a range of opinions, views and insights in answer to our first question “How did children behave and communicate in groups when working with augmented reality?” We observed increase of intentional communication and cooperation between children within small groups and between them as well as with the teacher and researcher. Also, during the focus group reflection time, we noticed the increase of intentional

Uses of Augmented Reality in Primary Education

97

communication (in the whole classroom) as well. Children mostly exchanged their views and experiences, but they also gave advice when they were asked for it and they asked for advice from either the teachers, the researchers or the other children, as well, and/or were helping other children in other groups and researchers. Regarding their behavior, we have to also mention the increase of children’s physical activity and frequent changes of their physical positions and poses during the activities with augmented reality. It was due to necessity of finding the right angle of focus of tablet, and also because of communication which was going on between groups and exchanging of colored worksheets between them. This increase of children’s physical activity we consider a positive thing, because usually children just sit during their class, which could be a bit boring for them. On our second question about “What surprises children while using augmented reality?” Children were surprised by several things when handling AR Quiver, firstly, that the objects “came out” of the tablet as if they were in our real space, the secondly, that they were “alive” – moving (e.g., the car was moving along the road, the butterfly waving wings and flying, the volcano erupting) and thirdly, that the children could control the AR objects (e.g., as the fire truck moved along the road, the children could speed it up, slow down, turn on lights, and direct the flow of water in different directions). On our third question about “What did they (children) enjoy most what they were enthusiastic about? When, and why did children react with enthusiasm?” our observation results made unambiguous outcome that children reacted joyfully and with enthusiasm to masks and hats that they could try on and take virtually augmented pictures with them. And on our fourth question about “What disappointed the children, or with what were they dissatisfied?” our research findings indicate that children didn’t really have any disappointments and when they had any problems (the only problem experienced was an incorrect Wi-Fi password) they moved quickly and efficiently towards solution (they were not easily discouraged). The activities with the AR were so interesting for the children that they continued playing with them also in their home environment: The children asked questions such as: “Can we download it at home, on the tablet, on the mobile?” And the next day many of them came talking about how they taught their parents and siblings about augmented reality and how their parents printed for them various pictures from Quiver, so they can bring them to school to use the next day with tablets. Therefore, we were able to observe and indicate impact of usage of augmented reality in education activities on children which was formulated as our goal.

98

Csandová et al.

What we consider to be the most beneficial is potential of inner and outer motivational circle of reciprocity where children’s enthusiasm about learning through and with augmented reality increases their inner motivation for learning in general which also motivates the teacher for better performance, willing to try new, innovating methods, to self-education and professional development. Therefore, increase of children’s inner motivation for learning increases also the inner motivation of teacher for professional development and further learning. Another benefit of augmented reality usage in educational activities is necessary communication and cooperation between children, which can support and strengthen the group of children as whole (classroom) and create a good, proactive learning environment for them.

4

Discussion

A constructivist approach to teaching according to von Glasersfeld should be the cause of more children being “better” scientists. The aim is to build up the autonomous ability of both children and teachers to prioritize the most challenging aspects of their educational experience; consider alternative approaches to addressing the challenge; identify and analyze the evidence that provides the most information about a specific problem; and consider alternative solutions that can be implemented quickly (von Glasersfeld, 1993). A specific technique for providing this type of support is cognitive coaching (Stansbury & Zimmerman, 2000), which is, according to us, also applicable in the relationship with teacher and child. In the short term, it can and will make children profit by solving specific problems; but in the long term both child and teacher can benefit that they know how to think constructively about any problem that occurs in teaching process. The question is whether or not teachers are ready for this kind of support. The critical self-reflection induced by cognitive coaching might ideally, according to Stansbury and Zimmerman (2000), have a wider impact, and a certain tendency to collective peer action. When a group of children under the leadership of a teacher in a given school is capable of critical self-reflection, it is not a problem to openly talk to each other about their progress and/or problems, and in the case of a team of teachers, about their performance in classes. Then the ability of both teachers and children to identify and solve the problems in the educational process and other important issues dramatically increases. Within the teaching team, this kind of dialogue allows each teacher in primary (or other) schools extend into the details of each class and see the bigger

Uses of Augmented Reality in Primary Education

99

picture of what is happening in the school or a particular class. For example, when a teacher notices that children in his class have problems, for example with space orientation, he can assume that the problem is idiosyncratic. However, when all teachers of a given age category in a primary school meet to discuss the teaching and learning of children and realize that an inappropriate number of their children have the same problem, they can more effectively address both the given problem and its causes. Therefore, this enhancement of learning process and teaching methods ultimately benefits a child’s comprehensive development.

Acknowledgement The chapter was written the support of the grant KEGA 012UK-04/2018 “The Concept of Constructionism and Augmented Reality in the Field of the Natural and Technical Sciences of the Primary Education (CEPENSAR).”

References Bodner, G. M. (1986). Constructivism: A theory of knowledge. Journal of Chemical Education, 63(10), 873–878. Cresswell, J. (2009). Research design: Qualitative, quantitative, and mixed methods approaches (3rd ed.). Thousand Oaks, CA: Sage. Fern, E. F. (1982). The use of focus groups for idea generation: The effects of group size, acquaintanceship, and moderator on response quantity and quality. Journal of Marketing Research, 19, 1–13. Gavora, P. (2007). Sprievodca metodológiou kvalitatívneho výskumu. Bratislava: Polygrafické stredisko UK. Held, Ľ., Pupala, B., & Osuská, Ľ. (1994). Konštruktivistický prístup k učeniu a vyučovaniu (empirický pohľad). Pedagogická revue, XLVI(7–8), 319–327. Hendl, J. (2008). Kvalitativní výzkum. Základní teorie, metody a aplikace. Praha: Portál. Hess, J. M. (1968). Group interviewing. In R. L. King (Ed.), ACR Conference Proceedings (pp. 187–193). Chicago, IL: American Marketing Association. Krueger, R. A., & Casey, M. A. (2000). Focus groups: Practical guide for applied research (2nd ed.). Thousand Oaks, CA: Sage Publications. Miovský, M. (2010). Kvalitativní přístup a metody v psychologickém výzkumu. Praha: Grada publishing a. s. ISBN 80-247-1362-4. Morgan, D. L. (1997). The focus group as qualitative research (2nd ed.). Thousand Oaks, CA: Sage Publications.

100

Csandová et al.

Morgan, D. L. (2001). Ohniskové skupiny jako metoda kvalitativního výzkumu. Boskovice: Albert. Ondrejkovič, P. (2002). K metodologickým otázkam kvalitatívneho a kvantitatívneho výskumu. Pedagogická revue, 54(2), 101–120. Piaget, J., & Inhelderová, B. (1993). Psychológia dieťaťa. Bratislava: SOFA. Reese, E. (2002). Social factors in the development of autobiographical memory: The state of art [Electronic version]. Social Development, 11(1), 124–142. Renström, L., Anderson, B., & Marton, F. (1990). Students’ conceptions of matter. Journal of Educational Psychology, 82(3), 555–569. Stansbury, K., & Zimmerman, J. (2000). Lifelines to the classroom: Designing support for beginning teachers. San Francisco, CA: WestEd. Švaříček, R., Šeď ová, K., Janík, T., Kaščák, O., Miková, M., Nedbálková, K., Novotný, P., Sedláček, M., & Zounek, J. (2007). Kvalitativní výzkum v pedagogických vědách. Praha: Portál. Tóthová, R. (2014). Konštruktivistický prístup vo výučbe ako možnosť rozvoja myslenia žiakov. Bratislava: MPC. Dostupné z https://mpc-edu.sk/sites/default/files/ projekty/vystup/tothova.pdf von Glasersfeld, E. (1993). Questions and answers about radical constructivism. In K. Tobin (Ed.), The practice of constructivism in science education (pp. 23–38). Hillsdale, NJ: Lawrence Erlbaum Associates. Žoldošová, K., & Prokop, P. (2007). Primary pupil’s preconceptions about child prenatal development. Eurasia Journal of Mathematics, Science and Technology Education, 3(3), 239–246.

CHAPTER 5

Augmented Reality Applications in Early Childhood Education Lilla Korenova, Zsolt Lavicza and Ibolya Veress-Bágyi

Abstract Today, digital technologies, and mobile technologies in particular, are already part of the everyday life of children even at the preschool age. Joanne G. Sujansky and Jan Ferri-Reed (2009) in their book Keeping the Millennials claim that today’s young people are continuously multitasking for example, working on a laptop while also watching TV, listening to their iPod, and chat or sms at the same time. Therefore, even in pre-primary education, it is important for educators to explore the uses and opportunities of smartphones and other technologies. Among the currently developing mobile technologies are virtual and augmented realities and their uses in various areas of life. The aim of this chapter is to discuss the possibilities of using mobile apps with augmented reality in pre-primary education. We will introduce some of the most popular and widely used applications with examples of uses in kindergarten. We can use the camera of mobile phones or tablets to explore 3D objects. These applications offer an immersive experience and edutainment. AR technology in education is developing rapidly and we think that the alpha generation classrooms could be enhanced and transformed by this technology.

Keywords mobile applications – mathematics education – kindergarten

1

Introduction

Kalas (2010) talks about the need to teach digital technologies in kindergarten. He writes in his book about children’s curiosity, which involves asking questions, and telling and listening to stories about themselves, other people and things. Children like to draw houses, animals, trees, their parents, and fairy-tale © koninklijke brill nv, leideN, 2020 | DOI: 10.1163/9789004408845_005

102

Korenova et al.

heroes, etc. Also, they enjoy creating something, like playing and interacting with other people and animals. ICT (Information Communication Technology) can help children do many of such things mentioned above. ICT can provide content and activities that originate and support strong and productive emotions for children and Kalas (2010) shows that they can serve as the environment and tools for the development of a child. However, if we are to use ICT to support in early learning across the curriculum, then technology should be integrated to support the development of positive dispositions of children toward learning. One of the most appropriate curriculum models is to draw upon, therefore, arguably an emergent one (Sylva & Siraj-Blatchford, 2006). We should encourage young children to apply ICT tools for their own purposes in their play. If we adopt an integrated approach, we can provide opportunities for children to develop in all areas of learning (Kalas, 2010). Sylva and Siraj-Blatchford (2006) identify four key areas of learning in Early Childhood Environments (ECE), which is a branch of education theory about the teaching (formally and informally) of little children up through the age of eight, and reflect on how ICT could support them. These are: – communication and collaboration – they naturally appear in collaborative problem solving, drawing, video recording, or construction, using screenbased applications, in experimenting with programmable toys; – creativity – to be creative, children need to acquire a repertoire of schemes, and they need the playful disposition to try out these schemes in new contexts; – socio-dramatic play – there is an enormous scope for the integration of ICT into young children’s play environments; – learning to learn – there is a strong evidence that computers can be applied to help even young children think about thinking (as suggested by Papert, 1980), and that ICT applications that support the development of metacognition and learning to learn are also those that most effectively support communication and collaboration and socio-dramatic play (Sylva & Siraj-Blatchford, 2006). We believe that visual imagery is more capable of spreading knowledge than other communication channels. According to Piaget’s research (Piaget, 1978; Piaget & Inhelder, 1999), the evolution of thinking is manifested simultaneously in several areas: speech, music and expression, shape, form and field of vision, ability to represent a drawing, symbolic communication skills, etc. Facts and ideas can be transmitted more deeply by visual language than any other communication tool (Kepes, 1979). Arnheim (2004) argued for the unity of perception and abstract thinking, according to which mental processes do

Augmented Reality Applications in Early Childhood Education

103

not only consist of operations with words and numbers, but also imaginary thinking. For instance, augmented reality technology can transform a simple sandbox into an innovative interactive equipment for education and entertainment. Interactive Sandbox (iSandBOX) consists of a box filled with regular beach sand, a depth sensor and a projector on top that produces interactive images onto the sand (see Figure 5.1). By moving and building the sand, users can create erupting volcanoes, rivers, oceans, mountains and other geological formations. In 2017, Lego launched its new AR magazines and the LEGO AR-Studio application offers a new AR experience. LEGO AR-Studio includes six virtual sets that tap into the popularity of games like Pokémon Go by images coming to life on a table or living room floor. Using a mobile device and the Lego AR app, we can show our children (or maybe, by themselves, they can display) 3D models of Lego structures and they can decide which is most interesting for them. This is also aligned with the goals and purposes of technology integration at his age. Augmented reality children’s books are also popular even for kindergartenage children. For instance, in Slovakia, Slovak translations of Disney Princess, Mythical Beings and Dinosaurs Come Alive have been published. In Hungary,

figure 5.1  iSandBox

104

Korenova et al.

BOOKR Kids application have made children’s books interactive while the content remained the same, only the presentation method and the platform changed. The aim of these books is to make literature part of children’s lives.

2

DIFER – Skills Development Program for 4 to 8-Year-Olds in Hungary

Researchers at the Department of Educational Science at the University of Szeged under the leadership of József Nagy have shown that the development of basic skills determines the success of school learning in the long term, and that there are major differences in the level of intellectual and social development of students at the same age. Based on these, the DIFER (Hungarian acronym of Diagnostic System of Assessing Development) was constructed by József Nagy and his colleagues in 2003 for 4 to 8-year-olds. The first data collection takes place individually, at the end of the first school year. The data collection is done by class teachers and external experts. The purpose of developing the program was to create a tool to help nursery and school skills for children’s development (Figure 5.2 and Figure 5.3). The Diagnostic System of Assessing Development (DIFER) tests provide a diagnostic picture of the skill level, covering all its components. The diagnostic map of skills development can show which skills a child has already mastered and what kind of development work is still needed.

figure 5.2 Detail from the DIFER – Speech and sound hearing

Augmented Reality Applications in Early Childhood Education

105

figure 5.3 Detail from the DIFER – elemental numeracy skills

DIFER supports the development of seven elementary basic skills. All of them can be considered a prerequisite to personal development and school learning. These are the following: 1. The prerequisite for the acquisition of writing skills is the critical elementary skill called “writing movement coordination.”

106 2. 3. 4. 5. 6. 7.

3

Korenova et al.

Speech and sound hearing and understanding is essential for reading and writing. development of the relational vocabulary, one of the determining factors for the acquisition of the information communicated. The development of mathematics learning and elemental numeracy skills. The development of discovery-based learning The development of experimental deduction The development of social relationships (with peers, adults), so-called “sociality,” the decisive criterion for effective school integration and learning.

The DIFER Test

Diagnostic mapping of skills development takes between 45 and 50 minutes to complete. This is followed by the preparation of a diagnosis in the light of the results and the compilation of the individual development proposal and the beginning of continuous development. In each case, after 4 to 8 months of development a control measurement is also performed. Comparative analysis of results of the control measurement and the input measurement, determines the degree of development of the individual. Experience in testing showed that a significant number of parents are interested in the results of the measurements and support and appreciate measurements and development work. This could be highly relevant for developing and evaluating AR applications.

4

Augmented Reality in Education

The use of mobile AR technology applications is a method with which the work of educators can be enhanced if used well and can enable pupils to become more actively involved in the education process. It is hoped that AR can make studying both in class and outside class become more attractive and experiential. It may be an effective tool for reaching students who have been raised with extensive access to visually stimulating entertainment. Besides the innovative nature of the technology and the multifaceted opportunities for its usage in education, an additional advantage is that it does not require special financial or human resources. There is no need to involve

Augmented Reality Applications in Early Childhood Education

107

a separate informatics expert in the development of study materials, because teachers can learn produce these themselves without deep programming skills. It is to be expected that within a few years, a significant proportion of children will have a device, such as a smartphone, which is able to run AR applications, so there will no need any for particular acquisition of hardware appliances either. All these could enable a rapid spread of technology in education, too. For instance, the Street Museum iPhone application produced by the Museum of London, or “Nowy wymiar Sukiennic” (New dimension in the Cloth Hall) from the Muzeum Narodowe Krakowie in Poland presented successful applications of AR technologies for education. For augmented reality in education, a variety of theories and a spectrum of solutions and analyses have been proposed (see, for example, Azuma, 1997; Azuma, Baillot, Behringer, Feiner, Julier, & MacIntyre, 2001; Coimbra, Cardoso, & Mateus, 2015; Diegmann, Schmidt-Kraepelin, Van den Eynden, & Basten, 2015; Ferko, Martinka, Sormann, Karner, Zara, & Krivograd, 2004; Haladova, Szemzö, Kovačovský, & Žižka, 2015; Prodromou, Lavicza, & Koren, 2015; Rambli, Matcha, & Sulaiman, 2013). The visual impact of AR is important because with little children, motivation is a crucial factor. For example, when colored images appear in space and start to move (Quiver 3D Augmented Reality coloring apps), then we can prompt kindergarteners to color some more, thus developing their manual skills. With schoolchildren, it can also be highly motivating, if AR can help make difficult fields, such as geometry, more comprehensible for pupils.

5

M-Learning with App

Using mobile devices in learning can have ample benefits. We can learn anywhere, anytime, anything, without taking our notes or books. This learning format is also called mobile learning or m-learning. In the following, we studied that area of m-learning which relies on the use of AR apps. How do we start introducing mobile learning with applications? The main steps for introducing mobile learning with applications that schools/teachers should consider (Fegyverneki & Aknai, 2017): 1. search application 2. mobile devices: involvement of students’ tools (BYOD) 3. internet: in many cases internet is needed only for downloading the app 4. setting out the rules for mobile usage and the exit points 5. methodological awareness: we only use it if it has added value

108

Korenova et al.

We will review these steps in detail and offer some insights. 1. Searching for applications can be done thematically, for instance, we can try to find geometry-related apps. We need to know that the number of applications is growing steadily every day. For example, in May 2018 in the AppStore, the number of apps related to education is over 500,000, and the number of applications related to mathematics is over 3000. Depending on the number of data listed, it is worth to narrow down the set of applications and to get access with different filtering techniques to a reduced list of applications. Another option is that we hear at a conference, workshops or from colleagues about effective and free-of-charge applications that they consider to be useful. 2. The BYOD (Bring Your Own Device) allows use of AR when there are no devices in the classroom. 3. If you are unsure how the Internet will work, then download the applications in advance 4. The rules have to be discussed in advance. Exit points are the termination of device usage, which is discussed in advance. 5. The conscious use of equipment is one of the great challenges of our time. Not only for students but for teachers it can also be a problem. Particular emphasis should be put on when, for what and why we use the device. Methodological awareness means that we need to try to decide which part of the curriculum would be really useful for the acquisition of knowledge to involve the tools. At this stage, the teacher can decide on his/her own ideas and can shape the lesson. It must be kept in mind that the device – as its name suggests – is just a tool for us and not the basic of the lesson. Practically, if you have a difficult specific theme in case of which the usage of mobile devices can make it easier, then it is reasonable to choose it.

6

AR Applications and Their Use for Kindergartners

We are talking about the usage of augmented reality in education when with the aid of an application on a mobile device, virtual objects appear in the space and this information helps pupils understand new materials, complements the teachers’ work, and motivates learning. We propose a list of applications categorized by different elementary skills groups. To determine these groups, we used the Hungarian program and its seven elementary basic skills: writing movement coordination, speech hesitation/hearing, development of the relational vocabulary, mathematics learning and elemental numeracy skills, experimental deduction, development of the

Augmented Reality Applications in Early Childhood Education

109

understanding of the empirical context, development of social relationships (with peers, adults), sociability. Related applications groups are: development of literacy, development of numeracy, experience-based learning and discovery-based learning, development of social abilities. At the end of this section, we show these four groups in a table and we attach examples for each. We present some applications that can be used both in kindergarten and out-of-kindergarten activities, managed by parents. We have tried these apps and we have observed which one captures them. There was one criterion, motivation. In other words, raising and maintaining attention. We use our selfmade photos (Figures 5.2–5.12). 6.1 AR Dragon AR Dragon (Figure 5.4) is a pet simulator. The educational purpose of this AR app is the care and responsibility of a pet. It helps children learn to take care of a creature, with the intention of helping children develop social abilities. For children have the responsibility of both feeding and training the dragon, after which it acts friendly to the children. It is able to console when someone is sad. Caring for the AR dragon is similar to Tamagotchi, a Japanese keychainsized virtual pet simulation toy from the 1980’s. The little dragon changes its color, can fly, swear, walk around, give different voices randomly (in our observations, children laugh at this). It indicates when

figure 5.4 AR Dragon on the street

110

Korenova et al.

it would like to play with the ball, will act sad if it doesn’t play. It acts grateful for being touched. If they do the right things, they children can put it on their hands. 6.2 Sketch AR This app puts virtual images on white paper to help children draw, in hope that the success will inspire children to continue their drawings (Figure 5.5). Children see a virtual image on the paper, which they can make a sketch. Sketch AR works correctly only on an A4 or A5 size paper, in a sufficiently bright room. It is necessary to fix the paper because it may move during the process of sketching. The app helps developing the writing movement coordination of pupils as well as provides a sense of success. 6.3 Quiver – Platonic Solids The Platonic Solids tool is part of the Quiver Education application (Figure 5.6). With its help, students can visualize the five Platonic Solids: Tetrahedron,

figure 5.5 Drawing a horse with aid of Sketch AR

Augmented Reality Applications in Early Childhood Education

111

figure 5.6 The dodecahedron and appearance of the aithêr

Hexahedron, Octahedron, Dodecahedron and Icosahedron. The benefits of using this app are both the development of experimental deduction and development of mathematics learning. On the printed pages are displayed only the surface of the bodies, but with the help of the app, they are presented in 3 dimensions associated with four classical elements (earth, water, air, fire) and the fifth element, aithêr (aether), added by Aristotle. Our test of this app was impressive, and we think that it offers more than just presentation of the polyhedrons. It teaches not only geometry, but also the history of science. 6.4 AR FlashCards Shapes and Addition AR FlashCards is an application family. We tried the AR Flashcards Addition (Figure 5.7) and AR Flashcards Shapes (Figure 5.8) applications. Although the cards to be printed are free on the webpage, the apps are paid. These apps can help both in the development of experimental deduction and in the development of mathematics learning and elemental numeracy skills.

112

Korenova et al.

figure 5.7  Addition app’s card

figure 5.8 Multiple shapes at the same time

Augmented Reality Applications in Early Childhood Education

113

Firstly, the Addition app clearly illustrate the addition of digits from 0+1 to 9+9. We can show the addition of numbers followed by an image, which is for children and if they see that null animal plus six lions visualized the sum is more understandable for them. Secondly, the Shapes application (Figure 5.8) give possibility for coloring the shape. Additionally, we can hear the name of the color and the name of the shape in English. 6.5 Aurasma, Walla Me, QR Code With these applications (Aurasma, Walla Me and QR code) (Figure 5.9) we can make a treasure hunt game for kindergarteners. The teacher may hide something (for instance, solutions for a task) with one of these apps in the classroom and children using the relevant app can decode the solution or the message. The coding and decoding function is similar in the three applications. Differences

figure 5.9 The Walla Me private message

114

Korenova et al.

can discover in tools offered by them. For instance, with Walla Me we can hide a secret AR message which can be a drawing on the wall. The apps assist in experience and discovery-based learning, which is desired in many curricula. 6.6 Stack AR This is a game. The task is to stack up the blocks generated in the AR as high as you can (Figure 5.10). This is not a special education app but it visualizes the construction of the tower and at the same time it improves “writing movement coordination”—fine motor control—and the development of experimental

figure 5.10  The tower on the table made by Stack AR

Augmented Reality Applications in Early Childhood Education

115

deduction. Children put the AR blocks in the real word, for example the tower appears in their room, on their table with the help of augmented reality and they experience that only the well-placed parts remain, and the others fall. Using this app, we increase attention and concentration as well. 6.7 Animal 4D+ With this app we can show animals in 3D on our table (Figure 5.11). Children can look at these animals. Unfortunately, when we teach about pets and wide animals, we rarely have the opportunity to visit them in their natural habitat or a zoo to observe them. This app will help both in the development of experimental deduction and in the discovery-based learning. It is exciting for children when the animals come to life in front of their eyes. If the sound is enabled on the smart device, they can hear the sound of the animals as well. If we put two cards next to each other (a plant and an animal) that are likely to join together and we allow the interaction, then the animal moves next to the corresponding plant. This way we can show the nutrition of different animals. The language of the application is English, so it is a great opportunity for language practice. As with several other apps, this could enhance experience- and discovery-based learning.

figure 5.11 A turtle on my table

116

Korenova et al.

figure 5.12 Lego product 3D animation

figure 5.13  A colored animal who eats his food when it comes to life with the app

6.8 Lego 3D Catalogue Lego 3D Catalogue shows the new Lego products’ 3D animations (Figure 5.12) which children find exciting. They can explore the elements of a construction from all sides. The Lego catalogue can help in teaching math visually for several reasons. For instance, Lego bricks can teach children about counting and measuring, can show patterns and visualize arithmetic problems. Finally, the

Augmented Reality Applications in Early Childhood Education

117

table 5.1 Skills and suitable applications

Skills

Applications

Writing movement coordination (fijine motor control) Speech and sound hearing and understanding The development of mathematics learning and elemental numeracy skills

Sketch AR, Quiver Coloring, Walla Me, Stack AR, Lego 3D Catalogue

The development of discovery-based learning The development of experimental deduction

The development of social relationships

StackAR, Augemnted Polyhedron, AR FlashCards Addition and Shapes, Quiver Platonic Solids WallaMe, Aurasma, QR Code, Animal 4D+, Lego 3D Catalogue Ar Dragon, Quiver Platonic Solids, AR FlashCards Addition and Shapes, WallaMe, Aurasma, QR Code, Stack AR, Animal 4D+ AR Dragon

app assists in development of fine motor control and discovery-based learning as well. 6.9 Quiver – 3D Coloring App Quiver is a coloring app (Figure 5.13) supporting both learning and fun. Every colored page comes to life in its uniquely colored way. The coloring activities assist in the development of fine motor control. The Quiver – 3D Coloring App combines physical coloring with augmented reality technology. Pages for painting can be downloaded and printed free of charge from the http://www.QuiverVision.com website. In Table 5.1 we summarize the categorization of applications by skills which they develop. The groups are: writing movement coordination (also called fine motor control), speech and sound hearing and understanding, development of mathematics learning and elemental numeracy skills, the development of discovery-based learning, development of experimental deduction and development of social relationships. There are no apps listed for development of speech and sound hearing and understanding because these apps are in English and our trials were with Hungarian children.

118 7

Korenova et al.

Conclusions

For those in the same generation as the writer of this article, it is almost automatic to use Google for searches; and today’s teenagers already search directly on YouTube. Kindergartners nowadays, the alpha generation, differ to an even greater extent from the generation preceding them with regard to their digital development and habits and have no trouble using mobile devices, almost demanding that the education system introduce them alongside traditional methods. We know that there are many who are skeptical about accepting the use of mobile devices in education, so we should like to stress that AR technology (or even Gamification) is a means available to an educator, but it does not replace the work of teaching. It is important to emphasize that the opportunities offered by applications based on augmented reality technology do not need to be used at all costs or every day, but the opportunity must be taken when we judge that they may greatly contribute to the children’s development and the teacher’s work. Because of their visual appeal, we expect that AR will fascinate kindergartners, to the point it is difficult for them to stop the activity and they have an immersive learning experience. Perhaps they are touched by the Flow experience which Csíkszentmihályi (1997) defines as the phenomenon when our consciousness is harmonically ordered, and we want to continue the activity we are pursuing just for the sake of it. AR technology is still so new, that a teacher can become just as engrossed in the use of the applications, and it can seem incredible to possibly expand children’s knowledge in ways never dreamt of earlier. Today, thanks to augmented reality and holographic technology, a teacher can even demonstrate a whale splashing in the water in the middle of the group room. We think that alpha generation classrooms will be transformed significantly, and although the use of the new technologies may appear costly at first sight, in the long term, as the technology becomes more common, its costs will fall.

Acknowledgement The chapter was written the support of the grant KEGA 012UK-04/2018 “The Concept of Constructionism and Augmented Reality in the Field of the Natural and Technical Sciences of the Primary Education (CEPENSAR).”

References Arnheim, R. (2004). A vizuális élmény [Visual thinking]. Budapest: Gondolat Kiadó. Azuma, R. T. (1997). A survey of augmented reality. Presence: Teleoperators and Virtual Environments, 6(4), 355–385.

Augmented Reality Applications in Early Childhood Education

119

Azuma, R. T., Baillot, Y., Behringer, R., Feiner, S., Julier, S., & MacIntyre, B. (2001). Recent advances in augmented reality. IEEE Computer Graphics and Applications, 21(6), 34–47. Coimbra, M. T., Cardoso, T., & Mateus, A. (2015). Augmented reality: An enhancer for higher education students in math’s learning? Procedia Computer Science, 67, 332–339. Csíkszentmihályi, M. (1997). Flow – Az áramlat. Budapest: Akadémia KIadó. Diegmann, P., Schmidt-Kraepelin, M., Van den Eynden, S., & Basten, D. (2015). Benefits of augmented reality in educational environments: A systematic literature review. Wirtschaftsinformatik, 3(6), 1542–1556. Fegyverneki, G., & Aknai, D. O. (2017). A mobiltanulás ábécéje pedagógusoknak [The Alphabet of Mobile Learning for Teachers – Methodological and technical ideas for mobile students]. Budapest: Neteducatio. Ferko, A., Martinka, J., Sormann, M., Karner, K., Zara, J., & Krivograd, S. (2004). Virtual heart of Central Europe. Vienna, Austria: Selbstverlag des Instituts für EDV-gestützte Methoden in Architektur und Raumplanung der Technischen Universität Wien. Haladova, Z. B., Szemzö, R., Kovačovský, T., & Žižka, J. (2015). Utilizing multispectral scanning and augmented reality for enhancement and visualization of the wooden sculpture restoration process. Procedia Computer Science, 67, 340–347. Kalas, I. (2010). Recognizing the potential of ICT in early childhood education. http://unesdoc.unesco.org/images/0019/001904/190433e.pdf Kepes, G. Y. (1979). A látás nyelve. Budapest: Gondolat Kiadó. Papert, S. (1980). Mindstorms – Children, computers and powerful ideas. New York, NY: Basic Books. Piaget, J. (1978). Szimbólumképzés a gyermekkorban [Play, dreams and imitation in childhood. New York, NY: Norton, 1962]. Budapest: Gondolat. Piaget, J., & Inhelder, B. (1999). Gyermeklélektan [La psychologie de l’enfant]. Budapest: Osiris Kiadó. Prodromou, T., Lavicza, Z., & Koren, B. (2015). Increasing students’ involvement in technology-supported mathematics lesson sequences. The International Journal for Technology in Mathematics Education, 22(4), 169–178. Rambli, D. R. A., Matcha, W., & Sulaiman, S. (2013). Fun learning with AR alphabet book for preschool children. Procedia Computer Science, 25, 211–219. Sujansky, J., & Ferri-Reed, J. (2009). Keeping the Millennials: Why companies are losing billions in turnover to this generation-and what to do about it. Hoboken, NJ: John Wiley & Sons. Sylva, K., & Siraj-Blatchford, I. (2006). Capturing quality in early childhood through environmental rating scales. Early Childhood Research Quarterly, 21(1), 76–92.

PART 2 Advanced Education

∵

CHAPTER 6

Mathematics Learning and Augmented Reality in a Virtual School Gilles Aldon and Corinne Raffin

Abstract Virtual reality and immersive worlds come together with augmented reality (AR) in the educational landscape with crucial questions regarding their construction, their potential educational uses, and their implications for teaching and learning. The context of this research is a dual space comprising a secondary school, and a 3D, virtual world analogue of the school in which students and teachers act through evolving avatars. This study examines an augmented reality secondary school, especially its educational uses and conditions necessary for its successful development and adoption. The objectives are to assess the pedagogic contributions of an immersive space, and participate in the development of new pedagogical strategies. A main part of this evaluation will focus on the impact of this tool on the pupils’ working methods, on their acquisition of competences and knowledge, and on their relationships with mathematical objects through augmented reality embedded in the virtual world.

Keywords immersive pedagogy – virtual world – augmented virtuality – augmented reality – mathematics – dynamic mathematics

1

Introduction

It is generally accepted from Azuma (1997) and other research (Yuen, Yaoyuneyong, & Johnson, 2011; Zhou, Duh, & Billinghurst, 2008) that augmented reality is characterized by three main properties: – the combination of real-world and virtual elements, – an interaction in real time, – a 3D representation. © koninklijke brill nv, leideN, 2020 | DOI: 10.1163/9789004408845_006

124

Aldon and Raffin

In contrast to AR, augmented virtuality is a representation of the current state of real world elements in technological environments. In a virtual world, realworld tools or components are at the user’s disposal in a virtual environment in real-time interaction. The virtual school discussed in this chapter is in the midpoint of augmented reality and augmented virtuality, in the sense that it proposes a virtual environment in which teachers and students can deal with “real” mathematical objects and tools that can be augmented by properties given by the digital environment. But also, when students work together in the same classroom and, in parallel, circulate in the virtual school, through their avatars, acting on mathematical objects, the combination of real world and virtual elements is present and gives augmented information about the objects at stake. The word avatar has been used in digital games since the 80s after having been used in the religious or mythological context of the appearance of a god in human or animal form. But, in a 3D world, the player doesn’t become the avatar, as for example Zeus became the swan to seduce Lena. The relationship of the player and his/her avatar is more an embodied commitment of the player in the 3D world (Gazzard 2009): [Zeus’s] positioning became the swan’s positioning; he was not present separately as both himself and as a swan in two separate places. This is not possible in a virtual world. The screen does not allow for the translation of our bodies into the virtual body, therefore we are present in dual realities, of both the quotidian and the virtual world. […] Two opposite directions determine the position of the learner within the world: the avatar stays external to the learner and his/her engagement is only linked to the resources present in the world without acting on them or the avatar becomes an actor of the world itself taking into account its possible interactions with the world. (p. 191) Exploiting 3D worlds in the context of education leads us to consider the main issues regarding, first, the way teachers organize and orchestrate the virtual environment to suit teaching objectives and, second, the relationship with and role of a virtual-environment avatar in processes of effective learning. The questions that arise naturally in that context are linked both to the ergonomic aspect of adopting and adapting the platform in a teaching and learning context and to the cognitive aspect when students are confronted with a digital 3D world. Is it easy to use? Is it acceptable in the learning institution in which students and teachers evolve? Does the world bring a new learning experience to the students, allowing them to reach their learning goals? Is it an obstacle or is it a force motivating a successful learning experience?

Mathematics Learning and Augmented Reality

2

125

General Context

This study takes place in a dual virtual space of the low secondary school of Fontreyne in Gap in the south-east of France. Since 2014 this school has been a “connected school,” which means that a particular effort has been made by the local authority to equip the school with digital infrastructure (network, computers, tablets, etc.) and simultaneously, the headmaster and teachers develop innovative pedagogy using technology. In this context, most of the pupils have their own tablet which they can use in and out of school. The company “Immersiv colab,” a partner in this experiment, supplies an immersive 3D platform built on the technology of “Second Life,” which allows the participants to build virtual objects from simple geometric shapes. Teachers can put documents and other resources at the students’ disposal, while students can build, manipulate, and explore mathematical objects in this collaborative work environment. Tools facilitating and promoting the collaboration between actors are available such as collaborative boards, special virtual places, interactive software, and chat (Figure 6.1). The purpose of virtual reality, according to Fuchs (1996), is to “allow someone (or several persons) to have a sensorimotor and cognitive activity in a virtual world, created digitally, which may be imaginary, symbolic or a simulation of the real life” (Fuchs, 1996, p. 6). In fact, this virtual world looks like a digital version of a school with rooms, schoolyard, sandbox, places where students can meet, exhibition rooms, a “language room,” a “maths room,” a “documentation room,” etc. The various actors (students, teachers, educators, researchers) are present in this space via avatars. This environment is considered “partially

figure 6.1 A place of collaboration with shared boards

126

Aldon and Raffin

immersive” because the user doesn’t need a special interface requiring stereoscopic glasses or gloves such as would be necessary for virtual reality. The visitor exists by means of his/her avatar, an image of an adult that can be entirely customized. He or she moves in the world according to diverse sensorimotor modalities: in subjective, or external sight, by walking, running or flying and even by teleporting to a particular place. Participants can communicate within the world through writing (chat) or orally using the audio capabilities of their computer. And last but not least, via their avatars, users can build and manipulate objects and can also choose to supplement them with additional information. We refer to the functional classification of the augmented reality (Fuchs, Hugues, & Nannipieri, 2010), to describe functionalities of this virtual environment. This taxonomy identifies two distinct groups of virtual environments. The first concerns environments where functionalities inform the real environment. In such cases, there is an augmented perception of reality. The second group concerns environments where the construction of an imaginary world is possible with appropriate tools. Each of the two types can be divided into subtypes depending on the information given by AR. For example, in the first type, the augmented reality can be seen as a documented virtuality when there is no interaction between the objects and their additional information, or as augmented visibility when additional information appears, augmenting their visual perception, or as a copy of the real world in which virtual objects appear without interactions with the user. In the case of our experiment, we can say that the platform belongs to the second group. Students and teachers create an imaginary world augmented with objects coming from pictures of the real world, or, sometimes, they give to their avatar an appearance coming from fantasy literature. However, this platform is a copy of the real world where the mathematical objects may appear with mathematical information in a window next to the object, or with outlines for a better perception and understanding of geometrical properties (Figure 6.2). In this case, we can say that the environment offers a reality with augmented visibility and documented virtuality. This technology includes tools for augmented perception and for a better understanding of the reality. This augmented visibility is reinforced by the possibility given to users to manipulate the objects. Users can move, turn, enlarge, or reduce them, proportionally or not. For these manipulations, several mathematical tools are available: graduated rules, grids, and geographic or geometrical coordinates, which appear on the screen, integrated within the virtual landscape or only in the

Mathematics Learning and Augmented Reality

127

figure 6.2 Object augmented with geometrical data

properties of the geometrical figure. This virtual world brings in mathematical tools that are common in mathematics classrooms, as well as some that are less common, like the dynamic display of avatars’ coordinates at the top of the screen. Using this technology for learning purposes requires specific digital competencies. Students who are already heavy online gamers adapt easily to the virtual world, quickly learning to refine the interface to suit their preferences. For all students, even if the interface is intuitive and they are experienced, a phase of instrumentation is necessary to understand and use the possibilities and transform those possibilities into usable tools for learning. This collaborative space may cultivate peer support and thereby promote the use of the technology in or out of the classroom. In interviews, students expressed their belief that gamers have technical skills and can be instructional resources for one another. This study unfolds within an associated educational place (LéA, Lieu d’éducation associé in French; Chabanne, Monod-Ansaldi, & Loisy, 2016), translated into “AeDeP” at FIE (standing for Associated Educational Designexperiment Places at the French Institute for Education). Les LéA ont été définis dans le programme scientifique de l’IFÉ comme des lieux à enjeux d’éducation, rassemblant un questionnement des acteurs, l’implication d’une équipe de recherche, le soutien du pilotage de l’établissement, et la construction conjointe d’un projet dans la durée. Il s’agit de considérer l’éducation comme un fait social total et de fonder des recherches en éducation sur l’action conjointe entre chercheurs et acteurs du terrain.1

128

Aldon and Raffin

This experiment began two years ago, and since then this technology has been actively tested and developed in several contexts. It aims at investigating a new design of the educational territory, first, by establishing a link between primary and secondary school and between French and foreign schools. This virtual world became a meeting place, allowing students from different schools, abroad or in the region, to interact and collaborate. Visits of exhibitions, cultural exchanges, and games are organized, aiming at developing language skills (Figure 6.3). Second, this initiative responded to the requests of the participating educational institution by offering a special place for students’ individual homework with a teacher’s assistance. At the same time, pedagogical scenarios are developed, the goal being to work differently in order to differently approach pupils’ learning. For example, a class project has been the design of a game with students; the theme of this game is the “seven wonders of the world.” Five school subject areas, English, mathematics, history, French, and plastic arts collaborate to develop this project. Our study is based on one mathematics lesson which has been designed to integrate this technology.

3

Theoretical Framework

In order to answer the questions regarding the adoption and adaptation of the 3D world to serve teaching and learning goals, it is necessary to deal both with ergonomic and learning theoretical frameworks. The first gives a frame allowing explanation of the way teachers and students use and adapt the 3D world and the latter allows better understanding of the relationship between the teachers’ intentions and the students’ learning within such an environment. The second models the way teachers put students into a situation where, confronted with the environment, they can build their own knowledge in an a-didactical situation (Brousseau, 1986, 1998, 2006) that is to say, a situation built on the confrontation of the student with the environment (the “milieu”) that the teacher planned.

figure 6.3  English homework with a teacher

Mathematics Learning and Augmented Reality

129

The first issue concerns the platform itself. Three main questions can be posed about the potential to work with it: is the environment acceptable? Is it usable? Is it useful? The first question concerns the norms, values, motivation, and moods of users that will decide on whether to use the platform or not (Bétrancourt, 2007). Acceptability (Tricot, Plégat-Soutjis, Camps, Amiel, Lutz, & Morcillo, 2003) can be defined as the value of the mental representation of usefulness and usability. It is directly linked to both these concepts but also takes into account external perceptions of the digital tool. In the case of virtual school, because it is built on a game engine, the question of acceptability is particularly crucial, not only for students and teachers but also for the school community. In order to evaluate the acceptability of the platform, motivations and values have to be taken into account. The relationship to the institutions is an important issue. By “institutions” we refer to social or educational organizations which are connected with the school. At a large scale, this can be the national institution of education, or at a local scale, an individual school or even a class within the school. In the case of the introduction of a virtual environment in an institution, the difficulties can be examined at both the large and the local scales. In one case, the issue is the acceptability of such an innovation within the national curriculum, and in the other, the acceptability in the local educational institution: first, the school community, in general, including teachers, headmaster, students, and parents, and second, any specific class considering the use of a virtual school in teaching specific subject matter. Regarding an Intelligent Tutoring System (ITS), the two concepts of usefulness and usability (Chughtai, Zhang, & Craig, 2015) are used to explain how and whether users handle the digital environment effectively. Usefulness refers more precisely to the pedagogy and the didactics: does the ITS allow teaching (and learning) of the prescribed knowledge in a certain curriculum and what does it bring on top of that? It is necessary to assess whether the ITS achieves the pedagogical intentions (Tricot et al., 2003). In that sense, the augmented reality and the augmented virtuality within the virtual environment are key points of the usefulness: what does it bring in terms of learning, and how is it possible to assess whether the AR or the VR favour the learning of knowledge? These are the main issues of usefulness analysis. “Usability” concerns the facility or difficulty actors experience when using the virtual environment. The coherence of the platform ergonomy can be assessed both by inspection and empirically: – looking at the possible ways to perform given actions, for example walking, flying, speaking with others, communicating and so on,

130

Aldon and Raffin

– observing how teachers and students deal with the potentialities of the environment. When confronted with any new tool, its use and application for a certain aim may be neither immediate nor obvious. The transformation of an artefact, that is to say an object without didactical intention (Sensevy, 2010), into an instrument as a construct associating the artefact and schemes of utilization is a long process. This process, studied initially in an ergonomic context (Rabardel, 1995) has been adapted and developed in a didactical context (Artigue, 2002; Drijvers, Doorman, Boon, Reed, & Gravemeijer, 2010; Guin & Trouche, 1999; Lagrange, 2000). The process of transformation of an artefact into an instrument, called instrumental genesis, needs time and links the artefact to the actors’ activity through a double movement of instrumentation and instrumentalization. Instrumentation links the artefact’s properties to the actors’ activity, augmenting their possibilities of action. Instrumentalization is the necessary adaptation of the artefact by the actors for their own use. It can include the transformation of the artefact, changing menus, adding programming shortcuts, and otherwise modifying the behaviour of the artefact for their own use. But also, instrumentation is developed using the tools provided by the environment in order to create personal tools or places, personalizing the environment, such as this house that was created by a student (Figure 6.4). Finally, “instrumentalization is a differentiation process directed towards the artifacts themselves” (Trouche, 2004, p. 293). The research questions deal with the use of a 3D platform in a teaching and learning context. We ground our observations in constructivist theories and more particularly in the Theory of Didactical Situations (TDS, Brousseau, 1986, 1998, 2006). The generic learner, confronted with a specific environment, learns by adapting his/her knowledge to the feedback from the situation environment. Taking a didactical point of view, we consider with Brousseau, that a didactical situation is the set of conditions for the utilization of mathematical knowledge. In this view, a situation can be modelled as a win-win game about a mathematical knowledge. Students win when their knowledge is modified and enhanced at the end of the game. Teachers win when all students learn something. When speaking of a game, it is necessary to define both the game play – the rules and the goal – as well as the specific board where the players can act. In the case of this didactical game, the rules focus on the knowledge at stake. The difficulty inherent in a learning situation is that the goal, that is to say the mathematical knowledge at stake, cannot be revealed before students encounter it. So, the didactical contract calls for the teacher to give to the student the ownership of his/her own learning and for the student to accept the game, trusting in the teacher that he/she will learn.

Mathematics Learning and Augmented Reality

131

figure 6.4 House built by students in the virtual world

The teacher and the student thus enter into what at the time we saw as the negotiation of a type of ‘didactical contract’. Neither party could make the contract explicit, or even maintain it. It was always being broken and renewed, and it was through that contract that the student’s knowledge was created. (Brousseau, 2006, p. 23) The “board” of that game is the whole environment in which the teacher plunges the students. It is called the milieu by Brousseau and includes the material or virtual environment as well as the set of knowledge that students can enlist in order to play the game. The knowledge is then modified, and enhanced through the feedback that the milieu gives to the students’ actions. This knowledge in action has to be institutionalized, that is to say, recognized as common knowledge by the teacher in the particular institution of the class so that students can learn from the situation what they ought to know. In the case of our experiment, it is clear that the milieu is directly linked to the augmented reality provided to students in the 3D platform. And the role of teachers is to organize, to orchestrate in a coherent scenario, the availability of mathematical concepts linked to the mathematical tasks. The teacher wants

132

Aldon and Raffin

to give the best conditions to promote a student’s learning and in parallel the student accepts the responsibility of learning: “Devolution is the act by which the teacher gives to the generic student the responsibility of learning within the situation (a-didactic) or the problem and accepts himself/herself the consequences of this transfer” (Brousseau, 1990, p. 325).

4

Methodology

This study aims at assessing the contribution of an immersive virtual space to learning as part of a larger aim of developing new pedagogical strategies. The educational team of the AeDeP Fontreyne works with researchers in order to develop the use of this 3D world. And so, every actor brings his/her expertise, expectations, and questions. The research questions form the basis for our research design and data collection strategies. Therefore, our methodology had to allow us an ergonomic and a cognitive study of the use of this 3D platform. That’s why the observed participants are both secondary and primary school students and teachers of the participating schools. Teachers involved in the experiment are all volunteers who designed their own pedagogical activities for this technology. The data for analysis are comprised of different sources of information: first, some interviews with students at the beginning of this experience; second, observation of the actions of participants or their avatars in both the virtual world with sound recorded and in the real world in the classes when the technology is involved; and, third, the log books that teachers kept during the experimentation, as well as interviews with teachers. We discuss these data collection methods in the sections that follow.

5

Observations

Audio and video recordings of lessons in the virtual college or in the class constitute data whose analysis provides information about the instrumental genesis and more particularly about instrumentation and students’ learning strategies. We, as researchers are present in the virtual world through our own avatars and so we are able to observe students’ avatars in the 3D space. When the observations take place in the class, we can also observe the same students in person. These two kinds of observations are complementary. We can follow how students integrate this tool in their activity in class and their activities

Mathematics Learning and Augmented Reality

133

as avatar. This confrontation highlights the instrumental genesis of students: what are their intentions with the tool and what do they effectively do with it? We hypothesize that it is in the going back and forth between the virtual space and the classroom where mathematical formalization takes place that learning occurs. The comparison between these two kinds of observations allows us to study the relationship between the teachers’ intentions and the students’ learning within the environment.

6

Log Books and Interviews with Teachers

A methodological difficulty comes from the fact that using the virtual world is a long process and periodic observations don’t show the evolution of the work day after day. In order to follow the experiment more closely, we decided to ask teachers to participate in the data collection in a perspective of reflexive investigation (Gueudet & Trouche, 2006). A shared data folder for the teachers and researchers was created where documents linked to the experiment are available. Within this drive, teachers fill in a common log book indicating the lessons in the virtual world, the didactical and pedagogical intentions, their immediate feedback about difficulties or successes, and also their own feelings about the sequence of events during the lesson. Interviews with teachers complete this data collection. These interviews take place in the school or in the virtual world, just after observations, and are based on both our observations of the sequence and teacher’s feelings. The combination of direct observations and analysis of teachers’ testimonies gives us information regarding both the ergonomic properties of the environment and the cognitive possibilities offered by the augmented reality of the 3D world: What about its usability? Is this tool easy to learn and use? When do students join this virtual world? Do they log in only during school hours or do they also log in from outside of class? How do teachers accept and integrate this technology in their pedagogical environment: easily or laboriously?

7

Students

The didactical situation that we are analysing takes place in a grade 7 class where all students have a tablet available that they can bring home as well as use during courses. In the mathematics classroom they use, in particular, a notepad, OneNote, shared with their teacher. This notepad allows them to

134

Aldon and Raffin

organise their note-taking in class; their teacher can correct it and can share documents with students electronically. Interviews of students aim at knowing their own technology habits; do they often play with virtual games? Are they going on the platform outside of school time? To do what? What are their feelings about this new pedagogy? These questions and those under the previous paragraph are related both to the ergonomic approach of the platform and to the consideration of the platform as an element of the milieu of didactical situations. The milieu is complex and its adoption and use by students depends on their instrumentalization level regarding the functionalities as well as depending on their specific activities with object, and the feedback given by the augmented reality within the virtual world.

8

Sessions Observed

The goal of the work that is at stake in this chapter is to build a virtual rough scale model of one of the Pyramids of Giza in a reserved space of the virtual school. Two sessions were observed. The first session concerned the creation of a ground plane in a special place in the virtual platform to serve as the site used in the second exercise, in which students build their own model of one of the pyramids. For the first exercise, students used maps of Giza found on the Internet. Films of screens of students’ tablets, of the classroom and of a group of 4 students at work, and pictures of students’ work, were collected. The second exercise continued from the first session, and this session and observation of a pair of students took place within the virtual world: the teacher and students were not at the same physical location, so the 3D space was the sole meeting place for the work. During this lesson, students used their OneNote, sharing with the teacher who consulted and corrected it directly.

9

Analysis of the Situation

In the first session, the didactical situation leans on previous research that students had to carry out at home: they arrived at school with information about the Giza site: pictures, maps, plans, etc. The first phase was the presentation of the situation by the teacher, which consisted of gathering and sharing information through the network of tablets. So, the didactical milieu was the material gathered by students and shared in class, the aim of the work (building a maquette of the Giza site) and the

Mathematics Learning and Augmented Reality

135

previous work done on geometrical polyhedrons, and particularly on the square pyramid. It is important to note that the tablets and tools used – One Note, the Virtual School, and drawing software – are part of the habitual milieu of these students. This initial context contributed to the so-called devolution phase where the responsibility for learning is given to the students, and the problem is no longer the teacher’s problem but becomes progressively the students’ one. The fact that the teacher showed the place in the virtual school using the beamer and his own avatar, further contributes to this devolution phase. This is also the first encounter with augmented reality. Indeed, the teacher showed the coordinates that evolve with the movement of the avatar. So, when walking in the virtual school, the position of its own avatar is written in a system of coordinates on top of the window. The game board is now this desert field, including mathematical information, say coordinates of the avatar position and objects’ dimensions, etc., where the maquette will be built. This space is roughly a rectangle of 190mx190m, bordered by rocks, as shown in Figures 6.5a and 6.5b. All the characters of the game are in place. The second phase is actually the interaction of students with this milieu. After separating the class into groups, the teacher asks the students to delimit the field that will be the base of the maquette in order to find the proportionality coefficient. The goal was to mark the four corners of the site according to the map, in order to make a ground plane. First, students have to measure the virtual field. In the real world, measuring a large length requires the use of a tool: either, for example, measuring the length of one’s step and then counting paces, or carrying a suitable measuring tool, or even calculating distances using a coordinate system. Each of these techniques is based on very different theories. In the virtual world, one technique is to build a ruler and to carry it forward, another is to use coordinates. It is this second method that the teacher emphasized. It is important to note that the coordinates given by the virtual space are related to a coordinate system built-in to the system. The coordinate system is available only by the coordinates of the avatars evolving in the virtual space. Axes of the coordinate system are not drawn in the virtual world, and the origin of the coordinate system is outside the field where the experiment takes place. So, the mathematical work was different from the usual exercises about points or shapes in a coordinate system, which are habitually given when a coordinate system is determined by an origin that is visible and axes of coordinates that are (or can be) drawn. Without a visible coordinate system, students have to predict from the positions of their own avatar, moving on the ground to create the dimensions of the rectangle. It was not the only method allowing them to measure the field; and even if the teacher suggested this method, an important part of the

136

Aldon and Raffin

(a)

(b) figure 6.5 (a) The teacher screen showing the available space. (b) A side of this field

activity is that students did try to measure the field using habitual techniques, such as carrying a measuring stick along the sides of the rectangle. In this part, the central mathematical concept is the analytical geometry used in the measurement of the rectangle, that is to say, to locate objects on a plane within a Cartesian coordinate system. Students have to infer distance on straight lines parallel to the axes. Using coordinates to deduce a length, even if the axes of coordinate are parallel to the sides of the rectangle is unusual

137

Mathematics Learning and Augmented Reality

and difficult for 7th graders, as noted by the teacher in his diary: “During the first half hour, students thought about the organisation of the ground plane of the Giza site. There is important work done on the concept of coordinates.” As a remark, the teacher noted “important difficulties of students about coordinates.” Through this experiment in augmented reality, students discovered the links between measurement and systems of coordinates through their avatar’s movement: a student said: “If I walk, the size moves.” This understanding helped the student to visualize the road travelled and to associate the system of coordinates with length and distance. The back and forth from virtual environment to paper and pencil reinforced the pathway from experiment to its interpretation in the mathematical domain. Figures 6.6 and 6.7 show this pathway of two students in this experiment. In the second phase, students had to extract from the map of the site, a rectangle including all the pyramids, and then to transform these measures relative to the available rectangle. Here the mathematical concept at stake was the concept of proportionality. The last part of the lesson was dedicated to the effective placement of the four border markers of the rectangle in the field. The teacher wrote in his diary: “there is a real interest for the modelling work which is inducted by the virtual school.” In addition to remarking that students were happy with this kind of work, the teacher pointed out the modelling work provoked by the environment itself. The motivation induced by the use of the 3D world is important evidence related to acceptability of the platform. What is also important to notice here is the relationship between the real world, the virtual world, augmented reality and the mathematical world. Students adapt methods learnt in the real world to access mathematical methods through the augmented reality present in the virtual world. In this case

figure 6.6  Students extract the coordinates

138

Aldon and Raffin

figure 6.7  Students infer the field’s dimensions

the movement goes from reality to mathematics by the means of augmented reality. The second part of the observation was made within the virtual world. We were present as avatars in the virtual school. During this session, the students had to place and to build a model of one of the pyramids of Giza. We were following two students who were in two different physical places. The goal of their work was first to mark the four corners of the pyramid on the field and second to build a maquette of the pyramid using the correct dimensions. The teacher was also present as an avatar within the virtual school, communicating orally and also sharing documents and information with the tools of the virtual school. T: The bottom of the pyramid is down on the right. With your map, you place the three other points … with the scale … In this case, the teacher used vocabulary which doesn’t make reference to the coordinate system. However, the students began to work with the coordinates of the border stones and when the teacher came back to ask what they had done, there is the following dialog (T is the teacher and E1 and E2 the students):

Mathematics Learning and Augmented Reality

139

T: Do you hear me? E1: Yes. T: the first vertex, (45, 55), is correct. The second, (45, 195), is also correct. But, if you come back down on the right, (245, 35) is incorrect. Could you find the right coordinates? You can talk orally or write in the chat. E1: So, we have found (180, 255) for the vertex down right … T: The border stone is (220, 30). 220 for the horizontal axis and 30 for the vertical one. How do we have to go up? E2: 5. T: No, vertically, how long do you go up? E1: Yes we wrote 5. T: What is on the other one? E1: 25. T: So, the vertical coordinate will be …? Yes I have heard. E2: 55. T: Yes perfect! And for the first coordinate you have to move forward or backward? E1: I move forward. T: You move forward? E1: No I move backward. T: Then you’ll be at which ordinate? E1: Euh … 120. At first, one might think that this dialogue could have been done over the phone. However, we can at least consider three functions of the virtual environment: the first function is here to provide a place where students and teacher meet, modifying the learning space. Thus, doing mathematics with a teacher is no longer limited to the classroom space but also takes place outside of school. The second is that it gives a reference spot allowing expression of movements referring to a place: “you move forward, backward, ….” This important function gives a visual spatial intelligence that students can associate with the mathematical concept of coordinates. In this example, the mathematical concepts that have been used to calculate the position of the pyramid are practiced in the virtual world. The milieu of the situation gives feedback through the visualization of the positions of the avatars in the virtual field. The third is more global and has to do with the experiment of a new learning space which is between the school and the outside world. It foretells a continuity of learning supported by technology. In this case, students are working from outside school but with the presence of their teacher. The virtual world becomes a learning bridge linking school and outside in a continuity of learning.

140 10

Aldon and Raffin

Discussion

The first issue that has been evoked is related to the acceptability of a 3D virtual world and particularly of this world. As explained earlier, one aspect of acceptability is institutional acceptability of the usefulness of the platform in terms of mathematical learning. The observations, showing the role of the virtual world and augmented reality as a medium between the real world and the mathematical model, give evidence about the possibility of using the virtual world to build bridges between reality and mathematics. In that sense and based on this evidence, the school community is in agreement that the platform can be useful, subject to a clear definition of pedagogical purposes and uses. The example discussed in this chapter of the construction of a maquette of the Pyramids of Giza and the way the teacher alternates work on the platform and work with traditional tools shows clearly a new pedagogical approach for the mathematical concepts of proportionality and coordinate systems, as well as for visual spatial intelligence. Acceptability is also linked to the usability of the platform. Both the interviews with students and the observations in the classroom show that learning to manipulate the platform is easy, even if both teachers and students are struggling with issues of instrumentation and instrumentalization. One important result is related to the avatar’s personification by students: not only does the personification of the avatar give students the opportunity to make the platform their own through their own representation and the way they want to be seen by their peers; it also allows them to understand how objects are described and manipulated within the object-oriented programming used to implement the platform. Changing the avatar’s appearance introduces students to the idea that objects in the augmented reality can be described by specific properties, and changed by changing those properties. The acceptability seen as the value of the mental representation of usefulness and usability leans on these possibilities given by AR to view objects’ properties and to transform them as an aid for the understanding of mathematical concepts. On a separate level, the support of the headmaster and the local authorities plays an important role in the acceptability of the platform as a teaching and learning tool. The whole educational community (teachers, students and parents) is impacted by the headmaster support. The coordination of institutional support and the redesign of pedagogical scenarios that take AR into account is a fundamental step for AR’s acceptance as a tool for teaching and learning. The second issue of this chapter enters more deeply into the effective learning that occurs when a situation is proposed to students in the sense of the theory of didactic situations (TDS; Brousseau, 1998). Throughout the observations,

Mathematics Learning and Augmented Reality

141

the main evidence is that effective use of augmented reality depends on the mathematical aim of the activity. If we consider augmented reality as an artefact, its appropriation by students in a process of instrumental genesis transforms the artefact into an instrument, giving students the opportunity to build new reasoning by using the feedback of the situation’s milieu. As an example, when students create a solid, a window appears that gives mathematical information about the solid. This information is dynamic in the sense that students can act on it and immediately see the effect of the modifications on the solid itself; they can also change the shape of the solid and see the changes of the quantitative information about the solid. Augmented reality plays two different roles with respect to concrete or mathematical objects: the first is to give additional information about an object, and the second is to give a new representation of this object. As seen in the second observation, calculation and reflection about the positioning and the dimension of the pyramid have been made in a previous lesson, and augmented reality changes the mathematical strategies employed to solve the problem as it simultaneously participates in the construction of knowledge. A second example that shows clearly the effect of the feedback of AR on the construction of knowledge is a different exercise given by the teacher, in which students were asked to build a cube whose volume is eight times the volume of a given cube. At the beginning of this problem, a student multiplied the length of the cube by eight and immediately saw the cube on the field. The feedback of the milieu, that is to say a cube whose volume has been multiplied by 512, was enough for the student to understand her mistake and to begin to think more deeply about the problem. The augmented reality, as an instrument, exists between the mathematical world and the virtual world. It opens a pathway between mathematical objects and their representations in the virtual world. AR plays a role in supporting the representation of mathematical objects and via its function as a verification tool from mathematics to reality: Maths + AR ==> gives an aid to interpret reality On the other hand, and looking to our first observation, AR plays a role of translation from the virtual reality, for example the dimension of the field, to the mathematical world; here the location of the four vertices within a coordinate system and the determination of the lengths from these coordinates. In that sense, the pathway goes from reality to mathematics. The AR, as an instrument, is between the two worlds, but now, leading from reality to mathematics by way of virtuality. The possibility to collect coordinates through AR facilitates the feedback from the milieu and promotes a mathematical strategy

142

Aldon and Raffin

figure 6.8 Back and forth from mathematical world to virtual world

that is uncommon for ninth graders: using coordinates to determine dimension. This thinking path reinforces the understanding of the coordinates, and thus modifies the students’ system of knowledge. But also, the instrumentalization of the AR is different from representation and control as seen before. In this case, the instrumentalization leans to the construction of a new mathematical method. AR can be seen as a pathway from the virtual world, a place of data collection, to the real world, where didactical transposition will take place. Constructions in virtual world + AR ==> gives an aid for mathematical formalization Figure 6.8 illustrates this double movement from mathematics to reality and from reality to mathematics with the help of AR.

11

Conclusion

The main questions at stake with augmented reality in a virtual space concern the adoption of the tool by both teachers and students. This chapter’s review of two years’ experimentation provides insight into the acceptability of such an environment within the school community and also about the potentialities in terms of pedagogical tools. The whole investigation from which we have extracted the above observations and analyses shows that preliminary conditions are necessary for realizing good practices with this tool. Particularly, the

Mathematics Learning and Augmented Reality

143

engagement of students within the project is of great importance. In one interview, a teacher was sorry about the fact that a particular class didn’t play the game and as a consequence missed a positive learning experience. The analysis of the situation, compared with other similar classes that did profit from the experience, showed that the presentation of the materials (tablet, virtual world) had been made without pedagogical intentions, in a rush, with more attention to selling the system than for pedagogical aims or intentions, and the consequence was disinterest from the whole class that teachers did not succeed in changing. The acceptability of the platform as well as the acceptability of the experiment depend on engagement that can only be achieved through a deep understanding of the pedagogical stakes. However, once this crucial stage was overcome, our study clearly shows benefits for mathematical learning that the augmented reality promises. When appropriate pedagogical scenarios are built, that is to say scenarios taking into account the potentialities of the virtual world and particularly the augmented reality included in it, students confronted with mathematical tasks can use AR to develop bridges between the mathematical abstraction and the concrete manipulation of objects. We have seen that AR is the engine of a double movement; the first from the mathematical world to the real world, giving an opportunity to catch different representations of mathematical objects; the second movement starts from the real and the virtual world, heading towards the mathematical world and giving an opportunity to better understand the mathematical objects through experiences involving the objects and their mathematical properties. The virtuality as well as augmented reality can be considered as artefacts that become instruments in the process of instrumental genesis, giving to the mathematical activity different roles and providing students feedback allowing better understanding of the mathematical concepts at stake. The next step of the experiment will be to consider the didactic transposition and the role that AR can play in the evolution of mathematical praxeologies (Chevallard, 1992), that is to say both the techniques allowing a solution of a certain type of task and the theoretical justifications for these techniques. From the observations of this two-year investigation, we suggest that AR can be a lever to reach the logos of praxeologies from experiments made in a virtual world.

Note 1 “These places created to meet educational challenges gather a set of conditions: questions raising from actors supported by the place managing staff (school director,

144

Aldon and Raffin

local authorities …), involvement of an FIE research team and joint construction by all the actors of a long-term project designed to find answers to these questions.” Retrieved from http://ife.ens-lyon.fr/ife/recherche/lea-ancien-site/201caedep201dassociated-educational-design-experiment-places-at-the-french-institute-foreducation

References Artigue, M. (2002). Learning mathematics in a CAS environment: The genesis of a reflection about instrumentation and the dialectics between technical and conceptual work. International Journal of Computers for Mathematical Learning, 7, 245. Azuma, R. T. (1997). A survey of augmented reality. Presence: Teleoperators & Virtual Environments, 6(4), 355–385. Bétrancourt, M. (2007). L’ergonomie des TICE: Quelles recherches pour quels usages sur le terrain? In B. Charlier & D. Peraya (Eds.), Transformation des regards sur la recherche en technologie de l’éducation (pp. 77–89). De Boeck Supérieur. Brousseau, G. (1986). Fondements et Méthodes de la didactique. Recherches en Didactique des Mathématiques, 7(2), 30–115. Brousseau, G. (1990). Le contrat didactique: Le milieu. Recherches en Didactique des Mathématiques, 9(9.3), 309–336. Brousseau, G. (1998). Théorie des situations didactiques: Didactique des mathématiques 1970–1990. Grenoble: La Pensée Sauvage. Brousseau, G. (2006). Recherches en éducation mathématique [Research in mathematics education, first translation by Ginger Warfield]. Quaderni di Ricerca in Didattica, 16, 12–26. Chabanne, J. C., Monod-Ansaldi, R., & Loisy, C. (2016). Faire le lien entre la pratique et la recherche pour transformer l’école? Le dispositif LéA de l’IFE comme laboratoire de l’innovation en recherche-intervention-formation. Analyse du cas particulier d’un LéA impliquant une ESPE. In B. Marin & D. Berger (Eds.), Recherches en éducation, recherches sur la professionnalisation: Consensus et dissensus (pp. 279–295). Paris: Editions Le Réseau National des ESPE. Chevallard, Y. (1992). Fundamental concepts in didactics: Perspectives provided by an anthropological approach. In R. Douady & A. Mercier (Eds.), Research in didactique of mathematics, selected papers (pp. 131–167). Grenoble: La Pensée Sauvage. Chughtai, R., Zhang, S., & Craig, S. D. (2015). Usability evaluation of intelligent tutoring system: ITS from a usability perspective. Proceedings of the Human Factors and Ergonomics Society Annual Meeting, 59(1), 367–371. https://doi.org/10.1177/ 1541931215591076

Mathematics Learning and Augmented Reality

145

Drijvers, P., Doorman, M., Boon, P., Reed, H., & Gravemeijer, K. (2010). The teacher and the tool: Instrumental orchestrations in the technology-rich mathematics classroom. Educational Studies in mathematics, 75(2), 213–234. Fuchs, P. (1996). Les interfaces de la réalité virtuelle. Paris: Les Presses des Mines. Fuchs, P., Hugues, O., & Nannipieri, O. (2010, December). Proposition d’une Taxonomie Fonctionnelle des Environnements de Réalité Augmentée. AFRV2010. Cinquième Journées de l’Association Française de Réalité Virtuelle et de l’Interaction 3D, Orsay, France. Gazzard, A. (2009). The avatar and the player: Understanding the relationship beyond the screen. In 2009 Conference for games and virtual worlds for serious applications (pp. 190–193). New York, NY: IEEE. Gueudet, G., & Trouche, L. (2010). Teaching resources and teachers’ professional development: Toward a documentational approach of didactics. In V. Durand-Guerrier, S. Soury-Lavergne, & F. Arzarello (Eds.), CERME 6, 2010 (pp. 1359–1368). Lyon, France: INRP. Guin, D., & Trouche, L. (2002). Mastering by the teacher of the instrumental genesis in CAS environments: Necessity of intrumental orchestrations. Zentralblatt für Didaktik der Mathematik, 34(5), 204–211. Lagrange, J. B. (2000). L’intégration d’instruments informatiques dans l’enseignement: Une approche par les techniques. Educational studies in mathematics, 43(1), 1–30. Rabardel, P. (1995). Les hommes et les technologies: Approche cognitive des instruments contemporains. Paris: Armand Colin. Sensevy, G. (2010). Formes de l’intention didactique, collectifs, et travail documentaire. In G. Gueudet & L. Trouche (Eds.), Ressources vives: Le travail documentaire des professeurs en mathématiques (pp. 147–161). France: Presses Universitaires de Rennes/ INRP. Tricot, A., Plégat-Soutjis, F., Camps, J. F., Amiel, A., Lutz, G., & Morcillo, A. (2003). Utilité, utilisabilité, acceptabilité: Interpréter les relations entre trois dimensions de l’évaluation des EIAH. In Environnements Informatiques pour l’Apprentissage Humain (pp. 391–402). ATIEF; INRP. Trouche, L. (2004). Managing the complexity of human/machine interactions in computerized learning environments: Guiding students’ command process through instrumental orchestration, International Journal of Computers for Mathematical Learning, 9, 281–307. Yuen, S. C.-Y., Yaoyuneyong, G., & Johnson, E. (2011). Augmented reality: An overview and five directions for AR in education. Journal of Educational Technology Development and Exchange ( JETDE), 4(1), 119–140. doi:10.18785/jetde.0401.10

146

Aldon and Raffin

Zhou, F., Duh, H. B. L., & Billinghurst, M. (2008, September). Trends in augmented reality tracking, interaction and display: A review of ten years of ISMAR. In Proceedings of the 7th IEEE/ACM International Symposium on mixed and augmented reality (pp.193–202). New York, NY: IEEE Computer Society.

CHAPTER 7

Engaging Students in Covariational Reasoning within an Augmented Reality Environment Osama Swidan, Florian Schacht, Cristina Sabena, Michael Fried, Jihad El-Sana and Ferdinando Arzarello

Abstract This chapter considers the application of augmented reality (AR) technology for developing covariational reasoning and aims also to show general potentialities of AR technology and to give some hint of the theoretical principles behind them. Specifically, the chapter discusses the first design-cycle from a design-based research study that aims at both the development of an AR toolkit within iterative research cycles, and at its scientific investigation. Our main claim is that engaging students in coordinating continuous real phenomenon (e.g., a moving object along inclined plane) with its mathematical representations (e.g., plotting points of graph, ordered pairs in a table of values) through visual-kinesthetic activities may help the students to experience the multiple levels of sophistication and develop the multiple meanings of covariational reasoning. Insights from the first design cycle are presented and ideas for future design-cycles are also discussed.

Keywords augmented reality – covariational reasoning – design-based research – real phenomenon – visual-kinesthetics activities

1

Introduction

Covariational reasoning is important for students to conceptualize functional relationships at all levels of schooling (Thompson & Carlson, 2017). At the same time, covariational reasoning involves complex cognitive acts that students, teachers, and researchers can engage in at multiple levels of sophistication and with multiple meanings (e.g., discrete covariation, chunky © koninklijke brill nv, leideN, 2020 | DOI: 10.1163/9789004408845_007

148

Swidan et al.

continuous covariation, smooth continuous covariation). In the last decades, educators have sought to design learning environments that develop covariational reasoning (Doorman, Drijvers, Gravemeijer, Boon, & Reed, 2012; Zieffler & Garfield, 2009). This chapter contributes to these efforts by showing how AR environments may help deepen covariational reasoning. Specifically, we emphasize the capability of AR to bring together, (1) continuous dynamic features of a real phenomenon, (2) their mathematical representations, which can be displayed discretely, and (3) the learners’ own perspectives with respect to the phenomenon (Figure 7.1). What is essential is that these elements – continuous variational phenomena, mathematical representations, and a learner’s own perspectives – are brought together at the same time so that they become, in effect, parts of a single unified experience. Augmented reality-based learning environments thus draw learners into visual-kinesthetic interactions with physical objects in their own real environments while, simultaneously, engaging them with symbolic, or more generally semiotic representations in various registers. According to current cognitive theories and neuroscience research (de Freitas & Sinclair, 2014; Gallese & Lakoff, 2005; Radford, Arzarello, Edwards, & Sabena, 2017; Rizzolatti, 2005; Rizzolatti & Craighero, 2004) these interactions play a central role in fostering mathematical thinking. Based on these theories and through

figure 7.1 Illustration of the AR prototype. The inclined plane illustrates the real phenomenon; the Cartesian systems and the table of values illustrate the covariation of distance-time of the dynamic object; the left distance-time Cartesian system and the table of values illustrate discrete covariation; the middle distance-time Cartesian system illustrates chunky continuous covariation; the moving ball illustrates smooth continuous covariation

Engaging Students in Covariational Reasoning

149

design-based research, we claim that engaging students in coordinating the continuous real phenomenon with its mathematical representations through visual-kinesthetic activities may trigger the mirror neuron system of the students and thus help them to experience the multiple levels of sophistication and develop the multiple meanings of covariational reasoning. Specifically, the main assumption underlying the proposed contribution is that a combination of physical and virtual experiences together with simultaneous symbolic representations can aid learning and, in particular, facilitate variational and covariational reasoning. In this chapter, we will discuss the use of AR during real-world experiments in the students’ own physical environments, such that the technology provides data and symbolic representations of real-world phenomena, which the students can manipulate and coordinate during their observations. Our general approach involves adopting a design-based research perspective to frame a study allowing us to develop an AR prototype that addresses covariational reasoning by means of collecting real-time data of a dynamic phenomenon while simultaneously augmenting the students’ experience with mathematical representations of the dynamic phenomenon. As with all design-based research, we hope also, in the course of developing the prototype, to gain insight into learning and thinking in the context of AR. In this connection, we are guided by a research paradigm informed by current cognitive theories and neuroscience reserach, as already mentioned, as well as current ideas about semiotics, multimodality, and mirror neurons (Arzarello, 2006; Keysers & Gazzola, 2007; Radford, 2014; Sabena, 2018). However, the emphasis of the chapter is both concrete and theoretical. For its chief task is to set out the first cycle of our design-based research, namely, the design principles for the first prototype and the tasks’ design. The significance of the chapter lies, first and foremost, then, in its potential to reveal the concrete benefits of AR technology, in supporting making sense of pre-calculus concepts (e.g., variational and covariational), which are crucial in mathematics and other STEM fields.

2

Theoretical Perspective

2.1 Augmented Reality in Mathematics Education The use of AR in educational settings remains in its infancy (Wu, Lee, Chang, & Liang, 2013) and the potential of this technology for learning and teaching mathematics, specifically, has not been sufficiently explored. Cheng and Tsai (2013), in their literature review of the use of AR in education, note that the

150

Swidan et al.

few studies on AR in science education have mainly focused on the use of AR for motivating students, explaining topics, and adding information. Dunleavy, Dede, and Mitchell (2009) observes that AR with ready-made materials (movies, explanations, figures, etc.) helps students engage in authentic learning and in interacting with real-life environments outside the classroom. AR is commonly used by science educators to help students visualize scientific processes, such as chemical reactions, which cannot be observed easily in the real world (e.g., Wu et al., 2013). Importantly, Cheng and Tsai (2013) conclude that while research on the use of AR in educational settings has examined issues such as development, usability, and initial implementation (Blake & Butcher-Green, 2009; El Sayed, Zayed, & Sharawy, 2011), students’ inquiry skills and the learning processes within AR environments have largely been ignored in the scientific literature. Since AR enables the alignment of real objects or places and digital information in 3D, research on learning and teaching mathematics using AR has mainly focused on how students perceive and experience virtual 3D objects. Sommerauer and Müller (2014) found that most studies on mathematics education using AR examined the effect of this technology on learning spatial abilities (e.g., Kaufmann, Steinbügl, Dünser, & Glück, 2005); this finding is not surprising, as 3D is one of the key affordances of AR. For example, Kaufmann and Schmalstieg (2003) developed a learning environment that allows students to act on and interact with geometrical objects in space. Similarly, Orozco and colleagues (Orozco, Esteban, & Trefftz, 2006) developed an AR-rich learning environment to assist students in visualizing multivariable function curves and reported that it helped develop spatial reasoning and visualizability of 3D mathematical objects. The next logical – and crucial – step in this line of research is to examine how these components of learning fit together into complete acts of learning, such as those which combine real phenomena, semiotic systems, and the body interaction, where we believe AR shows its strongest potential.

3

Design-Based Research in the Learning Perspective

Compared to descriptive or reconstructive research approaches, design-based research offers the possibility to connect research and development in the mathematics classroom (Bakker & van Eerde, 2015). There are several different approaches of design-based research, understood as Design Science (Wittmann, 1995), Design-Based Research (Barab & Squire, 2004; Brown, 1992; Collins, 1992), Design Research in the learning perspective (Freudenthal, 1991;

Engaging Students in Covariational Reasoning

151

Gravemeijer, 1994; Gravemeijer & Cobb, 2006; Hußmann & Prediger, 2016; Prediger, Gravemeijer, & Confrey, 2015), or Engineering Research (Burkhardt, 2006). Common aspects of these different approaches are their interventionist direction, their iterative design and the understanding of theory-driven research (van den Akker, Gravemeijer, McKenney, & Nieveen, 2006, p. 5). The approaches are different concerning their specific emphasis: realization within natural learning designs (Design-Based Research), the iterative development giving both respect to conceptual understanding and design (Design Research), the development of learning designs within and for real mathematics classroom situations (Engineering Research) and the use of empirical data of students’ learning processes for the design of learning materials (Design Science). The aim of our study is to develop a mathematical design including both the tasks and an AR prototype and to carefully examine the underlying learning processes within an iterative process. That was the main reason why we choose the framework of design research in the learning perspective (Gravemeijer & Cobb, 2006; Prediger et al., 2015). Such a mathematical design consists of learning activities embedded in mathematical tasks. Hence, the role of the design principles concerning the tasks is crucial. As the ICME Study on task design in mathematics education points out, design principles include the use of “several representations, several kinds of sensory engagement, and several question types” (Watson et al., 2013, p. 10). In addition, we are convinced that the positive role of interactions should be taken into account in the design principles of the tasks as well as the benefits of what Douady (1987) called ‘games of frames’ which is active when one faces different representations of the same phenomenon. Augmented reality technology has great potential in respect to these design principles. Such technology can be used in order to initiate learning processes in which implicit concepts- and theorems-in-action (Vergnaud, 1982) become explicit and make sense within the mathematical conceptual fields. In order to achieve such substantial learning results, Barzel, Leuders, Prediger, and Hußmann (2013) suggest three epistemological phases that can be used for the design of tasks within learning arrangements: – The exploration phase consists of rich and open exploration tasks (Freudenthal, 1973), which allow students “to actively and collaboratively re-invent ideas, concepts, procedures and relations” (Barzel et al., 2013, p. 288). For the use of AR technology this means that students will have the possibility to carry out experiments in meaningful mathematical situations, use the technological devices in order to collect data within several representations (Duval, 2006).

152

Swidan et al.

– The next phase aims at the organization of knowledge in order “to establish a shared understanding of the core concepts, theorems and procedures” (Barzel et al., 2013, p. 288). For the context of using AR technology it will be one aim to connect the information given within the different representations and establish and integrate the information into a stable individual concept of functional reasoning. – In order to deepen the acquired mathematical concepts, the third phase consists of practice tasks in order to render students’ “knowledge and skills more stable and flexible by repeated practice and transfer” (Barzel et al., 2013, p. 289).

4

Design Principles Both for Designing the Technological Tool and the Tasks

This section sets out the design principles for the learning design on three different layers: On the mathematical level with respect to the concept of covariation, on the cognitive-semiotic level concerning the role of multimodality within the developed learning design and on the technological level concerning the use of augmented reality technology. We will describe mathematical taskdesign principles (MTDP) and technological task-design principles (TTDP), concluding the literature review. These principles will then be the basis on which the mathematical tasks for this leaning environment will be developed and redeveloped within the design cycles. 4.1

Covariation: Idea (Literature Review), Tech Principles, Task Principles When we think in terms of covariation, we think of two (or more) variables whose changes are coordinated so that the variation of one is related in some way to the variation of the other. Where we consider, more specifically, a functional relation, we can think of an independent variable running through the domain set and inducing, as it were, the dependent variable to run through a range set. Thompson and Carlson (2017) described five major levels of covariational reasoning: (a) Pre-coordination of values; (b) Gross coordination of values; (c) Coordination of values; (d) Chunky continuous covariation; and (e) Smooth continuous covariation. In the first level, “Pre-coordination of values,” the student can predict the change of every variable value separately, but doesn’t create pairs of values from both variables (x and y). In the second level, “Gross coordination of values,” the student perceives a loose link between the overall changes in two quantities’ values. In the third level, “Coordination of

Engaging Students in Covariational Reasoning

153

values,” the student can match the values of one variable (x) with values of another variable (y), creating a discrete collection of pairs (x,y). In the fourth level, “Chunky continuous covariation,” the student may perceive that the changes of both variables happen simultaneously, and that they are varying smoothly but only in discrete intervals, from 0 to 0.5, from 0.5 to 0.8, from 0.8 to 2, and so on. In the fifth, “Smooth continuous covariation,” the student can perceive increases or decreases in variable’s value as happening simultaneously with changes in another variable’s value throughout its domain and see both variables varying smoothly and continuously. It seems that the five levels of co-variational reasoning are developmental in a sense that, through them, learners can relate operational to structural understanding of function. Sfard (1991) distinguishes operational and structural concepts, the first concerning mathematical processes and the latter mathematical objects. A function may operationally be seen as a computational process, as a recipe to transform one number into another, whereas, structurally, a function can be thought of as a set of ordered number pairs. Indeed, some of the students’ difficulties in perceiving the function concept, are brought about by the instructional approaches and the resources used by the teachers. Mathematics educators found that using dynamic technological tools may help students in perceiving covariation (Thompson & Carlson, 2017). Previous researches showed that the learning arrangement with a computer tool helped students to overcome the difficulty of integrating operational and structural aspects of the function concept and supported explorative activities for coordinating and investigating the dynamics of co-variation (Kieran & Drijvers, 2006; Doorman et al., 2012). On this basis, we identify the following mathematical task-design principles (MTDP) and technological task-design principles (TTDP) concerning the mathematical concept of covariation: – MTDP1: Covariation is explored within a process of data collection (exploration phase) – MTDP2: Covariation is systematized with operational and structural aspects of function (organization of knowledge phase) – TTDP1: Different sensors allow the coordination of real processes. – TTDP2: The distance-time function graph displays in both discrete or continues modes 4.2 Multimodality: Theory, Tech Principles, Task Principles In the last two decades, many studies in mathematics teaching have increasingly emphasized the contribution of the body and of the sensory-motor experiences in the formation of mathematical thinking and in the mathematics

154

Swidan et al.

teaching-learning processes (de Freitas & Sinclair, 2014; Nemirovsky, 2003; Radford, 2014; Roth, 2009; for an overview see Gerofsky, 2015, and Radford et al., 2017). A great impulse in this direction has come out of the embodiment strand of cognitive psychology following the publication of the provocative book Where Mathematics Comes From by George Lakoff and Rafael Núñez. In this book, Lakaoff and Núñez elucidated crucial perceptual and bodily aspects of the formation of abstract concepts, including mathematical concepts (Lakoff & Núñez, 2000). More recently, embodied stances seem to receive a certain confirmation by neuroscientific results on “mirror neurons” and “multimodal neurons,” which are neurons that fire when the subject performs an action, when he observes somebody else doing the same action, as well as when he imagines it (Gallese & Lakoff, 2005). On the basis of these results, Gallese and Lakoff provide a new theoretical account on how the brain works, according to which “action and perception are integrated at the level of the sensory-motor system and not via higher association areas” (p. 459). In particular, such an integration would appear to be crucial not only for motor control, but also for planning actions, an activity typical of what is generally understood as “thinking.” On the other hand, mathematics and mathematics teaching-learning processes are deeply shaped by the cultural plane (Radford, 2014), and develop within social processes. On the basis of these considerations, Arzarello and colleagues frame the teaching-learning of mathematics in a multimodal perspective (Arzarello, Paola, Robutti, & Sabena, 2009; Sabena, 2018). This perspective highlights the coexistence and importance of different modes or resources in the learning-teaching processes: words (written or spoken), specific symbols of the discipline (such as those for writing numbers or algebra), diagrams and graphs, but also sketches, gestures, body positions, tones of voice and all those aspects related to the embodied nature of knowledge. The term “multimodal” has been borrowed from neurosciences, and in particular from the mentioned studies of Gallese and Lakoff (2005), where it indicates a characteristic of human cognitive functioning highlighted by recent findings on mirror neurons. On the other hand, in the field of communication, “multimodal” is used in reference to the multiple ways we have to communicate and express meanings to our interlocutors: words, sounds, images, and so on (Kress, 2004). With the spread of the visual richness given by recent technology (web, games, tablets, etc.), and the development of the possibilities of interaction with them through our body, a multimodal perspective on the communicative and cognitive dimensions assumes an increasing importance. Transposed into the educational context, the paradigm of multimodality considers the learning and teaching of mathematics as characterized by the

Engaging Students in Covariational Reasoning

155

activation of different cognitive and semiotic components. In particular, the perceptual-motor and embodied activities such as body movements, gestures, manipulation of materials or artifacts, drawing, rhythms, take on a prominent role for “abstract thinking,” including mathematics. In particular, the role of gestures in the learning of mathematics has emerged through the observation of teaching-learning processes through video recordings and has been supported by the flourishing of studies on gestures in communication processes. Today research in different disciplines (from psychology to cognitive linguistics, to anthropology) converges in highlighting the fundamental importance of gestures not only in communication, but also in thought processes (McNeill, 2005). Verbal language and gestures contribute to the formation of thought and its communication each according to its own specifications. Since gestures, however, involve motion, and thus space, time, form and trajectory, they have an intrinsic dynamicity that language lacks. In order to analyse their role in mathematical thinking and put it in relation with other important resources in mathematics, the theoretical tool of the semiotic bundle has been developed. A semiotic bundle consists of the set of different types of signs produced by one or more subjects in interaction while performing a mathematical activity (Arzarello, 2006; Arzarello et al., 2009). The semiotic bundle includes both the classical registers with precise and codifiable rules of production and transformation (Duval, 2006), and the embodied ones, allowing us to provide a semiotic account of the multimodal processes occurring while learning and teaching mathematics. An example can be constituted by students’ words, gestures and drawn figures while solving a geometrical problem. Including AR technology in the students’ activities may offer new potentials for their multimodal exploration and understanding. For instance, blending phenomena within the semiotic bundle, between the iconic and the symbolic aspects of the mathematical model and of the modelled situations could be fostered both in the exploration and in the organizing knowledge phase (Sabena, 2010), and investigation will be needed to study their didactic potential. Specifically, we identify the following mathematical task-design principles (MTDP) and technological task-design principles (TTDP) based on the multimodal perspective: – MTDP3: Different semiotic resources will be enacted by students in order to explore the concept of covariation (exploration phase); these resources will include the coordination and synergy between embodied signs such as gestures, gazes, body posture, etc., with natural language and typical mathematical signs, including those in algebraic, tabular, and graphical registers.

156

Swidan et al.

– TTDP3: The exploration within an AR environment will allow students to gesture in a blended way between the modelled phenomena and the modelling settings. – TTDP4: The AR environment will influence the multimodal resources enacted and will offer new possibilities to students in order to organize their knowledge.

5

First Design Cycle

Technological tools offer opportunities for thinking and learning as they can mediate the learning activities in which students engage (Sfard & McClain, 2002). Technological tools themselves are not a guarantee for supporting students’ learning and thinking, but rather the integration of appropriate tasks may support the students’ learning. We designed a learning arrangement, which includes AR tools and a set of tasks for the covariation topic that goes over all the five levels mentioned in Thompson and Carlson (2017), that provides room for reasoning, and that leads to the higher levels of covariation. As for the latter, this is possible because of the capability of AR to merge continuous dynamics features of a real phenomenon (e.g., a moving object along inclined plane) with those of its mathematical representations, which can be displayed discretely (e.g., plotting points of graph, ordered pairs in a table of values). 5.1 Technological Tools The technological tool that will be used in this environment is an AR toolkit that collects real-time data regarding a dynamic phenomenon (a cube moving on an inclined plane) during a physical experiment (Figure 7.2). The data is collected by the sensors, analysed, and its mathematical representation is displayed simultaneously to the students on designated eyeglasses. The students will thus be able to observe both the real-world experiment and the mathematical model of the dynamic object immediately and in real time (Figure 7.3). In addition, the learning environment includes tasks (see the next section) leading students to engage in an activity that prompts covariational reasoning. The learning tasks will be derived from aspects of the high-school syllabus in mathematics and will be connected to functions. Thus, the tasks are geared to centre on major mathematical concepts that are crucial for understanding pre-calculus concepts. The design principles of the tasks were motivated by the idea of covariation, and especially with respect to the five major levels of covariational reasoning described by Thompson and Carlson (2017). Each task focuses on a specific

Engaging Students in Covariational Reasoning

157

figure 7.2 The dynamic phenomenon: A cube moving on an inclined plane during a physical experiment

level of covariational reasoning. The main idea of the design is to allow the students to move gradually through all levels, with particular emphasis on levels three through five. For example, the first task requests students to explore the relationship between the dots in the Cartesian system, a table of values, and the movement of the cube. In this situation, the students assume to be engaged in third level of the covariational reasoning namely, matching the values of one variable (x) with values of another variable (y), in a discrete collection of pairs (x, y). Similarly, the second task requests students to explore the same situation as in task 1, but this time with new segments added, which connect the separated dots. In accordance with the covariational reasoning frame, we assumed that adding segments that connect the dots would prompt the students to engage in the fourth level of the covariational reasoning, i.e., chunky continuous covariation. The third task aims to engage the students in the fifth level of the covariational reasoning. Doing so, the students are requested to explain the relationships between the continuous movement of the cube and a continuous function graph. 5.2 The Tasks Task 1 Your task is to come up with conjectures about the relationship between the dots in the Cartesian system, table of values, and the movement of the cube.

158

Swidan et al.

figure 7.3 The real-world experiment and the mathematical model of the dynamic object displayed immediately and in real time

– (exploration phase) Set the cube on the highest point in the inclined plane, and then release it. Observe the creation of the dots in the Cartesian system. – (Organization knowledge phase) Describe the relationship between the dots in the Cartesian system, table of values, and the movement of the cube.

Task 2 Your task is to come up with conjectures regarding the relationship between the segments connecting the dots in the Cartesian system and the movement of the cube. – (exploration phase) Set the cube on the highest point in the inclined plane, and then release it. Observe the creation of the segments in the Cartesian system.

Engaging Students in Covariational Reasoning

159

– (organization knowledge phase) What is a relationship between these segments; the values in the table; and the cube movement?

Task 3 Your task is to find out the relationship between the function graph and the movement of the cube. – (exploration phase) Set the cube on the highest point in the inclined plane, and then release it. Observe the creation of the function graph in the Cartesian system as you look at the cube. – (organization knowledge phase) What is the relationship between the graph and the movement of the cube? Explain how the graph was created.

6

Evaluation of the AR Prototype

6.1 Version 1 of the Prototype These paragraphs describe the features of the first version of the developed prototype with respect to the three principles technology, covariation, and multimodality. On that basis we will describe not only the features of the first version but also the reasons for changes being made within the first design cycle. We used MATLAB software in our first attempt to design the AR prototype. In the first version of AR prototype, the user is able to conduct a real experiment while the AR prototype identifies the dynamicity of an object and represents

160

Swidan et al.

it graphically and numerically. We designed an AR prototype to measure the stretch of a spring when masses were hanging at its end (Hooke’s law). To activate the measurement option, a video camera with high resolution is required. The masses and the spring should be color coded. In this way, the specific colours given to the masses and spring are traced allowing the dynamicity of the physical objects to be recognized. For example, a mass hanging on a spring was wrapped by yellow coloured paper as well as the beginning and the end of the spring are coded by another colour (Figure 7.4). In this case, a camera followed the color-coded papers. To recognize the color-coded paper, the users must calibrate and define the color to be traced in each new experiment. For recognizing the color properly – and thus to collect the data of the dynamic object correctly – a uniform distribution of light in the room was required. In fact, small changes in the lighting conditions changed the results significantly so that the values and graphs in the different augmented representations showed different results depending on the lighting conditions of the specific point of view.

figure 7.4  Coded color method for recognizing the dynamic object

Engaging Students in Covariational Reasoning

161

From the covariation perspective, we recognized that by hanging masses one by one, students might be able to engage the third level of covariation, but not the advanced levels. The process of adding a mass one by one and then sampling the data collected, may strengthen the third level of covariation connected with the coordination of values. Being unable to add weight to spring in a continuous way, however, might impair the development of students’ sense of chunky continuous covariation and smooth continuous covariation. From the multimodality perspective, although the AR prototype displayed the stretch-mass function graphically and numerically, it failed in merging the real phenomenon with the mathematical representations, as we intended. This failure occurred due to the technological constraints of MATLAB that require users to sample the stretch measurement of the spring each time the users hang a mass at the end of the spring. The sampling action does not occur automatically, but must be done manually by pressing a specific key on the computer keyboard. Having to look at the keyboard rather than the physical objects undermines the process of merging the mathematical representations with the real objects. We found that the implementation of this demand was complex and difficult in an authentic class space. For this reason, we looked for new ways of enabling us to recognize the dynamic object. Luckily, the AR toolkit library provides print-codes allowing the recognition of dynamic objects without the constraints of lights. In the next section, we describe briefly the steps we did to overcome the challenges we met in the first version. 6.2 Version 2 of the Prototype In version 2, we used the AR toolkit library instead of MATLAB software. AR toolkit is an open-source computer tracking library for creation of strong augmented reality applications that overlay virtual imagery on the real world. The developer’s mission was to write an AR code to detect a moving object. For reasons of continuity of motion, in this version, we used a cube that moves on an inclined plane. The cube’s movement is detected with aid of fiducial markers (hereafter, “markers”). The AR toolkit provides many methods for tracking and detection objects in real world. The most primitive is marker-based tracking and detection. Here, the marker is treated as pre-defined input by means of which the program can extract various features. Additionally, the program, in each frame, detects these features of the marker. The markers were placed on the cube in our experiment thus allowing the detection of cube movement in real-time while using a camera (see Figure 7.5).

162

Swidan et al.

figure 7.5 Cube detection by markers

From the covariation perspective, by conducting the cube experiment and augmenting it with real time data, covariational reasoning can be advanced in different ways. On the one hand, covariation can be explored through the third level of covariation, i.e., coordination of values level. This exploration can be done with each one of the mathematical representations. That means, for example, asking the students to explain a discrete graph of the distance-time function (e.g., task 1). On the other hand, exploring the smoothly continuous level of covariation is also possible. This can be done by asking students to explore the seemingly continuous graph and its relationship with the moving cube, as the students experience the emerging process of the different representations within the process of experimentation itself. In this case, the emergence of real-time data within the experiment may allow the students to connect, for example, the features of the real process with the emergence of the graph. Having developed this second version, we realized that students might have difficulties in experiencing the emergence of the representations (and the relations between their features within the process of the experiment) and in carrying out a detailed analysis of the representations at the same time. A central problem is the short timeframe in which the experiment is conducted. This will be changed for the third version. In order to give the user the possibility to reflect on the process of emergence of the representations within the experiment, a video (by using screen-capturer technology) should be generated. From the multimodality perspective, the second version, like the first, presents the collected data concerning the dynamic object graphically and numerically. Owing to the marker glued on the cube’s surfaces, the linked mathematical representations emerged as the students were looking at the cube. In this way, we insured that the real phenomenon and the mathematical representation inform one another and became parts of a unified experience. Moreover, on a more practical level, since the markers can be seen from

Engaging Students in Covariational Reasoning

163

a distance of three meters, students can conduct the experiment and communicate their ideas with the others. This means interaction among the students through the physical as well as the virtual objects is possible.

7

Concluding Remarks

In this chapter we have presented an AR environment that aims at advancing covariational reasoning among middle school students. The design of the environment takes into account current cognitive theories, specifically those which consider learning as a multimodal process. Inspired by design-based research methodology, we presented two iterations of designing the learning environment and discussed the reasons for redesigning the environment in the second iteration. The process of designing and redesigning the learning environment has demonstrated to us that, among other things, the design of an AR-based environment, to foster multimodal interaction, requires the following three considerations: (A) On a practical level, the environmental constraints (e.g., light separation, camera resolution) that may affect the operation of the AR model need to be carefully controlled. To this end, the AR Toolkit library, which allows the recognition of a moving object and collecting its dynamic data (i.e., speed, height, traversed distance), has proven to be of great advantage. (B) The design should include open space allowing students to move and interact freely. (C) To prompt advanced covariational reasoning, the dynamic phenomenon chosen for the experiment must involve continuous motion. By choosing proper representations of the dynamic data (discrete, continuous) to be merged with this kind of physical phenomenon, the student may be able to progress from one level of covariational reasoning to another and finally arrive at the highest level involving smooth continuous covariation. Although we illustrated the design of an AR environment when the physical model is a cube moving on an inclined plane (Galileo’s experiment), this was merely a convenient example. The AR environment presented here is applicable to any experiment in which an object is moving in space. Even when the object is not moving continuously as in the example cited above concerning Hooke’s law, we expect that students will at least be able to develop the third level of covariational reasoning. The development of an AR-based learning environment such as we have described seems promising theoretically and consistent with the principles of multimodality. Clearly, though, conducting empirical experiments will be necessary to strengthen the theoretical foundation of the design. Thus, clinical

164

Swidan et al.

experiments with students will be the next round in our design-based research. The findings of these experiments, we hope, will provide data to improve the learning environment and to establish its theoretical roots.

Acknowledgement This study was supported by the Israel Science Foundation (Grants No. 1089/18).

References Arzarello, F. (2006). Semiosis as a multimodal process. In L. Radford & B. D’Amore (Guest Eds.), Revista Latinoamericana de Investigación en Matemática Educativa, Special Issue on Semiotics, Culture, and Mathematical Thinking (pp. 267–299). Arzarello, F., Paola, D., Robutti, O., & Sabena, C. (2009). Gestures as semiotic resources in the mathematics classroom. Educational Studies in Mathematics, 70(2), 97–109. Bakker, A., & van Eerde, D. (2015). An introduction to design-based research with an example from statistics education. In A. Bikner-AhsBahs, C. Knipping, & N. Presmeg (Eds.), Approaches to qualitative research in mathematics education (pp. 429–466). London: Springer. Barab, S., & Squire, K. (2004). Design-based research: Putting a stake in the ground. Journal of the Learning Sciences, 13(1), 1–14. Barzel, B., Leuders, T., Prediger, S., & Hußmann, S. (2013). Designing tasks for engaging students in active knowledge organization. In C. Margolinas, A. Watson, M. Ohtani, J. Ainley, J. Bolite Frant, M. Doorman, C. Kieran, A. Leung, P. Sullivan, D. Thompson, & Y. Yang (Eds.), ICMI study 22 on task design – Proceedings of study conference (pp. 285–294). Oxford. Blake, M. B., & Butcher-Green, J. D. (2009). Agent-customized training for human learning performance enhancement. Computers & Education, 53(3), 966–976. Brown, A. L. (1992). Design experiments: Theoretical and methodological challenges in creating complex interventions in classroom settings. Journal of the Learning Sciences, 2(2), 141–178. Burkhardt, H. (2006). From design research to large-scale impact. Engineering research in education. In J. van den Akker, K. Gravemeijer, S. McKenney, & N. Nieveen (Eds.), Educational design research (pp. 121–150). London: Routledge. Cheng, K.-H., & Tsai, C.-C. (2013). Affordances of augmented reality in science learning: Suggestions for future research. Journal of Science Education and Technology, 22, 1573–1839.

Engaging Students in Covariational Reasoning

165

Collins, A. (1992). Towards a design science in education. In E. Scanlon & T. O’Shea (Eds.), New directions in educational technology (pp. 15–22). New York, NY: Springer. de Freitas, E., & Sinclair, N. (2014). Mathematics and the body: Material entanglements in the classroom (Learning in doing: Social, cognitive and computational perspectives). Cambridge: Cambridge University Press. Doorman, M., Drijvers, P., Gravemeijer, K., Boon, P., & Reed, H. (2012). Tool use and the development of the function concept: From repeated calculations to functional thinking. International Journal of Science and Mathematics Education, 10(6), 1243–1267. Douady, R. (1987). Jeux de cadres et dialectique outil-objet. Recherches en didactique des mathématique, 7(2), 5–31. Dunleavy, M., Dede, C., & Mitchell, R. (2009). Affordances and limitations of immersive participatory augmented reality simulations for teaching and learning. Journal of Science Education and Technology, 18(1), 7–22. Duval, R. (2006). A cognitive analysis of problems of comprehension in a learning of mathematics. Educational Studies in Mathematics, 61(1–2), 103–131. El Sayed, N. A. M., Zayed, H. H., & Sharawy, M. I. (2011). ARSC: Augmented reality student card an augmented reality solution for the education field. Computers & Education, 56, 1045–1061. Freudenthal, H. (1973). Mathematics as an educational task. Dordrecht: Reidel. Freudenthal, H. (1991). Revisiting mathematics education. Dordrecht: Kluwer. Gallese, V., & Lakoff, G. (2005). The brain’s concepts: The role of the sensory-motor system in conceptual knowledge. Cognitive Neuropsychology, 22, 455–479. Gerofsky, S. (2015). Approaches to embodied learning in mathematics. In L. D. English & D. Kirshner (Eds.), Handbook of international research in mathematics education (3rd ed., pp. 60–97). London: Routledge. Gravemeijer, K. (1994). Educational development and developmental research in mathematics education. Jounal for Research in Mathematics Education, 25(5), 443–471. Gravemeijer, K., & Cobb, P. (2006). Design research from the learning design perspective. In J. van den Akker, K. Gravemeijer, S. McKenney, & N. Nieveen (Eds.), Educational design research (pp. 17–51). London: Routledge. Hußmann, S., & Prediger, S. (2016). Specifying and structuring mathematical topics: A fourlevel approach for combining formal, semantic, concrete, and empirical levels exemplified for exponential growth. Journal für Mathematik-Didaktik, 37(Suppl. 1), 33–67. Kaufmann, H., & Schmalstieg, D. (2003). Mathematics and geometry education with collaborative augmented reality. Computers & Graphics, 27, 339–345. Kaufmann, H., Steinbügl, K., Dünser, A., & Glück, J. (2005). General training of spatial abilities by geometry education in augmented reality. Annual Review of CyberTherapy and Telemedicine: A Decade of VR, 3, 65–76. Keysers, C., & Gazzola, V. (2007). Integrating simulation and theory of mind: From self to social cognition. Trends in Cognitive Sciences, 11, 194–196. doi:10.1016/ j.tics.2007.02.002 .

166

Swidan et al.

Kieran, C., & Drijvers, P. (2006). The co-emergence of machine techniques, paper-andpencil techniques, and theoretical reflection: A study of CAS use in secondary school algebra. International Journal of Computers for Mathematical Learning, 11(2), 205–263. Kress, G. (2004). Reading images: Multimodality, representation and new media. Information Design Journal, 12(2), 110–119. Lakoff, G., & Nùñez, R. (2000). Where mathematics comes from: How the embodied mind brings mathematics into being. New York, NY: Basic Books. McNeill, D. (2005). Gesture and thought. Chicago, IL: University of Chicago Press. Nemirovsky, R. (2003). Three conjectures concerning the relationship between body activity and understanding mathematics. In N. A. Pateman, B. J. Dougherty, & J. T. Zillox (Eds.), Proceedings 27th Conference of the International Group for the psychology of mathematics education (Vol. 1, pp. 105–109). Honolulu, HI: PME. Orozco, C., Esteban, P., & Trefftz, H. (2006). Collaborative and augmented reality in teaching multi-variate calculus. Proceedings of IASTED Conference on Web Based Education (WBE), Puerto Vallarta, México. Prediger, S., Gravemeijer, K., & Confrey, J. (2015). Design research with a focus on learning processes – An overview on achievements and challenges. ZDM Mathematics Education, 47(6), 877–891. doi:10.1007/s11858-015-0722-3 Radford, L. (2014). Towards an embodied, cultural, and material conception of mathematics cognition. ZDM – The International Journal on Mathematics Education, 46, 349–361. Radford, L., Arzarello, F., Edwards, L., & Sabena, C. (2017). The multimodal material mind: Embodiment in mathematics education. In J. Cai (Ed.), Compendium for research in mathematics education (pp. 700–721). Reston, VA: National Council of Teachers of Mathematics. Rizzolatti, G. (2005). The mirror neuron system and its function in humans. Anatomy and Embryology, 210, 419–421. doi:10.1007/s00429-005-0039-z Rizzolatti, G., & Craighero, L. (2004). The mirror-neuron system. Annual Review of Neuroscience, 27, 169–192. doi:10.1146/annurev.neuro.27.070203.144230 Roth, W. M. (Ed.). (2009). Mathematical representation at the interface of body and culture. Charlotte, NC: Information Age Publishing. Sabena, C. (2010). Are we talking about graphs or tracks? Potentials and limits of ‘blending signs’. In M. M. F. Pinto & T. F. Kawasaki (Eds.), Proceedings of the 34th Conference of the International Group for the psychology of mathematics education (Vol. 4, pp. 105–112), Belo Horizonte: PME. Sabena, C. (2018). Exploring the contribution of gestures to mathematical argumentation processes from a semiotic perspective. In G. Kaiser, H. Forgasz, M. Graven, A. Kuznia, E. Simmt, & B. Xu (Eds.), Invited lectures from the 13th International Congress on mathematics education (pp. 541–559). Springer Open.

Engaging Students in Covariational Reasoning

167

Sfard, A. (1991). On the dual nature of mathematical conceptions: Reflections on processes and objects as different sides of the same coin. Educational Studies in Mathematics, 22, 1–36. Sfard, A., & McClain, K. (2002). Analyzing tools: Perspectives on the role of designed artifacts in mathematics learning. Journal of the Learning Sciences, 11(2), 153. Sommerauer, P., & Müller, O. (2014). Augmented reality in informal learning environments: A field experiment in a mathematics exhibition. Computers and Education, 79, 59–68. Thompson, P. W., & Carlson, M. P. (2017). Variation, covariation, and functions: Foundational ways of thinking mathematically. In J. Cai (Ed.), Compendium for research in mathematics education (pp. 421–456). Reston, VA: National Council of Teachers of Mathematics. van den Akker, J., Gravemeijer, K., McKenney, S., & Nieveen, N. (Eds.). (2006). Educational design research. London: Routledge. Vergnaud, G. (1982). Cognitive and developmental psychology and research in mathematics education: Some theoretical and methodological issues. For the Learning of Mathematics, 3(2), 31–41. Watson, A., Ohtani, M., Ainley, J., Bolite Frant, J., Doorman, M., Kieran, C., … Yang, Y. (2013). Introduction. In C. Margolinas, A. Watson, M. Ohtani, J. Ainley, J. Bolite Frant, M. Doorman, C. Kieran, A. Leung, P. Sullivan, D. Thompson, & Y. Yudong (Eds.), ICMI study 22 on task design – Proceedings of study conference (pp. 9–16). Oxford, UK. Wittmann, E. (1995). Mathematics education as a ‘design science’. Educational Studies in Mathematics, 29(4), 355–374. Wu, H. K., Lee, S. W. Y., Chang, H. Y., & Liang, J. C. (2013). Current status, opportunities and challenges of augmented reality in education. Computers & Education, 62, 41–49. Zieffler, A. S., & Garfield, J. B. (2009). Modeling the growth of students’ covariational reasoning during an introductory statistics course. Statistics Education Research Journal, 8(1), 7–31.

CHAPTER 8

Uses of Augmented Reality in Biology Education Mária Fuchsová, Miriam Adamková and Miroslava Pirháčová Lapšanská

Abstract One of the options for increasing the appeal of STEM subjects among children and students, the future work force from Generations Z and Alpha, is using different innovative and engaging methods when teaching STEM subjects. Information technology provides us with many opportunities in the field of education, of which Augmented Reality (AR) is one. Quantitative results indicate that, compared to the human-guidance-only model, augmented reality-based learning systems successfully stimulate positive emotions and improve learning outcomes among learners. In addition, AR technology supports students in complex understanding of the subject nature, in our study, the subject of Biology. Based on experiments of previous authors we believe that augmented reality can be used in the field of human, animal or plant anatomy, morphology, and physiology, and not only in secondary education, but also in primary education. In our study, we focus on undergraduate students, future primary school teachers, future teachers from Generation Z in the field of biology education as one part of the science-oriented education. We focus on students’ awareness of the Augmented Reality Apps, their engagement with them as well as their attitude to them and their willingness of using them and incorporating them into education, taking into account the developmental stages of kids’ thinking.

Keywords STEM subjects – biology – augmented reality – primary school teachers

In today’s rapidly changing world there is an increasing requirement for supporting teaching STEM (Science, Technology, Engineering and Mathematics) subjects at all levels of education as there are many professionals from these fields needed in the job markets and the number is not increasing as hoped. One of the options for increasing the appeal of STEM subjects among children © koninklijke brill nv, leiden, 2020 | doi: 10.1163/9789004408845_008

Uses of Augmented Reality in Biology Education

169

and students, the future work force from Generations Z and Alpha, is using different innovative and engaging methods when teaching STEM subjects starting at the earliest education levels. Information technology provides us with many opportunities in the field of education; we will discuss one of them: Augmented Reality (AR). The main focus of this chapter is on using AR in the context of Biology education, one of the STEM areas. The chapter is focused on the perception of this method by Future Primary Teachers who will teach Biology as one of the areas of Primary Education and who were also the subjects of the research described later in the chapter. It is additionally concerned with the elements influencing AR use in educational processes, whether at the undergraduate level or in teaching practice in Biology. With an additional survey following our research, we also want to stress the socio-cultural aspect of using AR. We also want to mention several already existing Biology-themed AR applications, most of them free and their features which seem to be interesting, enhancing and beneficial in the context of Biology education at different levels. AR technology provides students the improved access to the subject. It mobilizes the learning environment irrespective of location and time, allowing flexibility of learning (Ganguly, 2010). AR technology also supports the idea of constructivist theory of learning. Constructivist theory assumes that each person himself creates (constructs) his own knowledge of the world in which he lives (Tóthová, Kostrub, & Ferková, 2017). The person constructs knowledge through interaction with others, including more knowledgeable peers and adults in the existing socio-cultural context (Clark, 2016). Many authors support the positive impact of AR in educational contexts. The coexistence of virtual objects and real environments allows learners to visualize complex spatial relationships and abstract concepts (Arvanitis et al., 2007). Radu highlighted the positive impact of AR applications on students’ understanding of the content as well as memory retention and improved collaboration and motivation (Radu, 2014).

1

AR in Human Biology Education

AR provides improved access to the subject because it mobilizes the learning environment and allows flexibility of learning. In addition, AR technology features support students in learning complex subjects in general, in particular the subject of human anatomy as one part of Biology education (Ganguly, 2010). Learning human anatomy involves learning in the practical dissection laboratory, where the structure of the human body and internal organs can be

170

Fuchsová ET AL.

revealed. The practical session, facilitates students to learn more complex parts of the body structure. Nevertheless, after the practical sessions in the learning curriculum, most students have difficulty in recalling the subject matter. Ganguly found that human anatomy lectures followed by practical dissections did not generate long-lasting understanding of the subject, while AR applications had a positive impact on students’ understanding of the content as well as memory retention. When speaking about AR applications in more detail, the visualization of the internal structures may serve as a stronger memory trigger in the students (Radu, 2014). AR visualizations do not only improve students’ knowledge of the subject human anatomy but also improve clinical skills in the laboratory (Akçayır, Akçayır, Pektaş, & Ocak, 2016; Garrett, Jackson, & Wilson, 2015). Garrett and his co-workers found that students’ knowledge acquisition, self-directed learning, and laboratory skills improved by using AR technology. Carlson and Gagnon similarly describe the use of AR technologies to improve clinical skills. They use the ARISE (Augmented Reality Integrated Simulation Education) system, which enables virtual clinical scenes to be set up for students to diagnose virtual patients. It is a system used in the education of medical students. The ARISE system is an emerging, versatile way to educate students. It is an innovative way to enhance simulation, provide authentic interactions, and potentially assist learning (Carlson & Gagnon, 2016). AR technologies can ultimately be used for real diagnostic purposes as well as for surgical planning and intraoperative guidance (Marescaux, Soler, & Rubino, 2005). As our research was focused on students studying human anatomy and physiology, we would like to start with applications focused on this particular area. As already mentioned, we present a list of some available AR applications on human anatomy and physiology that can help students understand the content of the curriculum and that can be further used in their future profession. One of the applications focused on human body (male and female) is Anatomy 4D, which enables students to experience the interactive 4-D environment of human anatomy. In this application, it is possible to view all the systems simultaneously and separately. It reveals the spatial relationships of the individual internal organs and enables students to understand the physiological processes that arise between and within individual organs. It is also very good for showing the detailed structure of the organs themselves. Based on the above-mentioned facts, this simple learning environment is good for use in the classroom and it is widely used by teachers, students, medical professionals and medical practitioners. Through this application biology tutors can visualize one of the most complex systems, human anatomy (Anatomy 4D, 2018). 4D Human Anatomy is another useful application for biology and medical

Uses of Augmented Reality in Biology Education

171

students that presents human body structure and its parts through augmented reality (4D Human Anatomy, 2018). An outstanding recent AR application is the Human Anatomy Atlas 2018 (Human Annatomy Atlas, 2018). One of the most exciting applications that display the human body in the AR environment is Virtuali-Tee by Curiscope. The application works in conjunction with an augmented reality t-shirt. The anatomical parts of the human body can be recognized by the students on their own body. The application is interesting as well with its feature of working in selfie and mirror modes (Virtuali-Tee by Curiscope, 2018). For young learners, a nice anatomy and physiology-oriented application is Parker Bear by Seedling. It is and engaging application for children as it uses play in the context of Biology education. In case of this application attention of children is attracted trough the plush toy Parker Bear. Children help the toy bear to solve his problems, and after solving his problems they see how his happiness changes with augmented reality effects (Parker Bear by Seedling, 2018). In addition to applications that focus on anatomy generally, there are some applications that focus exclusively on a few systems or parts of the human body. Two applications that provide detailed information about the brain are The Brain iExplore AR and The Brain App. The Brain iExplore AR application shows how the brain reacts to sounds, as well as the upside-down images of the world that the eyes actually see before the visual cortex of the brain processes them (Brain iExplore AR, 2018). Through this application, it is possible test fine motor skills and find what part of the brain deals with short-term memory. The Brain App allows exploring the layers of the head from skin, muscle and skull down to the inner areas of the brain. The application provides an interactive view of the brain and it is good for educational demonstrations and learning (Brain App, 2018). Another pair of AR apps help students learn the endocrine system and the intra-uterine development of the fetus. These applications are VR Endocrine Glands Male APK (VR Endocrine Glands Male APK, 2018) and VR Fetus in the Uterus apk (VR Fetus in the Uterus APK, 2018). These applications include quiz games for naming parts of the male endocrine glands and of the fetus in the uterus. The snap feature allows saving the image of the 3D or augmented reality model. This image can be used for later study. The AR Liver application is also worth mention (AR Liver, 2018). AR Liver is appropriate for use by secondary students, undergraduate and graduate students, and medical and biology professionals. It is one of a series of applications using actual human CT imaging data, and the most accurate 3D modelling technology available. Unlike other anatomical applications and programs, there are no pre-rendered animations. Therefore, the student can place

172

Fuchsová ET AL.

the incredibly detailed organ in any position and zoom in to any location to explore the anatomical structures. The students can learn by using the extensively labelled sections and by drawing directly on the liver in 3D. AR Liver offers an augmented reality viewing mode that bridges the gap between the purely computer-based interactive experience and the real world. AR technologies are also used to understand the more abstract bio-macromolecular structures of the human body (Arvanitis et al., 2007). One of these abstract concepts in the subject of human anatomy is protein structures (Berry & Board, 2014). Zheng and Waller developed the ChemPreview application to interact with large bio-macromolecules at the atomic level in an augmented reality environment (Zheng & Waller, 2017). ChemPreview presents a visual representation of a protein, and enables a student to manipulate atoms, and make measurements all by using hand gestures (ChemPreview, 2018). AR technology for biology can be used not only to understand anatomy and morphology, but also to preserve these systems in a healthy physiological process (Feng, Duh, & Billinghurst, 2008). Bayua, Arshada, and Alib (2013) proposed and developed AR prototype applications for nutritional information for Android mobile. This AR application provides information about the number of calories contained in food, such as carbohydrates, proteins, and fats, using a mobile phone camera to scan the image of desired object. The AR application concerning nutritional information can improve health of people who need to regulate their diet. Rollo, Bucher, Smith, and Collins (2017) evaluated the impact of an AR tool, ServAR. This application serves even as a tool for portion size estimation of foods in order to guide individuals in serving and consuming appropriate amounts of foods (Domhardt, 2015).

2

AR in Environmental Education

One part of biology education is the development of students’ environmental sensitivity. With modern trends causing highly polluted environments, there is urgent need for environmental and ecological education. While many countries stress ecological education, they perform poorly in instituting effective education. Elementary and middle school ecological and environmental education is still usually carried out in classrooms (Chang, Chen, & Hsu, 2011; Gurevitz, 2000). This lack of real-world interactions and exploration makes it difficult for students to develop an emotional attachment to or interest in ecology, thus limits their enthusiasm for practicing environmental protection (Hautecoeur, 2002). One of the ways to improve this relationship would be to give access to a botanical garden, which provides opportunities for learners to experience complex ecosystems. However,

Uses of Augmented Reality in Biology Education

173

traditional display cards at such gardens typically only render information via text or graphics, often providing a very limited introduction to specific plants. Although audio tours have become quite common in recent years, and provide users with more in-depth explanations than conventional labels, such systems still cannot provide systematic and interactive learning in outdoor learning environments. Theoretically, additional technology which provides learners with expanded access to practical real-world information might allow a more effective approach for implementing ecological educational activities in contexts such as botanical gardens (Chang et al., 2014). For example, Huang, Chen and Chou designed and implemented an ecological learning system based on experiential learning theory using AR technology (Huang, Chen, & Chou, 2016). The AR feature in the system presented an interactive virtual plant silhouette and students were prompted to learn about related plants, guiding them to experience and explore the surrounding environment. These authors found that students who use AR technology with an audio commentary demonstrated the best learning performance. Although the technology enables transmission of the information, interpersonal interaction and support are still crucial to increase learner engagement. The mentioned AR learning system allows real spaces to be integrated with anthropomorphic learning scenarios, thus enhancing students’ learning experience and positive emotions. The AR system enhances students’ interest in exploratory learning and aroused their curiosity. Another interesting example of using AR in ecological education is the Eco MOBILE project which combines an augmented reality (AR) experience with use of environmental probeware during a field trip to a local pond environment (Kamarainen et al., 2013). The EcoMOBILE (Ecosystems Mobile Outdoor Blended Immersive Learning Environment) project [EcoMOBILE project, 2018] is funded by the National Science Foundation and by Qualcomm, Inc. The augmented reality experience was created using the FreshAiR augmented reality development platform (FreshAiR, 2018) designed by MoGo Mobile, Inc. The FreshAiR platform allows an author to create augmented reality games and experiences with no programming experience required. The augmented reality program specifically supported students’ use of the probes by helping them navigate to a location to collect a sample, providing them with timely introductory information, step-by-step instructions for the use of probes, and immediate feedback related to the measurements performed by the student. The technology supported independence, as students were navigated to the AR trigger locations to explore and learn at their own pace. This enabled the teacher to be a facilitator, which is one of the advantages when using AR in educational context. The teachers indicated that the technology promoted more interaction with the pond environment and with classmates compared

174

Fuchsová ET AL.

to field trips in past years. The findings of Kamarainen et al. (2013) suggest that combining AR with use of probes inside and outside of the classroom holds potential for helping students to draw connections between what they are learning and new situations (Kamarainen et al., 2013). Tarng, Yu, Liou, and Liou (2013) used the Augmented Reality and the mobile learning technologies to develop a virtual butterfly ecological system by integrating the host plants and nectar plants on campus. Using this system, students can observe the life cycle of virtual butterflies at different growing stages. With the available space on campus, a virtual butterfly garden was designed as a greenhouse where the students can observe different species of butterflies using the tracking telescope. The virtual butterfly ecological system can increase the learning motivation and interest of students through virtual breeding and observation activities. It has both educational and entertaining functions, and is therefore a suitable assistant tool for science education in elementary and high schools. One part of ecological education is the process of plant photosynthesis. Photosynthesis is complicated and difficult to understand for students. Sakulphon, Srisawasdi, and Wangsomnuk (2015) reported that most students have misconceptions about this process, for example, that the chloroplast only has chlorophyll, or that photosynthesis is the same process as plant respiration. The authors therefore designed and created a mobile AR representing process of photosynthesis. Their investigation suggested that students’ understanding of the importance of photosynthesis by AR technology has improved. For understanding of the animal world, there are zoo games using Augmented Reality features (Chen, Ho, & Lin, 2015; Zarzuela, Pernas, Martínez, Ortega, & Rodríguez, 2013). The game of Zarzuela et al. (2013) consists of different activities which are performed in a virtual zoo that can be projected anywhere with a mobile device and a printed marker. During the game, the student is able to choose among different scenes, each of them with a different task related to the animal selected in the main one. This application offers a new way of learning about different kinds of animals living in a zoo. This application was very attractive for the target group of students who tried it. The application seemed to provide students with very good long-term knowledge about the animals. Using the animal games with AR, students and children get basic information about animals, and it is also a good way they can eliminate their fear of small animals, so this technology can also serve as prevention against phobias for insects or other animals (Wrzesien, Botella, Bretón-López, Río González, & Alcañiz, 2015). Chien, Su, Wu, and Huang (2017) included AR-based learning material in their study to provide different views on plants being studied, thereby

Uses of Augmented Reality in Biology Education

175

enhancing the learning experience. Their students’ learning outcomes when using AR were measured according to Bloom’s cognitive levels, showing the degree of learning achievement for different stages. According to their results, students in the experimental group observing the plants through AR had significantly greater comprehension of leaf arrangement compared to those in the control group. The findings suggest that compared with the traditional learning approach involving plant observation, AR-based learning material can significantly enhance students’ higher-level cognitive capabilities, enabling them to scaffold knowledge about plants in the observational learning activity more effectively. To enhance knowledge of zoological and botanical biology, we also include the following AR applications. Firstly, we focus on AR applications that help students and pupils recognize individual animal species. These applications are Zookazam (2018), Animal 4D +, AR Safari Zoo, AR-Animals book and Froggipedia. Zookazam is a simple educational AR application for learning about animals; therefore it is suitable not only for teachers, students, but also parents or children (Zookazam, 2018). Animal 4D+ is an application that allows users to scan printed animal cards, and then provides interesting and informative facts about those animals. It even allows users to scan a number of cards at once and make an AR zoo (Animal 4D+, 2018). The next two applications, AR Safari Zoo and AR-Animals book, which show animals in AR environments, require a book to be purchased that contains individual markers for loading the application. These books contain farm animals, forest animals, jungle-wild animals and dinosaurs. With the ARAnimals application, teachers can teach pupils how different animals walk, how they move, how they eat, what they eat, whether they are vegetarians or carnivores, and the sounds they make (AR Animals Book, 2018; AR Safari Zoo, 2018). Froggipedia is an interactive learning application which helps users explore the intricate anatomical details of a frog and its life cycle (Froggipedia, 2018). The AR application Skin & Bones is used to study the mechanisms of animal bones. Skin & Bones is an application that was developed for visitors to the Bone Hall at the Smithsonian’s National Museum of Natural History. Modified mobile application forms create the same 3-D experience in the classroom. It can serve teachers in explanation of animal anatomy and evolution (Skin & Bones, 2018). There are other interesting botanical applications. One of them is Arloon Plants (Arloon Plants, 2018). This application is designed to meet the aims and content of the primary education syllabus, tailored to students of 8 years of age. To do so, it includes the following syllabus content: the classification of plants (e.g., grasses, shrubs and trees); the parts and their functions and examples of

176

Fuchsová ET AL.

each part of the plant (e.g., root, stem, leaf, flower and fruit); the plant’s growth process; the processes of nutrition, photosynthesis and respiration; the processes of sexual (flowering or non-flowering) and asexual reproduction; and adaptation to different ecosystems as plants change throughout the seasons. An interesting application that evaluates air quality and the general wellbeing of plants is Plant Life Balance (Plant Life Balance, 2018), which helps teachers explain to students how specific plants affect the environment, for example in their classroom. The app’s creators reviewed more than 100 scientific articles on the impact of plants on people, developing what they call the “plant life balance index.” An interesting twist is that the application can also judge combinations of large and small plants in the same space, determining if they work well together to achieve a desired level of air quality. Students take a picture of their classroom and record the number and size of the existing plants in classroom. The application then rates the “health” of the classroom based on its size and plant distribution and allows students to improve air and wellbeing ratings by dragging and dropping new plants onto the image. To see the structure of animal and plant cells, for young learners, the Quiver Education application is an interesting option. As mentioned earlier, this application is designed so young learners are attracted through the play elements, in case of this particular one by colouring pages which are animated in AR after children colour them (Quiver Education, 2018). Based on experiments and the experience of authors mentioned previously, we believe that augmented reality can be used for education in the fields of human, animal, and plant anatomy, morphology, and physiology. Complex understanding of structures and processes which take place in human, animal and plant body might be crucial for students’ specialisation in their future profession. With use of the AR application features, students should be able to enhance their learning environments and improve their ability to retain information. The use of this technology might be very effective in motivating students’ learning and nurturing their ability to become passionately involved in their own learning process.

3

Augmented Reality in Human Biology Education for Future Primary School Teachers

In the following study, we evaluate the effectiveness of using the Brain iExplore and Anatomy 4D AR applications of human anatomy in teaching of students; pre-service teachers for primary level. We examine whether a demanding subject such as human anatomy and physiology is more understandable for

Uses of Augmented Reality in Biology Education

177

students, and whether it is more motivating for them to learn this challenging subject using AR technology. We use the constructivist theory of learning which assumes that each person himself creates (constructs) his own knowledge of the world in which he lives (Kostrub, 2017). Constructivism as an educational theory has many applications in the digital school environment. Constructivist instructional design, according Kalaš (2013), aims to provide generalised mental representations embedded in relevant learning environments that facilitate knowledge construction by learners. 3.1 Materials and Methods We conducted the research with a group of 61 students in the first year of the bachelor’s teacher-training program for future primary school teachers. The experiment was realized in November 2017 at the Faculty of Education of Comenius University in Bratislava. Our research was focused on analysing the use of mobile AR technology and manipulation activities in teaching biology. Our main focus was the issue of incorporating the use of tablets and AR technology in education. We believe that using this method, the students’ understanding is deeper, their motivation is greater, and, last but not least, their creativity is strongly supported. Two AR applications were used The Brain iExplore and Anatomy 4D with cards/markers on tablets (Figure 8.1). Students worked in groups of 3–4 persons (17 groups) during the lesson using the constructivist learning theory approach. In the course of the qualitative research, we wanted to see the effect of the AR applications on learning. Using these applications on the tablet, students could use augmented reality to study the anatomy of neural and endocrine systems that appeared in

figure 8.1 Application the Brain iExplore and Anatomy 4D

178

Fuchsová ET AL.

three dimensions. The students worked with neural and endocrine systems to understand neurohumoral regulation of growth and development during the lesson of “somatic child development.” The students’ task was to identify parts of the brain in the Brain iExplore application and to describe their functions and to identify endocrine glands in individual anatomical systems in the application Anatomy 4D. Each group obtained two cards/markers, one for the Brain iExplore application and one for the Anatomy 4D application. They worked for 60 minutes. A particular interest of the research was to see how students tried to find those parts of neurohumoral system on the internet or in the anatomy book which were unknown to them. Some students tried to translate the English names of anatomy systems to Slovak language on the internet. At the end of the lesson, students submitted a protocol (one for the each group). Our qualitative pedagogical research is based on the description of the teacher’s observations and on the video recording of the students’ work. Through the use of the open coding method and video-recording transcription, we defined three categories and their sub-categories (Table 8.1). The defined categories referred to constructivist teaching and the students’ manipulation activities. The categories we defined were given names which best represented the group of related expressions. Having grouped and identified data in this way, we tried to understand and evaluate them from our perspective. 8.2 Findings We summarized the results of the open coding research in Table 8.1. The categories from Table 8.1 referred to constructivist teaching and the students’ manipulation activities. Constructivist teaching takes place through didactically considered but conceptually open teaching activities, and through discourse (controlled argumentation, handling facts) in the form of individual as well as group exploration (learning groups), thanks to which common knowledge and understanding is established. The first task of the students was to recognize the colour of anatomical parts of the brain in the Brain iExplore application. Students recognized only a few parts of the brain through augmented reality. Only five groups from seventeen correctly recognized all the parts of the brain that the application offered. One group had problems determining which part of the brain is the front and which back. It was interesting when some students revealed some inner parts of the brain in the application. It should be noted that in this application, the brain was moving, changing colours, and the work with this application was therefore more challenging for students. Some students had almost no interest in completing tasks relating to the functions of the individual parts of the brain

Uses of Augmented Reality in Biology Education

179

table 8.1  Categories of students’ activities

Category

Subcategory

1. Mutual 1. Students support learning each other among students – constructivist 2. Mutual learning approach helps them to understand terms and relations

2. Influence of mobile AR technology

3. They do not collaborate with each other 1. Mobile AR technology helps them understand the anatomy

Expressions and activities ‟We could determine it”; ‟We will see how it can be done …”; ‟We will try it this way”; ‟We need to fijind out what it is”; ‟We should try it”; ‟We solved it.” ‟You have to come and see”; ‟You can set up your tablet so we can see it”; ‟I do not understand it, can you explain it to me?”; ‟You will not see these parts of the brain on the tablet, because …”; ‟Try to give something else to compare it.” ‟I think …”; ‟I will overwrite it”; ‟I do not see anything there.”

‟There is an emphasis on the whole body”; ‟The middle brain is the smallest”; ‟This purple and yellow part has two hemispheres”; ‟There are no pathways of the nervous system”; ‟It will be the whole brain stem”; ‟We found the pancreas in the digestive system.” 2. Thanks to mobile ‟Do you know how to say it in Slovak?”; AR technology, ‟Do you know how to say it in English?”; they recognize the ‟Pons is the bridge, Cerebellum is the nomenclature of brain, The pituitary gland is ‘hypophysis,’ the anatomical ‘Skeletal’ means ‘skeleton’ in Slovak.” parts in diffferent languages 3. It helps them when ‟I have made several attempts, several they can make solutions”; ‟I can correct myself and try it several attempts diffferently.” thanks to the technology (cont.)

180

Fuchsová ET AL.

table 8.1  Categories of students’ activities (cont.)

Category

Subcategory

3. Manipulation 1. Improving activities manipulation ability with the help of AR technology

2. Students are more motivated

3. Disinterest of mobile AR technology

Expressions and activities ‟When you adjust a tablet like this, you see …”; ‟Look at it from the side”; ‟Try to go closer to see …”; ‟Try to zoom it”; ‟Click something here”; ‟The eye icon means we see them”; ‟You can choose something here”; ‟Are you looking at the internet?” ‟It would be interesting if …,” ‟It took me unawares”; it is good”; ‟Try it too”; ‟We solved it”; ‟We are wise”; ‟I already understand it now.” ‟That‘s weird”; ‟That‘s disgusting”; ‟That‘s awful”; ‟It is not working”; ‟I do not know”; ‟It does not matter.”

(Table 8.1, category no. 1, subcategory no. 3). However, especially when completing this task, students collaborated most and did not hesitate to use other sources complete the task (anatomy book or smartphone, internet). Through the transcription of the video-recordings and structured observations, we found that the students frequently helped each other. In most cases, they used the plural form when they talked to each other, which meant that the students did not consider their tasks to be individual ones (Table 8.1, category no. 1, subcategory no. 1). Table 8.1 (category no. 1, subcategory no. 2) clearly shows the cooperation of the students. During the classes, they explored together and determined interconnections. The environment supporting constructivist approach significantly helped the students in their work and students’ mutual teaching with the help of this approach was efficient. The second task for the students was to identify endocrine glands in individual anatomical systems in the application Anatomy 4D. The application offered English anatomical names. Some students accepted it as a challenge and began to search the terms on the internet (Table 8.1, category no. 2, subcategory no. 2). Based on the video-recordings and observations, it was quite clear that the students collaborated to solve their tasks; they explained some terms and relations to each other and they performed activities without any major interventions from the teacher (Table 8.1, category no. 2, subcategory no. 1 and no. 3).

Uses of Augmented Reality in Biology Education

181

The application Anatomy 4D also caused some reluctance to look at some systems in some students. They expressed it by expressions like “That’s weird,” “That’s disgusting” or “That’s awful” (Table 8.1, category no. 3, subcategory no. 3). However, the results of the last category suggest (Table 8.1, category no. 3, subcategory no. 1 and 2), that most students enjoyed the manipulation activities. They improved their manipulation ability through mobile AR technology. It was again something new to them. For the students, it was easier to understand the contents of the child’s somatic development and similarly, it was more motivating for them to learn this challenging subject using AR technology.

9

AR Shaping in Educational Context

When using AR as one of the prospective and highly effective methods within education process, it is always important to take into consideration all elements influencing the educational process itself. It is vital to see it as the system, though quite coherent inside, but still continuously changing, adapting and reacting somehow to outside conditions. We believe that everyone participating in educational processes and considering or using AR for teaching should definitely take these issues into consideration. In the context of this chapter, by “participants of the educational process,” we mean future teachers, teachers at all educational levels, parents and other family members of those being educated, as well as those who develop the AR applications for education. Firstly, we think that it is essential to characterize the generation of those who are being educated as well as of those participating on their education, thus the socio-cultural environment surrounding them. Secondly, we also believe that it is important to take into consideration different pedagogical, psychological, and didactic issues to using AR in the most effective way possible. If we take a closer look at environments by which those being educated nowadays are surrounded, stemming from the Bronfenbrenner’s model of ecological systems theory of child development, we can see that in his model of the child’s development Bronfenbrenner conceptualized four ecological systems, which we can imagine as four circles each being a part of bigger one with an individual being a common part of all of them, positioned in the middle and interacting with them. A child interacts firstly with the microsystem, being the closest to him, encompassing relationships, interpersonal interactions and immediate surroundings like his family or school environment. Other circles

182

Fuchsová ET AL.

are the mesosystem, encompassing the different interactions between the characters of the microsystem (family – school); the exosystem, encompassing elements of the microsystem which do not affect the individual directly (e.g., parents’ jobs); and the macrosystem, encompassing cultural and societal beliefs, for example gender norms (Bronfenbrenner & Morris, 2006). For better understanding of the microenvironments of those being nowadays fully in educational process as well as on those creating the microenvironments that make their educational background, we can talk about several age ranges as one of the criteria for describing several significantly different generations. From this point of view we can think of them as the Generation of the Baby Boomers, born between 1946–1964; Generation X, born between 1965–1979; Generation Y or so called Millennials, born between 1980–1994; Generation Z, born between 1995–2009; and Generation Alpha, born 2010–2024. These last two groups are the kids being now educated by those from the earlier generations (McCrindle, 2018). Those in generation Z are now in their undergraduate studies a preparing for future teaching profession. With regard to using AR methods in educational contexts, we want to emphasize the characteristics of the last two, especially the last one, when talking about children who are fully immersed in a technological world since they have been born. For the purpose of this chapter and when thinking about AR apps in broader educational context and in more details we adapted Bronfenbrenner’s model of ecological theories on child development. In our model (Figure 8.2), we think about an individual as being a part of each of these systems, in our case rectangles, which shape an individual from upper side. The individual is represented by child being part of an educational process as well as being in interaction with ICT, particularly with AR apps, and at the same time with a supervisor in the person of the teacher. From the lower side the individual is shaped by psychological, pedagogical, and didactical aspects which are represented by three squares of equal size. An AR app represents the method and means for achieving educational goals and helping the individual grow. The effectiveness of this means is influenced by environments which the individual is being a part of, which we consider as the external elements influencing using AR as well as by pedagogical, psychological, and didactic aspects representing the internal factors which, according to us, comprise of relatively stable set of rules using any teaching method and teaching means in the educational process. We think that all elements mentioned above are always important to remind to the Future Primary School Teachers as well to anyone involved in education or schooling focused on ICT use, including our interest in the use of AR apps.

Uses of Augmented Reality in Biology Education

183

figure 8.2 AR shaping in educational context

AR app is a method or means which can help us to motivate kids or students, to support their creativity, to enhance their play and learning as well as the method or means which can help better information memory retention, which is the basic level of Bloom’s cognitive taxonomy. All elements mentioned in our model (AR Shaping in Educational Context) together with emphasis on the social nature of the learning are the facts which are really important to when thinking about AR as the method that Future Primary School Teachers might use in their practice.

10

Perception of Using AR as an Effective Educational Method and Means

Based on our model AR Shaping in Educational Context, stressing the social nature of learning, we conducted a survey focused on environments which can influence use of AR in education as well as on several psychological, pedagogical and didactical factors regarding using ICT and using AR apps in general and AR apps in Biology education in particular. The survey called Perception of Using AR as an Effective Educational Method and Means was conducted in May 2018 on a sample of 59 male and female adult respondents, of which none was a student. The aim of the survey was to receive very basic information on

184

Fuchsová ET AL.

perceptions of using AR as an innovative method in home education as well as in schools, not only from teachers but also from parents, grandparents, people working in different working areas, and other people who also influence the effectiveness of using this method in education. Results obtained provide additional information regarding the pedagogical, psychological, and didactic aspects, as well as the environmental aspects of using this method, in our adapted model. For this group of respondents, we administered a survey questionnaire, which was previously tested in a presurvey. We received information on the attitude of respondents towards the use of information and communication technologies (ICT) in educational process, in home-schooling, as well as towards innovative methods related to them. Most of the respondents (38) worked in the areas of education, IT and media. The questionnaire contained 15 questions, a combination of 11 closed questions, 2 semi-open and 2 open questions, where respondents could express their attitude towards the subject. From our survey sample, 57 respondents agreed with the use of information and communication technologies (ICT) in the process of education in the school environment. When asked which age categories were considered appropriate for ICT use, among our survey sample we found that 2 respondents would only use ICT at pre-primary level (2 to 6 years); 7 respondents would implement it at the primary level (6–10 years) of education only; another 6 respondents would use ICT at the lower secondary level of education only (11–15 years); and only 1 respondent would use ICT at higher secondary education (15–20 years) levels only. Some respondents considered using ICT as a prerequisite for future use of AR as appropriate at more levels of education: 15 respondents out of the total agreed with the suitability of ICT use at all levels of education; 14 respondents agreed with the use of ICT from the primary level of education onwards; and 7 respondents agreed with the use of ICT from lower secondary level and upwards. Thirty-two of our respondents had heard about AR. Of these, 21 respondents said they knew some AR applications. The most frequently mentions applications were: Pokemon Go, Ikea Place, Anatomy 4D, Quiver, Atlas 2018 (Human Body), and Parker by Seedling. Five respondents came into contact with the AR in the process of education in the school environment. Forty-eight respondents believed that AR applications can improve the educational process itself, and 48 respondents thought that AR can improve educational outcomes (longterm learning, the ability to apply information). Of all respondents surveyed working in the field of education, only 5 mentioned specific AR applications (Quiver, InovEduc, Anatomy 4D, The Brain iExplore, Fetus in the Uterus, Endocrine glands Male, 4D Biology, Squiggle Fish, Parker by Seedling, etc.). Only 12

Uses of Augmented Reality in Biology Education

185

respondents, of whom 8 work in IT, education, or media, have been using the AR application for the play and education of their children. They have been working with these AR applications: Quiver, 4D Human Anatomy, Atlas, Pokemon Go, and Sky Guide. Fifty-three respondents think that AR applications developed for a particular educational goal and a school education program can improve the educational process itself. Even up to 56 respondents think that an AR application for a particular educational goal and a school curriculum can improve the outcomes of the educational process. Forty-one respondents replied that they would like an AR method to be integrated into their children’s educational process using t applications developed for specific educational goals. Another 12 respondents would like existing applications to be used in addition to applications developed for specific educational goals. As we are focused mainly on Biology education in our study, which is a part of STEM education, we asked the respondents whether they think that AR applications might improve the quality of Biology teaching: 56 participants responded affirmatively. Of the total number of respondents, 51 think that AR applications should have a Slovak language version. Some respondents added comments related to AR topics. They stated the following ideas about using AR apps in educational process: – Too much use of it means raising a “tablet generation.” – Pieces of information need to be dosed in a proper way; otherwise there is a risk of overloading the system. – There are only few apps in Slovak. You need to pay for better apps. Children play in a digital world that is connected to the real world. Children have enough movement because they walk and move when using these applications. – It is definitely an advantage if schools provide opportunities like this. Unfortunately, in our kindergarten, we do not have such opportunities. We don’t even have interactive board in every class as well as Internet so it is hard to think about aids like these. It is about lack of finance, old colleagues unable to work with methods like this, etc. – I haven’t met with AR and I haven’t downloaded applications of this kind. But I am interested now. – Teachers need to be informed about the options of using this method as well as parents do. Sufficient technical equipment for schools is necessary – WiFi connection, current versions of technical devices – tablets. It is important for everyone who uses AR app for educational purposes not only know it, but also know how to implement it into the educational process, what is the purpose of using an app. It should not be only about spending one’s free time, but also about education and play enhancement.

186

Fuchsová ET AL.

– I think applications made by being focused on educational goal can be helpful in the educational process and can have a positive impact on its results. On the other hand, it is pity that children are becoming more and more involved in the world of ICT, and sometimes ICT maybe replaces other activities through which they can also get to know the world in a much easier and healthier way, walking to nature, observing real animals and plants, visiting monuments, etc. Until the activities are balanced as well as guided at home and at school, nowadays they still can be an excellent and essential teaching aid. – Today’s fast and modern time period requires a careful reform of education. It is important to emphasize the changes in the educational process taking into consideration the form, content, and the way of teaching children from the first grade of primary school. The implementation of AR applications into education enriches it and draws children’s attention to a specific subject, topic. It improves the educational process itself, and definitely also the outcomes of the educational process. Applications would definitely be interesting also in another language, adapted to the educational process at bilingual schools. – I cannot imagine any benefit in education (for example: I cannot see any difference whether the pupil sees the frog on his own table or on a white background). I can imagine it as a game element because not only children but also adults need to have game elements as being part of their lives. – Schools lack technical equipment. I am a proponent of using the AR in the education since early childhood but only for the purpose of learning through games. – I see an AR as a great opportunity to create interesting classes even when teaching subjects that the most children see as not very attractive. We live in the world of information technology moving forward every day. They provide us with space for developing creativity. I personally think that the AR in education field has tremendous potential in education and can also play an important role in remembering the teaching staff. – I think that AR developed with regard to the educational goal would be probably the best solution for education. I am not entirely decided which age category is AR appropriate to implement into educational processes even though today’s generation of children is getting into contact with IT and smart technology at a very early age. I think it is very individual and it is upon consideration of both parents in the home environment or teachers in the educational process. Although, according to the developments on the smartphone field, this direction is experiencing a boom. – It is clear that AR is a certain sub-group of audio-visual communication. It is true that the more perceptions people have in the learning process, the

Uses of Augmented Reality in Biology Education

187

more durable the record in human memory is. I can confirm that in the 80s of the last century, at lower secondary school level, we were sometimes taught by watching films from biology and physics. I can confirm that those films were popular among us pupils. Unfortunately, this form was little used as a didactic aid. But I will go back to AR. The main meaning of these didactic tools should not be based on AR, as I consider it only one possible variant of the graphical interface of application. The more important criterion in each learning application is the way knowledge is transferred, knowledge revision, playability, and motivation factor. And here is the stumbling block. Many teaching programs fell into oblivion because their motivation factor was low. To avoid this, you have to follow the rules of the “Gamification” theory. Referring to the previous findings about AR and using AR applications when teaching biology classes in general, and human anatomy in particular, a topic which is difficult to comprehend, especially in cases like previously mentioned in our chapter when future primary teachers need to get acquainted with the subject despite the fact that biology won’t be their future specialisation but will be one they need to cover in their future science teaching practice, it is important to mention that it is beneficial to incorporate methods like AR into their undergraduate studies as much as possible to engage them more in AR applications, which they themselves might then use in their future jobs as primary school teachers. As they have the opportunity to experience AR during their studies and directly use AR applications and interact with them during biology and other classes, it definitely provides them with challenging ways they might use AR applications later on. It is vital to discuss AR methods used with students, so we can draw more of their attention to this way of teaching, as it offers an opportunity for attracting their future students, as well as motivating them to be more involved in STEM subjects. Using AR applications during undergraduate studies of future Primary School Teachers is also a great way to introduce them to the didactic, pedagogical, and psychological issues regarding using AR as a method for teaching generation Alpha kids who are completely immersed in the technological world.

11

Discussion

The simplicity and mobility of the mobile device allows for more effective learning and retainment of knowledge (Balog & Pribeanu, 2010). With use of

188

Fuchsová ET AL.

the AR application features, students should be able to enhance their learning environments and improve their ability to retain information. Our findings confirm that mobile AR technology helped students to repeat the lesson and helped them understand biology, specifically anatomy. It appears that the AR technology learning experience was effectively delivered to students. They are moving from instructed-learning to a self-centred learning method. Although there have been a variety of technological interventions in education, there has been a lack of adoption of AR technology (Azuma, Billinghurst, & Klinker, 2011). In addition, many of the previous studies state that AR technology has been ignored in the learning environment in general, and particularly at the university level (Chu, Hwang, & Tsai, 2010; Tsai, Yen, & Yang, 2012). According to Billinghurst (2017), this technology is still underutilized because there are not enough experts available who are able to develop the content of the subject. They do not have the required level of skill needed to develop 3D modelling, programming knowledge and a detailed understanding of the subject for content development (Dunser, Walker, Horner, & Bentall, 2012). Despite still being underutilized, as previously mentioned, we consider AR use in educational contexts a very challenging method and means, which was also confirmed in our additional survey of those creating the microenvironments for those being educated nowadays. Most of them, who are in the role of parents and thus influencing using AR within educational context in some way, have positive attitudes towards using AR apps in educational context. The majority of our respondents from different working fields believes that these apps can improve the educational process itself as well as its outcomes. They highlighted the importance of apps developed for specific educational goals which they would like to be integrated into the educational contexts of their children. This seems to us to represent a big challenge and need for cooperation of AR app developers and educators. Similar positive attitudes to AR use in educational contexts were also observed by Cascales, Pérez-López and Contero (in Fleer, 2017) in their research, where parents’ perceptions of their kindergarten child’s use of AR were positive, and they considered AR an enriching and valuable addition to the early childhood program. Parents supported the use of AR (Fleer, 2017). Another issue of introducing AR technologies into the teaching process is the lack of material and technical provision of the university. For instance, Lin, Hsieh, Wang, Sie, and Chang (2011) stated that students find AR complicated, and that they often encounter technical problems while using it. During our research we have experienced technical problems (particularly with Wi-Fi connections), which sometimes made it difficult to use the AR applications, reducing the effectiveness of using tablets and AR applications. Without a well-designed interface and guidance for the students, AR technology can

Uses of Augmented Reality in Biology Education

189

be too complicated for effective use (Squire & Jan, 2007). Most of the studies reported that AR technology in education leads to enhancement of learning achievement in educational settings. AR can enhance learning motivation, help students to understand, enhance positive attitude, and enhance satisfaction (Akçayır & Akçayır, 2017). Talking about technology in general, AR in particular, as well as about the social nature of learning, we agree with Waite (2017) that technology can help us to overcome constraints of classroom learning environments by bringing the outdoor world into classrooms, by sharing and gathering information with others outside school community, by allowing the opportunity to become inspired learners by gaining access to international sources of information, or by contributing to different contexts outside school environment, a beneficial part of STEM education. In the context of our research focused on future primary school teachers who will cover biologythemed education and their future students as well as in the context of some of the biology-themed applications mentioned, we see the AR as one of the tools for developing play, an important form of learning in childhood, crucial in developing the creativity and imagination which is so important in STEM education and learning. While play is mainly viewed as an activity prevailing at preschool level, it is worth mentioning that school children also socialise through play and are motivated by play, but play is not always supported in school environments (Fleer, 2017). Play supports constructivist learning, as it serves as a tool for using what is already known in flexible and imaginative ways (Bruce, 2011). We are convinced, that play should be incorporated into educational contexts and that the combination of biology-themed education together with AR provides many opportunities to do that, as we have observed during our research. In general, researchers in educational technology are in agreement that AR as a learning method needs to be studied in more detail. The use of this technology could be very effective in motivating students’ learning and nurturing their ability to become passionately involved in their own learning process. The AR applications will assist them in learning biology using enhanced materials which stimulate their interest and help them to retain information longer. Based on the study results, we encourage higher education institutions to adopt and implement AR applications as a teaching and learning tool imperative in enabling effective and positive learning for the future.

12

Conclusion

In the chapter, we summarize information on the use of the augmented reality in biology education for university students who are future primary education

190

Fuchsová ET AL.

teachers, as well as on the elements shaping using AR educational process. We explore how AR technology can contribute to better understanding of curriculum regarding human, animal and plant anatomy and physiology, and how it can enhance students’ interest in ecological or environmental education. In the introductory part of the chapter, we also presented information on some free AR applications from different areas of biological education that students might use not only for deeper understanding of curriculum but also in their future profession of primary education teacher. Based on our qualitative research (Augmented Reality in Human Biology Education for Future Primary School Teachers), we confirmed that students’ understanding was deeper, their motivation was greater, and their creativity was strongly supported by the use of AR technology. The students were motivated by the new method, they cooperated very well, and their learning was constructive. They gained new knowledge and skills and using this method supported their mutual cooperation as well. In our chapter, we also point out to the importance of knowing the elements shaping the use of AR in the educational process, the issue which is also important to remind to future teachers as well to all being involved in educational process in the context of AR use. The information obtained from our survey that was conducted among respondents from different work areas, served as an additional source of information regarding the use and perception of AR in the educational process. The survey supported the idea of using AR in the educational process in general, as well as in teaching biology in particular, and at the same time pointed out to the need to develop applications for particular educational objectives. Survey findings can also serve as a basis for the future primary school teachers’ training along with information on free AR applications from different areas of biological education.

Acknowledgement The chapter was written the support of the grant KEGA 012UK-04/2018 “The Concept of Constructionism and Augmented Reality in the Field of the Natural and Technical Sciences of the Primary Education (CEPENSAR).”

References 4D Human Anatomy. (2018, May 31). Retrieved from https://play.google.com/store/ apps/details?id=com.ar.human.anatomy, marker: http://papp.tech/ARGames/ arhmnanatomy.html

Uses of Augmented Reality in Biology Education

191

Akçayır, M., & Akçayır, G. (2017). Advantages and challenges associated with augmented reality for education: A systematic review of the literature. Educational Research Review, 20, 1–11. Akçayır, M., Akçayır, G., Pektaş, H. M., & Ocak, M. A. (2016). Augmented reality in science laboratories: The effects of augmented reality on university students’ laboratory skills and attitudes toward science laboratories. Computers in Human Behavior, 57, 334–342. Anatomy 4D. (2018, May 31). Retrieved from https://play.google.com/store/apps/ details?id=com.daqri.d4DAnatomy&hl=sk, marker: http://getrealscience.org/ tzhao/2015/11/28/a-magic-application-anatomy-4d/ Animal 4D+. (2018, May 31). Retrieved from https://play.google.com/store/apps/ details?id=com.OctagonStudio.Animal4DPlus&hl=en, marker: https://www.octagonstudio.com/en/1/animal-4d-cards AR Animals Book. (2018, May 31). Retrieved from https://play.google.com/store/apps/ details?id=com.gt.aranimals&hl=en, marker: http://www.ar-animals.com/free_ marker, book: http://www.ar-animals.com/index_en.html AR Liver. (2018, May 31). Retrieved from http://www.iso-form.com/apps/ARLiver/, marker: http://www.iso-form.com/AR.pdf Arloon Plants. (2018, May 31). Retrieved from https://play.google.com/store/apps/ details?id=com.Arloon.PlantsAR&hl=en_US AR Safari Zoo. (2018, May 31). Retrieved from https://ar-safari-zoo-augmented-realitycardboard-ios.soft112.com/download.html, marker: http://e-itfunia.blogspot.sk/ 2015/11/safari-marker.html Arvanitis, T. N., Petrou, A., Knight, J. F., Savas, S., Sotiriou, S., & Gargalakos, M. (2007). Human factors and qualitative pedagogical evaluation of a mobile augmented reality system for science education used by learners with physical disabilities. Personal and Ubiquitous Computing, 13(3), 243–250. Azuma, R., Billinghurst, M., & Klinker, G. (2011). Special section on mobile augmented reality. Computers & Graphics, 35, vii–viii. Balog, A., & Pribeanu, C. (2010). The role of perceived enjoyment in the students’ acceptance of an augmented reality teaching platform: A structural equation modelling approach. Studies in Informatics and Control, 19(3), 319–330. Bayua, M. Z., Arshada, H., & Alib, N. M. (2013). Nutritional information visualization using mobile augmented reality technology. Procedia Technology, 11, 396–402. Berry, C., & Board, J. (2014). A protein in the palm of your hand through augmented reality. Biochemistry and Molecular Biology Education, 42(5), 446–449. Billinghurst, M. (2017, October 20). Augmented reality in education. Retrieved from http://www.solomonalexis.com/downloads/ar_edu.pdf Brain App (The). (2018, May 31). Retrieved from https://play.google.com/store/apps/ details?id=uk.co.harmony.brainapp&hl=cs, marker: https://www.harmony.co.uk/ project/the-brain-in-3d/

192

Fuchsová ET AL.

Brain iExplore AR (The). (2018, May 31). Retrieved from https://play.google.com/store/ apps/details?id=com.carltonbooks.iexplorethebrain&hl=en_US Bronfenbrenner, U., & Morris, P. A. (2006). The bioecological model of human development. In W. Damon & R. M. Lerner (Eds.), Handbook of child psychology (pp. 793–828). Hoboken, NJ: John Wiley & Sons, Inc. Bruce, T. (2011). Learning through play. London: Trans-Atlantic Publications. Carlson, K. J., & Gagnon, D. J. (2016). Augmented reality integrated simulation education in health care. Clinical Simulation in Nursing, 12, 123–127. Chang, C. S., Chen, T. S., & Hsu, W. H. (2011). The study on integrating WebQuest with mobile learning for environmental education. Computers & Education, 57(1), 1228–1239. Chang, K. E., Chang, C. T., Hou, H. T., Sung, Y. T., Chao, H. L., & Lee, C. M. (2014). Development and behavioral pattern analysis of a mobile guide system with augmented reality for painting appreciation instruction in an art museum. Computers & Education, 71, 185–197. ChemPreview. (2018, May 31). Retrieved from https://omictools.com/chempreview-tool Chen, C. H., Ho, C. H., & Lin, J. B. (2015). The development of an augmented reality game-based learning environment. Procedia – Social and Behavioral Sciences, 174, 216–220. Chien, Y. C., Su, Y. N., Wu, T. T., & Huang, Y. M. (2017). Enhancing students’ botanical learning by using augmented reality. Universal Access in the Information Society, 17(63), 1–11. Chu, H. C., Hwang, G. J., & Tsai, C. C. (2010). A knowledge engineering approach to developing mindtools for context-aware ubiquitous learning. Computers & Education, 54, 289–297. Clark, R. M. (2016). Childhood in society for the early years (3rd ed.). Thousand Oaks, CA: Sage. Domhardt, M. (2015). Training of carbohydrate estimation for people with diabetes using mobile augmented reality. Journal of Diabetes Science and Technology, 9(3), 516–524. Dunser, A., Walker, L., Horner, H., & Bentall, D. (2012). Creating interactive physics education books with augmented reality. In V. Farrell, G. Farrell, C. Chua, W. Huang, R. Vasa, & C. Woodward (Eds.), Proceedings of the 24th Australian Computer-Human Interaction Conference (pp. 107–114). Melbourne, Australia. EcoMOBILE project. (2018, May 31). Retrieved from http://ecomobile.gse.harvard.edu Feng, Z., Duh, H. B.-L., & Billinghurst, M. (2008). Trends in augmented reality tracking, interaction and display: A review of ten years of ISMAR. Paper presented at the 7th IEEE/ACM International Symposium on Mixed and Augmented Reality (ISMAR) (pp. 193–202). Cambridge: IEEE/ACM. Fleer, M. (2017). Play in the early years. Cambridge: Cambridge University Press. FreshAiR. (2018, May 31). Retrieved from http://www.playfreshair.com

Uses of Augmented Reality in Biology Education

193

Froggipedia. (2018, May 31). Retrieved from https://froggipedia.en.softonic.com/iphone Ganguly, P. K. (2010). Teaching and learning of anatomy in the 21 century: Direction and the strategies. The Open Medical Education Journal, 3, 5–10. Garrett, B. M., Jackson, C., & Wilson, B. (2015). Augmented reality m-learning to enhance nursing skills acquisition in the clinical skills laboratory. Interactive Technology and Smart Education, 12(4), 298–314. Gurevitz, R. (2000). Affective approaches to environmental education: Going beyond the imagined worlds of childhood? Ethics, Place & Environment, 3(3), 253–268. Hautecoeur, J. P. (2002). Ecological education in everyday life: Alpha 2000. Toronto: University of Toronto Press. Huang, T. C., Chen, C. C., & Chou, Y. W. (2016). Animating eco-education: To see, feel, and discover in an augmented reality-based experiential learning environment. Computers & Education, 96, 72–82. Human Annatomy Atlas. (2018, May 31). Retrieved from https://www.visiblebody.com/ anatomy-and-physiology-apps/human-anatomy-atlas Kalaš, I. (2013). Premeny školy v digitálnom veku. Bratislava: Mladé letá. Kamarainen, A. M., Metcalf, S., Grotzer, T., Browne, A., Mazzuca, D., Tutwiler, M. S., & Dede, C. (2013). EcoMOBILE: Integrating augmented reality and probeware with environmental education field trips. Computers & Education, 68, 545–556. Kostrub, D. (2017). Postoje aktérov treba dokázať akceptovať. Konštrukcionistická didaktika výučby matematiky. Pedagogická revue, 64(3), 40–58. Lin, H.-C. K., Hsieh, M. C., Wang, C. H., Sie, Z. Y., & Chang, S. H. (2011). Establishment and usability evaluation of an interactive AR learning system on conservation of fish. The Turkish Online Journal of Educational Technology, 10(4), 181–187. Marescaux, J., Soler, L., & Rubino, F. (2005). Augmented reality for surgery and interventional therapy. Operative Techniques in General Surgery, 7(4), 182–187. McCrindle, M. (2018, May 31). Generation alpha: Mark McCrindle Q & A with the New York Times. Retrieved from https://mccrindle.com.au/insights/blog/generation-alphamark-mccrindle-q-new-york-times/ Parker Bear by Seedling. (2018, May 31). Retrieved from https://www.apple.com/ uk-business/shop/product/HLJ42ZM/A/parker-your-augmented-reality-bear-byseedling Plant Life Balance. (2018, May 31). Retrieved from https://play.google.com/store/apps/ details?id=au.com.plantlifebalance Quiver Education. (2018, May 31). Retrieved from http://www.quivervision.com/ education-coloring-packs/#education-starter-pack Radu, I. (2014). Augmented reality in education: A meta-review and cross-media analysis. Personal and Ubiquitous Computing, 18(6), 1533–1543. Rollo, M. E., Bucher, T., Smith, S. P., & Collins, C. E. (2017). ServAR: An augmented reality tool to guide the serving of food. International Journal of Behavioral Nutrition and Physical Activity, 14, 65.

194

Fuchsová ET AL.

Sakulphon, M., Srisawasdi, N., & Wangsomnuk, P. (2015). Examining relationship between biology attitudes and perceptions toward mobile augmented reality of photosynthesis and impact on gender difference. In H. Ogata (Ed.), Proceedings of the 23rd International Conference on Computers in Education (pp. 836–841). China: Asia-Pacific Society for Computers in Education. Skin & Bones. (2018, May 31). Retrieved from https://itunes.apple.com/us/app/skinbones/id929733243?mt=8, marker: https://naturalhistory.si.edu/exhibits/bonehall/Triggering_images.pdf Squire, K. D., & Jan, M. (2007). Mad City mystery: Developing scientific argumentation skills with a place-based augmented reality game on handheld computers. Journal of Science Education and Technology, 16(1), 5–29. Tarng, W., Yu, C. S., Liou, F. L., & Liou, H. H. (2013). Development of a virtual butterfly ecological system based on augmented reality and mobile learning technologies. In 2013 9th International Wireless Communications and Mobile Computing Conference (IWCMC) (pp. 674–679). Sardinia, Italy: IEEE. Tóthová, R., Kostrub, D., & Ferková, Š. (2017). Žiak, učiteľ a výučba (všeobecná didaktika pre študentov učiteľstva). Prešov: Rokus. Tsai, C., Yen, J., & Yang, J. (2012). The influence of employing augmented reality in course design on the learning achievement and satisfaction of the Aquatic animals unit. Business and Information, 822–832. Virtuali-Tee by Curiscope. (2018, May 31). Retrieved from https://play.google.com/ store/apps/details?id=com.curiscope.virtualitee&hl=en_GB, marker: https://www.curiscope.com/products/virtuali-tee VR Endocrine Glands Male APK. (2018, May 31). Retrieved from https://apkpure.com/ vr-endocrine-glands-male/com.maleendocrine.android, marker: http://www.edulusvr.com/marker/ VR Fetus in the Uterus APK. (2018, May 31). Retrieved from https://www.apkmonk.com/ app/com.fetus.android/, marker: http://www.edulusvr.com/marker/ Waite, S. (2017). Children learning outside the classroom: From birth to eleven. Thousand Oaks, CA: Sage. Wrzesien, M., Botella, C., Bretón-López, J., Río González, E., & Alcañiz, M. Á. (2015). Treating small animal phobias using a projective-augmented reality system: A single-case study. Computers in Human Behavior, 49, 343–353. Zarzuela, M. M., Pernas, F. J., Martínez, L. B., Ortega, D. G., & Rodríguez, M. A. (2013). Mobile serious game using augmented reality for supporting children’s learning about animals. Procedia Computer Science, 25, 375–381. Zheng, M., & Waller, M. P. (2017). ChemPreview: An augmented reality-based molecular interface. Journal of Molecular Graphics and Modelling, 73, 18–23. Zookazam. (2018, May 31). Retrieved from https://play.google.com/store/apps/ details?id=com.AtlantaAr.ZooKazam&hl=en_US, marker: http://www.zookazam.com/targets.html

CHAPTER 9

Uses of Augmented Reality in Tertiary Education Martina Siposova and Tomas Hlava

Abstract Since augmented reality (AR) in educational settings hopes to enhance the process of learning, this chapter tries to answer questions of the use of AR at the university level and its impact on learners’ knowledge attainment (cognitive domain) and engagement (affective domain). Based on 21 studies (2013–2017) focusing on university level students of various fields of academic interest, meta-analysis showed that (1) the added value of AR implementation in terms of learning achievements ranges between intermediate and very positive (the lowest value of Cohen’s d = 0.42); and (2) the impact of AR on increases in learners’ motivation and autonomy, students’ collaboration and teacher-student interaction was prominent, too, although not supported by statistical testing. However, it is not known whether the effect of AR would prove stable and long-lasting, as it is probable that the aspect of novelty could play an important role during the experiments.

Keywords tertiary education – augmented reality – AR – social sciences – natural sciences – formal sciences – foreign language teaching – gamification – writing skills – vocabulary learning – control group – experimental group – motivation

1

Introduction

Augmented Reality (AR), which has been defined as interactive blending real and virtual environments in real-time and registered in 3-D, has become the focus of research and scholars’ attention since the 1990s. Moreover, the research in this field has dramatically intensified since 2013 (Akçayır & Akçayır, 2017). This trend suggests that a similar level of interest will continue in 2019 and after. Systematic reviews of research on AR technology provide readers with the most common aspects, such as the definitions of AR as well as © koninklijke brill nv, leideN, 2020 | DOI: 10.1163/9789004408845_009

196

Siposova and Hlava

determination of why it can be considered an educational concept, rather than a certain type of technology (Azuma, 1997; Azuma et al., 2001; El Sayed, Zayed, & Sharawy, 2011; Martin et al., 2011; Wu, Lee, Chang, & Liang, 2013). N. A. et al., 2011). Many scholarly articles have dealt with general characteristics of utilising AR in real-life (especially workplaces) as well as in educational settings (i.e., formal learning, non-formal learning, informal learning as well as training in a workplace; Funk et al., 2017; Ira & Berge, 2008; Wild, 2016). Another aspect which has been discussed by scholars and researchers concerns the typology of AR technologies, particularly the identification of the most frequently used AR technologies in education. At present, it has been confirmed that the most commonly used delivery platform for AR is mobile devices (60%) due to the fact that up-to-date mobile devices offer an inexpensive, practical, flexible, easy-to-use, effective, and profitable platform for AR applications (Akçayır & Akçayır, 2016, 2017; Bressler, & Bodzin, 2013; Chiang, Yang, & Hwang, 2014a; 2014b; Furió, González-Gancedo, Juan, Seguí, & Costa, 2013; Hwang, Tsai, Chu, Kinshuk, & Chen, 2012; Johnson, Laurence, Smith, & Stone, 2010; Yu, Jin, Luo, Lai, & Huang, 2009). One commonly identified feature of the other set of scholarly articles is the aspect of learners’ motivation and attitudes which have been positive and improved while utilising AR in the teaching-learning process (Chiang et al., 2014a; Lu & Liu, 2015; Sotiriou & Bogner, 2008). Obviously, AR offers plentiful opportunities to expand the boundaries of formal as well as informal learning spaces to create new dimensions in mobile learning and to increase connectedness of learners in multiple contexts. AR is already being embraced across many disciplines (Lee, 2012). 1.1 The Purpose of This Chapter Vast technological progress in computer processing power and imaging allows augmented reality (AR) to enter into educational settings in order to enhance the process of learning. The purpose of this chapter is to cover the use of augmented reality as part of teaching-learning interface in tertiary education. Naturally, thanks to what AR does the best –visualisation and interaction with virtual content – it suggests implementation into courses of fields that are, logically, based on working with concrete objects built from real matter. However, while expertise and excellence (cognitive domain) still remain the trademarks of modern tertiary education, it posits a challenge for AR to aid the process of learning in the realm of disciplines being grounded in highly abstract concepts and multidimensional systems of relations. There is a real imperative for educators to explore ways of using AR to enhance student learning and develop a complex understanding of the augmented learning processes, taking into consideration both advantages and challenges. There are many studies

Uses of Augmented Reality in Tertiary Education

197

which have focused on a variety of learner types, including primary school learners, lower and upper-secondary students as well as adults training in a workplace. However, reviews of research on AR in tertiary education are less common. Although university students seem to be naturally prone to exploiting AR in their educational setting (Kücük, Aydemir, Yildirim, Arpacik, & Goktas, 2013; Wu et al., 2012), it seems the range of scholarly articles and studies aimed at examining the potential of AR at the university level education does not fully cover the variety of academic fields. To fill that gap in the literature, this chapter is aimed at analysing selected and relevant studies published in the Social Science Citation Index (SSCI) journal database, the Education Resources Information Centre (ERIC) journal database and Open Access Journals.

2

The Manuscript Selection and Analysis Process

Since the authors of this chapter are linguists focused on research in the field of foreign language teaching and learning at university level, the aim of this systematic review is to give a more detailed analysis of the implementation and effects of AR in foreign language teaching and learning (as part of humanities). However, the complex view on tertiary education cannot exclude sciences, which, based on the reviewed research results, outweigh humanities. Thus, we have systematically selected and analysed relevant scholarly articles and research studies aimed at AR exploitation. We used the following criteria: the subject matter is fully acknowledged AR, as defined by (Azuma, 1997; Azuma et al., 2001); the research subjects are university students (aged 19 to 26); different academic fields of university studies are covered; and quantitative as well as qualitative research methods have been used by the authors of the articles in order to prove the impact of AR on university students. In order to get to the source papers, we fed the search engine of ERIC and WoS databases with the key words AR, Augmented reality, and Augmenting reality, and also filtered the search for education research context written in English from 2013 to 2017. The search resulted in 93 papers that were further analysed by two researchers to eliminate the papers not meeting the set criteria (Table 9.1). After manual review, 21 papers were considered to meet the requirements. Out of 21 papers, ten (47.6%) either contained quantitative data that were used to calculate effect size (Cohen’s d or its alternative) or it was already supplied (Table 9.2). The rest of papers did not contain suitable statistics but offered results such as findings of a questionnaire based on one group, analysis of post-treatment interview, etc.

198

Siposova and Hlava

table 9.1  Inclusion and exclusion criteria

Inclusion criteria

Exclusion criteria

Must involve AR application in Title, keywords, topic confusing AR with VR educational context Must involve tertiary level students Reviews without original research fijindings Must be publicly available Editorials, conference papers, book chapters Must be published between 2013–2017

table 9.2  Descriptive characteristics of included papers with quantitative data

Publication

Year

Lin, T.-J. et al. Kose et al. Albrecht et al.

2013 40 d = 0.75 2013 100 d = 0.78 2013 10 r = 0.7 (Rosenthals’ r is equal to Cohen’s d ≥ 1) 2014 40 d = 0.51 2015 30 d = 0.81 2016 120 d = 1.95 2016 38 d = 0.1 2016 30 d = –0.75 (nonAR group performed better) 2017 103 g = 0.48 (Hedges’ g is equal to Cohen’s d = 0.42) 2017 60 d = 1.81

Wang et al. Jamali et al. Hanafiji et al. Juan et al. Santos et al. Wang, Y-H. Őzcan et al.

3

N

Efffect size

Branch natural formal professional/applied natural professional/applied formal/social professional/applied language formal language

Branches of Science Exploiting the Utilisation of AR in Tertiary Education

With regard to the fact that contexts are inevitable and required when utilising AR, in terms of tertiary education these contexts offer the possibility of having virtual experiences in highly professional sectors. Firstly, in order to categorize the main foci of the analysis we have decided to follow the taxonomy of academic disciplines elaborated by Becher (1994), Franklin (1999), and Wanner, Lewis, and Gregorio (1981). In this taxonomy are five major categories of academic disciplines: humanities, social sciences, natural sciences, formal

Uses of Augmented Reality in Tertiary Education

199

sciences, and the professions and applied sciences. Each discipline features different sub-disciplines. The humanities include history, languages and linguistics, literature, performing arts, philosophy, religion and visual arts. The social sciences include anthropology, archaeology, area/regional studies, cultural and ethnic studies, economics, gender and sexuality studies, geography, political science, psychology and sociology. The natural sciences include the space sciences, earth sciences, life sciences, chemistry and physics. The formal sciences include computer science, logic, mathematics, statistics and systems science. The professions and applied sciences include agriculture, architecture and design, business, divinity, education, engineering, environmental studies and forestry, family and consumer science, health sciences, human physical performance and recreation, journalism, media studies and communication, law, library and museum studies, military sciences, public administration, social work and transportation. 3.1 Social Sciences In the field of social sciences, research papers and studies with empirical evidence are scarce. From the original set of papers, only one met the requirements, although one other was formally classified as appropriate, since a group of respondents employed in the study were students in the field of social sciences. However, we shall pay full attention to this study (Hanafi et al., 2016) only later in the section on Formal sciences as the contents that the students were supposed to learn was based on Computer Science. Thus, first and last, it is a study by Bicen and Bal (2016) who reported opinions and attitudes of 97 volunteer undergraduate students on effects of AR on their learning. On the basis of a 5-point Likert-based questionnaire survey (1 – complete disagreement to 5 – complete agreement), students enrolled in psychology and teacher-training studies reported that the AR application was helpful in increasing their interaction (mean: 4.52, SD: 0.50), motivation (mean: 4.58, SD: 0.49) and attention (mean: 4.05, SD: 0.68). Furthermore, they perceived the learning process supported by AR more enjoyable (mean: 4.55, SD: 0.49), which was implicitly linked with discovering more information (mean: 4.12, SD: 0.64), better understanding of content (mean: 4.19, SD: 0.68), and overall increased course success (mean: 4.15, SD: 0.66). However, no group of students opining the traditional, non-AR tasks and exercises was involved in the study and thus there is no evidence to which these results could be compared. 3.2 Natural Sciences In natural sciences, in the sub-discipline physics, a paper by Lin et al. (2013) explored the effectiveness of using a developed mobile AR system for collaborative learning called “AR Physics.” Singaporean undergraduate students’

200

Siposova and Hlava

knowledge of elastic collisions was examined through an experimental group consisting of 8 males and 12 females (age range 21–26) and a control group consisting of 7 males and 13 females (age range 21–24). In the study, a t-test was conducted to compare learners’ achievements in terms of their post-test scores between experimental (AR Physics) and control groups (2D Physics). The calculation of Student’s t-test showed that students who learned with the AR Physics system had significantly better learning achievement than those who learned with the traditional 2D simulation system (t (38) = 2.36, p < 0.05), with the practical effect (d = 0.75) of AR intervention being responsible for 75% of the differences between groups. Regarding the analysis of the learners’ epistemic dimension of knowledge construction, it was revealed that the AR Physics system served as a supportive tool and enabled the learner dyads to respond quickly to the displayed results and supported their knowledge construction processes. In the Wang et al. (2014) research paper on collaborative inquiry learning behaviours during physics class exercise, in which 40 university students formed random dyads, the findings show that learning through computerbased simulations (either standard 2D or AR simulation) can support higherorder inquiry learning. On the one hand, in case of five observed inquiry behaviours, namely orientation, hypothesis, experiment, interpretation, and conclusion, statistical testing did not indicate any significant differences between the experimental group (dyads working with AR simulations) and the control group (dyads using standard 2D simulation tools); both approached the learning process in similar fashion. The processes of data organization and interpretation, which normally lead to students’ own knowledge construction, were, from the perspective of the distribution of five inquiry-learning behaviours, constitutive elements and the percentage of their exploitation dissimilar. Differences were, however, found in individual behavioural patterns. In case of the experimental group, nine significant behavioural patterns were observed and in case of the control group, the number was six. While both groups shared five behavioural patterns, the extra four of them observed in AR group compared to only one extra in the control group is a clear quantitative advantage. Although this did not translate to statistically significant difference (X2 = 2.52, p < .05), the common language effect size indicator equalling to 69%, representing the effect size of d = 0.51, stands in favour of desired effects. As the measure of raw knowledge gains was not the aim of the study, it cannot be explicitly stated whether the AR would be more beneficial in these terms. Also, the higher number of higher-order learning behaviours applied during the activity seems to be in contrast with what many studies have reported, namely lower cognitive load (see Kücük, Yilmaz, & Göktas, 2014; Lin & Yu, 2012;

Uses of Augmented Reality in Tertiary Education

201

Santos et al., 2016). However, the higher number of significant behavioural patterns does not seem to be counter-productive, since more processes are not to be viewed as creating redundancy, but rather as exploiting the possibilities of human brain’s processing power and thus leading to better learning results. Such views suggest that AR simulation can be more supportive than traditional 2D simulation (Wang et al., 2014). Another survey finding based on opinions of 12 geology students, conducted by Woods, Reed, Hsi, Woods, and Woods (2016), refer to an AR application having positive impact. As described by authors, students unanimously agreed on effective learning of the contents of topographic maps class, stated its meaningfulness from the real-life perspective and considered the learning experience as interactive, attractive and refreshing. Although the report does not offer any rigorous statistical analysis, general conclusions imply positive effects of AR on learning experience. 3.3 Formal Sciences Concerning formal sciences, in the study by Wang (2017), the researcher adopted a comparative research approach with a total of 103 college students participating in a software editing course. The result of this study cannot be considered unequivocal. Although the experimental group (59 learners) learning with the AR-based contents expressed aroused interest in becoming active learners, increased motivation, and vivid peer-learning interaction, the control group (54 students) learning with the online-based support also showed deep concentration in doing the self-directed software operational practice. Calculation of effect size (Glass’ delta because of difference in variation and Hedges’ gamma because of difference in sample size) for the difference in groups’ results after the third week of instruction (d = 0.42, g = 0.48) compared to the results after the first week of instruction (Cohens’ d = 0.27 because of nondifferent variation and similar sample size) showed 8% increase in performance. Hence, the quasi-experiment findings demonstrated the great potential of AR techniques for supporting students’ learning motivation and peer learning interaction and focus on the subject matter. Hanafi et al. (2016) conducted a study where 120 undergraduates undertook an ICT course called Computer Systems. The participants’ process of learning was supported either by an AR application installed in their mobile phones, enabling students to collaborate, or by a conventional PC program installed in computers in a computer laboratory. Multiple-choice pre-tests and post-tests were administered to search for any increase in learning achievement. Individual test scores of the experimental group (pre-test mean: 63.20, SD: 1.86; posttest mean: 81.80, SD: 5.71; d = 4.38) and of the control group (pre-test mean:

202

Siposova and Hlava

61.53, SD: 4.48; post-test mean: 72.99, SD: 2.85; d = 3) changed significantly. The difference in increase of post-test performance between experimental and control group was reported as statistically significant with the effect size d = 1.95 what equals to approximately 96% of control group subjects performing below the average of experimental group. Thus, implementation of the AR application used by students in experimental group allowed them to outperform students learning by conventional means. However, observed differences were said to be gender sensitive, since it was only males whose performance differed significantly based on the learning condition. In case of females, no differences were observed based on learning condition. In the findings by Kose et al. (2013), 100 university students undertook the AR supported learning process in the field of computer technologies, education, and engineering, resulting in improved students’ academic achievement coexistent with improved learning experiences. Students’ learning gains were claimed on the basis of survey-based approach and on the number of students that passed university courses connected to the topic of the research. Students subjectively considered AR applications supporting their learning enabled them to learn faster and to reach higher academic achievement, which relates to objectively higher number of students passing the thematic university courses (84 in AR group with Mean: 73.88 and SD: 14.19 to 61 in control group with Mean: 60.28 and SD: 19.80). Although the significance of the difference between the groups was not calculated, our calculation of effect size (d = 0.78) refers about 76% of control group students achieving results below the average of experimental group. Thus, evaluation of the AR group students who considered the course to be better and more enjoyable, to whom the materials seemed more attractive and who felt more self-confident during their learning process, has statistical support. These can be considered influential characteristics positively influencing the learning gains through increasing learning satisfaction, which is in accordance with Santos et al. (2016). 3.4 Professions and Applied Sciences In the professions and applied sciences, in the sub-discipline health sciences, there has been a huge increase in AR use in medical settings to train and teach medical knowledge. The results of the study conducted by Albrecht, Folta-Schoofs, Behrends, and von Jan (2013) confirmed that those research subjects of the experimental group (6 third-year medical students) who were assigned to the self-developed mobile AR tool learned more than the members of the control group (4 students) who used the textbook in forensic medicine, particularly being given a 10-question standard multiple choice test about “gunshot wounds.” Statistically significant, the mARble group showed

Uses of Augmented Reality in Tertiary Education

203

greater knowledge gain than the control group (Wilcoxon z = 2.232, P = .03) with high practical effect (r = 0.70 equal to Cohen’s d≥1) when following Hattie’s educational interpretation (Hattie, 2009). The item analysis of the SC test showed a difficulty of P = 0.768 (s = 0.09) and a selectivity of RPB = 0.2. For mARble, fatigue (z = 2.214, P = .03) and numbness (z = 2.07, P = .04) decreased with statistical significance when comparing pre- and post-tests. Vigour rose slightly, while irritability did not increase significantly. Changes in the control group were insignificant. Regarding hedonic quality (identification, stimulation, attractiveness), there were significant differences between the mARble (mean 1.179, CI –0.440 to 0.440) and the book chapter (mean –0.982, CI –0.959 to 0.959); however, the pragmatic quality mean differed only slightly. The authors conclude that although the small sample size (10 students) reduces the potential of conclusiveness, its design seems appropriate for determining the effects of interactive eLearning material with respect to emotions, learning efficiency, and hedonic and pragmatic qualities using a larger group. Jan, Noll, Behrends, and Albrecht, (2012) proposed that AR within the medical learning environment is important because it allows users to “immerse themselves in the scenario, being active learners and also becoming learning objects at the same time, interacting within a learning scene in the real world.” Furthermore, AR learning tools in medical settings can offer an effective complement to the conventional anatomy training tool repertoire (namely anatomy models). Jamali, Shiratuddin, Wong, and Oskam (2015) carried out pilot testing for measuring the reliability of the prototype HuMAR (Human Anatomy in Mobile-Augmented Reality) application. In their study the contents of the prototype HuMAR application began with bone descriptions, bone joint locations, bone part labels, and reference links which were gathered following advice from discussions with anatomists. These features were built-in for students studying a Forensic Anatomy and Anthropology unit at Murdoch University in order to improve their learning environment by using the mAR prototype as a learning tool for identifying the osteological features of the lower appendicular skeleton. The pilot testing included 30 university students equipped with the prototype HuMAR application using tablets. To determine the significance of mAR technology, pre- and post-usage tests were conducted, and it was found out that there was a substantial difference in values between the two groups: the first group (control group, non-technology, using the human skeleton), the second group (experimental) was exposed to mAR technology in their learning. The results showed that mAR technology enhanced the understanding of the subject and increased motivation. In terms of educational achievements, improved students’ learning performance was found to be significant in both groups (control group: t (29) = 6.123, p < 0.05; experimental group: t (38) = 8.629,

204

Siposova and Hlava

p < 0.05)). However, calculated greatness of effect between the groups (d = 0.81; calculated de sensu, Klauer, 2001) showed 79% of control group performance being below the average of experimental group performance. Similarly, in the study by Patzer, Smith, and Keebler (2014), the use of digital AR anatomy models (particularly a labelled AR heart) was compared to the use of an unlabelled AR heart with a textual reference and an unlabelled fiberglass model of the heart with a textual reference while training on the functional anatomy of the heart-pumping to propel blood from the heart’s chambers, through the major vessels, the lungs, and back throughout the body. Questionnaire results (20 respondents) showed that learning the anatomy of the heart with the AR training tools presents more enjoyable, curiosity inducing, and easier to use learning tools than the fiberglass model. Similar results were obtained by Kücük, Kapakin, and Goktas (2015) whose study was aimed at determining the medical faculty students’ views on anatomy learning via “mobile augmented reality” technology. An explanatory design with mixed methods was applied in their study. The purposive, random sample consisted of 34 sophomore students, studying medicine. As data collection tools, questionnaires and interview forms were used. Descriptive analysis methods were used for the data analysis. The results showed students’ views towards mAR-based learning were highly positive. Students especially emphasized how mAR-based learning generated a sense of reality, materialized the subjects, increased interest in the lesson, and was beneficial for individual study by providing a flexible learning environment. The influence of AR applications on learning gains in the field of dentistry was researched by Juan (2016). In the study, 38 undergraduates of the School of Dentistry undertook the learning process supported by either a video lesson or an AR application. Results of statistical analysis were twofold. Firstly, statistically significant differences were found between the pre-test and post-test of both experimental (mean: 8.23; SD: 1.43 – mean: 9.60 SD: 0.72; d = 1.21) and control (mean: 7.54 SD: 2.00 – mean: 9.00 SD: 1.56; d = 0.81) groups. Secondly, no statistically significant differences were found between the post-tests of the experimental AR group (mean: 9.60 SD: 0.72) and the control video group (mean: 9.00 SD: 1.56) with effect size (d = 0.1; calculated de sensu, Klauer, 2001) indicating only a 4% increase in performance of the experimental group compared to the control group. Thus, AR formally demonstrated its positive impact on learning. The other side of the coin, however, reflects the fact that it was possible for participants to achieve significantly better results by the more traditional learning methods, too. Based on the calculated effect size values for individual groups, one could still consider the traditional video-presentation to be less effective, but the counter-argument holds that (a) the experimental group achieved better performance than control group in the pre-test in terms

Uses of Augmented Reality in Tertiary Education

205

of both mean score and variation resulting in effect size (d = 0.39) standing for 12% of the difference; and (b) the difference in the control group pre-test – post-test performance (d = 0.81) is a solid increase in learning achievement (29%). Similar positive feedback was reported in the field of medicine, anatomy learning in particular. In their research (Huang & Liaw, 2014), answers of 55 university students were gathered in order to study the influence of AR on motivation. Statistical analysis revealed that immersion and interactivity had a much bigger positive effect on learners’ motivation than presence; immersion was found to be statistically most important predictor of motivation (F (2.52) = 15.72, p < 0.001, R2 = 0.38). From the perspective of pre-service teachers, stronger evidence on AR having a positive impact on social and affective domain is reported by Delello (2014). In a class of 31 undergraduates in a teacher-training programme, the pre-service teachers prepared a content for a field-based science class in which an AR application was to be utilised. Based on the students’ reflection papers a qualitative analysis was performed, specifically focusing the use of the AR application. The author reported a positive impact on classroom experiences, and an increase in motivation and students’ engagement. In this case, horizontal form of social learning was present as well, in that the practice was supportive of community interactions. This effect was supported by a narrative saying that “students found it useful to learn from and with each other.” According to other pre-service teachers’ narratives, the students’ attention span was prolonged, and their active participation increased. Moreover, pre-service teachers subjectively observed AR’s positive effects on the cognitive domain as well, since the students seemed to “perform extremely well on the assessment at the end of the lesson” (Delello, 2014). A similar impact of AR on social learning and interaction was reported by Albrecht et al. (2013) in their forensic medicine experiment. The experimental groups were characterised by a high level of interaction, by contrast, participants in traditional textbook based condition lacked any form of social interaction and preferred individual learning, even though they were encouraged to interact.

4

Areas of Foreign Language Teaching Curricula Exploiting the Utilisation of AR in Tertiary Education

Recently published literature emphasizes the benefits of a foreign-languagelearning applications using mobile-augmented reality based on the gamification didactic method and text recognition (Dită, 2016; Liu, Holden, & Zheng, 2016; McGonigal, 2011; Perry, 2015). According to McGonigal (2011), globally, 3

206

Siposova and Hlava

billion hours a week are spent gaming. Perry (2015) and Gee (2003) add that a well-developed game keeps a player enthralled and engaged for hours. This view is also supported by pedagogical researchers (Erenli, 2013; Kumar, 2012) who are in favour of gamification as a didactic method, especially regarding the impact of gamification in different learning contexts. Among commonly stated benefits of gamification are: increased motivation, immersion, social aspects of learning as well as general impact the games have on students’ emotional, cognitive, and social areas. Consequently, learning a foreign language becomes much easier and more enjoyable using a gamification approach. Undoubtedly, educators have already been focusing on creating language learning experiences that extend beyond traditional classroom and one of these key ways has been through gamification. Gamification in language education has taken a dominant position due to the fact that “a good video game provides rich opportunities for players to experience the problem-solving and goal-pursuing processes” (Gee, 2005). Players need to interact, establish relationships with their partners, negotiate with each other for a collaborative action, as well as react immediately. Regarding this, Liu, Holden, and Zheng (2016) created an AR mobile game called “Guardians of the Mo‘o” in order to enhance cultural understanding, linguistic awareness, and ultimately to promote students’ active language learning. Players of the game are the helpers or guardians of the Mo‘o (who is a gecko or Lizard Goddess in Hawaiian culture) who is ill and in need of help. Using both virtual objects such as drawings or notes and physical items such as the trees or works of art on campus, the three research subjects (university students – South Koreans, intermediate English proficiency language level) were able to experience interaction in both the virtual and physical space as well as being naturally involved in a foreign language negotiation, thus coming to a group consensus resulting in action in the end. The use of the iPad afforded them the chance to discuss how it would be best utilized in order to accomplish their goals. Post-game interviews identified the following benefits of the game: real-life negotiation in the foreign language, experience in visiting new areas of the campus, and opportunities to speak with strangers in the real world. Dită (2016) proposed a foreign language learning application using mAR based on text recognition and gamification. In his study, the text recognition system allowed identifying the text written on a card by applying an Optical Character Recognition (OCR) technique and an algorithm to match the results of OCR with available words. This system also used the Text-To-Speech functionality in order to pronounce the translation of the word identified by the text recognition system. Although Dită’s study does not explicitly define the research sample (number of students, timing), he concludes that his study

Uses of Augmented Reality in Tertiary Education

207

showed the motivation of players was increased and they considered learning more enjoyable when using the gamification system. Apart from English language learning, other languages have been inspired by utilising AR tools in gamification too. In the case study by Perry (2015), eleven volunteering first-year University French students were involved in the gamified system Explorez. Using GPS, Explorez transforms the University of Victoria, B.C. campus into a virtual francophone world, where students interact with characters, items, and media as they improve their French language skills and discover their campus. These interactions (a player in this treasure hunt is hired as the personal assistant to a famous French celebrity) take place either in the form of written text or audio and video recordings to which the student must respond (both in writing and orally). Explorez allows learning take place outside the classroom, with the goal of providing a contextual and immersive language experience. Explorez was inspired by Mentira (Holden & Sykes, 2011), the first place-based, AR mobile game for learning Spanish in a local neighbourhood in the Southwestern United States. The main aim of the game was to explore both the complexities and benefits of integrating mobile games in language-learning contexts. From the cultural evolutionary psychology perspective, currently represented by Heyes (2018), gamification had a well observable sociocultural learning effect of horizontal character in that a more advanced student helped a less advanced one to get better orientation within the system or quest, and supplied the necessary word or information (Perry, 2015). The focus group interview and post-activity questionnaire indicated that the subjects considered the learning activity fun, motivating, useful, and relevant. Participants were particularly appreciative of the activity taking place out of the classroom settings and, as stated by the author, found the intrinsic motivation in collaboration and completion of the assigned quests related to real life (meaningfulness and relevancy). Although no direct impact on learning outcomes in cognitive sphere was stated, positive effects in sociocultural and affective dimensions were present. Apart from the group of games utilising AR application, there are also tools for developing writing skills of language learners related to the fact that when learners practice their writing skills, they often face a number of problems (e.g., insufficient vocabulary, lack of experience expressing the content as well as difficulty using appropriate stylistic and grammar structures). With regard to this, Liu and Tsai (2013) adopted AR technology to support outdoor language learning activities and English composition (particularly writing descriptive essays) for college students. In their study, the researchers adopted GPS with AR techniques to assist five 20-year-old undergraduate students in exploring university scenes. The AR-based mobile learning material assisted

208

Siposova and Hlava

the participants with English vocabulary and expressions needed for descriptive writing as well as content knowledge related to the composition subjects. After the students finished the exploration of the location, they were asked to write an English composition to introduce the scene. The results showed that the participants were engaged in the learning activities constructing linguistic and content knowledge and thus produced meaningful English essays. On the other hand, neither the study by Liu and Tsai (2013), nor the experiment by Ting (2015), carried out in a group of students learning Chinese writing, indicated significant differences in the performance of the learners’ writing content and skills. Both control and experimental groups produced meaningful pieces of writing. One observed advantage of utilising AR was that the learners in the AR group had better attitudes towards learning writing. In their study on augmenting a poster carousel through AR during the English course offered at Osaka University, Alizadeh, Mehran, Koguchi, and Takemura (2017) found that the opinion of the majority of 71 participants was that using BlippAR application would not directly contribute to the improvement of students’ English language performance. A very similar case was spotted in the research report by Furio, Juan, Seguí, and Vivó (2015). According to their findings, although the case of students’ motivation related to learning as a result of successful adoption of AR application was observed, learning outcomes in terms of gained knowledge were arguably in the range from similar to the very same. An indirect proof of AR applications’ effectiveness was proposed by Solak and Cakir (2015) study primarily focused on examining motivation of 130 Turkish university students who were attending elementary English course. In order to extend participants’ vocabulary, classes were enriched by the use of an AR application. Unfortunately, no details about the application or vocabulary items were stated. A positive correlation of statistical significance was reported between academic achievement (vocabulary learning) and motivation. However, the calculated Pearson’s coefficient (r = 0.21, p < 0.05) representing a weak uphill linear relationship does not provide us with much rigorous information on the background of inter-relatedness of participant’s learning gains and their motivation. On the other hand, results of study by Santos et al. (2016) showed that AR does not necessarily offer one-sided results in favour of AR applications. In their experiment on situated vocabulary learning, significantly better results of the control group (non-AR condition) compared to the experimental group (AR condition) in immediate post-test were reported (d = 0.75). What Santos et al. (2016) consider to be a moderate effect is seen as large from the perspective of educational outcomes (Hattie, 2009). Different results were observed in the

Uses of Augmented Reality in Tertiary Education

209

delayed post-test, in which the decrease in retention of learned vocabulary was significant in the case of control group (d = 0.84) but not in case of experimental group (d = 0.14). What is considered a positive impact of AR on learning is that subjects’ retention almost did not change in pre-test – post-test condition (low forgetting rate). While the control group forgot more of the contents they had learned compared to members of AR group, the AR group had not learned as much in the first place, so there was not as much they could forget (note: a lower cognitive load was observed in many research reports experimenting with AR, thus learning less might be an explanation). The research paper states that during the period between post-test and delayed post-test, learned vocabulary was not actively used, which resulted in decreased retrieval in control group subjects. While both groups had the same conditions in terms of not using the vocabulary during that period, it was the control group that could have potentially achieved lower retention decrease, thus remembering more if the learned knowledge would have been meaningfully used in real life scenarios. There was, however, no such possibility for the subjects of experimental group since, as we have already mentioned, they had not learned as much in the first place. It is not an unknown fact that knowledge remains significantly more available for its active use when exploited meaningfully (Prince, 2004) and it applies well in the case of foreign language vocabulary learning too. Under such conditions, the research follows the Popperian conception (see Munnerley et al., 2012) asking for the control of as many variables as possible, giving us the answers to the questions being based on very restricted conditions, mostly incompatible with real life settings or at least creating environments that are far from it and do not reflect the full-range of real-life interactions. There has also been some research related to examining effects of AR applications on reading literature texts and the comprehension of vocabulary used in them by university students. In the experiment by Őzcan et al. (2017) carried out with 60 freshman students in the Department of Turkish Education, University of Agri, the control group (30 students) read Turkish poems using a booklet with Ottoman spellings of poems, which were printed, while the experimental group (30 students) used QR-codes and 3D graphics (animated) of difficult words. When measuring the academic achievement of the students, it was found out that the test score of experimental group was higher than the control group (d = 1.81, de sensu Glass). At the end of the experiment, a motivation questionnaire showed that the students in experimental group expressed high motivation towards the material as well as positive attitude towards AR application. The authors concluded that AR applications made Turkish lesson more enjoyable and easier. Similar results were obtained by Kücük et al. (2014) as well as Shelton and Hedley (2004) who showed that the attitude of

210

Siposova and Hlava

students is positive about utilising AR applications. Obviously, benefits of AR applications when teaching and learning a foreign language are proliferating, and being explored and tested. Nevertheless, in accordance with Blyth’s (2018) reflection, “[AR] technologies, including virtual immersion, will help teachers meet these challenges, but they will also force teachers to ponder what it means to know a language.” Blyth presents an example of the pair of wireless headphones (launched by Google) featuring real-time language translation, a powerful form of AR technology that at first blush appears to eliminate the need to actually learn a foreign language. Thus, two foreign language speakers will be able to “converse” in real time without knowing a word of the other’s language. This leads Blyth to ask: “Will foreign language learning become obsolete?” Notwithstanding, Blyth concludes that “In the future, however, the claim that context [of the language usage] cannot be learned will be challenged by immersive technologies whose simulations of language within increasingly authentic context will make experiential language learning a reality.” Based on our review of research studies, experiments, and quasi-experiments, there seems to be a consensus that AR applications can support language learning in many positive ways. The applications do not necessarily guarantee huge gains in the cognitive domain, but, on the other hand, their use can lead to increased “simulated real-life language experience” and gains in the affective domain as well as social skills.

5

Conclusion

This survey of a heterogeneous body of research findings, observations, and subjective interpretations recognized the roster of patterns typical for educational reality when supported by AR applications. The spread in negative – neutral – positive continuum did appear to be non-random, and a tendency towards thematic grouping was visible – thus, both cognitive outcomes and experience in terms of affective domain improved. Learning achievements seen from the perspective of the cognitive domain focusing on declarative knowledge ranged from intermediate to very positive (interpreting the magnitude of effect sizes by Cohen, 1988). For the field of education, Hattie’s understanding of effect size values categorizes the findings as being in the range of desired effect. Although the findings seem to be completely one-sided, we advise the educational effect of AR be understood as generally superior in terms of pre-test – post-test, while having slight – intermediate superiority in AR vs. traditional teaching conditions. However, cases of the control group being superior to the AR group, or the AR group consisting of students

Uses of Augmented Reality in Tertiary Education

211

achieving better results in pre-tests, and control groups achieving significant improvement in pre-test – post-test conditions were observed too. From this perspective, application of AR demonstrated a considerable impact on learning achievement, but, to paraphrase Hattie (2009), sometimes it is not worth pursuing the small extra gains since at the end of the day, they are not quite cost efficient. The limits of the AR were identified in several aspects. Firstly, there is lack of experience with AR interaction and applications. This applies for both teachers and learners. Although current generation of youth and young adolescents are considered technology-literate natives, subjects of experiments had to be instructed in order to become successful users. Extrapolated to real-life scenarios, we may expect not hundreds, but thousands of various AR applications to cover the full range of curriculum, each highly likely to have specific peculiarities requiring its users to get instruction. As the teachers would be necessarily involved as well, the prognosis of university-level academics in their most productive age undergoing the required instruction necessary for productive exploitation of up-to-date technological innovations – (mobile) AR applications seems to be at least far-fetched. Secondly, technical aspects need to be incorporated as well. The slow speed of the Internet in schools and affordances of mobile learning devices seem to be easy to overcome, but such issues go further. Optimization of download/upload ratios, screen size, frame-rates and many more technical aspects need to be guaranteed. Thirdly, the topic of individual differences and how they influence the needs and learning outcomes of individual students should be transferred into AR contexts as well. Before integrating AR applications into a course, educational researchers should take the target learning content design into consideration, since dominance of visual input is not necessarily the most suitable learning scenario for learners that benefit from employing other types of intelligence. There is but one area where traditional teaching appeared clearly inferior. It was in the dimension of the affective domain and social learning and social skills that AR shined. What present-day schooling generally lacks (and has been probably lacking for decades, if not centuries) is the ability to catch students’ interest, keep the flow of enthusiasm and motivation running while allowing their autonomy to take care of the knowledge discovery and creation from the constructivist point of view. If there is a way for tertiary-level students to achieve high level of intellectuality and excellence in more effective and enjoyable ways, application of AR is a good answer since it embodies means enabling students to develop their reasoning and knowledge base to at least the same extent (and even better) as conventional praxis allows. The added value making such learning conditions psychologically more feasible are motivation,

212

Siposova and Hlava

enthusiasm, social-interaction and collaboration, creativity, problem-solving, critical thinking, relevancy, presence, immediacy, immersion, and autonomy. AR technology, as it has been demonstrated, offers truly great opportunities and possibilities for its exploitation in students’ self-development, respecting the fundamental, self-encapsulated qualities of the process of learning – its social character together with greater time and space independence.

Acknowledgement The chapter was written with the support of the grant KEGA 012UK-4/2018 “The Concept of Constructionism and Augmented Reality in the Field of the Natural and Technical Sciences of the Primary Education (CEPENSAR).”

References Akçayır, G., & Akçayır, M. (2016). Research trends in social network sites’ educational use: A review of publications in all SSCI journals to 2015. Review of Education, 4(3), 293–319. Akçayır, G., & Akçayır, M. (2017). Advantages and challenges associated with augmented reality for education: A systematic review of the literature. Educational Research Review, 20, 1–11. Albrecht, U. V., Folta-Schoofs, K., Behrends, M., & von Jan, U. (2013). Effects of mobile augmented reality learning compared to textbook learning on medical students: Randomized controlled pilot study. Journal of Medical Internet Research, 15(8), e182. http://doi.org/10.2196/jmir.2497 Alizadeh, M., Mehran, P., Koguchi, I., & Takemura, H. (2017). Learning by design: Bringing poster carousels to life through augmented reality in a blended English course. In K. Borthwick, L. Bradley, & S. Thouësny (Eds.), CALL in a climate of change: Adapting to Turbulent Tlobal conditions – Short papers from EUROCALL 2017 (pp. 7–12). Research-publishing.net. Azuma, R. (1997). A survey of augmented reality. Teleoperators and Virtual Environments, 6(4), 355–385. Azuma, R., Baillot, Y., Behringer, R., Feiner, S. K., Julier, S., & MacIntyre, B. (2001). Recent advances in augmented reality. IEEE Computer Graphics and Applications, 21(6), 34–47. Becher, T. (1994). The significance of disciplinary differences. Studies in Higher Education, 19(2), 151–161.

Uses of Augmented Reality in Tertiary Education

213

Bicen, H., & Bal, E. (2016). Determination of student opinions in augmented reality. World Journal on Educational Technology: Current Issues, 8(3), 205–209. Blyth, C. (2018). Immersive technologies and language learning. Foreign Language Annals, 51, 225–232. Bressler, D., & Bodzin, A. (2013). A mixed methods assessment of students’ flow experiences during a mobile augmented reality science game. Journal of Computer Assisted Learning, 29(6), 505–517. Chiang, T. H., Yang, S. J., & Hwang, G.-J. (2014a). An augmented reality-based mobile learning system to improve students’ learning achievements and motivations in natural science inquiry activities. Journal of Educational Technology & Society, 17(4), 352–365. Chiang, T. H., Yang, S. J., & Hwang, G.-J. (2014b). Students’ online interactive patterns in augmented reality-based inquiry activities. Computers & Education, 78, 97–108. Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Erlbaum. Delello, J. A. (2014). Insights from pre-service teachers using science-based augmented reality. Journal of Computers in Education, 1(4), 295–311. http://doi.org/10.1007/ s40692-014-0021-y Dită, F. A. (2016). A foreign language learning application using mobile augmented reality. Informatica Economică, 20(4). Retrieved from http://revistaie.ase.ro/ content/80/07%20-%20Dita.pdf El Sayed, N. A. M., Zayed, H. H., & Sharawy, M. I. (2011). ARSC: Augmented reality student card. Computer Education, 56, 1045–1061. Erenli, K. (2013). The impact of gamification-recommending education scenarios. International Journal of Emerging Technologies in Learning (iJET), 8(S1), 15–21. Franklin, J. (1999). Structure and domain-independence in the formal sciences. Studies in History and Philosophy of Sciences, 30(4), 721–723. Funk, M., Bächler, A., Bächler, L., Kosch, T., Heidenreich, T., & Schmidt, A. (2017). Working with augmented reality? A long-term analysis of in-situ instructions at the assembly workplace. In Proceedings of the 10th International Conference on PErvasive Technologies Related to Assistive Environments, Island of Rhodes, Greece (pp. 222–229). New York, NY: ACM. Furió, D., González-Gancedo, S., Juan, M.-C., Seguí, I., & Costa, M. (2013). The effects of the size and weight of a mobile device on an educational game. Computers & Education, 64, 24–41. Furió, D., Juan, M.-C., Seguí, I., & Vivó, R. (2015). Mobile learning vs. traditional classroom lessons: A comparative study. Journal of Computer Assisted Learning, 31(3), 189–201. Gee, J. P. (2003). What video games have to teach us about learning and literacy. New York, NY: Palgrave Macmillan.

214

Siposova and Hlava

Gee, J. P. (2005). Good video games and good learning. Phi Kappa Phi Forum, 85(2), 33. Hanafi, H. F., Said, C. S., Ariffin, A. H., Zainuddin, N. A., & Samsudin, K. A. (2016). Using a Collaborative Mobile Augmented Reality Learning Application (CoMARLA) to improve student learning. IOP Conference Series: Materials Science and Engineering, 160(1). doi:10.1088/1757-899X/160/1/012111 Hattie, J. (2009). Visible learning. New York, NY: Routledge. Heyes, C. (2018). Cognitive gadgets: The cultural evolution of thinking. Cambridge, MA: Belknap Press. Holden, C., & Sykes, J. (2011). Leveraging mobile games for place-based language learning. International Journal of Game-Based Learning (IJGBL), 1(2), 1–8. Huang, H.-M., & Liaw, S.-S. (2014). A case study of learners’ motivation and intention to use augmented reality learning system. WIT Transactions on Information and Communication Technologies, 49, 995–1002. Hwang, G.-J., Tsai, C.-C., Chu, H.-C., Kinshuk, K., & Chen, C.-Y. (2012). A context-aware ubiquitous learning approach to conducting scientific inquiry activities in a science park. Australasian Journal of Educational Technology, 28(5), 931–947. Ira, K., & Berge, Z. (2008). Online learning’s future in the workplace with augmented reality. In A. Cartelli & M. Palma (Eds.), Encyclopedia of information communication technology (pp. 637–646). Hershey, PA: IGI Global. Jamali, S., Shiratuddin, M., Wong, K., & Oskam, C. (2015). Utilising mobile-augmented reality for learning human anatomy. Procedia – Social and Behavioral Sciences – 7th World Conference on Educational Sciences, 197, 659–668. Jan, U. V., Noll, C., Behrends, M., & Albrecht, U. (2012). mARable: Augmented reality in medical education. Biomedical Technology, 57, 67–70. Johnson, L., Laurence, F., Smith, R., & Stone, S. (2010). The 2010 horizon report. Austin, TX: The New Media Consortium. Juan, M.-C., Alexandrescu, L., Folguera, F., & García, I. (2016). A mobile augmented reality system for the learning of dental morphology. Digital Education Review, 30, 234–247. Klauer, K. J. (2001). Handbuch kognitives training. Göttingen: Hogrefe. Kose, U., Koc, D., & Yucesoy, S. A. (2013). An augmented reality based mobile software to support learning experiences in computer science courses. Procedia Computer Science, 25, 370–374. http://doi.org/10.1016/j.procs.2013.11.045 Kücük, S., Aydemir, M., Yildirim, G., Arpacik, O., & Goktas, Y. (2013). Educational technology research trends in Turkey from 1990 to 2011. Computers & Education, 68, 42–50. Kücük, S., Kapakin, S., & Goktas, Y. (2015). Medical faculty students’ views on anatomy learning via mobile augmented reality technology. Journal of Higher Education and Science, 5(3), 316–323. http://doi.org/10.5961/jhes.2015.133 Kücük, S., Yilmaz, R., & Göktas, Y. (2014). Ingilizce Őgreniminde Artirilmis Gerceklik: Őgrencilerin Basari, Tutum ve Bilissel Yük Düzeyleri. Egitim ve Bilim, 39(176), 393–404.

Uses of Augmented Reality in Tertiary Education

215

Kumar, B. (2012). Gamification in education: Learn computer programming with fun. International Journal of Computers & Distributed Systems, 2(1), 46–53. Lee, K. (2012). Augmented reality in education and training. TechTrends, 56(2), 13–21. Lin, T.-J., Duh, H. B.-L., Li, N., Wang, H.-Y., & Tsai, Ch.-Ch. (2013). An investigation of learners’ collaborative knowledge construction performances and behaviour patterns in an augmented reality simulation system. Computers & Education, 68, 314–321. Liu, E. P.-H., & Tsai, M.-K. (2013). Using augmented-reality-based mobile learning material in EFL English composition: An exploratory case study. British Journal of Educational Technology, 44(1), 1–4. Liu, Y., Holden, D., & Zheng, D. (2016). Analyzing students’ language learning experience in an augmented reality mobile game: An exploration of an emergent learning environment. Procedia – Social and Behavioral Sciences, 228, 369–374. Lu, S.-J., & Liu, Y.-C. (2015). Integrating augmented reality technology to enhance children’s learning in marine education. Environmental Education Research, 21(4), 525–541. Martin, S., Diaz, G., Sancristobal, E., Gil, R., Castro, M., & Peire, J. (2011). New technology trends in education: Seven years of forecasts and convergence. Computer Education, 57(3), 1893–1906. McGonigal, J. (2011). Reality is broken: Why games make us better and how they can change the world. New York, NY: Penguin Books. Munnerley, D., Bacon, M., Wilson, A., Steele, J., Hedberg, J., & Fitzgerald, R. (2012). Confronting an augmented reality. Paper presented at ALT Conference: A confrontation with reality, Manchester, England. http://dx.doi.org/10.3402/rlt.v20i0.19189 Őzcan, M., Őzcan, A., & Sahin, N. (2017). The influence of the augmented reality application on students’ performances in Ottoman Turkish readings. Universal Journal of Educational Research, 5(12B), 27–33. Patzer, B., Smith, D. C., & Keebler, J. R. (2014). Novelty and retention for two augmented reality learning systems. In Proceedings of the Human Factors & Ergonomics Society 58th Annual Meeting (pp. 1164–1168). Retrieved from https://www.researchgate.net/ profile/Joseph_Keebler/publication/271728422_Novelty_and_Retention_for_Two_ Augmented_Reality_Learning_Systems/links/5717791808aefb153f9ef377/Noveltyand-Retention-for-Two-Augmented-Reality-Learning-Systems.pdf Perry, B. (2015). Gamifying French language learning: A case study examining a questbased, augmented reality mobile learning-tool. Procedia – Social and Behavioral Sciences, 174, 2308–2315. Prince, M. (2004). Does active learning work? A review of the research. Journal of Engineering Education, 93(3), 223–231. Santos, M. E. C., Wolde Lübke, A., Taketomi, T., Yamamoto, G., Rodrigo, M. T., Sandor, C., & Kato, H. (2016). Augmented reality as multimedia: The case for situated vocabulary learning. Research and Practice in Technology Enhanced Learning, 11(4), 2–23. http://dx.doi.org/10.1186/s41039-016-0028-2

216

Siposova and Hlava

Shelton, B. E., & Hedley, N. R. (2004). Exploring a cognitive basis for learning spatial relationships with augmented reality. Technology, Instruction, Cognition and Learning, 1(4), 323–357. Solak, E., & Cakir, R. (2015). Exploring the effect of materials designed with augmented reality on language learners’ vocabulary learning. The Journal of Educators Online, 13(2), 50–72. Sotiriou, S., & Bogner, F. X. (2008). Visualizing the invisible: Augmented reality as an innovative science education scheme. Advanced Science Letters, 1(1), 114–122. Ting, H. Y. (2015). The effects of applying augmented reality on fourth-grade students’ writing and attitude (Unpublished master’s dissertation). National Taipei Education University, Taipei. Wang, H., Duh, H., Li, N., Lin, T., & Tsai, C. (2014). An investigation of university students’ collaborative inquiry learning behaviours in an augmented reality simulation and a traditional simulation. Journal of Science Education and Technology, 23(5), 682–691. Retrieved from http://www.jstor.org/stable/24019783 Wang, Y.‐H. (2017). Using augmented reality to support a software editing course for college students. Journal of Computer Assisted Learning, 33, 532–546. http://dx.doi.org/ 10.1111/jcal.12199 Wanner, R. A., Lewis, L. S., & Gregorio, D. I. (1981). Research productivity in academia: A comparative study of the sciences, social sciences and humanities. Sociology of Education, 54(4), 238–253. Wild, F. (2016). The future of learning at the workplace is augmented reality. Computer, 49, 96–98. Woods, T., Reed, S., Hsi, S., Woods, J. A., & Woods, M. R. (2016). Pilot study using the augmented reality sandbox to teach topographic maps and surficial processes in introductory geology labs. Journal of Geoscience Education, 64, 199–214. Wu, H.-K., Lee, S. W.-Y., Chang, H.-Y., & Liang, J.-C. (2013). Current status, opportunities and challenges of augmented reality in education. Computers & Education, 62, 41–49. Wu, W.-H., Wu, Y.-C. J., Chen, C.-Y., Kao, H.-Y., Lin, C.-H., & Huang, S.-H. (2012). Review of trends from mobile learning studies: A meta-analysis. Computers & Education, 59(2), 817–827. Yu, D., Jin, J. S., Luo, S., Lai, W., & Huang, Q. (2009). A useful visualization technique: A literature review for augmented reality and its application, limitation & future direction. In L. M. Huang, V. Q. Nguyen, & K. Zhang (Eds.), Visual information communication (pp. 311–337). Boston, MA: Springer US.

CHAPTER 10

An Augmented Reality Based Intelligent Diagnosis Platform for Medical Training Utku Köse and Omer Deperlioglu

Abstract This chapter introduces Augmented Reality and artificial intelligence oriented approaches for developing intelligent platforms on which students can perform diagnosis or operations by using some tasks to understand more about diseases (Barfield, 2015; Douglas, Wilke, Gibson, Boone, & Wintermark, 2017; Hemanth, Kose, Deperlioglu, & de Albuquerque, 2018; Maier-Hein et al., 2011; Neira-Tovar, Cavazos, Carrasco, & Barrera-Aldana, 2018; Shuhaiber, 2004; Zou et al., 2017). Using Augmented Reality-based features, students can interact with real-world objects to perform their tasks on medical diagnosis. Training infrastructure for the platform can be improved with pre-defined data from experts. Some tests done in order to evaluate the platform accuracy on diagnosis and effectiveness for training processes reveal positive results considering the obtained findings. The chapter briefly gives more emphasis to use of both Augmented Reality and artificial intelligence technologies to ensure innovative training platforms in the future.

Keywords augmented reality – medical education – medical training – medical diagnosis – artificial intelligence

1

Introduction

Augmented Reality (AR) is one the most effective technologies in improving interactivity and efficiency in educational environments. Since it employs both the real and the virtual world in a common ground, it has been very effective for learning and understanding subjects, especially applied and technical ones. In this context, some triggering technologies such as mobile devices have enabled © koninklijke brill nv, leideN, 2020 | DOI: 10.1163/9789004408845_010

218

Köse and Deperlioglu

users to experience Augmented Reality-based applications wherever they go. Starting from games and entertainment-oriented systems, Augmented Reality has taken an active role in self-learning and open education approaches. One key goal here is to employ Augmented Reality in technical and abstract fields and the corresponding courses, in order to reach to the desired educational outcomes effectively. In that context, the objective of this chapter is to introduce an Augmented Reality-based platform for improving efficiency and effectiveness of medical training. As is known, training in medicine needs use of expensive medical equipment and training by experts – doctors must provide guidance for students. At this point, simulation-based solutions would be very effective to enable students for understanding the concepts better (Hammond, 2004; Kincaid, Hamilton, Tarr, & Sangani, 2003; Morfoot & Stanley, 2018; Wayne et al., 2008). As Augmented Reality has become a powerful technology and tool for achieving the mechanism of simulation, it is widely used to develop a platform on which students can perform diagnosis or operations-based tasks to understand more about some diseases (Barfield, 2015; Douglas et al., 2017; Hemanth et al., 2018; Maier-Hein et al., 2011; Neira-Tovar et al., 2018; Shuhaiber, 2004; Zou et al., 2017). In contrast to the usual employment of Augmented Reality, the platform developed here also uses Artificial Intelligence to support an intelligent diagnosis function, using signal processing processes or Machine Learning infrastructure based on widely-used Artificial Neural Networks. Using Augmented Reality-based features, students can interact with real-world objects to perform their medical diagnosis tasks. Training infrastructure for the platform can be improved with pre-defined data from experts. Some tests were done in order to evaluate the platform’s accuracy on diagnosis and effectiveness for training processes and positive results were achieved according to the obtained findings.

2

Augmented Reality in Medical Education

Augmented reality has become widespread in recent times and is used to enhance quality and productivity in many areas such as industry, commerce, marketing, engineering, and education. AR combines computer-generated content with user media, and user perception to improve perception in real world, applied problems. An AR system allows users to interact by integrating existing and virtual realities in their environment. The real-time interaction with existing and virtual realities in these environments allows the user to evaluate multiple parameters at the same time and analyse the problem efficiently. AR creates an interface based on a local environment that is enhanced

Augmented Reality Based Intelligent Diagnosis Platform

219

with information from a variety of geographically distributed sources. Objects that co-exist in the same realm with the real world were recognized by MIT as one of the ten technologies that emerged in 2007 (Jonietz, 2007). Full AR systems generally include three main components: monitoring, recording, and visualization. AR technology is a programmable reality that develops existing knowledge in real time with virtual support. In the past decade, AR applications have moved from desktop computers to mobile platforms (Barsom, Graafland, & Schijven, 2016; Billinghurst, Clark, & Lee, 2015; Li, Nee, & Ong, 2017; Van Krevelen & Poelman, 2010). The content prepared with AR applications has been proven to encourage students to develop their learning experience (Dunleavy & Dede, 2014; Wu, Lee, Chang, & Liang, 2013). The integration of AR’s virtual objects into the physical reality of the users makes the simulations more realistic and impressive. It has a great potential especially in medical education (Herron, 2016). AR not only contributes to the education of students, but AR can also affect patient care through better medical education. Students and educators can benefit from resources that include increased reality applications in libraries in medical education institutions. For many years, many AR-based studies have been conducted for medicine (Birkfellner et al., 2002; Ha & Hong, 2016; Kilgus et al., 2015; Kipper & Rampolla, 2012; Peters, Linte, Yaniv, & Williams, 2018; Ploder, Wagner, Enislidis, & Elwers, 1995; Sielhorst, Feuerstein, & Navab, 2008; Székely & Satava, 1999; Viirre, Pryor, Nagata, & Furness, 1998). For example, digital images and virtual reality environments are often used to visualize complex 3D spatial relationships. AR applications are associated with digital data such as real and virtual images to extend the scope of real information. In the AR echocardiography simulator, a 3D surface model of the human heart is integrated with echocardiographic volume datasets (Weidenbach et al., 2000). In another image-processing and AR fusion study, CT and MRI images acquisition for breast cancer diagnosis, standard three-plane cross-sectional interpretation studies were examined. In such studies, image processing plays the role of analysing image-oriented medical data for better diagnosis and AR supports the diagnosis by visualizing it and providing a combined real and virtual environment. Advanced 3D imaging techniques such as surface processing, volume creation, D3D, VR and AR have been investigated, and have found that using AR technology can increase awareness, save costs and improve patient care (Douglas et al., 2017). In another study, a training environment was developed that included AR with assistive devices for finger extension rehabilitation. The system consists of three components: AR equipment, software, and a body-based prosthesis. The therapist’s interface consisted of visual, auditory and power feedback and

220

Köse and Deperlioglu

monitoring, and a control interface. The evaluation of the work was presented with the results of a pilot case study of patients who had experienced a stroke (Viirre, Pryor, Nagata, & Furness, 1998). These AR studies in medicine have also been a source of inspiration for those working in medical education. For example, Morgan and his colleagues made an AR-based 3D viewing tool to make bone dynamics more understandable during body movement to teach dynamic 3D anatomy. Thus, they have developed a system that allows 3D imaging and recording of the complexity of human anatomy and combinations of bone structures (Rolland, Wright, & Kancherla, 1997). Yeo et al. (2011) attempted to develop a tool for correct placement of a needle in percutaneous facet joint injection in medical training using AR. They used laser guidance systems with an AR image-overlay feature in the study. Thus, they examined whether AR contributed to medical learning. Another simulation study describes the use of a birth simulator for medical training using AR. Sielhorst, Obst, Burgkart, Riener, and Navab (2004) added a user interface to a previously developed AR simulator. By adding sound and visual feedback for doctors’ training on this interface, they provided important physiological data such as the blood pressure, heart rate, pain, and oxygen sources. See, Billinghurst, Rengganaten, and Soo (2016) developed a mobile AR heart rate murmur simulator that can be used in clinical teaching for medical education fields. They developed a mobile hearing, wearable apparel system that provides an ARbased heart murmur simulation to show patients all kinds of heart disease and facilitate medical learning (See et al., 2016). In order to strengthen the positive aspects of traditional learning environments with AR, a web-based learning environment was prepared and given the name of the practice. At this moment, a basic lesson in traditional medical education has been realized in order to support students’ learning experience. In the evaluation results, it was observed that the application contributed to student motivation and increased the efficiency in medical education (Von Jan, Noll, Behrends, & Albrecht, 2012). AR applications can include 2D and 3D images as well as audio and video files, and textual information. Thus, they can be seen and understood directly (Yuen, Yaoyuneyong, & Johnson, 2011). Due to these features, there is a clear need for students and medical specialists to further investigate the use of AR in health education, especially since they require more situational testing in clinical care, especially in terms of patient safety. In recent years, AR’s attention to the following beliefs has drawn attention to the growing interest in research (Frenk et al., 2010; Rolland et al., 2003; Sielhorst et al., 2004; Thomas, John, & Delieu, 2010; Zhu, Hadadgar, Masiello, & Zary, 2014):

Augmented Reality Based Intelligent Diagnosis Platform

221

– AR enhances students’ decision-making abilities by using collaborative work and making the best use of available resources, enabling them to work in complex clinical training, in a real hands-on training-like environment. – AR offers more unique learning opportunities and experience for learning, providing a more personalized and engaging learning environment for students. – In case of mistakes that may occur during skill training with AR, patients are protected because real patients are not used. Applied medical education in class is very expensive. It requires a complex educational environment because it is a complex education. The students are expected to gain professional competence by increasing their skills with the physical environment and appropriate patient applications. Medical training activities require expertise gained from experiences that enable effective implementation of expected standards in real-life environments. To achieve this level of expertise requires the opportunity to experience many cases covering many variations of the subject being studied. However, meaningful learning is a precondition for the realization of the learning and covers the following requirements (Kamphuis, Barsom, Schijven, & Christoph, 2014): – Active: Requires interacting with the real world and learning by sight. – Constructive: Requires integration of existing information with new experiences. – Intentional: It should be aimed at a target. – Collaborative: It requires mutual communication and cooperation. Of course, AR is not a new technology, and has been widely used in medical education for many years. But in medical education, which has a wide application area, there are many new fields that can be adapted. In a systematic literature review, AR applications developed in seven different medical education fields were encountered. AR applications allow students in medical education to better understand relations among different medical applications and also learn essential concepts. Thus, by providing a learning environment in practice, it provides a better understanding of the prominent aspects of the subject. It can be said that AR applications provide a valid and reliable learning environment for medical education. In addition, the AR with their original simulation experiences aims to improve the motivation of the students, to support and improve their educational performance (Barsom et al., 2016). The diversity of cases and situations in the medical setting is not only concerned with students, but also with their doctors. It is not always possible to create a real physical environment in all medical training areas. Creating a physical environment for each case variety or searching for solutions with existing simulations is very expensive. In this context, the first methods that

222

Köse and Deperlioglu

come to mind when considering continuous and sustainable medical education are distance education and/or e-learning. The systems to be developed should support both traditional and e-learning. In other words, blended learning should also be possible. In general, when it comes to medical education, it should be structured in a similar way to the real world, including the phases of diagnosis, treatment and rehabilitation. Today, computer technology is predominantly mobile technology, which emphasizes AR-based intelligent diagnostic systems that support mobile learning.

3

Intelligent Diagnosis Platform

Information technology has the potential to improve the clinical learning environment. It is the question of IT increasing health professionals’ work performance, affecting both the learning of students and patient treatment outcomes. Clinical training is practiced in the hands of experts with professional skills. It is especially suitable for clinical training using clinical environments as teaching environments (Chow & Chan, 2010). As mentioned before, medical education is a multi-faceted learning structure that requires multiple adaptive physiological systems, specific adaptability, and interoperability. The best example of the complexity of medical education is the modern clinical decision support system. The infrastructure of the intelligent diagnostic training system to be created will be almost the same as that of the clinical decision support system. 3.1 Clinical Decision Support Systems Because physicians’ workloads are so great, with many patients and each patient with his or her own special needs, each requiring a lot of the physician’s time, physicians often do not have enough time to examine past cases of patients’ conditions and to make a specific diagnosis. Clinical decision support systems (CDSS) are used to help physicians make inferences. Figure 10.1 shows the architecture of a modern clinical decision support system. The main task of a CDSS is to help physicians diagnose patients using up-to-date and patientspecific information. These systems try to make special deductions by evaluating the basic clinical information that the clinicians have entered. Thus, patient management, consultation functions, and rehabilitation can be performed depending on the specific characteristics of the patient (Deperlioglu, 2018). A modern CDSS is primarily designed to provide clinical information, laboratory information and, if necessary, medical images and data related to the current state of the patient. We need a data warehouse management system

Augmented Reality Based Intelligent Diagnosis Platform

223

figure 10.1 The architecture of modern clinical decision support systems (from Deperlioglu, 2018)

that has sufficient infrastructure for this and holds numerical data. Taking into consideration the previous expert opinions, which will take and evaluate the present data, machine learning is the basis of the sub-system which will make the patient-related inferences. In other words, the structure that will provide adaptability according to the patient’s situation decides due to the required rule base and expert opinion using Artificial Intelligence (AI) algorithms. These systems can be used efficiently on both desktop and mobile devices, in web-based software that provides a suitable environment for interactive work in all conditions. Here, Web-based Clinical Decision Support Systems (WBCDSS) can be defined as interactive information systems that provide information and datamanagement tools and support for decision-makers. The objective of WBCDSS is to increase productivity by increasing adaptable expertise to help doctors and clinicians. The WBCDSS enables doctors and clinicians to see alternative solutions and review data while diagnosing disease. Users systematically send solution options for each patient to the WBCDSS. The WBCDSS evaluates this data by comparing it with the rule base and previous similar cases and tries to present the best proposal. The components of an exemplary WBCDSS are given in Figure 10.2 (Deperlioglu, 2018). These components are a Data server, an Application Server, and an Evaluation, Optimization or Classification unit. The data server contains the database management system to serve digital data such as clinical or laboratory analysis, medical images, pharmacy catalogues, etc. The application server includes the necessry system software and user interface. The evaluation, optimization or Classification unit manages the process steps and makes inferences using an AI algorithm for the patient’s status.

224

Köse and Deperlioglu

figure 10.2 Components of web based clinical decision support systems (from Deperlioglu, 2018)

3.2 E-Learning, Blended Learning and Mobility With the growth of information technology, the concept of e-learning has emerged. E-learning allows students to study in the environments they want, when they want. Thus, those living away from school have the opportunity to learn without leaving their homes, work, or family. Research about e-learning has shown that e-learning is a big contributor to student success (Deperlioglu & Yıldırım, 2009). Often, e-learning can provide much more effective solutions than face-to-face training methods. There is a growing interest in distance education around the world. Electronic learning (e-learning) can be applied not only to educational institutions but to all those who want to improve themselves in lifelong learning. It is also a very good solution for trainings that increase professional competencies, which large institutions will desire for their staff (Deperlioglu & Arslan, 2010; Elango, Gudep, & Selvam, 2008; Kavrakoğlu, 2002; Roy & Raymond, 2008). In e-learning, scientists focus on interaction, personalization, and control (Piccoli, Ahmad, & Ives, 2001). In particular, an e-learning platform or environment should be designed for all students in harmony with different objects and past experiences, and the student’s own activity should be enhanced. The development of today’s internet infrastructure and the increasing use of the internet have made the software become web-based. In today’s world, use of e-learning systems, animated images, electronic books, e-mail, and teleconference calls are very common and are widely used for ensuring interactive training methods. The network-based structure obtained by increasing and

Augmented Reality Based Intelligent Diagnosis Platform

225

continually improving these methods is called a Web Based Distance Learning System (WBDLS). One of the main advantages of a WBDLS is the ability to create a virtual school and provide synchronous or asynchronous training. Whenever students want, they can access and use the educational content provided by the trainers. Students may repeat a lesson that they do not understand. These advantages make WBDLS easy to spread (Hodges, 2004). Face-to-face education – in which teachers communicate with students and teaching and learning activities are conducted in the classroom – is the most commonly used learning strategy today. However, teachers often have too many students in the same class, which can cause some problems in educational activities (Gálvez, Guzmán, & Conejo, 2009; Gibbs & Jenkins, 2014). The WBDLS had developed to solve these problems. With the WBDLS, teachers will be able to continuously publish course content, give quizzes and assignments, and receive feedback on the course. Thus, it is possible to make a blended training. Thus, the shortcomings of face-to-face education are addressed by blended education. Blended learning is an educational model that supports face-to-face training in combination with e-learning to increase the efficiency and effectiveness of education and training (Alammary, Sheard, & Carbone, 2014; Deperlioglu & Kose, 2013). The use of mobile devices has increased significantly over the past few years. For this reason, mobile AR applications in e-learning environments have also increased. The increase in the technological capabilities of smartphones and tablets – their cameras, microphones, GPS, sensors, etc. – make mobile devices more attractive for AR applications. In addition, mobile devices are increasing the location independence of AR applications for educational purposes. Educators and corporate human resources are choosing these platforms to create training applications that provide new learning experiences. Blended learning and AR applications are a new method of developing learning activities for face-to-face and e-learning on mobile devices. Multiple users can experiment with 3D mixed reality from different objects, interact with objects and share insights. This can make learning a more collaborative experience. Such skills can be based on new educational experiences (Billinghurst & Duenser, 2012). The use of e-learning methods to improve teaching and learning activities has long been part of medical education. However, previous e-learning modules in medical education generally only include visuals or e-books. They do not have experiences in applications such as bedside teaching or operation. On the other hand, on-the-job training may not always be possible. Due to the condition of the patient, time, and restrictions on the physical infrastructure, it may not always be possible to describe all symptoms and findings. However, not all cases can be appropriately used for teaching due to ethical constraints.

226

Köse and Deperlioglu

For visually focused subjects, applications based on AR can provide a way out of this dilemma. This is because they have a potential to improve the learning experience and to show all the desired aspects. It makes it possible to create virtual cases similar to a real-world case (Billinghurst & Duenser, 2012). The increased availability and computing power of mobile phones and tablets increase the efficiency and effectiveness of intelligent diagnosis platforms by supporting the AR on mobile devices. 3.3 Intelligent Learning Environments Intelligent learning environments (ILE) are software applications designed to enhance the effectiveness of e-learning and adaptation to the student. ILEs offer further training techniques to improve the learning process of students. They allow the student to make self-discoveries and develop himself/ herself in a collaborative work environment. For this reason, ILEs are e-learning environments that enhance students’ general and field-specific thinking and problem-solving skills (Crews, 1995, 1997). Today, ILEs are widely used in advanced distance education applications because they perform their learning activities more efficiently than traditional face-to-face teaching (Bloom, 1984). By using ILEs, blended learning models can be developed to provide more powerful learning experiences. These environments are interactive software that students can study with the lessons they like. Thus, students will be able to make more efficient and effective learning by realizing face-to-face and e-learning based on artificial intelligence activities. These environments, which allow students to discover themselves, also allow students to increase their self-esteem and academic achievement (Deperlioglu & Kose, 2013). 3.4 Structure of the Intelligent Diagnostic Platform Medical diagnosis of diseases is a process typically performed by field specialists. This process can be very variable in relation to the current state of the patient and requires taking advantage of past experience or research from very large data sets. Expert information is a necessary process, as medical diagnoses are made by doctors or computer-aided decision support systems. Not every physician has the same level of expertise and physicians do not have enough time to consult experts in every situation, nor do physicians always have enough time to investigate large data sets for a single current case. Likewise, in medical education, it is not possible to find suitable cases for each disease. There are situations in which the physical environment and time are not appropriate for existing cases. In this context, an Intelligent Diagnosis Platform (IDP) is seen as the most suitable solution for both physicians and

Augmented Reality Based Intelligent Diagnosis Platform

227

medical students to develop their skills, to gain analytical thinking habits, and to make their expertise sustainable. Functionally, the intelligent diagnostic platform for medical education consists of three parts, an intelligent learning environment, an augmented reality unit, and a modern CDSS. Figure 10.3 illustrates the structure of the IDP. The ILE is the communication and interaction interface between the AR unit and the CDSS as well as the users. The IDP entry is a record containing the current state of the patient. This data may be the outline of laboratory data, patient’s current state, general clinical data, medical images, and the patient’s medical history.

figure 10.3 Structure of the intelligent diagnostic platform

In the IDP, there is a need for a system to make deductions based on the results of the examination of the patient. At this point, a Web Based Clinical Decision Support System (WBCDSS) that uses machine learning or artificial intelligence can be used to solve the problem. A WBCDSS provides a solution recommendation based on the entered medical data. An IDP’s intelligent learning environment is a web-based application and provides a realistic presentation of findings about the AR infrastructure. It enables both traditional face-to-face training and independent e-learning for individuals to create blended-learning environment. It can be used for preteaching as well as for individual learning. It also allows for group work. The IDP has structure that will appeal to both medical education and medical professionals. While one side develops the clinical application skills of the students, the other side will enable the experts to follow up on new developments in their fields and to increase their competence. To better understand the components of the IDP, we can update Figure 10.2 as in Figure 10.4. The IDP is a web-based adaptive learning system. The IDP infrastructure consists of three main components. The intelligent learning environment (ILE) also functions as a user interface while communicating between the decision support system and the augmented reality (AR) units. The ILE also runs the user interface task as an adaptive learning management system. It

228

Köse and Deperlioglu

figure 10.4 Components of the intelligent diagnostic platform

communicates with the web-based decision support system (WBDSS) according to the systematic input of the users. The WBDSS evaluates the received data and transmits the results to the AR unit via the ILE. The AR unit provides reallife, mixed reality by combining virtual data. An ILE is essentially an adaptive web-based learning management system (AWBLMS). It allows users to easily access the content they want with system entry. One of the most important advantages of an AWBLMS is that it can create a virtual campus. Synchronous and asynchronous training is provided. Students can access and benefit from the educational content delivered to the system by trainers whenever they want. It should be aimed not to force users while working inside the ILE. In the ILE, the user interface for system users and the administrator interface for administrators must be presented. A membership in an AWBLMS system should have several features, including a forum, instant messaging (chat), and web addressing to members to allow collaborative work. Considering student needs in the intelligent education system, all the structures that the student needs will be placed in the easiest way to use the main page. In particular, it is necessary to permit students to navigate the system more conveniently by using the usual site structure at the time of link placement. This allows users to work interactively. An ILE should also be organized so that users are able to provide facilities according to their behaviour and feedback while they are working within the site. Adaptability can be achieved

Augmented Reality Based Intelligent Diagnosis Platform

229

using artificial intelligence techniques such as artificial neural networks. With a user-friendly interface structure, even users with very little medical knowledge can learn effectively. A WBDSS realizes data evaluation processes over users’ input. A WBDSS is an expert system that uses rule bases created by experts for decision-making processes. An expert system consists of two main components: the knowledge base (or rule base), and the patient information. The knowledge base is a database of clinical experiences for similar cases and includes diagnoses made by experts in previous cases. The patient information component is the unit in which the data concerning the current status of the patient is entered. This component uses different artificial intelligence algorithms such as fuzzy logic, artificial neural networks, genetic algorithms, particle swarm optimization, ant colony optimization, DNA computation, and bee colony to make inferences. In short, this system tries to simulate expert decision-making ability. The difference is that they have the ability to make a faster decision by evaluating large quantities of up-to-date data. The WBDSS sends the required data to the AR unit for visualization. AR systems are often used to solve medical problems in clinical training areas (Figure 10.5). The AR unit extracts the medical field information for the design and implementation of the mixed reality, prepares the reality model and transfers it to LME as sound and image. An AR unit receives a reality model

figure 10.5 Some AR views from the diagnosis oriented medical training system

230

Köse and Deperlioglu

from a WBDSS that is required for a precise definition of the real environment, and then transfers the virtual knowledge to the real world. Data transferred to an AR unit may include body temperature, medication status, heart rate, 2D or 3D images, images in Picture Archiving and Communications System (PACS) format, and related sounds. The AR system consists of three main components for monitoring, recording, and visualization of a medical task. An AR unit also has a database containing related digital data such as 3D images, videos, and simulations. The simulation component in the AR system allows the use of animations to show the behaviour of medical situations. For example, it may show the first step of the intervention for a high-fever patient. Simulations prepared using external software are stored in the database. Depending on the incoming data, the relevant simulations are added to the mixed reality.

4

Conclusion

An AR-based intelligent diagnostic platform includes intelligent learning environments and adaptive clinical decision support systems. Both of these systems offer user friendly interfaces, and interactive, collaborative, adaptive instructional learning environments using AI methods. Because all components of the intelligent diagnostic platform include network-based systems, they can easily operate over the Internet, intranet or cloud systems on desktop or mobile computers. Therefore, these systems can be used for both faceto-face in traditional education and distance education. It is possible to use both together for blended education. Thus, the IDPs provide real interaction as if they were in face-to-face training, allowing them to receive feedback. At the same time, they may also reduce the lack of training in non-homogeneous groups of students in face-to-face instruction. An IDP offers significant potential for face-to-face training as well as for e-learning, thanks to the ability to dynamically tailor intelligent diagnostic scenarios to personalize and provide real-time assessment of applied clinical education to students. Over the next few years, IDPs, which will become more powerful by increasing student’s case identification competencies, will find a widespread use in medical training centres, health care facilities, and hospitals. With the developing technologies and societies, it will be able to diagnose and treat new diseases. The interactive structure allows medical students and physicians who want to improve themselves to collaborate independently of time and place. An AR-based IDP which today can demonstrate sustainability and lifelong learning skills that are often used in medical education will become indispensable educational tools for the future.

Augmented Reality Based Intelligent Diagnosis Platform

231

In future work, IDP systems can be improved by adding capabilities such as sensory and smell detection by taking advantage of developing web technologies. The response of the patient can be measured and compared to feedback from a virtual patient. In the same way, it can be permissible to customize by developing user-oriented interfaces.

References Alammary, A., Sheard, J., & Carbone, A. (2014). Blended learning in higher education: Three different design approaches. Australasian Journal of Educational Technology, 30(4). Barfield, W. (2015). Fundamentals of wearable computers and augmented reality. Boca Raton, FL: CRC Press. Barsom, E. Z., Graafland, M., & Schijven, M. P. (2016). Systematic review on the effectiveness of augmented reality applications in medical training. Surgical Endoscopy, 30(10), 4174–4183. Billinghurst, M., Clark, A., & Lee, G. (2015). A survey of augmented reality. Foundations and Trends in Human–Computer Interaction, 8(2–3), 73–272. Billinghurst, M., & Duenser, A. (2012). Augmented reality in the classroom. Computer, 45(7), 56–63. Birkfellner, W., Figl, M., Huber, K., Watzinger, F., Wanschitz, F., Hummel, J., … Bergmann, H. (2002). A head-mounted operating binocular for augmented reality visualization in medicine-design and initial evaluation. IEEE Transactions on Medical Imaging, 21(8), 991–997. Bloom, B. S. (1984). The 2 sigma problem: The search for methods of group instruction as effective as one-to-one tutoring. Educational Researcher, 13(6), 4–16. Chow, M., & Chan, L. (2010). Development and evaluation of a compartmental picture archiving and communications system model for integration and visualization of multidisciplinary biomedical data to facilitate student learning in an integrative health clinic. Computers & Education, 54(3), 733–741. Crews, T. R. (1995). AdventurePlayer: An intelligent learning environment. In Conference companion on human factors in computing systems (pp. 43–44). ACM. Crews, T. R. (1997). Intelligent learning environments: Using educational technology to assist complex problem solving. In Frontiers in education Conference, 1997. 27th Annual Conference. Teaching and learning in an era of change. Proceedings (Vol. 2, pp. 911–916). IEEE. Deperlioglu, O. (2018). Intelligent techniques inspired by nature and used in biomedical engineering. In U. Kose, G. E. Guraksin, & O. Deperlioglu (Eds.), Nature-inspired

232

Köse and Deperlioglu

intelligent techniques for solving biomedical engineering problems (pp. 51–77). IGI Global. Deperlioglu, O., & Arslan, Y. (2010). Design principles of web-based distance education system and sample application in Afyon Kocatepe University. IET Software, 4(4), 283–293. Deperlioglu, O., & Kose, U. (2013). The effectiveness and experiences of blended learning approaches to computer programming education. Computer Applications in Engineering Education, 21(2), 328–342. Deperlioglu, O., & Yıldırım, R. (2009). Mesleki eğitimin uzaktan eğitim ile desteklenmesi ve örnek uygulama. Afyon Kocatepe Üniversitesi Fen Ve Mühendislik Bilimleri Dergisi, 9(1), 61–70. Douglas, D. B., Wilke, C. A., Gibson, J. D., Boone, J. M., & Wintermark, M. (2017). Augmented reality: Advances in diagnostic imaging. Multimodal Technologies and Interaction, 1(4), 29. Dunleavy, M., & Dede, C. (2014). Augmented reality teaching and learning. In J. M. Spector, M. D. Merrill, J. Elen, & M. J. Bishop (Eds.), Handbook of research on educational communications and technology (pp. 735–745). New York, NY: Springer. Elango, R., Gudep, V. K., & Selvam, M. (2008). Quality of e-learning: An analysis based on e-learners’ perception of e-learning. Electronic Journal of E-Learning, 6(1), 31–43. Frenk, J., Chen, L., Bhutta, Z. A., Cohen, J., Crisp, N., Evans, T., & Kistnasamy, B. (2010). Health professionals for a new century: Transforming education to strengthen health systems in an interdependent world. The Lancet, 376(9756), 1923–1958. Gálvez, J., Guzmán, E., & Conejo, R. (2009). A blended e-learning experience in a course of object oriented programming fundamentals. Knowledge-Based Systems, 22(4), 279–286. Gibbs, G., & Jenkins, A. (2014). Teaching large classes in higher education: How to maintain quality with reduced resources. London: Routledge. Ha, H. G., & Hong, J. (2016). Augmented reality in medicine. Hanyang Medical Reviews, 36(4), 242–247. Hammond, J. (2004). Simulation in critical care and trauma education and training. Current Opinion in Critical Care, 10(5), 325–329. Hemanth, J. D., Kose, U., Deperlioglu, O., & de Albuquerque, V. H. C. (2018). An augmented reality-supported mobile application for diagnosis of heart diseases. The Journal of Supercomputing, 1–26. Herron, J. (2016). Augmented reality in medical education and training. Journal of Electronic Resources in Medical Libraries, 13(2), 51–55. Hodges, C. B. (2004). Designing to motivate: Motivational techniques to incorporate in e-learning experiences. The Journal of Interactive Online Learning, 2(3), 1–7. Jonietz, E. (2007). TR10: Augmented reality (10 breakthrough technologies). MIT Technology Review. Retrieved from http://www2.technologyreview.com/ news/407473/tr10-augmented-reality/

Augmented Reality Based Intelligent Diagnosis Platform

233

Kamphuis, C., Barsom, E., Schijven, M., & Christoph, N. (2014). Augmented reality in medical education? Perspectives on Medical Education, 3(4), 300–311. Kavrakoglu, I. (2002). The importance and method of e-learning. Human Resources, 9, 23–25. Kilgus, T., Heim, E., Haase, S., Prüfer, S., Müller, M., Seitel, A., … Hornegger, J. (2015). Mobile markerless augmented reality and its application in forensic medicine. International Journal of Computer Assisted Radiology and Surgery, 10(5), 573–586. Kincaid, J. P., Hamilton, R., Tarr, R. W., & Sangani, H. (2003). Simulation in education and training. In M. S. Obaidat & G. Papadimitriou (Eds.), Applied system simulation (pp. 437–456). Boston, MA: Springer. Kipper, G., & Rampolla, J. (2012). Augmented reality: An emerging technologies guide to AR. Waltham, MA: Elsevier. Li, W., Nee, A. Y. C., & Ong, S. K. (2017). A state-of-the-art review of augmented reality in engineering analysis and simulation. Multimodal Technologies and Interaction, 1(3), 17. Luo, X., Kenyon, R. V., Kline, T., Waldinger, H. C., & Kamper, D. G. (2005). An augmented reality training environment for post-stroke finger extension rehabilitation. In 9th International Conference on Rehabilitation Robotics, 2005 (ICORR 2005) (pp. 329–332). IEEE. Maier-Hein, L., Franz, A. M., Fangerau, M., Schmidt, M., Seitel, A., Mersmann, S., … Meinzer, H. P. (2011). Towards mobile augmented reality for on-patient visualization of medical images. In H. Handels, J. Ehrhardt, Th. M. Deserno, H.-P. Meinzer, & Th. Tolxdorff (Eds.), Bildverarbeitung für die Medizin 2011 (pp. 389–393). Berlin: Springer. Morfoot, C., & Stanley, H. (2018). Simulation-based education for neonatal skills training and its impact on self-efficacy in post-registration nurses. Infant, 14(2), 77–81. Neira-Tovar, L., Cavazos, J. A. E., Carrasco, J. A. L., & Barrera-Aldana, S. (2018). Proposal of virtual and augmented reality supported by a data structure for giving tracing to chronic diseases treatments. Case: DM2. In F. Torres-Guerrero, J. Lozoya-Santos, E. Gonzalez Mendivil, L. Neira-Tovar, P. Ramirez-Flores, & J. Martin-Gutierrez (Eds.), Smart technology (pp. 59–71). Switzerland: Springer. Peters, T. M., Linte, C. A., Yaniv, Z., & Williams, J. (Eds.). (2018). Mixed and augmented reality in medicine. Boca Raton, FL: CRC Press. Piccoli, G., Ahmad, R., & Ives, B. (2001). Web-based virtual learning environments: A research framework and a preliminary assessment of effectiveness in basic IT skills training. MIS Quarterly, 25, 401–426. Ploder, O., Wagner, A., Enislidis, G., & Ewers, R. (1995). Computer-assisted intraoperative visualization of dental implants. Augmented reality in medicine. Der Radiologe, 35(9), 569–572. Rolland, J. P., Davis, L., Hamza-Lup, F., Daly, J., Ha, Y., Martin, G., … Imielinska, C. (2003). Development of a training tool for endotracheal intubation: Distributed augmented

234

Köse and Deperlioglu

reality. In J. D. Westwood, H. M. Hoffman, G. T. Mogel, R. Phillips, R. A. Robb, & D. Stredney (Eds.), Studies in health technology and informatics: Medicine meets virtual reality (Vol. 11, pp. 288–294). Amsterdam: IOS Press. Rolland, J. P., Wright, D. L., & Kancherla, A. R. (1997). Towards a novel augmented-reality tool to visualize dynamic 3-D anatomy. In K. S. Morgan, H. M. Hoffman, D. Stredney, & S. J. Weghorst (Eds.), Studies in health technology and informatics: Medicine meets virtual reality (Vol. 39, pp. 337–348). Washington, DC: IOS Press. Roy, A., & Raymond, L. (2008). Meeting the training needs of SMEs: Is e-learning a solution? Electronic Journal of E-Learning, 6(2), 89–98. See, Z. S., Billinghurst, M., Rengganaten, V., & Soo, S. (2016, November). Medical learning murmurs simulation with mobile audible augmented reality. In SIGGRAPH ASIA 2016 mobile graphics and interactive applications (article 4). ACM. doi:10.1145/ 2999508.2999527 Shuhaiber, J. H. (2004). Augmented reality in surgery. Archives of Surgery, 139(2), 170–174. Sielhorst, T., Feuerstein, M., & Navab, N. (2008). Advanced medical displays: A literature review of augmented reality. Journal of Display Technology, 4(4), 451–467. Sielhorst, T., Obst, T., Burgkart, R., Riener, R., & Navab, N. (2004). An augmented reality delivery simulator for medical training. International Workshop on Augmented Environments for Medical Imaging-MICCAI Satellite Workshop, 141, 11–20. Stredney, D., & Weghorst, S. J. (1998). The virtual retinal display: A new technology for virtual reality and augmented vision in medicine. Medicine Meets Virtual Reality: Art, Science, Technology: Healthcare (R) Evolution, 50, 252. Székely, G., & Satava, R. M. (1999). Virtual reality in medicine. British Medical Journal, 319(7220), 1305. Thomas, R. G., John, N. W., & Delieu, J. M. (2010). Augmented reality for anatomical education. Journal of Visual Communication in Medicine, 33(1), 6–15. Viirre, E., Pryor, H., Nagata, S., & Furness, T. A. (1998). The virtual retinal display: A new technology for virtual reality and augmented vision in medicine. In J. D. Westwood, H. M. Hoffman, D. Stredney, & S. J. Weghorst (Eds.), Studies in health technology and informatics: Medicine meets virtual reality (Vol. 50, pp. 252–257). Amsterdam: IOS Press. Van Krevelen, D. W. F., & Poelman, R. (2010). A survey of augmented reality technologies, applications and limitations. International Journal of Virtual Reality, 9(2), 1–20. Von Jan, U., Noll, C., Behrends, M., & Albrecht, U. V. (2012). mARble augmented reality in medical education. Biomedical Engineering/Biomedizinische Technik, 57(SI-1 Track-A), 67–70. Wayne, D. B., Didwania, A., Feinglass, J., Fudala, M. J., Barsuk, J. H., & McGaghie, W. C. (2008). Simulation-based education improves quality of care during cardiac arrest team responses at an academic teaching hospital: a case-control study. Chest, 133(1), 56–61.

®

Augmented Reality Based Intelligent Diagnosis Platform

235

Weidenbach, M., Wick, C., Pieper, S., Quast, K. J., Fox, T., Grunst, G., & Redel, D. A. (2000). Augmented reality simulator for training in two-dimensional echocardiography. Computers and Biomedical Research, 33(1), 11–22. Wu, H. K., Lee, S. W. Y., Chang, H. Y., & Liang, J. C. (2013). Current status, opportunities and challenges of augmented reality in education. Computers & Education, 62, 41–49. Yeo, C. T., Ungi, T., Paweena, U., Lasso, A., McGraw, R. C., & Fichtinger, G. (2011). The effect of augmented reality training on percutaneous needle placement in spinal facet joint injections. IEEE Transactions on Biomedical Engineering, 58(7), 2031–2037. Yuen, S. C. Y., Yaoyuneyong, G., & Johnson, E. (2011). Augmented reality: An overview and five directions for AR in education. Journal of Educational Technology Development and Exchange ( JETDE), 4(1), 119–140. Zhu, E., Hadadgar, A., Masiello, I., & Zary, N. (2014). Augmented reality in healthcare education: An integrative review. PeerJ, 2, e469. doi:10.7717/peerj.469 Zou, Y. B., Chen, Y. M., Gao, M. K., Liu, Q., Jiang, S. Y., Lu, J. H., … Zhang, D. H. (2017). Coronary heart disease preoperative gesture interactive diagnostic system based on augmented reality. Journal of Medical Systems, 41(8), 126.

CHAPTER 11

Augmented Reality and Future Mathematics Teachers Martina Babinská, Monika Dillingerová and Lilla Korenova

Abstract The chapter is devoted to the application of augmented reality (AR) in teachertraining programmes at universities. The first part describes the opportunities of future mathematics teachers at the Faculty of Mathematics, Physics and Informatics of Comenius University in Bratislava to work with digital technologies. The second, main part, describes the research of the AR application: Augmented Polyhedrons – Mirage 2.2. We conducted two studies with 40 future teachers. Results support the suitability of the selected application and AR in general. However, the implementation has to be precise, with carefully chosen and formulated tasks to solve. The last part of the chapter summarizes useful AR applications for secondary schools.

Keywords augmented reality – teacher-training – mathematics teachers – digital technologies

1

Augmented Reality and Future Mathematics Teachers

Digital technologies (DT) are becoming more and more important in our society every day. Teachers, as mentors and students’ advisors, should take advantage of this growing power (Pimmer, Mateescu, & Gröhbiel, 2016; Shin & Kang, 2015). To fulfil this expectation, teachers have to be familiar with digital technologies and their implementation into the learning process (Akcayir & Akcayir, 2017). The most direct means of helping teachers develop these skills would be to systematically implement digital technologies throughout the teacher-training programme (Wells, 2018).

© koninklijke brill nv, leiden, 2020 | doi: 10.1163/9789004408845_011

Augmented Reality and Future Mathematics Teachers

237

On the following pages, we briefly describe the situation of future mathematics teachers at the Faculty of Mathematics, Physics and Informatics of Comenius University in Bratislava from the point of view of digital technologies. We are focusing on courses offering space for DT implementation in the learning process. Augmented reality in general has not found a place on these courses yet. The next part of the chapter is devoted to the AR application named ‘Augmented Polyhedrons – Mirage 2.2’ (APM) and its implementation in the teacher-training programme. In November 2017 and April 2018, we conducted two studies with 40 future teachers participating in the implementation of the application into the preparation course. Our results support the suitability of the selected application and AR in general. However, similar to previous studies (Kaufmann & Schmalstieg, 2003), we also observed, that the implementation has to be precise, with carefully chosen and formulated tasks for students. Based on the results of the conducted research we created an implementation method for the APM application in teachers’ training programme mathematical courses. The last part of the chapter offers a view of secondary school mathematical problems, teaching of which could be supported by AR applications. We match mathematical problems with specific existing applications, as well as suggesting new AR applications. that could be created and developed in the future.

2

The Teacher-Training Programme at Comenius University in a View of Mathematics and Digital Technologies

The teacher-training programme has a rich history at Comenius University in Bratislava. To be fully qualified, future teachers have to get a master’s degree. Teachers for primary education are studying at pedagogical faculties only. Teachers for secondary and upper secondary education can choose a pedagogical or scientific faculty and they have to be specialists in two subjects. If the faculty does not support one of the subjects chosen by a student, the student has to study at two faculties. The Faculty of Mathematics, Physics and Informatics offers the following combined mathematics study programmes: 1. Mathematics – Descriptive geometry, 2. Mathematics – Physics, 3. Mathematics – Informatics, 4. Mathematics – Physical education (in collaboration with the Faculty of Physical Education and Sports).

238

Babinská et al.

That faculty is also responsible for mathematical teachers’ preparation for the teacher-training students of the Faculty of Natural Sciences, which offers: 1. Mathematics – Chemistry, 2. Mathematics – Biology, 3. Mathematics – Geography programmes. The teacher-training programme has a bachelor’s and a master’s degree. Study includes three main parts: pedagogical basics, mathematics, and their subject of specialisation. Digital technologies take part in both the pedagogical basics and mathematics part of training. Courses are divided into compulsory, compulsory optional and elective. The only compulsory course required for the bachelor’s degree, Digital technology (1), consists of 28 lessons. Students are introduced to the academic information system, the faculty’s e-learning system, library services usage, and other different digital resources. They also have an opportunity to work with advanced graphical information, text editors, spreadsheet calculators, and presentation software (FMPH UK, 2017b). Subsequently, students can pick up compulsory optional courses Digital technologies (2–5), 28 lessons each. The first two courses are general, the last two are subject oriented and focused on DT implementation into their profile subject. Students have an opportunity to use modern digital technologies to support the achievement of their educational goals. They: – analyse multimedia educational software in terms of the subject, decide critically about its inclusion into the teaching process, – critically evaluate educational and support software and other digital content; formulate educational software requirements and digital content, – assess and decide why, when, where and how DT will contribute to the achievement their educational goals, – have an overview of how to adequately and productively use DT to help achieve the educational goals of their subject, manage classroom teaching so that teamwork by using DT’s support to the cognitive process of students, achieve their didactic goals to communicate with colleagues or students using appropriate and effective tools, use modern DTs in assessing student education, use DT to collect and analyse data on student learning progress, and interpret their results. (FMPH UK, 2017c) The last compulsory optional course related to digital technologies is offered for master’s degree students. Didactics of Teaching Mathematics in Digital

Augmented Reality and Future Mathematics Teachers

239

Environments, with 28 lessons, aims to show students a variety of materials available online and their reliability. Students in the course analyse educational software and digital content. To successfully finish the course, they have to prepare a lesson for one mathematical topic in an e-learning environment. The prepared lesson consists of explanations, training, some mathematical games, and exercises as well as testing and examination. Materials from the internet are permitted, but students are responsible for their correctness (FMPH UK, 2017a). The described courses offer many opportunities to familiarize future teachers with digital technologies. However, as we can see, augmented reality has not taken a place in these courses yet. As a great supportive and motivational teaching aid (Bujak et al., 2013; Kaufmann & Schmalstieg, 2003; Wu, Lee, Chang, & Liang, 2013) it should not be forgotten. In the following pages we describe a selected AR application implementation in the teacher-training process at Comenius University.

3

Augmented Polyhedrons AR Application and Future Mathematics Teachers

In November 2017 and April 2018, we conducted two independent studies focused on an augmented reality application named Augmented Polyhedrons – Mirage 2.2. The aim of these studies was to explore reactions of teacher-training students to the selected application and AR in general. We were also interested in usability limitations of the application, its strengths and weaknesses. The research questions asked: Is the selected application suitable for heuristic, constructionist student activities? If yes, what is the best way to implement it in the teachertraining programme course? Based on their own experience, do future teachers see a concept of AR implementation into the mathematics teaching process? 3.1 Augmented Polyhedrons – Mirage 2.2 Augmented Polyhedrons is an AR application for visualizing 3D mathematical polyhedrons. If user looks through the installed application to the paper printed marker, he/she can see a 3D model of a polyhedron (Figure 11.1). The application was created as a part of their Education category by SOFT12 group. It offers 12 different polyhedrons: cube (number 1), cuboid (2), sphere (3), cylinder (4), cone (5), pyramid (6), tetrahedron (7), triangle prism (8), prism with a trapezoid base (9), hexagonal prism (10), truncated cuboid (11), and prism with a deltoid base (12) prepared as a set of markers. The whole application as well as a set of markers are free to download and print (Chardine & Corneille, 2018; Softland, 2018).

240

Babinská et al.

figure 11.1 Augmented polyhedrons

Part of the free materials are 3 different working sheets for students (Baliros, 2018; Gallien, 2018; Pierrot, 2018). These materials are available only in French and for that reason we did not use them in our research. 3.2 Research Methodology The conducted research included two independent studies handled in November 2017 and April 2018. Both studies engaged teacher-training programme students. The first group included 27 second-year students in the bachelor’s degree level of the primary teacher-training programme from the Faculty of Education at Comenius University in Bratislava. The second one was a group of 13 secondary and upper secondary teacher-training programme students in the first-year of the master’s degree at Faculty of Natural Sciences and at the Faculty of Mathematics, Physics and Informatics. The execution of studies involved two main parts: problem solving (where an AR application was used) and presentation of findings. During these parts, students worked in 3–6-member groups, created of their own accord.

Augmented Reality and Future Mathematics Teachers

241

At the beginning, every group received a tablet with the installed AR application and a set of AR markers printed on cards. The first goal of every group was to identify the type of polyhedron represented by a selected marker, find its characteristics, volume, surface and net.1 The primary teacher-training students also received a building kit Polydron (Lakeside Business Park, 2018) to form these polyhedron(s). This activity was optional; groups had an opportunity to use it for their own benefit. At their request, students were allowed to use the Internet. At the end, every group presented their findings in front of the whole class. Each group was asked to create a video record of their working process, and the whole class was independently observed by a researcher. The researcher did not interfere with any part of participants’ problem-solving process. The first and the second study differed in the content of activities and the scheduling of activities. The primary teacher-training students in the first study were working with 12 polyhedrons in two classes (90 minutes each). The presentation of findings was held during the second class, one week after the problem-solving exercise. Students presented two polyhedrons, chosen by the researcher. One of the chosen polyhedrons was supposed to be from the elementary group of polyhedrons (marked by the application’s creators with the numbers 1–6) and one from the auxiliary group (marked by numbers 7–12, see Section 3.1). The secondary and upper secondary teacher-training programme students in the second study were only working with 3 polyhedrons chosen by a researcher. Two of these chosen polyhedrons were supposed to be from the elementary group of polyhedrons (numbers 1–6) and one from the auxiliary group (7–12) (see Section 3.1). The time for problem solving was shortened to 30 minutes plus instructional time (15 minutes). The presentation of findings followed immediately. This change from the structure followed by the primary teacher students was made because of scheduling constraints faced by some of the participating students. From the mathematical point of view, the main idea of the requested activities was to identify parameters necessary to find a polyhedron’s volume, surface, and net. However, the results suggested that some students focused more on exact measurements and counts. The results also indicated that the elementary/auxiliary classification suggested by the app’s makers was problematic.2 Based on these results we suggest some improvements for the implementation of the Augmented Polyhedrons application in teacher-training programmes (see Section 3.5).

242 3.3

Babinská et al.

Data analysis

For qualitative data analysis, we used video recordings of students working process as well as the researcher’s observations. Each of the student groups was analysed separately as well as in comparison to other groups, with respect to teamwork, approach, motivation, and knowledge. A priori, we established four main categories of observed situations: Influence of mobile technology (ICT), Constructionist approach (CA), Motivation (M) and Mathematics (MTH). The students’ presentations of findings were coded and analysed in view of the whole work but also separately with a focus on presenters’ attitude. 3.3.1

Secondary and Upper Secondary Teacher-Training Programme Students The students involved in the research were first year master’s degree students who already completed bachelor’s degrees in a teacher-training programme at the Faculty of Natural Sciences and at the Faculty of Mathematics, Physics and Informatics. All 13 students participated in the research voluntarily in 4 mixed (men and women) groups (4, 3, 3, 3 members). 3.3.1.1 Group 1 This group of four members was evaluated as the most paradoxical group. On the one hand, it showed great motivation, cooperation and support. Passionate discussions and great effort to solve requested problems were often accompanied by initial individual ideas, jokes, and overall by a very good atmosphere. The group work was not weakened even by difficulties with the group’s challenging polyhedron, the truncated cuboid (number 11). On the other hand, we observed mathematical problems, unsolved tasks, individual problems in group ethics, and the strongest discouraging influences. The analysis showed that these problems were caused by an Augmented Reality Zooming Problem3 and by the fact that the group did not discover/identify this problem. Group members repeatedly measured polyhedron’s edges and argued on measurement accuracy and mathematical rounding. Rounding to one decimal place, the strategy finally chosen by the group, meant that exact calculations of volume and surface were extremely challenging and boring. The students were under time pressure and they lost motivation. One very negative argument between group members was identified (for more details see Table 11.1). The group’s presentation confirmed our observations. The presenter pointed many times to measurement accuracy problems and the time necessary for measuring. She was focusing mostly on measurements and the

signifijicant

very minor

yes + + + + + Truncated Cuboid (11) good strong no

43

Group 2

minor

very good very strong no

yes + + + + + no

51

Group 3

good strong medium (beginning) strong (the end) minor

no + – – + – Tetrahedron (7)

68

Group 4

a If the electronic device with an AR application is closer to the visualized object, the object seems to be bigger. If the device is further, the object seems to be smaller.

Mathematical difffijiculties[very signifijicant/ signifijicant/minor/very minor/none]

Group ethics [very good/good/bad] Working motivation [very strong/strong/medium/week/no] Working demotivation [very strong/strong/medium/weak/no]

The most challenging polyhedron

no + – – – – Truncated Cuboid (11) very good very strong strong (the end)

Zooming problem discovera [yes/no] Requested tasks [+: solved correctly for 2 or more polyhedrons/–: unsolved or solved incorrectly for 2 or more polyhedrons] classifijication volume surface net characteristics

56

The percentage of a time used to measure the edges of polyhedrons [+/– 5%]

Group 1

table 11.1  Main results, secondary and upper secondary teacher-training programme students

Augmented Reality and Future Mathematics Teachers

243

244

Babinská et al.

polyhedron’s exact dimensions instead of its characteristics. In conclusion, the group decided that it was not pleasant to work with the cone (5) and the truncated cuboid (11). 3.3.1.2 Group 2 This group of three members was confident, professional, and mathematically very strong. They showed very nice cooperation and presentation skills. All three members were involved in the problem solving, they did not interrupt each other, and they clearly explained their ideas and solutions to each other and to their schoolmates. This group discovered the application’s zooming problem and they realized the limitations of the application connected with this problem. Because of this, the group did not focus on exact measurements of polyhedrons’ edges, and instead discussed which parameters are necessary to find the volume or surface of a polyhedron if we know the formula for volume or surface of this polyhedron and how to find these formulas if not known. As the group was very mathematically strong, we expected that the work with easier polyhedrons like a cube (1) would be a little bit boring. However, the students were not bored or demotivated. They discussed each polyhedron’s advanced characteristics like: the possibility of inscribing a sphere to the cube, symmetries, Plato’s solid identification, section planes, volume and surface ratios, and formulas for counting diagonals. 3.3.1.3 Group 3 This group of three members was very similar to Group 2 in cooperation, group ethics and professional approach. The zooming problem’s impact on accuracy was recognized from the beginning. The group attended to it while making measurements and on a few occasions the students noted that the application is not aimed to exact measurements. This group had more mathematical challenges than the previous one. We observed two very common mathematical mistakes. One terminological, the other one purely mathematical. Students were using “cube” instead of “square” for a polyhedron’s square base and they discussed that it is possible to create the net of a sphere.4 Both these mistakes were explained and understood by co-members. As we can see in Table 11.2, the group did not work with very complicated polyhedrons. For this reason, students finished tasks in less than half allocated time and no big/passionate discussion developed. Similarly, to the previous group, they were not bored or discouraged by easier polyhedrons. In case of polyhedrons’ characteristics, these students also focused on advanced

245

Augmented Reality and Future Mathematics Teachers table 11.2  Polyhedrons’ difffijiculty, secondary and upper secondary teacher-training programme students

Group 1

Group 2

Group 3

Group 4

IM VS N CH IM VS N CH IM VS N CH IM VS N CH Cube (1) Cuboid (2) Sphere (3) Cylinder (4) Cone (5) 1 3 3 0 Tetrahedron (7) Triangle prism (8) Prism with a 2 2 1 0 trapezoid base (9) Truncated 3 3 1 0 Cuboid (11)

1

1 1

1

1

1 1

1

IM – Identifijication and measurement VS – Volume and Surface N – Net CH – Characteristics

RED – incorrect approach GREEN – correct approach BLUE – we did not identify whether the approach was correct or incorrect

1 2

1 1 1 2

1 2 2 2 3

2

3

0 3

1 1

1 1 1 1 0 0

0 0 0

2

3

Difffijiculty: 0 – unsolved task 1 – easy 2 – medium 3 – challenging

characteristics like volume and surface formula optimization or sphere’s section planes. 3.3.1.4 Group 4 The last group, also consisting of three members, was definitely the group with the strongest discouragement. In comparison to the previous ones, students were solving problems much less enthusiastically and we observed negative reactions, e.g., “wait, we have to turn that stupid cone,” from the beginning. Even awkward situations caused by the application5 were perceived more as a problem than as an amusing experience. It should be noted that the members of this group were students identified by their teacher as hard to motivate. From this point of view, we greatly appreciate group’s attempt to solve the requested tasks, the funny names they created for polyhedrons (e.g., “puck” for a cylinder and “coloured pyramid” for tetrahedron), and the jokes they made while working.

246

Babinská et al.

As the time went by, students showed more and more passion to find a correct solution and frustration when they did not have success (Table 11.1). Many of these students’ problems were caused by their lack of understanding of the application’s zooming problem. While making measurements of the last polyhedron, students started to realize “some” problem but, unfortunately, only one member discovered the core of it and he did not know how to explain it to his co-members. Irritation from this situation was expressed by the presenter, who said, “this polyhedron was problematic; it was frustrating for me a bit.” 3.3.2 Primary Teacher-Training Programme Students Students involved in this part of the research were second year bachelor’s degree students in the primary teacher-training programme at the Faculty of Education. The group, consisting of 27 female students, cooperated on the research voluntarily. The students worked in 5 groups of 5–6 members (5, 6, 5, 6, 6). They were asked to make video records of their working process, but this activity was only partially executed (students recorded only some parts of their work). Results from the study, focused on pedagogical-psychological aspects, have already been presented at the Conference on Applied Mathematics APLIMAT 2018, in February 2018 (Koreňová & Gunčaga, 2018). Now we are looking at the data with a focus on the use of the Augmented Polyhedron application. 3.3.2.1 Group 1 The five students in this group worked with a strong motivation and showed a very good group ethics. Also, mathematically the group members did not have any problems. They did not use the building kit Polydron and spent most of the working time trying to find the exact values of a volume and a surface of polyhedrons. After the first few attempts, the students discovered the AR Zooming Problem. 3.3.2.2 Group 2 The second group, consisting of six members, was the group with the lowest motivation and the worst group ethics: mostly only 2–3 students cooperated. However, these cooperating students worked hard and enthusiastically on solving tasks. They used the building kit Polydron for better visualization, and also created and used paper models of polyhedrons. Unfortunately, the task for presentation was not figured out correctly. The students did not present the second assigned polyhedron, instead, they chose a prism with deltoid base (12) (Table 11.3).

247

Augmented Reality and Future Mathematics Teachers table 11.3  Main results, primary teacher-training programme students

Group 1 Group 2 Group 3 Group 4 Group 5 Zooming problem discovera [yes/no] Requested tasks classifijication [+: solved volume correctly/ surface –: unsolved net or solved characteristics incorrectly] The building usage kit Polydron implementation to the presentation The most challenging polyhedron Group ethics [very good/good/bad] Working motivation[very strong/ strong/medium/week/no] Working demotivation [very strong/strong/medium/weak/no] Mathematical accuracy (presentation) [very good/ good/medium/bad/very bad]

yes

no

yes

yes

no

+ + + + +

+ + +

+ + + + +

+ + +

+ + + +

no

yes

yes

yes

yes

no

no

yes

yes

yes

–b very good very strong no

bad

very good very strong no

good

very good

bad

very good medium very strong strongc no

medium very good

strong no good

a If the electronic device with an AR application is closer to the visualized object, the object seems to be bigger. If the device is further, the object seems to be smaller. b As the video records were not complete, it was not possible to identify polyhedrons’ difffijiculty. c Two members started to get bored in the middle of the work.

There are few possible reasons for this disorganized group approach. It is possible that there were too many group members in this group or/and their mathematical skills were too diverse, but the main cause was not found. 3.3.2.3 Group 3 This five-member group showed strong cooperation and motivation but, compared to group 1, worse mathematical skills. At the beginning of their work,

248

Babinská et al.

students started to measure the edges of polyhedrons and after a time, they discovered the zooming problem. During the presentation, students divided the roles. One of the group members controlled the video recording playback, another presented the paper model, while a third was writing on a board and explaining details of the creation of formulas for the polyhedron’s volume and surface. 3.3.2.4 Group 4 Group 4, the second six-member group, was the best cooperating and systematically working group. Students took notes, and ordered work with polyhedrons based on markers’ numbers. The work was divided between group members: while one student was searching for information on internet, two other students were recording video, two others were erecting models from the building kit Polydron, and the last student was building paper models and nets. Additionally, their presentation was mathematically accurate, and both graphically and logically well executed (Table 11.3). In conclusion, the group was highly motivated while working with the AR application as well as while presenting and they demonstrated the best digital competencies. 3.3.2.5 Group 5 The last group, with five members, can be evaluated as a good, standard team. Students were interested in the requested tasks, cooperated well, and created polyhedrons’ models from the building kit Polydron, but only for more challenging polyhedrons. The group took notes, but again, only about more challenging polyhedrons. The presentation was mathematically accurate but missing important information. 3.3.3 Polyhedrons’ Difficulty Compared to the secondary teachers described in the previous section, we are missing the information about polyhedrons’ difficulty for primary teacher-training programme students (Table 11.2, Table 11.3). This is caused by incompleteness of the video records. In this phase of the study, students did not record the whole working process, so it was not possible to analyse their manipulation of some polyhedrons. 3.4 Conclusion The conclusions can be divided into two main categories. The first one is related to augmented reality in general (Sections 1–3). The second one is related to the researched application – Augmented Polyhedrons – Mirage 2.2 (Softland, 2018) (Sections 4, 5).

Augmented Reality and Future Mathematics Teachers

249

3.4.1 Augmented Reality Brings a Great Opportunity for Group Work Augmented reality offers a great opportunity for training students’ group work. This way, students benefit from natural space to train cooperation, individual argumentation and management skills, ones of the crucial future teachers’ skills. The research showed that most of the groups of students were strongly motivated to work in a positively inspired environment with mostly highly positive group ethics. Students did their best to find solutions for the requested tasks and cooperate. Given the high motivation of the student participants during the study, we conclude that the AR did not have any negative impact on motivation, and may, in fact, have had a positive impact.6 3.4.2 Positive Memories Can Be Useful in the Future AR applications’ implementation into the teachers’ training process can increase future teachers’ tendency to implement augmented reality and also digital technologies in general into the teaching process. 3.4.3 Zooming Problem of AR7 Has to Be Discovered The provided analysis a showed strong correlation between discovery of the zooming problem and both group ethics and working motivation/discouragement. Students who did not recognize this problem, sometimes failed to understand the requested tasks’ main point and they felt frustrated. It is crucial to find a way for students to discover the zooming problem at the beginning of their work with AR. Students have to realize that the visualized object scales up/down if they move an electronic device closer/further to the printed marker. If they understand this dependency, they understand that they cannot measure the edges of polyhedrons precisely and they will not focus on this measurement. 3.4.4 Augmented Polyhedrons – Mirage 2.2: Start with Easy Solids As we can see in Table 11.2, there is a big difference between the different polyhedrons in the application. Some are very easy to work with, others are more challenging. To support students’ enthusiasm and motivation, it is crucial to start the work with easier solids. If students start with easier solids, they do not have to focus so much on mathematical tasks. These are quite easy for them to solve. Instead of mathematical knowledge, students can more focus on augmented reality to “play” with it and to discover how it works. Working with easier solids can then also help with the zooming problem. 3.4.5 Augmented Polyhedrons – Mirage 2.2: Easy Solids Are Interesting The last conclusion relates to the students’ mathematical skills. The research suggested that higher mathematical skills did not lead to boredom or

250

Babinská et al.

discouragement when working with easier polyhedrons. When the students with better mathematical skills worked with these polyhedrons, they used different approaches to the tasks. They focused on mathematically more challenging concepts and their working motivation did not decrease. Higher/lower mathematical skills did not influence students’ motivation to work with less challenging polyhedrons. This observation highly supports Augmented Polyhedrons – Mirage 2.2 application’s implementation in the learning process. It shows that it is possible to start work with basic polyhedrons without concern for students’ mathematical knowledge. Based on major, age, and mathematical skills, students may choose their own approach to the application and carefully requested tasks. 3.5

Augmented Polyhedrons – Mirage 2.2 Implementation in the Teachers’ Training Programme Mathematical Course The conclusions from our research showed, that the Augmented Polyhedron – Mirage 2.2 application can be a good application to use in a class of future teachers. The work with this application supports positively inspired group work. However, the implementation of this application into the learning environment has to be precise, with carefully chosen and formulated tasks for students to solve. Based on our experience during the research we found that the way we used the application (see Section 3.2) can be improved. We also found that the elementary/auxiliary classification of polyhedrons in the application suggested by the app’s makers is problematic. In the next section, we describe improved protocol of implementation of Augmented Polyhedrons app for teachers’ training programme mathematical class. In Table 11.4 we suggest a new classification of application’s polyhedrons. table 11.4  Polyhedrons’ classifijication

Difffijiculty

Polyhedrons

Category I (C1) Category II (C2) Category III (C3) Category IV (C4)

Cube (1) Cuboid (2) Triangle prism (8) Cylinder (4) Pyramid (6) Hexagonal prism (10) Sphere (3) Cone (5) Tetrahedron (7) Prism with a trapezoid base (9) Truncated Cuboid (11) Prism with a deltoid base (12)

Augmented Reality and Future Mathematics Teachers

251

3.5.1 Improved Protocol of Implementation The whole working process should be divided into two sessions: one for manipulation with an application (two classes, 90 minutes) and the other one for presenting findings (10 minutes for each group). We have two main suggestions for the manipulation class. First, the most important is for students to have enough time to familiarize themselves with the AR application at the beginning of their work. Second, students should start the work with elementary solids. From this reason we suggest splitting the work with application into three parts. The first one (Activity I), only 10 minutes long, is to discover the application’s features. The next, 35 minutes long (Activity II), allows students to manipulate the easier polyhedrons and familiarize themselves with the requested tasks. We hope that students can discover the zooming problem during this phase. After a 10-minute break students can continue with the last activity, 45 minutes long (Activity III), designed to apply gained knowledge to the more challenging polyhedrons. 3.5.1.1 Session 1 (Manipulation Class) At the beginning of the class, the teacher should ask students to create groups of 3–4 members. Every group has needs paper, a pen, and a ruler. After the groups are made, the teacher gives every group a tablet with the installed Augmented Polyhedrons application and the printed markers and then explains how to turn on and use this application. Students can also install the app to their own device. Activity I: Examine markers from category C1 and C2 (Table 11.48) in front of you. What kind of polyhedrons are hidden there? Can you do the same with polyhedrons from categories C3 and C4 (Table 11.4)? Activity II: Select one polyhedron from category C1 and one from category C2 (Table 11.4). For every chosen polyhedron answer next questions: How many edges, faces, and vertices has your polyhedron? Can you create its net? Is it possible to find more than one net for it? Can you find volume and surface of your polyhedron? What data do you need for exact volume and surface counting? Can you find them? Can you find and describe more characteristics of your polyhedron? Activity III: Select one polyhedron from category C3 OR C4 (Table 11.4). Take your previous experience and prepare a 6-minute presentation about selected polyhedron. Use short video record from your work if necessary, point out the most challenging questions for your group. Use the internet if you need to find some information.

252

Babinská et al.

3.5.1.2 Session 2 (Presentation Class) Students should present their findings from the previous class here. The teacher needs to prepare a presentation device and a board for writing. Each group should present for 6 minutes and have 4 minutes of discussion after the presentation. Groups should share and discuss discovered facts about chosen polyhedrons.

4

AR Applications for Secondary School Mathematics

There are many mathematical topics where AR applications can be usefully implemented in the teaching and learning process. The next pages describe secondary school mathematical problems that could be well explained using AR apps. We are identify current applications, but also suggest new ones by describing their functionalities. 4.1 Slovak Curricula In Slovak Curricula (Štátny pedagogický ústav, 2018), there are five main topics included in secondary school mathematics: A Numbers, variables, and counting with numbers, B Binary relations, functions, tables, graphs, and diagrams, C Geometry and measurement, D Combinatory, probability, and statistics, E Logic, argument, and proof. Each of these topics should be addressed every year to enlarge and deepen students’ learning. On the next pages we are going through selected mathematical problems divided by these topics and students’ age (10–14 years old), suggesting augmented reality applications’ implementation (Table 11.59). Our suggestions open a wide space for new AR application creation and use in the mathematical teaching and learning process. 4.2 Secondary School Mathematics Problems and Uses of AR 4.2.1 Ten-Years-Old Students10 Numbers, variables and counting with numbers Problem #1: 10-year-old students are counting with natural numbers and zero only. They learn to identify which number of a given set is the biggest (lowest) one and to represent it on a number axis. Suggestion: For this part of mathematics it would be beneficial to have an application reading two numbers and making a number axis showing not only

Augmented Reality and Future Mathematics Teachers

253

table 11.5  AR applications’ implementation into the secondary school mathematics

Age [years]

A. Numbers, variables, and counting with numbers

10

#1 Arrangement of natural numbers #2 Even/odd, fijigural numbers #6 Divisibility #7 Decimal numbers’ comparison #12 Fraction, how much is it? #13 Ratio #14 Proportionality #17 Arrangement of Whole numbers #22 Arrangement of Real numbers

11 12

13 14

C. Geometry and measurement

10

11

12 13

14

#3 Solids’ sorting #4 Cubes’ constructions, footprints, front view #5 Polygons’ naming #8 Are of a polygon (square grid) #9 Angles’ measuring and naming #10 Triangles typology #15 Parallel projection #16 Cube, cuboid (net, volume, surface) #19 Parallelograms #20 Thales’ theorem #21 Prism (net, volume, surface) #23 Similar triangles #24 Pythagorean theorem #25 Pyramid, cone, cylinder (net, volume, surface)

B. Binary relations, functions, tables, graphs, and diagram

#18 The coordinate system #26 Direct and inverse proportionality D. Combinatory, probability and statistics

#11 Possibilities’ listing, variate, permutate

these two, but also the nearest lower and the nearest bigger number with only one non-zero digit, or with the last digit zero. Example: Student write numbers 154 and 146. On the number axis represented in an AR app, following numbers: 100, 146, 154, 200, or 140, 146, 154, 160 should be shown.

254

Babinská et al.

Problem #2: Another possible AR usage is related to even/odd numbers. Suggestion: We suggest an application reading a number and constructing it via little stones, giving pairs of numbers into a rectangle with the width of two. In augmented reality, it is possible to make the process really quick and better to understand. We are waiting for the last stones. If there is only one stone, the number was odd. If there were two of them, the number was even. The application can also be helpful for a bigger numbers’ 2 and 10 division. It should read the dividend and the divisor from the exercise book and place the stones over these two numbers onto the paper. The same idea can be used for a multiplication by one-digit numbers. Geometry and measurement Problem #3: Ten-year-old students are describing polyhedrons, cylinders, and spheres. Suggestion: We can use the application Augmented Polyhedrons – Mirage 2.2 (Softland, 2018) to find the number of vertices, faces, and edges. It is possible, that students will better understand some of these 3D-bodies characteristics. Problem #4: To aid understanding of polyhedrons, students have to construct cubes and describe the construction. They also have to make and read different structures from cubes – blueprints (Figure 11.2). Suggestion: Scanning the picture of a construction and modelling a solid would be really helpful. The same could be if the application reads a code, displays the solid and students have to make the blueprint. The highest advantage of AR is the possibility to see the solid from all sides by moving the displaying device. The Lego AR Studio (LEGO, 2018) is a current application that could be used for this activity.

figure 11.2  Structure from cube, blueprint

Augmented Reality and Future Mathematics Teachers

255

Problem #5: Rectangles, triangles, polygons, and circles are partly new and partly old shapes. Students have to be familiar with recognizing them. Suggestion: Working with the application What is geometry (Watizeet, 2018) could be really motivating. With the camera focusing on objects, children have to find polygons in the real world. The application shows it as a mathematical object. Students can count vertices and sides. They can easily compare their findings and learn by experience. 4.2.2 Eleven-Year-Old Students11 Numbers, variables, and counting with numbers Problem #6: Students have to decide if a natural number is divisible by 2, 3, 4, 5, 6, 9, 10 and 100. Suggestion: To help students go through this problem, we can use the application with rectangular shapes of stones suggested above (problem #2). It could also be beneficial, if children can prepare and “teach” application their own rule for divisibility by 3, 4, 9 or even 6, created on their own observations. Problem #7: The topic of positive decimal numbers includes numbers’ comparison at this age. Suggestion: The application suggested for problem #1 could be similarly used for decimal numbers. Application shows the nearest lower and the nearest bigger whole number. Children can easily decide which number is smaller/greater. Geometry and measurement Problem #8: Students have to create polygons’ 3D and 2D square grids. The area of one grid’s square is named “unit.” Students have to decide how many units a polygon contains. Suggestion: Very helpful application could be one that adds a square grid to the displayed image of reality with drawn polygons. Ideally, the app could recognize edges and display boundaries within the square grid and fill the polygon with a colour. The first step – polygons’ recognition in a real world-is solved by the previously mentioned app What is geometry (Watizeet, 2018). The next step is to

256

Babinská et al.

measure lengths and angles. We can use apps with rulers. There are many of these available: AirMeasure – AR Tape & Ruler (Laan Labs, 2018), Measurement tool (Matei, 2018), AR Ruler – AR Ruler App (GRYMALA, 2018a) or AR MeasureKit (Khanov, 2018). Problem #9: The next topic in the curriculum is angle. Students have to measure angles, and identify whether an angle is acute, right, obtuse, straight, or reflex. Suggestion: While measuring angles students can experiment with the app Prime Ruler (GRYMALA, 2018b). This app can measure length and angles. Problem #10: After recognizing angles, students work with triangles’ typology. Suggestion: The app Prime Ruler can also be useful here. It can work with pictured triangles, students can measure its edges and angles. They can also try to find acute, obtuse, or right triangles in a real life by picturing objects in the classroom or at home. Working with so many models leads to better understanding and remembering. Combinatorics, probability, and statistics Problem #11: The first experience with this topic for many students is connected to the listing of possibilities. Students have to find a way to find and list all possibilities without repeating any. If students do not have a system, they make many mistakes, especially in repeating. To find a mistake is very often a challenging task. Suggestion: An application can compare listed possibilities, one to each other. If application finds two equal possibilities, it draws a rectangle above each of them. 4.2.3 Twelve-Year-Old Students12 Numbers, variables, and counting with numbers Problem #12: Students work with fractions and percentages. At the beginning, it’s sometimes very challenging to recognize what a fraction really represents. Teachers then try to draw as many models (chocolate bar, pizza, cake, etc.) as possible.

Augmented Reality and Future Mathematics Teachers

257

Suggestion: Teacher’s work can be supported by an app. A fraction’s representation by chocolate bar, pizza, or cake can be visualized by an application. Binary relations, functions, tables, graphs and diagrams Problem #13: Students are confronted with the binary relations first time here. They have to divide a unit in a given ratio. Suggestion: A unit division can be visualized by an app. If the app is able to read the ratio and a unit, it can divide the unit with a line. This application can be used to better understand the method or even to correct student’s solutions. Problem #14: Students work with direct and inverse proportionality. Suggestion: To improve the understanding of these two proportionalities there can be used applications creating graphs from equations like Photomath (Photomath, 2018) or Mymath (Vladimirovich, 2018). Both these apps can picture an equation and draw a graph of it. If students see the graph, it could be easier for them to understand the difference between proportionalities. Geometry and measurement Students are working with cubes and cuboids and their characteristics here. Problem #15: Students have to construct objects in a parallel projection. Without having a cube (cuboid) in their hands, it is sometimes challenging to construct visible and invisible edges correctly. Suggestion: It could be nice, if students can visualize a cube (cuboid), which is possible with the previously mentioned application Augmented Polyhedrons (Softland, 2018) (problem #3). Problem #16: Students construct the net of a polyhedron, count its surface and volume. Suggestion: As there are many applets which can construct a cube from its net or spread the net from a cube, we recommend adding this visualization to any AR app showing real polyhedrons. One possibility could be an educational pack of platonic solids made by QuiverVision (QuiverVision, 2016). For the next

258

Babinská et al.

step, surface and volume counting, GeoGebra Augmented Reality (International GeoGebra Institute, 2018) or Arloon geometry (Arloon, 2017) applications can be used. 4.2.4 Thirteen-Year-Old Students13 Numbers, variables, and counting with numbers Thirteen-year-old students are confronted with negative numbers for the first time. Problem #17: Students have to decide, which number is greater. One of these numbers is positive and the other one is negative (e.g., 4 and –5). Suggestion: For problems #1 and #7, we suggested drawing a numeric axis, and that technique could be used here. To help students with this topic, the app should only read two numbers and draw a number axis with given numbers and zero in a proper orientation. Binary relations, functions, tables, graphs, and diagrams Functions and binary relations theme evolve from a table-oriented system to a graph-oriented system here. Problem #18: Students are working with coordinate system; coordinates of a point, and they are beginning to construct function graphs. Suggestion: It could be very helpful if some application can add a coordinate system to students’ drawings. Students can try to determine the coordinates then. Another useful application could create grid for a drawn coordinate system. Then students can easily mark some points given by their coordinates. Geometry and measurement Problem #19: Students learn about parallelograms here. Is a rectangle a parallelogram? What are the characteristics of a parallelogram? Suggestion: Using the Arloon geometry app (Arloon, 2017), students can deal with some pictures of parallelograms and try to describe why it is a parallelogram. They can experiment with the sides’ length, and angles’ scale and orientation.

Augmented Reality and Future Mathematics Teachers

259

Problem #20: Thales’s theorem. Suggestion: We have two possibilities. The first one is to prove Thales’s theorem. We can use measurement apps (problem #8) and try to measure angles. The second one is to construct a circle for an existing triangle. The app could visualize a circle based on a student’s triangle dimensions. The students would have to find a proper position for that circle and recognize perpendicular lines. Problem #21: Students learn about prisms’ surface and volume for quadrilateral prisms, triangle prisms, and hexagonal prisms. Suggestion: Necessary prisms are visualized by Augmented Polyhedrons application (Softland, 2018). 4.2.5 Fourteen-Year-Old Students14 Numbers, variables, and counting with numbers Students are confronted with irrational numbers here. They should already understand Euler’s number and π. Now they have to discover the square root of two and other roots. Problem #22: The problem of numbers’ comparison exists for continuous numbers. To see where the root of two lies on the number axis, we are dealing with approximate decimal numbers. Suggestion: We encounter number axes again (problem #1, #7, #17). Here the application should read the real number, draw a number axis with the given number and two decimal numbers, one lower and one bigger, in a proper orientation. We should have an opportunity to choose a desired approximation. Binary relations, functions, tables, graphs, and diagrams Problem #26: Student learn about direct and inverse proportionality. They should be able to create a graph of these two proportionalities. Suggestion: To discover one constants’ impact to another one, students can use applications creating graphs from equations (problem #14). Geometry and measurement Problem #23: Students are working with similar triangles here. They learn about proportionality of sides and equality of angles.

260

Babinská et al.

Suggestion: We would like to have an application with markers visualizing various triangles. The application creates a triangle, students then try to find a similar triangle to the created one among triangles represented by given markers. They have to move the AR device to make sure triangles are really similar. They also can move tablet higher/lower to get a bigger/smaller triangle and try to find ratios, or even get congruent triangles. Problem #24: Pythagorean Theorem. Suggestion: The previously mentioned application in problem #23 could have a second mode. To every visualized triangle (from a marker) it could display the appropriate squares. Students can try to find, the relationship between these squares. Can be the smaller squares’ area sum bigger/smaller than the area of the biggest square? Problem #25: New geometrical solids are pyramid, cone, and cylinder. Students learn about their surface and volume. Suggestion: A visualisation of these solids through AR apps can help to form a better understanding of the shape, net and solid itself. The biggest advantage is in the possibility to see solids from all sides. Students can again use the Augmented Polyhedrons app (Softland, 2018).

Acknowledgement The chapter was supported by a project 012UK-4/2018 “The Concept of Constructionism and Augmented Reality in the Field of the Natural and Technical Sciences of the Primary Education (CEPENSAR).”

Notes 1 The net of a polyhedron is also known as an unfolding, development, or pattern (Weisstein, 2018). 2 The students found the triangle prism (number 8) much easier to work with than the sphere (number 3), for example. 3 If the electronic device with an AR application is closer to the visualized object, the object seems to be bigger. If the device is further, the object seems to be smaller. 4 The sphere does not have a net with the same meaning like cube for example. It is impossible to make a perfect sphere (ball or globe) from a flat sheet of paper.

Augmented Reality and Future Mathematics Teachers

261

5 For example, the displayed polyhedron “jumps” if the ruler is crossing the number on a marker. 6 This positive impact can be influenced by novelty of AR. To be sure about the positive impact on motivation of AR, it is necessary to study a long-term, systematic implementation of AR in the learning process. 7 See Note 3. 8 Based on analysed polyhedron’s difficulty (Table 2), we are dividing application’s polyhedrons to four difficulty categories: C1 (the easiest), C2, C3, C4 (the most challenging). 9 We have no suggestion for the topic Logic, argument, and proof, so this topic does not figure in the table. 10 The first-grade students of lower secondary education (ISCED 2). The age of these students can vary. It depends on the age when students join school. 11 The second-grade students of lower secondary education (ISCED 2). 12 The third-grade students of lower secondary education (ISCED 2). 13 The fourth-grade students of lower secondary education (ISCED 2). 14 The fifth-grade students of lower secondary education (ISCED 2).

References Akcayir, M., & Akcayir, G. (2017). Advantages and challenges associated with augmented reality for education: A systematic review of the literature. Educational Research Review, 20, 1–11. Arloon. (2017). Arloon geometry. Retrieved from https://play.google.com/store/apps/ details?id=com.Arloon.Geometry.AR&hl=en Baliros, F. (2018). Les volumes de l’espace au collège. Retrieved from http://mirage.ticedu.fr/wp-content/uploads/2014/11/activite-RA1.pdf Bujak, K. R., Radu, J., Catrambone, R., MacIntyre, B., Zheng, R., & Golubski, G. (2013). A psychological perspective on augumented reality in the mathematics classroom. Computers & Education, 68, 536–544. Chardine, M., & Corneille, I. P. (2018). Augmented Polyhedrons-markers. Retrieved from http://mirage.ticedu.fr/wp-content/uploads/2014/07/00-marqueurs-universel.pdf FMPH UK. (2017a). Didactics of teaching mathematics in digital environment. Informational list. Retrieved from https://sluzby.fmph.uniba.sk/infolist/sk/1-UXX137_15.html FMPH UK. (2017b). Digital technology (1). Informational list. Retrieved from https://sluzby.fmph.uniba.sk/infolist/sk/2-UMA-151_15.html FMPH UK. (2017c). Digital technology (5). Informational list. Retrieved from https://sluzby.fmph.uniba.sk/infolist/sk/1-UXX-341_15.html

262

Babinská et al.

Gallien, V. (2018). Les solides de l’espace étudiés au collège. Retrieved from http://mirage.ticedu.fr/wp-content/uploads/2014/11/Appli_tablette_augmentedPolygons.pdf GRYMALA. (2018a). ARuler – AR Ruler App. Retrieved from https://appadvice.com/ app/aruler-ar-ruler-app/1326773975 GRYMALA. (2018b). Prime ruler – Length measurement by camera, screen. Retrieved from https://play.google.com/store/apps/details?id=com.grymala.photoruler International GeoGebra Institute. (2018). GeoGebra augmented reality. Retrieved from https://itunes.apple.com/us/app/geogebra-augmented-reality/id1276964610?mt=8 Kaufmann, H., & Schmalstieg, D. (2003). Mathematics and geometry education with collaborative augmented reality. Comuters & Graphics, 27, 339–345. Khanov, R. (2018). AR MeasureKit. Retrieved from https://itunes.apple.com/app/ id1258270451 Koreňová, L., & Gunčaga, J. (2018). Augmented reality in mathematics education for pre-service teachers in primary level. In D. Szarkova, P. Letavaj, D. Richtarikova, & J. Gabkova (Eds.), 17th Conference on applied mathematics APLIMAT 2018: Proceedings (pp. 597–605). Red Hook, NY: Curran Associates. Laan Labs. (2018). AirMeasure – AR tape & ruler. Retrieved from https://itunes.apple.com/us/app/airmeasure-ar/id1251282152?ls=1&mt=8&ct=airmeasure.com Lakeside Business Park. (2018). Polydron frameworks platonic solids set. Retrieved from http://www.polydron.co.uk/polydron-frameworks/polydron-frameworks-platonicsolids-set.html LEGO. (2018). The LEGO AR studio. Retrieved from https://itunes.apple.com/us/app/ lego-ar-studio/id1296734986?mt=8 Matei, A. (2018). Measurement tool. Retrieved from https://itunes.apple.com/in/app/ measurement-tool/id1086600799?mt=8 Photomath. (2018). Photomath. Retrieved from https://itunes.apple.com/us/app/ photomath/id919087726 Pierrot, M. (2018). Découverte des solides. Retrieved from http://mirage.ticedu.fr/wpcontent/uploads/2014/11/ACTIVITES-SOLIDES-6-.pdf Pimmer, C., Mateescu, M., & Gröhbiel, U. (2016). Mobile and ubiquitous learning in higher education settings. Computers in Human Behaviour, 63, 490–501. QuiverVision. (2016). Coloring packs. Retrieved from http://staging.quivervision.com/ coloring-packs/#quivervision-platonic-solids Shin, S. W., & Kang, M. (2015). The use of a mobile learning management system at an online university and its effect on learning satisfaction and achievement. The International Review of Research in Open and Distributed Learning, 16(3). Softland. (2018). Augmented polyhedrons – Mirage 2.2. Retrieved from https://augmented-polyhedrons-mirage.soft112.com

Augmented Reality and Future Mathematics Teachers

263

Štátny pedagogický ústav, S. (2018). Matematika – nižšie stredné vzdelanie. Retrieved from http://www.statpedu.sk/files/articles/dokumenty/inovovany-statny-vzdelavaciprogram/matematika_nsv_2014.pdf Vladimirovich, S. (2018). Photo solver of math problems. Retrieved from https://itunes.apple.com/sk/app/photo-solver-of-math-problems/id878221213?l= sk&mt=8 Watizeet. (2018). What is geometry. Retrieved from https://play.google.com/store/ apps/details?id=com.watizeet.geometry Weisstein, E. (2018, October). Wolfram MathWorld. Retrieved from http://mathworld.wolfram.com/Net.html Wells, R. (2018). Poster: 21stC mobile social learning. Retrieved from https://eduwells.com/2012/11/19/poster-21st-century-mobile-social-learning/ Wu, H.-K., Lee, S. W. Y., Chang, H.-Y., & Liang, J.-C. (2013). Current status, opportunities and challenges of augmented reality in education. Computers & Education, 62, 41–49.

PART 3 All Ages and outside the School

∵

CHAPTER 12

Enlivened Laboratories within STEM Education (EL-STEM): A Case Study of Augmented Reality in Secondary Education Ilona-Elefteryja Lasica, Maria Meletiou-Mavrotheris, Efstathios Mavrotheris, Stavros Pitsikalis, Konstantinos Katzis, Christos Dimopoulos and Christos Tiniakos

Abstract Inspired by the emerging technologies of Augmented and Mixed Reality (AR/ MR), the Enlivened Laboratories in Science, Technology, Engineering and Mathematics (EL-STEM) project aims to develop a new approach, integrating these technologies into school laboratories, for encouraging secondary school students’ STEM engagement. In particular, EL-STEM’s main objectives are to attract students who might not be interested in STEM-related studies/careers, enhance the interest of those who have already chosen these fields of studies/ careers, and improve students’ performance in STEM-related courses. Moreover, EL-STEM provides teachers with high quality professional development opportunities to acquire knowledge and skills to effectively embed AR/MR in teaching and learning. This book chapter aims to provide an overview of the ELSTEM project and describe use cases of Augmented Reality in STEM education.

Keywords augmented reality – laboratories – STEM – teacher training

1

Introduction

During the last decade, it has become critical to equip students from an early stage with the knowledge and skills necessary to upscale their profiles to a well-prepared workforce of the future labour market (National Science Teacher Association, 2011). However, lack of scientific competence for a considerable proportion of students has been indicated by cross-national studies © koninklijke brill nv, leideN, 2020 | DOI: 10.1163/9789004408845_012

268

Lasica et al.

on students’ achievement (e.g., TIMSS, PISA). In PISA 2015, for example, about 50% of EU countries that participated had significantly lower than average performance in basic skills, including science and mathematics (OECD, 2016b). Only two countries (Estonia and Finland) were included in the top-10 rated countries globally, with scores lower than 13% (OECD average concerning share of low achievers). Moreover, there is well-documented evidence of declining interest in key STEM topics and careers for students in the EU and internationally (EU Commission, 2016; Panorama, 2016). This situation calls for urgent action, since STEM skills have become critical to innovation and to the creation of a competitive edge in knowledge-intensive economies (OECD, 2015). According to Beers (2011), there is a natural match between STEM skills and 21st century skills. STEM skills are among the key competencies all individuals need in a knowledge-based society for employment, inclusion, subsequent learning, personal fulfilment, and development (OECD, 2016a). While in past years, some EU countries have made significant progress towards improving their students’ performance in science skills (e.g., Estonia, Finland), other countries are still behind (e.g., Cyprus, Greece, Portugal). In a series of EU summits and reports, EU policy makers have expressed the need to reduce the number of underachievers in member states (EU Commission, 2016; EU Commission, 2017; EU Commission, 2018; OECD, 2016a), culminating in the strategic targeting of resources to improve science education and ensure full participation among people from all backgrounds. Technological Innovations also play a critical role in STEM education, which is a field reflecting the philosophy of “technological literacy for all” (Sanders, 2009, p. 24). Efforts to reform current secondary education STEM curricula should aim at enabling students to design and conduct experiments (Felder & Brent, 2003). Schools and other educational authorities invest in laboratories for STEM-related courses in an effort to make their curricula more innovative, attractive, and competitive. Recent technological advances have provided the opportunity to create entirely new learning environments in STEM education, including laboratories. One promising approach lately explored, is the potential of integrating Augmented Reality (AR) and/or Mixed Reality (MR) within secondary STEM education (Lee, 2012), as a means of making the subjects more accessible and attractive for students (Bacca, Baldiris, Fabregat, Graf, & Kinshuk, 2014). AR/MR technologies not only offer new learning opportunities, but also create new challenges both in teaching and learning (Heradio et al., 2016; Wu, Lee, Chang, & Liang, 2013). In order to successfully integrate AR/MR in STEM education, innovative pedagogical approaches and re-contextualized learning environments should be applied, focused on inquiry-based learning and

Enlivened Laboratories within STEM Education (EL-STEM)

269

problem-solving (Chiang, Yang, & Hwang, 2014a; Pedaste et al., 2017). However, changing teaching practices is proving difficult, as inquiry-based learning is not widely implemented in EU schools (EU Commission, 2017). Research literature indicates a disconnection between calls for reform and actual classroom practice and persistence of traditional, teacher-centred approaches (EU Commission, 2007; Klette, 2009). Numerous studies have asserted that it is much more demanding for teachers to exploit the growing penetration of AR/MR and other digital technologies and their transformative potential in instructional settings than was originally anticipated (McNair & Green, 2016). Thus, the provision of high quality preservice and in-service teacher training that will equip teachers with the required knowledge and skills to effectively infuse AR/MR and other emerging technologies into teaching and learning is of utmost importance (Delello, 2014; Lasica, Meletiou-Mavrotheris, Katzis, Dimopoulos, & Mavrotheris, 2018).

2

Background to the EL-STEM Project

The Enlivened Laboratories for STEM Education (EL-STEM), an Erasmus+ Action 2 program funded by the EU, comes as a solution to the above-mentioned situation focusing on secondary education students and STEM teachers, and the effective integration of AR/MR technologies within the existing school curricula. More specifically, EL-STEM seeks to practically contribute towards meeting the 2020 EU target of reducing the percentage of underachievers in STEM education to below 13% and of motivating a bigger proportion of young Europeans to exhibit interest in STEM and to undertake scientific and technical studies and careers (Mavrotheris, Lasica, Pitsikalis, & MeletiouMavrotheris, 2018). This contribution is expected to be achieved by integrating Augmented Reality (AR), or more generally Mixed Reality (MR), within secondary STEM education to make the subjects more accessible and attractive to children, particularly those at special risk of exclusion from scientific studies and/or careers. The project consortium was intended to include partners originating from both EU countries that have made significant progress in reducing gaps in student achievement, and countries that are still lagging behind (OECD, 2015). A strategic partnership was set by a group of schools and other organisations interested in the promotion of STEM education (universities, research laboratories, local education authorities, and private e-learning firms). During the forming of the consortium, PISA results (OECD, 2016b) were considered. School partners and a university from the high achieving EU countries, Estonia

270

Lasica et al.

and Finland, have been included to provide strong paradigms for reducing the gap between European countries. The rest of the project partners have been selected from EU countries where the scores of low-achievers in science and mathematics are still significantly below the target average, such as Greece and Cyprus, or relatively close to it, such as Portugal. The complementarity of profiles and expertise allows the consortium to maximize the impact of the project and ensure high quality transferable results. In the context of the EL-STEM deliverables, an innovative in-service teacher training program is implemented, that will offer EU secondary school STEM teachers high quality professional development on how to effectively embed AR into instruction. This training provides an innovative methodological framework and related AR/MR learning resources to equip teachers with a wealth of practical experiences and methods of inquiry-based learning that can help foster children’s learning and motivation towards STEM-related studies and careers. AR is an innovative technology and, given its potential, it is already supported by the EU Commission: The Innovation Union-A Europe 2020 Initiative has funded several AR-related research projects in different fields including education (Marten, 2010). To sum up, the project aims at fostering an innovation “ecosystem” that will facilitate an effective and efficient user-centric design via the use of AR/MR resources for personalized STEM learning and teaching. Its priorities include strengthening current STEM curricula in secondary education with innovative methods and tools that can help reverse students’ lack of interest in STEM subjects and raise students’ attitudes and levels of achievement in these disciplines. In parallel, focus is maintained on training and motivating secondary school teachers to effectively integrate innovative technologies with core STEM curricular ideas, so as to transform the classroom into a smart-learning environment.

3

Methodological Framework of the EL-STEM Project: SAM Model

In the context of the EL-STEM project, the SAM (Successive Approximation Model) is suggested as the core framework for the project’s implementation, which refers to an agile development of the teaching method or more specifically, an effective strategy for designing learning events intending to produce greater performance (Allen, 2012). The need for effective management when referring to educational/training product-development projects, including technology-enhanced learning, has been identified in the instructional design literature (Allen, 2006; Van Rooij, 2010). Leadership, estimation of project

Enlivened Laboratories within STEM Education (EL-STEM)

271

requirements and risks, and development of processes and standards for the completion of the project (Li & Shearer, 2005) are only some of the project management skills an instructional designer requires (Stubbs, 2002; Yang, Moore, & Burton, 1995). Similar to the success criteria of any other project, a technology-enhanced learning project could be considered as successful when: (a) it is delivered on time, (b) it does not exceed the initial budget, and (c) it meets the expectations and requirements of the project stakeholders (Crawford & Pollack, 2007; Stubbs, 2002; Van Rooij, 2010). SAM is appropriate for innovative e-learning projects, since it provides the options, strategies and tools to be successful. The stages involved in the SAM process include (Figure 12.1): – Preparation – An initial meeting (locally or online) with representatives from all partners of the consortium to collect basic information about the learning outcomes and necessary actions. – Iterative Design – Designing, prototyping, and evaluating loops during the project’s implementation. – Iterative Development – Evaluation, development, and implementation loops. Probably one of the most notable features of SAM is the preparation phase, which consists of two main steps: (a) gathering information, and (b) holding a brainstorming and prototyping session (savvy start; Rimmer, 2016). Despite being still a new approach in developing educational/training product projects, SAM seems to be a promising and cost-efficient model, due to its adaptivity and agility (Allen, 2012). Moreover, it promotes collaboration as a critical characteristic during all phases of a project, which is very important when referring to consortiums within EU educational/training product projects. Within EL-STEM, which is obviously an e-learning project, there is a general need of setting a methodological framework, to guide the design and implementation of all relevant tangible and intangible outputs and results. As described above, taking into consideration the EL-STEM project background,

figure 12.1 SAM phases (from Allen, 2012)

272

Lasica et al.

figure 12.2 EL-STEM project’s approach based on the SAM model

SAM is appropriate for this purpose. The general framework of the project’s methodology consists of the plan described in Figure 12.2. At the same time, based on the SAM model, all the above-mentioned will be accompanied by an iterative work method, remodelling of the resources and evaluation of the quality of the resources and methods produced.

4

Theoretical Framework of the EL-STEM In-Service Teacher Training Program

The important role of teachers’ preparation in the instructional use of AR/ MR is highlighted in numerous studies (Delello, 2014; Dunleavy & Dede, 2014; McNair & Green, 2016), where it is indicated that many teachers reject new technologies for reasons such as lack of time or motivation for acquiring new technological skills, fear and lack of confidence in their use, lack of existing resources, and failure of materials to align with standards (Delello, 2014; McNair & Green, 2016). Teachers usually do not feel comfortable with the new style of learning provided by innovative technologies including AR/MR, which, with their greater focus on student-guided learning, are unlike more familiar teacher-centred approaches (Delello, 2014). Radical changes in teaching cultures are required to support students’ performance improvement and to reduce disparities in STEM outcomes between different EU countries. Dunleavy and Dede (2014) underlined the need of teachers’ preparation in using AR and the different pedagogical strategies that this task entails. Thus, training that involves applying AR/MR and other innovative technological tools through standards-based pedagogical approaches seems to be a priority.

Enlivened Laboratories within STEM Education (EL-STEM)

273

The theoretical framework underpinning the EL-STEM in-service teacher training program is grounded on and structured under the interrelated bodies of research of (a) Problem-Based Learning, (b) Inquiry-Based Learning, (c) Scaffolding in STEM education, (d) Principles of Adult Education and (e) Recontextualized TPACK framework (Lasica et al., 2018). 4.1 Problem-Based Learning (PBL) PBL is widely used in STEM education with multiple benefits for both students and teachers. In PBL, students need to define conceptual solutions to ill-structured problems and support them with appropriate arguments (Belland, 2017). Success in addressing authentic ill-structured problems within the educational process is possible when students are provided with: (a) appropriate information concerning the problem and (b) instructional scaffolding to extend and enhance their capabilities during their engagement in the learning process (Belland, 2017). Recently, AR appears to play a vital role in PBL, promoting the development of critical thinking and problem-solving skills through authentic activities (Dunleavy & Dede, 2014). 4.2 Inquiry-Based Learning (IBL) IBL is a student-centred approach based on the premise that education begins with the curiosity of the learner (Dewey, 1997), promoting active learning and focusing on questioning, seeking information, critical thinking and problem solving (Savery, 2015). Similar to PBL, IBL is frequently used within STEM education where students practice the scientific method while tackling authentic problems. Learning activities within this approach begin with a core question followed by investigating solutions, while new knowledge is constructed during the process of gathering information and discussing discoveries and experiences (Pedaste et al., 2015). Inquiry is not actually seeking for the right answer (there is usually no unique answer), but rather seeking appropriate resolutions to questions and issues (Pedaste et al., 2015; Pedaste et al., 2016). The role of the teacher in both PBL and IBL is that of facilitator of the whole process (encouraging students, rewarding teamwork, etc.), and of provider of information for the problem under investigation (Pedaste et al., 2016). There are some studies (Chiang, Yang, & Hwang, 2014b) highlighting that AR environments can guide students to share knowledge in IBL activities. 4.3 Scaffolding in STEM Education Scaffolding could be defined as the support provided by teachers, parents, peers or digital tools (Belland, 2017) to fill the gap between what learners could achieve on their own and what they achieve when being guided by others

274

Lasica et al.

(Van de Pol, Volman, & Beishuizen, 2010). The main steps for the implementation of this approach could be: (a) contingency--a teacher’s adaptation to the student’s specific needs and recognition of the cognitive situation; (b) fading scaffolding--a progressive reduction of scaffolding provided to prepare students for independence; and (c) transfer of responsibility from the teacher to the students to achieve the learning objectives (Van de Pol et al., 2010). In the context of AR technology, when referring to the scaffolded IBL approach, the literature indicates better results in learning and achieving high complexity task elements (Crippen & Archambault, 2012). Dunleavy and Dede (2014, p. 1) argue that AR “positions the learner within a real-world physical and social context while guiding, scaffolding and facilitating participatory and metacognitive learning processes such as authentic inquiry, active observation, peer coaching, reciprocal teaching and legitimate peripheral participation with multiple modes of representation.” 4.4 Principles of Adult Education During the design of a teachers’ training program, it is vital to take into consideration the main principles of adult education. There is general acknowledgement of the fact that in adult education: (i) the adults’ own competencies and potential for growth and development should be emphasized while any dependency of adults on teachers should be prevented; (ii) teachers should act as facilitators of learning and help adults learn to teach themselves; (iii) adults should take responsibility for their own learning in instructional settings as they do in everyday life situations; (iv) actual real-life situations are the source and the focus of learning; (v) learning starts in the actual lived-in situation of adults – in workplace settings or in social communities – and aims to develop knowledge and skills that are usable and applicable in these situations; and (vi) learning in informal ways in a meaningful setting is much more effective than learning in traditional classroom settings (Knowles, 1973; Mezirow, 1981). Thus, to make training more relevant and attractive for teachers and to increase their motivation, they should experience it as usable and applicable to their actual classrooms. Dunleavy and Dede (2014) underlined the need of teachers’ preparation in using AR and the different pedagogical strategies that this task entails. 4.5 Re-Contextualized TPACK Framework Building on Shulman’s idea of Pedagogical Content Knowledge (1986), the TPACK framework of Mishra and Koehler (2006) is based upon the premise that effective technology integration for pedagogy around specific subject matter requires developing understanding of the dynamic relationship between all three knowledge components (technological, pedagogical, and

Enlivened Laboratories within STEM Education (EL-STEM)

275

content). TPACK has, in recent years, become central to research into technology education and teacher professional development in many different disciplines, including STEM education. Within the EL-STEM project, the aim is to move teachers beyond technocentric strategies that focus on technology, and to promote their critical reflection on the use of AR/MR technologies in STEM education. Some studies (Phillips, 2013) illustrate the usefulness of TPACK as a research framework for facilitating and assessing teachers’ professional growth in the use of ICT in STEM education, including the utilization of AR/MR technologies as tools for the development of students’ scientific reasoning (Ke & Hsu, 2015). TPACK was recently re-contextualized by Phillips (2016), attending to the sociocultural influences on (a) pedagogical technology practices and (b) identity transformations, adding the key role of the place (school, educational institution, etc.) where the TPACK framework is implemented. The re-contextualized TPACK framework is being applied in the context of the EL-STEM teacher training program.

5

Enlivened Laboratory Methodological Guidelines (ELMG)

The Enlivened Laboratory Methodological Guidelines (ELMG) are primarily addressed to secondary education teachers across Europe on how to apply the methodology of the Enlivened Laboratories (EL-STEM), in order to create their own Lesson Plans (LP) and AR/MR Learning Objects (LOs) within their STEM-related courses. More specifically, they reply to core questions, including: 1. whom they are addressed to and to whom they can be applied, 2. where they could be implemented, 3. why they should be applied within the context of STEM-related disciplines, 4. what approaches and tools could be used, and finally, 5. how they could be generated to teach STEM-related disciplines in secondary education using AR/MR technologies. The ELMG that are already developed, are expected to: – Provide the necessary methodological framework and recommendations for teachers to understand how to: (i) increase European Youth (students 12–18 years old) skills in STEM-related courses and interest in STEM studies and careers, and (ii) make an appropriate use of ICTs and especially AR/MR technologies for this purpose. – Suggest interactive activities for students in Europe with disadvantaged backgrounds who need additional support to be motivated in STEM, as a vehicle of integration, inclusion and prevention of early school leaving.

276

Lasica et al.

– Suggest interactive AR/MR LOs that will be widely available to European teachers and general educators during and after the completion of the project. – Develop a framework and appropriate tools for identifying and assessing the AR competences for both STEM teachers and students. The framework will make reference to the EU framework of Key Competences for Lifelong Learning, and it will be broken down into different levels, according to the age of students. It will take into account the specificities of each national educational system. – Offer suggestions for teachers to create Augmented/Mixed self-evaluation activities to be carried out by STEM students. – Offer instructions and suggestions on how to transfer the approach and successfully implement it in a diverse range of non-formal educational sectors that are also interested in increasing young people’s motivation to STEM (e.g., organisations within STEM disciplines, youth associations, public libraries). The following sections describe the main axes of the ELMG in detail. 5.1

Who: To Whom the ELMG Are Addressed and to Whom They Can Be Applied As already mentioned, ELMG are useful to enhance the teaching and learning processes in the context of STEM-related disciplines by applying AR and MR technologies. The primary target group and key users for these guidelines are secondary education STEM teachers, aiming to or already teaching students between 12–18 years old. These teachers will make up the core of the community of Augmented Reality STEM teachers, who will aim not only to apply the guidelines in their teaching practices but also, share them with others and increase the community range. Valuable characteristics of these teachers could include (but are not limited to) the following: – They are not using Remote/Local Labs within their courses, in order to change their teaching cultures. – They are already using Remote/Local Laboratories, applying innovative approaches within their courses and/or willing to include AR/MR technologies to obtain full benefits of them. – They are aiming to apply innovative methodological approaches and integrate emerging technologies, namely AR/MR, in their teaching practices. Other potential users include third parties such as staff of educational institutions and/or schools, research centres, universities and development partners interested in AR/MR technologies in the field of STEM. The students targeted by these guidelines are secondary education students in the EU and beyond, between 12 and 18 years of age. These students will make

Enlivened Laboratories within STEM Education (EL-STEM)

277

up the core of the community of Augmented Reality STEM students, who will apply AR/MR technologies in their learning and share their experience with students from the other countries of the consortium. The ELMG (when applied by teachers) are expected to: – encourage students’ STEM engagement, – attract students who currently might not be interested in STEM-related studies/careers and enhance the interest of those who have already chosen such fields, – equip students with 21st-century skills through inquiry-based approaches in STEM-related courses, supported by AR/MR technologies, – improve students’ performance in STEM-related courses. While the initial target is EU secondary students, we anticipate that these theoretical principles could be widely applied to other locales and/or age groups. Other potential target groups to whom the ELMG could be applied are younger students, and also adults attending STEM-related community outreach activities and/or programs. 5.2 Where: The ELMG in and out of the Classroom The second important question that the ELMG aims to answer is “where can they be applied?,” referring to the location where a teacher can integrate an AR/MR Learning Object and/or implement all or part of a lesson plan concerning a STEM-related discipline. A strong advantage of the AR/MR technologies, as already mentioned, is the fact that their most common usage is as a layer on top of a smartphone’s and/or tablet’s field of view through its camera. This makes the AR technology easily accessible without spatial constraints. Taking into consideration the inquiry-based phases suggested by Pedaste et al. (2015), which were used as the basis for the development of the “Instructional Design for using MR/AR in STEM learning” guidelines in the EL-STEM project, Table 12.1 describes some possible locations where “Enlivened Laboratories” could be implemented. It is important to note that these are only suggestions; the final decision on where an educational intervention should take place is based on the teacher and could depend on numerous factors, such as the school directory, the parents, the school location and facilities, the curricula, the country’s educational context, etc. Also, the different inquiry-based phases could be implemented in more than one location. 5.3 Why: Reasons to Apply the ELMG Some of the main reasons to apply the ELMG within the educational process based on the existing studies concerning the affordances of this technology are the following (Chen, Liu, Cheng, & Huang, 2017; Wu et al., 2013):

The most common place where ELMG could be applied is a school’s classroom. Teachers could provide their students with “triggers”1 enabling the AR content through their devices in diffferent points (wall, board, etc.) or in diffferent objects (books, notepads, workbooks, library, desk etc.) in the classroom. The classroom could be considered as a safe location to introduce the students to the AR technology for the fijirst time. The main idea of the ELMG is to enliven STEM laboratories. Teachers could provide students with triggers enabling the AR content related to diffferent objects/ equipment (tubes, thermometers, beakers, flasks, models of anatomy or chemical structure, construction materials, etc.). Moreover, access could be provided to remote laboratories supported by AR technologies through a computer device (desktop or laptop). Local school laboratories could be considered as an ideal location to implement inquiry-based STEM learning by using AR/MR technologies, since there is usually a satisfactory stable internet connection while the teacher can use numerous triggers to enable AR/MR content.

Classroom

Laboratory

Description

Location

table 12.1 Where to use AR/MR to Enliven Laboratories in STEM

(cont.)

Orientation Conceptualization Investigation Conclusion Discussion

Orientation Conceptualization Conclusion Discussion

Inquiry-based learning phase

278 Lasica et al.

Description

Out of the class An additional suggestion is to get students out of the class, using mobile devices to investigate real environments and get an authentic learning experience. In this case, it is of critical importance to ensure that a sufffijicient data connection will be available to access the AR/MR content. Teachers could provide students with triggers enabling the AR content through their devices in diffferent locations around the school (e.g., school’s garden). Students could play the role of a real traveller or researcher through activities such as searching for the lost treasure through an experiential learning process. Moreover, out-of-class learning experiences could be implemented in other locations such as museums, zoos, or planetariums. In this case, teachers could prepare their own AR content based on the place to visit or design activities based on the AR/MR content provided by the location to be visited (e.g., most planetariums have already integrated AR/MR experiences in their exhibitions). Using location-based AR/MR educational applications could enrich the educational process, providing additional information to the students. Applying ELMG out of the class could be a good way to promote collaboration in learning. Home Using AR/MR at home is also possible. Teachers could provide students with triggers enabling the AR content through their devices in diffferent objects, such as workbooks, notepads, books. This content could focus on summarizing the inquiry-based process implemented in other locations, providing additional information or guidance on how to complete a project.

Location

table 12.1  Where to use AR/MR to Enliven Laboratories in STEM (cont.)

Conclusion Discussion

Orientation Conceptualization Investigation

Inquiry-based learning phase

Enlivened Laboratories within STEM Education (EL-STEM)

279

280

Lasica et al.

– AR and MR technologies overcome VR’s limitations (i.e., lack of realism) since the real and the virtual elements of the respective environments can be clearly distinguished by students (Johnson & Levine, 2008; Tall, 2017). – Students are engaged in the educational process in ways that were not possible in the past, since AR introduces rich contextual interactive learning environments (Garzón, Pavón, & Baldiris, 2017). – Problem-based and inquiry-based learning are promoted through AR, as students can take control of their learning and follow their own learning paths (Chiang et al., 2014a; Garzón et al., 2017; Klopfer, 2008). – There are already some studies that indicate improved student learning performance when AR technology is implemented during the educational process, especially in the context of science courses (Chiang et al., 2014a; Squire & Jan, 2007). – AR technology and AR games specifically, can be engaging for acquiring 21st century skills (Schrier, 2006). – Since AR is still a very recent technology, it can make the studying experience fun, while students sustain interest and curiosity in using it (Wu et al., 2013). – Simulations of authentic educational activities, including dangerous tasks (e.g., explosions), can be implemented without any consequences on the real environment (Garzón et al., 2017). – Scientific disciplines including abstract or complex concepts that could not be accessed (such as microworlds) or explained, can be explored (Akçayır, Akçayır, Pektaş & Ocak, 2016; Garzón et al., 2017). 5.4 What: Focus on STEM and AA/Mr through the ELMG Having personalized and located the ELMG, as well as having convinced teachers to apply them in their instructional practices, a critical remaining question is “what should teachers know in order to create their own Lesson Plans (LP) and AR/MR Learning Objects (LOs) within their STEM related courses?” There are two main axes here; the first one refers to STEM as an instructional approach and the second one refers to AR and MR as innovative technologies. These concepts are described in the following sections, while existing case studies of AR/MR in STEM education are presented to inspire teachers to come up with new ideas. 5.4.1 STEM-Related Concepts STEM overcomes the strict individual borders of Science, Technology, Engineering, and Mathematics and treats them as a “single whole” (Morrison, 2006). Changing STEM to STEAM (incorporating “Arts”), STEMM (incorporating

Enlivened Laboratories within STEM Education (EL-STEM)

281

“Music”) or STREAM (incorporating both “Reading” and “Arts”), has initialized numerous discussions between experts (Miller & Kimmel, 2012; Sanders, 2009). The fact is that students need a well-rounded education to acquire 21st century skills, which could be achieved through interdisciplinary approaches to real-life problems (Beers, 2011; Mishra & Kereluik, 2011). Real-life problems are composed of many disciplines, so it is not about adding any letter to the STEM acronym, but instead, adding to the relevancy of learning and providing students with skills that will help them apply concepts in a real context (Bertram, 2014). The terms cross-disciplinary, multi-disciplinary, inter-disciplinary, and transdisciplinary are the ones most commonly used in the literature to define the extent of integration and holism in a STEM approach (Jensenius, 2012). A crossdisciplinary approach refers to viewing one discipline from the perspective of another (Jensenius, 2012), without intending integration, but highlighting the aspects of one STEM discipline by means of studying the other. Within a multidisciplinary approach, students work on a common problem but maintain the boundaries of the individual STEM fields (Zhe, Doverspike, Zhao, Lam, & Menzemer, 2010). Each student contributes to the final solution from a different perspective, but, like with cross-disciplinary work, there is no attempt to cross the boundaries and generate integrated knowledge. An inter-disciplinary approach could be considered as a step up from multi-disciplinarity, where students work on a common “real-world” problem but in this case, they overlap disciplinary boundaries to create new knowledge, and jointly frame integrated solutions (Huutoniemi, Klein, Bruun, & Hukkinen, 2010). Finally, trans-disciplinarity is an approach where a problem is faced through sub-problems (Schmidt, 2008) creating a holistic approach by transcending disciplinary boundaries. EU policy makers (OECD, 2015) claim that the demand for STEM skills is expected to increase both in the short and medium term. According to Beers (2011), there is a natural match between the basic tenets of STEM and 21stcentury skills. More specifically, “exemplary science education can offer a rich context for developing many 21st-century skills, such as critical thinking, problemsolving, and information literacy” (National Science Teacher Association, 2011, p. 1). These skills can provide students with life competences, leading them to more in-depth understanding of the subjects under study and allowing them to effectively “combine” the different disciplines under consideration (Binkley et al., 2012). Within the context of the EL-STEM project, STEM is adapted, not simply referring to the individual subjects of the acronym, but providing an engaging and interdisciplinary (or at least multidisciplinary) way of teaching and learning.

282

Lasica et al.

5.4.2 Defining Virtual, Augmented, and Mixed Reality VR, AR, and MR technologies are still not clearly distinguished from each other within the literature, especially when referring to educational applications. Reality-altering terminology can get confusing as it includes concepts that are still fluid and concern different technologies and relative equipment that definitely overlap (Billinghurst, Clark, & Lee, 2015). Virtual Reality (VR) was introduced in education in professions mostly related to medicine and industry about three decades ago (Helsel, 1992; Wickens, 1992), while it became more popular through commonly used educational software such as “OpenSim” and “Second Life” a decade later (Dickey, 2005). Today, AR receives increased attention and interest within the educational process, while MR, which seems to be the future of AR, is not yet widely exploited within the educational field. Taking into consideration the crucial role of laboratories within STEM-related courses, in order to suggest an effective implementation of AR/MR technologies within a school environment, it is important to understand the main differences of these technologies (Billinghurst et al., 2015): – Virtual Reality (VR) is a digital world, fully accessible and exploitable through a computer device that keeps the users mostly isolated from the real world. Nothing is real within this “reality;” everything is virtual (digital). VR is accessible through 3D glasses or simply through a computer device. VR headsets usually require both a computer (to be cabled into) and a separate controller to function. – Augmented Reality (AR) consists of the real world “supplemented” by digital objects with which the users can interact. The objects are obviously not part of the real world; they exist in a layer on top of the existing reality, not integrated into it. A wide range of equipment is used for AR, including headsets, sensors etc., but the most common use is on a smartphone or tablet as a layer displayed on top of the field of view of the device’s camera. The visualization cannot be interacted with as part of the real environment, but only through the smartphone’s screen. – Mixed Reality (MR) is the real world with digital objects integrated, where all the users’ senses are enabled, and it looks as if the content interacts with the real world as real part of it instead of a layer on top of it. Users have the illusion that real and digital form a unity with which they can interact. Since MR refers to the ability to mix digitally rendered objects into the real environment, it needs more specific equipment to interact with, including headsets, sensors and in some cases, even whole rooms where the real and virtual content seem to be a single entity. Unlike in AR, with MR the visuali-

Enlivened Laboratories within STEM Education (EL-STEM)

283

zation can be interacted with as part of the real environment without any mediating device, just the users’ bare hands. Considering the EL-STEM project, it is important that partners and the target groups involved (teachers, students, stakeholders etc.) realize the main differences between these technologies and are able to recognize them. As mentioned above, these technologies overlap, and this is why AR/MR is mentioned within the project’s outputs and results. However, taking into consideration the fact that MR is still in early stages of development and implementation and the costs of adapting such technologies are still very high, the aim of EL-STEM is to integrate AR within the educational process and set the fundamentals for future research and development, concerning MR within education. To sum up, VR could be considered as the past that EL-STEM seeks to go beyond, AR the present that EL-STEM is claiming as an innovative solution and finally, MR the future that EL-STEM is investing in. 5.4.3 Laboratories within STEM Education There are several potential benefits to adapting laboratories in STEM education, including the attraction of students’ interest and the provision of multiple opportunities for the acquisition of practical knowledge (Krneta, Restivo, Rojko, & Urbano, 2016). Within this field, there is a number of studies (Odeh, Abu Shanab, & Anabtawi, 2015; Heradio et al., 2016; Lasica, Katzis, Meletiou-Mavrotheris, & Dimopoulos, 2016) concerning the different types of laboratories depending on their location (local and remote) and type of technology (real, virtual, augmented; Figure 12.3). These studies point to the need of employing a combination of the different types of laboratories during the different phases of an educational process, such as preparation, live lectures that involve experimentation, repetitive experimentation, etc. (Heradio et al., 2016). In addition, there are studies that highlight the potential of remote, virtual and augmented reality labs on the enhancement of the educational process, especially within e-learning and blended learning environments (Radhamani et al., 2014). Bringing AR technologies into local and remote labs within STEM education is an efficient way to achieve better learning outcomes (Radhamani et al., 2014), and to attract students to STEM related fields of study and careers (Krneta et al., 2016). AR labs overcome the issue of misunderstanding differences between real and virtual worlds, however, they are still immature, since they have only recently entered the educational field (Lasica et al., 2016). MR labs could enhance AR ones, adding the dimension of interaction

284

Lasica et al.

figure 12.3 Different types of laboratories presented by Lasica et al. (2016)

both with the objects of the laboratory and the students implementing the experiment. Within this framework, the EL-STEM project designs, develops, pilot tests, and implements local and/or remote “Enlivened Laboratories,” including Open Educational Resources for teachers and students across Europe and beyond to visit, learn, enjoy, and contribute. The aim is to augment the AR/MR experience with rich media content, increasing the engagement within STEM education. The EL-STEM platform is expected to become a dynamic portal integrating new AR/MR content produced by teachers and students. The Enlivened Laboratories will also be accompanied by social communication tools to foster the creation of EU communities of “AR STEM Teachers” and “AR STEM Students.” 5.4.4 Case Studies of AR/MR in STEM Education Since 2010, when Augmented Reality was integrated as a technology into common mobile devices (smartphones, tablets, etc.), there has been an increasing interest in applying AR in the educational process (Akçayir & Akçayir, 2017). This access to AR offers new learning opportunities but, at the same time, creates new challenges for both teaching and learning. There is already a wide range of emerging AR educational applications including skills training, discovery-based learning, AR gaming, modelling objects, and AR books (Bacca et al., 2014). STEM education is a field of high interest when referring to the studies that have already been implemented concerning AR integration within the

Enlivened Laboratories within STEM Education (EL-STEM)

285

educational process (Chen et al., 2017): More than 45% of the existing studies have been implemented in the context of STEM-related courses (Bacca et al., 2014). Since AR is still a technology under development, most AR educational applications are marker-based (Bacca et al., 2014), which is a stable technique. Marker-based AR makes use of visual cues (labels with coloured or black and white patterns) recognized by AR applications, that trigger the display of the virtual information (Park & Park, 2004). These applications usually include AR interactive books (Holley, Hobbs, & Menown, 2016) and 3D models emerging from educational objects such as worksheets, notebooks, AR cards (McGrath, Craig, Bock, & Rocha, 2011) and objects in STEM laboratories (Akçayır et al., 2016). Recently, many science museums and exhibitions of educational interest have included AR experiences to bring their exhibits to life and to engage students in discovering and learning additional information about what they see (Yoon, Elinich, Wang, Steinmeier, & Tucker, 2012). These kinds of applications advance inquiry-based learning as students can retrieve additional information on the content that interests them and interact with 3D models through actions such as rotation, customization, etc. Because of its complexity, marker-less AR is still lagging behind in educational applications. It is based on recognition of the objects’ shapes (for this reason, it is also known as object-based AR) and allows more complex applications of AR, since it overcomes the interactivity limitations placed on the range of images encapsulated within markers (Beier, Billert, Bruderlin, Stichling, & Kleinjohann, 2003). It is expected that marker-less AR will be widely exploited within education in the future (Garzón et al., 2017), promoting AR gaming (Li, van der Spek, Feijs, Wang, & Hu, 2017) and skills training (Chiang et al., 2014b). Moreover, explaining abstract and difficult concepts within the context of STEM-related courses could be enhanced through marker-less AR applications that overcome the restriction of using images encapsulated within markers. There are also some location-based AR educational applications in which the information is superimposed based on the geographical location of the user, detected through the GPS sensor of a mobile device (Bacca et al., 2014), enhancing the interaction with the real mobile learning environment of a student (Kamarainen et al., 2013; Squire & Jan, 2007). These AR applications could be used in wider contexts, including “learning out of the classroom,” providing students with on-the-spot information and additional knowledge. Within the context of the EL-STEM project, an AR game in the field of STEM is being designed and developed for implementation. The aim is to provide

286

Lasica et al.

the “Augmented Reality STEM Teachers” with a Learning Object, mainly in the object-based type of AR, promoting STEM education. This will constitute a case study of AR in secondary education in the field of STEM-related disciplines. 5.5 How: Using the ELMG to Teach STEM Using AR/MR The final critical question to answer is “how teachers could apply the ELMG to create their own AR/MR Learning Objects (LO) and Lesson Plans (LP) within their STEM related courses.” As with the “what” question, there are two main axes here. The first one refers to the pedagogical theories of implementing a STEM approach as well as integrating AR/MR technologies into the educational process. Wu et al. (2013) have highlighted some critical pedagogical issues that need to be taken into consideration when AR technology is integrated into a classroom. The educational activities implemented through AR are mainly based on innovative approaches, such as game-based learning, place-based learning, participatory simulations, inquiry-based learning, problem-based learning, role-playing, studio-based pedagogy, and the jigsaw method. Based on these approaches, Wu et al. (2013) categorize the instructional designs used to integrate AR in the educational process into three main dimensions: (a) emphasizing the “roles,” where students get into a role and collaborate with others to achieve a learning outcome, (b) emphasizing the “locations,” where students interact with the physical environment, getting a sense of authenticity, and (c) emphasizing the “tasks,” where students need to complete different tasks either on their own or in collaboration with others, in a problem-solving situation. Taking these into consideration, in the “Instructional Strategies” part of the LP, the “Instructional Design for using MR/AR in inquiry-based STEM learning” suggested in Intellectual Output 1 (see more about EL-STEM intellectual outputs at: http://elstem.eu/about-us/intellectual-outputs/) could be applied, combining inquiry-based learning with the contemporary learning approach, emphasizing the students’ “tasks” in a problem-based situation. Emphasizing on the students’ tasks within the EL-STEM project, teachers could describe the different tasks of the inquiry-based learning phases, including Orientation, Conceptualization, Investigation, Conclusion, and Discussion. These phases could also be enriched with components of other pedagogical approaches decided by the teacher. The second one refers to the “tools,” including applications and software that could be used to design and create an AR/MR LO.

Enlivened Laboratories within STEM Education (EL-STEM)

6

287

Concluding Remarks

In our contemporary knowledge-based society, the EL-STEM project, in line with the 21st century skills, aims to enhance secondary students’ engagement to STEM-related studies and improve their performance in STEM-related courses, while it provides teachers with a strong package of knowledge and skills to effectively embed AR/MR in teaching and learning processes. The need to prepare teachers for using innovative technological tools and the different pedagogical strategies that these tools entail, underlines the priority of an inservice teachers’ training program. In this context, teachers will develop and implement their own lesson plans and AR/MR Learning Objects, following the Enlivened Laboratory Methodological Guidelines. These guidelines provide a methodological framework to help teachers effectively engage European youth with STEM-related courses and make the appropriate use of ICTs and AR/MR technologies. In addition, the guidelines suggest interactive activities to promote inclusion and offer inputs, in regards with the transferability and implementation of the approach in a diverse range of non-formal educational sectors. The impact, the quality, and the transferability of the results are ensured by the project’s consortium which consists of partners from high and low achieving countries in the European Union. The expected project’s impacts on participating organisations, target groups and stakeholders, both during and after the active project period, are summarized below (Mavrotheris et al., 2018): Teachers’ capacity will be fostered to effectively integrate innovative technologies into teaching and learning through the acquisition of the skills to make optimal use of AR/MR technologies. Increased competence is also expected in dealing with students’ difficulties in mathematics and science and in cultivating their motivation towards STEM-related studies and careers. This will be achieved by coupling the development of theoretical knowledge and the use of “hands on” materials, with laboratory experimentation. In the medium and long term, this impact will be sustained by the availability of multilingual Open Educational Resources (OER) and the creation of communities of “Augmented Reality STEM teachers.” Moreover, the EL-STEM project activities will lead to greater implementation of inquiry-based teaching methods and interdisciplinary approaches as well as to greater responsiveness to the needs of disadvantaged groups through the adoption of an inclusive STEM education model. The EL-STEM project’s innovative, interdisciplinary approach is one in which scientific experimentation in authentic contexts serves as the machinery

288

Lasica et al.

for students to engage in problem solving activities. It is expected to result in enhanced learning experiences with the AR/MR learning materials, methods, and applications; increased motivation; and improved performance of students in STEM subjects. The expected improvements in critical thinking, problem solving, creativity, teamwork, and communication skills will promote the attainment of important 21st century competencies essential in modern society. The project outputs are expected to trigger innovative actions within organizations to better respond to the challenge of increasing students’ achievement in STEM related subjects and reducing disparities in learning outcomes. In the medium term, this should lead to a more motivated, competent, and dynamic staff which supports ongoing innovation. In turn, involved schools will acquire an overall better reputation, due to the ability to improve teaching standards and foster equity in learning achievements. Finally, the project will have a positive impact on numerous organizations, agencies, and groups across Europe. The project partners will actively engage national and European stakeholders during the project activities and will promote wide dissemination of the intellectual outputs.

Acknowledgements The Project EL-STEM was funded by the Erasmus+ Program Key Action 2 (Project No. 2017-1-CY01-KA201-026775) in 2017. A total of nine organizations from five EU member states are participating in the project, and the coordinating institution is the Open University of Cyprus.

Note 1 A “trigger” is usually the element (QR code, marker, object, image etc.) that includes the AR content which can be accessed through the appropriate application of a smartphone/tablet.

References Akçayir, M., & Akçayir, G. (2017). Advantages and challenges associated with augmented reality for education: A systematic review of the literature. Educational Research Review, 20, 1–11.

Enlivened Laboratories within STEM Education (EL-STEM)

289

Akçayır, M., Akçayır, G., Pektaş, H. M., & Ocak, M. A. (2016). Augmented reality in science laboratories: The effects of augmented reality on university students’ laboratory skills and attitudes toward science laboratories. Computers in Human Behavior, 57, 334–342. Allen, M. (2012). Leaving ADDIE for SAM: An agile model for developing the best learning experiences. Alexandria, VA: American Society for Training and Development. Allen, W. C. (2006). Overview and evolution of the ADDIE training system. Advances in Developing Human Resources, 8(4), 430–441. Bacca, J., Baldiris, S., Fabregat, R., Graf, S., & Kinshuk. (2014). Augmented reality trends in education: A systematic review of research and applications. Journal of Educational Technology & Society, 17(4), 133–149. Beers, S. (2011). 21st century skills: Preparing students for their future. Retrieved from https://cosee.umaine.edu/files/coseeos/21st_century_skills.pdf Beier, D., Billert, R., Bruderlin, B., Stichling, D., & Kleinjohann, B. (2003). Markerless vision based tracking for mobile augmented reality. In Proceedings of the Second IEEE and ACM International Symposium on Mixed and Augmented Reality (pp. 258–259). IEEE Computer Society. Belland, B. R. (2017). Instructional scaffolding in STEM education. Cham: Springer International. Bertram, V. (2014). STEM or STEAM? We’re missing the point. HuffPost Education. Billinghurst, M., Clark, A., & Lee, G. (2015). A survey of augmented reality. Foundations and Trends in Human–Computer Interaction, 8(2–3), 73–272. Binkley, M., Erstad, O., Herman, J., Raizen, S., Ripley, M., Miller-Ricci, M., & Rumble, M. (2012). Defining twenty-first century skills. In P. Griffin, B. McGaw, & E. Care (Eds.), Assessment and teaching of 21st century skills (pp. 17–66). Dordrecht: Springer. Chen, P., Liu, X., Cheng, W., & Huang, R. (2017). A review of using augmented reality in education from 2011 to 2016. In E. Popescu, Kinshuk, M. K. Khribi, R. Huang, M. Jemni, N.-S. Chen, & D. G. Sampson (Eds.), Innovations in smart learning (pp. 13–18). Singapore: Springer. Chiang, T. H., Yang, S. J., & Hwang, G. J. (2014a). An augmented reality-based mobile learning system to improve students’ learning achievements and motivations in natural science inquiry activities. Journal of Educational Technology & Society, 17(4), 352. Chiang, T. H., Yang, S. J., & Hwang, G. J. (2014b). Students’ online interactive patterns in augmented reality-based inquiry activities. Computers & Education, 78, 97–108. Crawford, L., & Pollack, J. (2007). How generic are project management knowledge and practice. ERA-Engineering and Environmental Sciences, 22. Crippen, K. J., & Archambault, L. (2012). Scaffolded inquiry-based instruction with technology: A signature pedagogy for STEM education. Computers in the Schools, 29(1–2), 157–173. Delello, J. A. (2014). Insights from pre-service teachers using science-based augmented reality. Journal of Computers in Education, 1(4), 295–311.

290

Lasica et al.

Dewey, J. (1997). How we think. New York, NY: Courier Corporation. Dickey, M. D. (2005). Three‐dimensional virtual worlds and distance learning: Two case studies of active worlds as a medium for distance education. British Journal of Educational Technology, 36(3), 439–451. Dunleavy, M., & Dede, C. (2014). Augmented reality teaching and learning. In J. M. Spector, M. D. Merrill, J. Elen, & M. J. Bishop (Eds.), Handbook of research on educational communications and technology (pp. 735–745). New York, NY: Springer. EU Commission. (2007). Directive 2007/2/EC of the European Parliament and of the Council of 14 March 2007 establishing an Infrastructure for Spatial Information in the European Community (INSPIRE), 25.4.2007. Brussels: European Parliament and Council. EU Commission. (2016). Education and training monitoring 2016. Directorate-General for Education and Culture. Retrieved from http://ec.europa.eu/education/sites/education/files/monitor2016_en.pdf EU Commission. (2017). Science with and for society. Horizon 2020-Work Programme 2016–2017. EU Commission Decision C (2017) 2468. Retrieved from http://ec.europa.eu/research/participants/data/ref/h2020/wp/2016_2017/main/ h2020-wp1617-swfs_en.pdf EU Commission. (2018). Science with and for society. Horizon 2020-Work Programme 2018–2020. Retrieved from http://ec.europa.eu/programmes/horizon2020/sites/ horizon2020/files/h2020-swafs-2018-2020_prepublication_2.pdf Felder, R. M., & Brent, R. (2003). Designing and teaching courses to satisfy the ABET engineering criteria. Journal of Engineering Education, 92(1), 7–25. Garzón, J., Pavón, J., & Baldiris, S. (2017, June). Augmented reality applications for education: Five directions for future research. In Proceedings International Conference on Augmented Reality, Virtual Reality and Computer Graphics (pp. 402–414). Cham: Springer. Helsel, S. (1992). Virtual reality and education. Educational Technology, 32(5), 38–42. Heradio, R., de la Torre, L., Galan, D., Cabrerizo, F. J., Herrera-Viedma, E., & Dormido, S. (2016). Virtual and remote labs in education: A bibliometric analysis. Computers & Education, 98, 14–38. Holley, D., Hobbs, M., & Menown, C. (2016). The augmented library: Motivating STEM students. Networks, 19, 77–84. Huutoniemi, K., Klein, J. T., Bruun, H., & Hukkinen, J. (2010). Analyzing interdisciplinarity: Typology and indicators. Research Policy, 39(1), 79–88. Jensenius, A. R. (2012). Disciplinarities: Intra, cross, multi, inter, trans. Retrieved from http://www.arj.no/2012/03/12/disciplinarities-2/ Johnson, L. F., & Levine, A. H. (2008). Virtual worlds: Inherently immersive, highly social learning spaces. Theory into Practice, 47(2), 161–170.

Enlivened Laboratories within STEM Education (EL-STEM)

291

Kamarainen, A. M., Metcalf, S., Grotzer, T., Browne, A., Mazzuca, D., Tutwiler, M. S., & Dede, C. (2013). EcoMOBILE: Integrating augmented reality and probeware with environmental education field trips. Computers & Education, 68, 545–556. Ke, F., & Hsu, Y. C. (2015). Mobile augmented-reality artifact creation as a component of mobile computer-supported collaborative learning. The Internet and Higher Education, 26, 33–41. Klette, K. (2009). Challenges in strategies for complexity reduction in video studies. Experiences from the PISA+ study: A video study of teaching and learning in Norway. In T. Janik & T. Seidel (Eds.), The power of video studies in investigating teaching and learning in the classroom (pp. 61–82). New York, NY: Waxmann. Klopfer, E. (2008). Augmented learning: Research and design of mobile educational games. Cambridge, MA: MIT Press. Knowles, M. (1973). The adult learner: A neglected species. Burlington, MA: Elsevier. Krneta, R., Restivo, M. T., Rojko, A., & Urbano, D. (2016). Evaluation of remote experiments by different target groups: NeReLa project case study. In Proceedings 13th International Conference on Remote Engineering and Virtual Instrumentation (REV) (pp. 326–331). New York, NY: IEEE. Lasica, I. E., Katzis, K., Meletiou-Mavrotheris, M., & Dimopoulos, C. (2016). Research challenges in future laboratory-based STEM education. Bulletin of the IEEE Technical Committee on Learning Technology, 18(1), 2. Lasica, I. E., Meletiou-Mavrotheris, M., Katzis, K., Dimopoulos, C., & Mavrotheris, E. (2018, March). Designing a teacher training program on the integration of augmented and mixed reality technologies within the educational process. In Proceedings 12th International Technology, Education and Development Conference (pp. 8943–8953). IATED Digital Library. Lee, K. (2012). The future of learning and training in augmented reality. InSight: A Journal of Scholarly Teaching, 7, 31–42. Li, D., & Shearer, R. (2005). Project management for online course development. Distance Learning, 2(4), 19–23. Li, J., van der Spek, E. D., Feijs, L., Wang, F., & Hu, J. (2017, July). Augmented reality games for learning: A literature review. In Proceedings International Conference on Distributed, Ambient, and Pervasive Interactions (pp. 612–626). Cham: Springer. Marten, H. (2010). Evaluating media literacy education: Concepts, theories and future direction. Journal of Media Literacy Education, 2(1), 1–22. Mavrotheris, E., Lasica, I. E., Pitsikalis, S., & Meletiou-Mavrotheris, M. (2018). Project EL-STEM: Enlivened laboratories within STEM education. In Proceedings 12th International Technology, Education and Development Conference (pp. 9099–9107). IATED Digital Library. McGrath, R. E., Craig, A. B., Bock, D., & Rocha, R. (2011). Augmented reality for an ethnobotany workbook. Technical Note, Institute for Computing in the Humanities, Social

292

Lasica et al.

Sciences and Arts (I-CHASS), University of Illinois, Urbana-Champaign. Retrieved from http://hdl.handle.net/2142/27715 McNair, C. L., & Green, M. (2016). Preservice teachers’ perceptions of augmented reality. Literacy Summit Yearbook, 74–81. Retrieved from http://www.texasreaders.org/ uploads/4/4/9/0/44902393/2016_literacy_summit_yearbook.pdf Mezirow, J. (1981). A critical theory of adult learning and education. Adult Education, 32(1), 3–24. Miller, J. D., & Kimmel, L. G. (2012). Pathways to a STEMM profession. Peabody Journal of Education, 87(1), 26–45. Mishra, P., & Kereluik, K. (2011). What 21st century learning? A review and a synthesis. In Proceedings of Society for Information Technology & Teacher Education International Conference (pp. 3301–3312). Mishra, P., & Koehler, M. J. (2006). Technological pedagogical content knowledge: A framework for teacher knowledge. Teachers College Record, 108(6), 1017. Morrison, J. (2006). TIES STEM education monograph series: Attributes of STEM education. Baltimore, MD: TIES. National Science Teacher Association. (2011). Quality science education and 21st-century skills. Retrieved from http://static.nsta.org/pdfs/PositionStatement_21stCentury.pdf Odeh, S., Abu Shanab, S., & Anabtawi, M. (2015). Augmented reality internet labs versus its traditional and virtual equivalence. International Journal of Emerging Technologies in Learning, 10(3), 4–9. OECD. (2015). EU skills Panorama 2014 – STEM skills analytical highlight. Organisation for Economic Co-Operation and Development Publications. OECD. (2016a). Education at a glance 2016. Retrieved from https://www.oecd.org/ education/skills-beyond-school/education-at-a-glance-2016-indicators.htm OECD. (2016b). PISA 2015 results in focus. Retrieved from https://www.oecd.org/pisa/ pisa-2015-results-in-focus.pdf Park, H., & Park, J. I. (2004, November). Invisible marker tracking for AR. In Proceedings of the 3rd IEEE/ACM International Symposium on Mixed and Augmented Reality (pp. 272–273). IEEE Computer Society. Pedaste, M., Lazonder, A., Raes, A., Wajeman, C., Moore, E., & Girault, I. (2016). Grand challenge problem 3: Empowering science teachers using technology-enhanced scaffolding to improve inquiry learning. In J. Eberle, K. Lund, P. Tchounikine, & F. Fischer (Eds.), Grand challenge problems in technology-enhanced learning II: MOOCs and beyond (pp. 17–20). Cham: Springer International Publishing. Pedaste, M., Mäeots, M., Siiman, L. A., de Jong, T., van Riesen, S. A. N., Kamp, E. T., Manoli, C. C., Zacharia, Z. C., & Tsourlidaki, E. (2015). Phases of inquiry-based learning: Definitions and inquiry cycle. Educational Research Review, 14, 47–61.

Enlivened Laboratories within STEM Education (EL-STEM)

293

Pedaste, M., Must, O., Leijen, Ä., Mäeots, M., Siiman, L., Kori, K., & Adov, L. (2017). Profiles of students who use mobile devices for the purposes of learning science and mathematics. Estonian Journal of Education, 5(1), 99. Phillips, M. (2013). Investigating in-service teachers’ workplace TPACK development. Australian Educational Computing, 28(2), 21–31. Phillips, M. (2016). Re-contextualising TPACK: Exploring teachers’ (non-)use of digital technologies. Technology, Pedagogy and Education, 25(5), 555–571. Radhamani, R., Sasidharakurup, H., Sujatha, G., Nair, B., Achuthan, K., & Diwakar, S. (2014). Virtual labs improve student’s performance in a classroom. In International Conference on E-Learning, E-Education, and Online Training (pp. 138–146). Cham: Springer. Rimmer, T. (2016). An introduction to SAM for instructional designers. Retrieved from https://community.articulate.com/articles/an-introduction-to-sam-forinstructional-designers Sanders, M. E. (2009). STEM, STEM education, STEMmania. The Technology Teacher, 68(4), 20–26. Retrieved from https://vtechworks.lib.vt.edu/bitstream/handle/10919/ 51616/STEMmania.pdf?sequence=1&isAllowed=y Savery, J. R. (2015). Overview of problem-based learning: Definitions and distinctions. In A. Walker, H. Leary, C. E. Hmelo-Silver, & P. A. Ertmer (Eds.), Essential readings in problem-based learning: Exploring and extending the legacy of Howard S. Barrows (pp. 5–15). West Lafayette, IN: Purdue University Press. Schmidt, J. C. (2008). Towards a philosophy of interdisciplinarity. Poiesis & Praxis, 5(1), 53–69. Schrier, K. (2006, July). Using augmented reality games to teach 21st century skills. In ACM SIGGRAPH 2006 Educators program (p. 15). ACM. Shulman, L. S. (1986). Those who understand: Knowledge growth in teaching. Educational researcher, 15(2), 4–14. Squire, K. D., & Jan, M. (2007). Mad City mystery: Developing scientific argumentation skills with a place-based augmented reality game on handheld computers. Journal of Science Education and Technology, 16(1), 5–29. Stubbs, S. (2002). How did you manage to do that? An instructional multimedia production management process. Journal of Interactive Instruction Development, 14(4), 25–32. Tall, T. (2017, October). Augmented reality vs. virtual reality vs. mixed reality – An introductory guide. Retrieved from https://www.toptal.com/designers/ui/augmentedreality-vs-virtual-reality-vs-mixed-reality Van de Pol, J., Volman, M., & Beishuizen, J. (2010). Scaffolding in teacher–student interaction: A decade of research. Educational Psychology Review, 22(3), 271–296.

294

Lasica et al.

Van Rooij, S. W. (2010). Instructional design and project management: Which competencies are which? In M. Simonson (Ed.), 33rd Annual Proceedings of the Association for Educational Communications and Technology (AECT) (Vol. 1, pp. 249–258). Wickens, C. D. (1992, October). Virtual reality and education. In IEEE International Conference on systems, man and cybernetics (pp. 842–847). New York, NY: IEEE. Wu, H. K., Lee, S. W. Y., Chang, H. Y., & Liang, J. C. (2013). Current status, opportunities and challenges of augmented reality in education. Computers & Education, 62, 41–49. Yang, C. S., Moore, D. M., & Burton, J. K. (1995). Managing courseware production: An instructional design model with a software engineering approach. Educational Technology Research and Development, 43(4), 60–70. Yoon, S. A., Elinich, K., Wang, J., Steinmeier, C., & Tucker, S. (2012). Using augmented reality and knowledge-building scaffolds to improve learning in a science museum. International Journal of Computer-Supported Collaborative Learning, 7(4), 519–541. Zhe, J., Doverspike, D., Zhao, J., Lam, P., & Menzemer, C. (2010). High school bridge program: A multidisciplinary STEM research program. Journal of STEM Education: Innovations and Research, 11(1–2), 61.

CHAPTER 13

Augmented Playgrounds: Questioning Simulations to Question Intuitions Chronis Kynigos, Zacharoula Smyrnaiou and Marianthi Grizioti

Abstract Pedagogical design principles of augmented reality games are discussed, including young peoples’ invocation of meanings and intuitions as they engage in collaborative play. Three games designed to expose students to intuitions formed by physical and virtual phenomena served as tools to consider potential pedagogical value in kinaesthetic engagement with virtual and physical game components. Our study showed that students prioritize physical movement to play the game but generate scientific language to explain their strategies. They alternate between physical and virtual phenomena when employing and questioning intuitions in order to form play strategies. We discuss the potential of designing games for students to purposefully question their intuitions.

Keywords AR games – meaning making – intuitions – simulations

1

Introduction

This chapter examines situations involving physical movement in a synchronous collaborative Augmented Reality context in order to interact with a virtual simulation. We discuss the pedagogical reflections underlying our design emphasis which were given to: – the type of physical movement as a means of expressing thoughts and meanings in time-critical situations – the embedded concepts in the interactions between bodily movement and the virtual simulation

© koninklijke brill nv, leideN, 2020 | DOI: 10.1163/9789004408845_013

296

Kynigos et al.

– the embedded concepts in the behaviours, properties, and interactions (such as collision) between the virtual objects as well as the influences of the simulated field they reside and move in – the possible verbal and bodily interactions of the learners as they collaboratively try to interact with the virtual environment directly or in competition with another group of learners We explore the pedagogical principles and parameters involved in such Augmented reality interfaces in our shift of attention from computer-based learning to embodied, collaborative learning in augmented game settings (Lee, 2012; Wu, Lee, Chang, & Liang, 2013). In this context, we discuss physical interactions like language, gestures, and full-body motion, as an integral means through which students can express thoughts and meanings while they interact with a set of digital games that are based on motion recognition. Moreover, we present specific factors which have been taken into consideration in the process of designing these games, in terms of mathematical and scientific meaning generation. Thus, our interest was in the role that language would play in this multi-semiotic, mixed-reality environment involving body motion. In addition, we were interested in understanding what meanings students developed through body-movement, gesture, language, and interaction with the games’ digital representations and how these meanings were related to the mathematical and scientific concepts embedded in the games. From that perspective, our aim was to study students’ physical exploration of concepts and systems by moving within and acting upon a digital environment. Finally, we explore the idea of achieving collaborative learning through the concept of augmented playgrounds (Kaufmann, 2003).

2

Theoretical Framework – Intuitions Revisited

There has been important research illuminating the notion of intuition in science (diSessa, 1983; Fischbein, 1987) and mathematics (Burton, 1999; Tall, 1991). In science, intuitions are perceived as persistent naïve experiential explanations of physical phenomena generated mainly at an early age as children become engaged with or interact with the phenomenon themselves. Researchers have studied the process of what happens to intuitions once they are formed or once they are invoked to explain phenomena and how they resist change with age as children and young learners immerse themselves in experiences contradicting intuitive understandings with phenomena that are more complex. In such a situation, learners need different kinds of explanations or perspectives than what these intuitions allow them to invoke.

Augmented Playgrounds

297

DiSessa (1983, 1988) discussed the process of questioning one’s own intuitions as they are challenged, the way they persist, and the way the persistence gradually subsides as we become more and more abstract and logical in addressing our experiences and explaining the phenomena we are engaged in. This fading-out process is inversely proportional to the importance of an intuition and its resistance to questioning. Most often, an intuition does not cease to exist, rather, it will remain, influencing one’s explanatory process but will be taken into consideration alongside progressively more alternative possibilities stemming from scientific, logical thought (Smith, diSessa, & Roschelle, 1993). Beth and Piaget talk about this fade-out phenomenon: “Although effective at all stages and remaining fundamental from the point of view of invention, the cognitive role of intuition diminishes during development … formalization … progressively limits the field of intuition” (Beth & Piaget, 1966, p. 255). So, for science education, students should be supported in the process of interpreting and explaining phenomena by means of a scientific abstract approach developing a readiness to question their intuitions and reducing the importance with which they are evoked. Pedagogically principled exposure to physical phenomena or augmented reality phenomena designed to challenge intuitions may influence scientific interpretations to become more readily invoked by learners. The way mathematicians and mathematics educators have perceived intuition and its status and role in mathematical thinking and learning is surprisingly different (Zaree, 2010). In an excellent paper analysing interviews with mathematicians, Burton (1999) reports that they perceived intuition to be an incomplete thought process, the opposite of rigor, convincing only in the absence of proof. On the other hand, intuition was considered valuable for mathematical thinking and creativity, for example the way connections and new ideas are made, a kind of feeling for a solution to a problem before rigor and proof are applied (Rhodes & Wellman, 2013). In contrast to the perspective from science, intuition in mathematics is perceived as growing with experience and necessary yet not sufficient for mathematical thinking (Wilder, 1984) – something which is missing from mathematics education but which can and should be cultivated and taught in school. So, for mathematics educators, intuition is welcome and should come along with or after a learner is able to engage in rigorous mathematical processes. In this study, we take an interdisciplinary lens and openly discuss the value of visiting the idea of intuitions again in a world where children and learners encounter virtual phenomena as well as physical phenomena in learning mathematical problems. Students grow up exposed to and even immersed in different kinds of phenomena, including virtual phenomena in digital worlds

298

Kynigos et al.

involving digital simulations of objects. Ubiquitous communication affordances involve learners in discourse about these digital simulations and games finding themselves in collectives with different roles and sharing their experiences. (Dede, 2010). So, students are faced with a world where intuitions are built regarding both physical and virtual phenomena and where mathematical intuitions may be put to use to explain and to solve problems in both of these terrains. It goes without saying that digital phenomena may embed rules and behaviours which are at odds with or even contradictory to physical phenomena, which can be anything from confusing to even physically dangerous for a young person. Adopting an educational perspective, specially designed digital simulations can be built with an aim to support children to question their own intuitions and to enrich and enhance the process of the fading out of intuitions regarding physical or virtual phenomena. Students using the simulations can thus find themselves being sensitive to invisible things around the phenomenon by visualizing or manipulating the digital representations therein, especially in the case when there are diverse interdependent representations. Students may thus immerse themselves in different cases of the same phenomenon with different embodiments, being able to use and observe dynamic measurements during the evolution of the phenomenon and to play with time – to pause it, slow it down, and speed it up – and also to use simulations as objects over which students engage in scientific experimentation and engage inquiry learning and practice in having a scientific perception of the world.

3

Questioning Simulations

Let us call these digital or augmented reality contexts ‘questionable simulations,’ simulations, that is, where it’s not a given that they constitute an accurate representation of an existing physical phenomenon (diSessa, 2013; Linn, Chang, Zhang, & McElhaney, 2010). Maybe the simulation and the way it’s mediated to the students purposefully represents some real phenomena poorly. Maybe the simulation represents a virtual phenomenon, not a real phenomenon, and the point of the exercise is to think about the differences between the real phenomenon, the physical phenomenon, and the virtual phenomenon. In such a case, for example, the exercise could be communicated to learners as follows: Take the simulation of a virtual phenomenon and change the rules so that it is simulates a physical phenomenon. What do you need to think about, what are the aspects of one phenomenon in relation to the other?

Augmented Playgrounds

299

The object of the exercise would be to question the behaviours of the object in the simulation, their relationships, and the field properties by experimentally running the simulation to see what would happen (Sherin, 2001). For instance, what if gravity did not point to the centre of the earth, or what would happen if the displacement of an object in free fall was not given by vi x t + 1/2 x g x t2 and there was some other kind of function explaining it? Students could try out these things to see whether they are unnatural or not (Parnafes, 2007). So, the consideration of the potential formation or questioning of intuitions in Augmented Reality (AR) designs such as the ones discussed in this chapter is interesting. Students’ experiences with phenomena beyond the physical world co-exist with phenomena from the physical world. The intuitions generated in this newly complex and confusing situation are also formed in a world which is becoming more and more dense in communication. Students are frequently finding themselves in collectives in different groups and in exchanging experiences through digital media and mediating these artefacts in a way that was unprecedented. We thus need to help students to find a language in which to do this and to be able to handle what happens when they see that phenomena and artefacts cannot be explained easily. So, our emergent considerations for intuitions is to study this ‘fade out’ process, and, at the same time, to study a world where, because there is such a plurality of all these diverse kinds of phenomena, there’s also a continual production of new intuitions. 3.1 Designing Simulations Affording Their Questioning An avenue we propose for addressing the above issues is the design and implementation of augmented reality game interfaces with the full-body recognition technology. These are game settings that are implemented in a physical space enhanced with digital elements and representations. In contrast to device-based AR applications, they are based on Natural User Interfaces which means that the player interacts with the game environment in a natural way by performing physical actions of the human body (talking, jumping, walking, gesturing etc.) without using any external device. The technology of the games integrates a variety of sensors (full body recognition, gesture recognition, weight tracking, etc.) in order to enable the body controlled functionality. In addition, the games we explore are also collaborative games in the sense that they cannot be played by only one person. Their gameplay usually requires 4–6 participants who will play cooperatively or competitively, in the same augmented physical space. The games stimulate physical activity, and encourage social interaction, creating a fun experience. From a pedagogical perspective, in our design we tried to bring and adapt the theories of embodied learning and game-based collaborative learning to the field of augmented reality, aiming to create an interactive augmented

300

Kynigos et al.

environment for students to generate meanings and develop fundamental skills in a fun and playful context. Before we present the three case studies of the games, we provide a brief description of the integration of learning theories into our designs.

4

Designing Augmented Learning Environments Based on Embodied Theory

The use of digital games (Kynigos, Smyrnaiou, & Roussou, 2010; Resnick, 1998) embedding kinaesthetic or tangibles interfaces in the learning process is nothing new (Antle et al., 2009) albeit not widespread. There have been some studies that focus on children’s use of gesture where acting, interacting, and transforming tangible artefacts affects digital representations (Nemirovsky, Tierney, & Wright, 1998) and cognitive schemata. Kinaesthetic interaction technique and its applications in the learning domain appear also with design frameworks for embodied interaction and embodied learning. Learners as embodied persons-in-the-world pose a clear sense of being, of location in a socio-material world in time-space. Being-in-the-world is predicated upon embodied beings in the world that act intentionally; it opens up the possibility of ‘being there’: a sense of personal presence in some place at some time and the possibility of co-presence with co-located others. This feeling of the mutual presence of another is part of our everyday experience. Embeddedness also transfers to technology-mediated experiences in virtual environments (Chee, 2007). The theoretical framework of embodied learning has lately affected humancomputer interaction (HCI) and the design of modern game-based devices as most human-machine interactions have begun to take into account the embodied impact and the phenomenology as shown by modern inquiries which we take into account in the design of digital environments. A review of research on this subject indicates that modern inquiries are focused : (1) on user’s daily experience and introduction of the concept ‘somaesthetics’ as a design approach of technological environments, (2) on the role of user’s emotions and user’s interaction with new technologies (affective computing), (3) on concepts of common experience and embodied gesture for the development of a tangible interface, (4) on shareability as a design principle, (5) on the role of embodied metaphors and their exploitation in HCI, (6) on the concept of ‘mounted action,’ (7) on the case of the extended mind (Extended mind), and (8) on the case of tangible objects (for instance, mathematical manipulatives) and others. The focus on the user’s daily subjective experience with technology provides privileges to games, to enjoyment, to fulfilment, and to sensational

Augmented Playgrounds

301

quality as described in the research of McCarthy and Wright (2004). Technology environments that support embodied and embedded interactions provide access to person-in-the-world experiences. Such environments allow people to actively experiment by observing and abstracting, following a reflective observation cycle of experiential learning. In embodied learning, the cognitive comprehension of mathematical or scientific concepts produces complex dynamic activities (bodily actions, gestures, handling of materials, or design and planning) and the kinaesthetic activities play a significant role. Conceptual metaphor, conceptual blending, etc. appear as if they are based on a basic and innate cognitive mechanism that is activated for the construction of concepts. Via these embodied cognitive mechanisms, the inductive drawings that emanate from the experience can extend in very concrete and precise ways and give a reason for new resulting inductive organization in more abstract sectors (Lakoff & Núñez, 2000). The idea transcending the three games in this chapter has been to put into practice these theoretical principles in order to form a test-bed for exploring and playing with digital extensions to realistic games where representations and interactions embedded scientific concepts and powerful ideas (Papert, 1980).

5

The Concept of Augmented Playgrounds

As mentioned before, collaboration is a fundamental skill for today’s students and it is quite important for an educational environment to promote and cultivate it. Kaufmann (2003) has studied and presented the potential benefits of collaborative Augmented Reality in learning. In these designs, multiple users may access a shared space populated by virtual objects while remaining grounded in the real world. We argue that the embodied perspective strongly supports participatory and collaborative modes of learning where knowledge is viewed in terms of the capacity for intelligent behaviour rather than the possession of any mental ‘thing’ (Rogoff, 1993). In our design, we implement the concept of interactive playgrounds as settings that promote collaboration and communication between children. An interactive playground can stimulate and motivate children to move and play together. It is designed as part of a public space, such as playgrounds, parks, or recreation areas. Such an environment is freely accessible by a child or a group of children on their own initiative, which is essential in order to make for an alternative leisure activity. Playful experience in an interactive playground can be described through the relationships between the player’s actions and the reactions of the system. Players’ actions are expressed by cognitive

302

Kynigos et al.

psychological, physical and emotional dimensions. System reactions are specified through the rules of the games. The relationships between players’ actions and the system reactions are within a cultural framework which specifies the player’s attitudes and prepositions to the playful experience (Figure 13.1). Culture includes the larger contexts engaged with and inhabited by the system (Salen & Zimmerman, 2003).

figure 13.1  Description of playful experience

Based on the idea of interactive playgrounds, we designed games to be played with multiple players in the same physical space. In addition, they stimulate physical activity, and encourage social interaction, creating a fun experience. The rules are often few and simple, easily learned by players of all age and knowledge backgrounds. Thus, each game provides an interactive area where any student can simply ‘jump in’ and immediately get involved and interact with the game without any preparation necessary. This forms an augmented game space where collaboration and learning occur as physical results of playing, like in physical playgrounds. To further elaborate on this approach, we discuss student views after playing with three different types of games which we called ‘Apples,’ ‘Wobble Board’ and ‘Air Hockey.’ They are all based on body recognition, however, they differ in both their modes of interaction (use of body, including gestures) and in the scientific concepts and powerful ideas they implement.

6

Context and Methodology of the Research

The main purpose of the conducted research was to examine the generation of scientific concepts from students through their interaction with the ‘Apples,’ ‘Wobble Board’ and ‘Air Hockey’ augmented games. We wanted to explore how

Augmented Playgrounds

303

the designed interactions of each game along with its educational characteristics affect the meaning that students construct around scientific concepts. The research and the analysis were also focused on the five elements which are fundamental for designing students’ interaction with game implementations: (1) the embodied interaction through motion and speech, (2) the play strategies formed by the students, (3) the concepts they perceived to be embedded in the digital game, (4) the principles behind the design of the game, and (5) the collaboration between students. Thus, the research tried to answer the following questions: – How a brief interaction of students with the three games could lead them to generate scientific meanings. – Which were the factors that may affect the generation of meanings and if these factors were perceived by the students. – To what extent the integration of the five elements of augmented game design mentioned above affects the generation of scientific meanings. For the data collection, we followed the technique of participatory observation along with structured interviews focused on students’ interactions with each game. In addition, we designed a questionnaire for each game that was given to the students. The questionnaires included both open and closed questions focused on 4 main elements: (1) The strategies that the students followed during the game in order to win; (2) The kinetic and verbal interactions both between students and between students and the game; (3) The scientific concepts of the game; and (4) Collaboration and students’ role in the team. We also created a number of codes and analysis categories according to the research questions which we used to analyse the collected data. For each of the three games, we subsequently begin with a brief description of its rules and the scientific concepts it endorses. Then we present the results based on the questionnaires and on the observation notes. 6.1 Case Study 1: Τhe ‘Apples’ Game In the presented research, 47 students from different schools played the game ‘Apples’: 30 girls and 17 boys, aged 12–15 years old. In this section, we first present the main rules and scientific concepts of the game and then the results according to the research questions. The ‘Apples’ game is one of four full-body games based on shadow recognition and one of two integrating concepts from science and mathematics. In this game, the players try to avoid the graphical shapes they see on the big screen coming from the right-hand corner of the screen towards their body shape which is also depicted on the screen. The way to do this is to either duck

304

Kynigos et al.

figure 13.2 Players trying to avoid the shapes moving at different heights in the game ‘Apples’

or jump at the right moment (Figure 13.2). The game, however, is not a classic Wii-style game where one individual interacts with the computer or two take turns. Three players try to avoid the objects in sync, and if an object ‘touches’ one player, they all miss a point. The players need to thus negotiate their positions, their strategy, maybe devise a management structure (e.g., one player calls out to the others what’s coming or what to do). 6.1.1 The ‘Hidden’ Scientific Concepts in the Game ‘Apples’ The concepts hidden behind this game are related to Newtonian mechanics and specifically, movement, speed, position, time, and whatever results from them. There are three concepts denoted by the same word: (1) the entity ‘Time,’ with the meaning of dimension, (respectively we have the concept of Space), that constitutes a component of the Universe; (2) the time variable, (respectively we have the concept of position), that determines at which instant the body is found when it is found at a specific place; and (3) the physical magnitude ‘time (t),’ (respectively the concept of motion) that characterizes the physical phenomenon of a body movement and determines its time duration. The position is the geometrical variable that determines at which point a body that makes a movement in a particular orbit is found. Furthermore, a body movement is characterized by an initial time instant, an old random time instant (i.e., in the past), a new random time instant (i.e., at present or in the

305

Augmented Playgrounds

future), etc. The same applies to the position, which must not be confused with displacement. The scientific definition of the concept ‘motion’ states that it is the continuous change of position of a body (in relation to another body that is arbitrarily considered fixed—the reference body). The question of which body is to be considered fixed has been of concern in Physics from early times. Finally, the General Theory of Relativity showed that in the Universe there is no absolutely fixed body and thus movement is always relative and its cause is an effect. 6.1.2 Results Concerning Strategies To the question which two strategies they would use if they played again (Table 13.1), we notice that the majority of the students’ answers (30%) focused on their body actions (Jump as high as possible). In addition, the percentages of the students who based their strategies on the scientific concepts of the game are rather high. Specifically, they claimed that they would focus on the height of the moving brick (16.25%), their reaction time (13.75%) and the exact moment they should move. If we examine the results according to which concept (height, time) affected the strategies of the students, we will notice that approximately 46% refer, consciously or unconsciously, to height, while 34% refer to time. If we combine the answers depending on whether the strategies include physical magnitudes regarding the student or physical magnitudes regarding the brick, we will notice that 58% refer to magnitudes regarding themselves and 21% to magnitudes regarding the brick. To the question ‘if the brick moved at medium height on the screen, which two strategies would you implement most often?,’ we noticed the students who gave no answer increased (32.25%), while the percentage of the students who table 13.1  Strategies you would implement if you replayed the game ‘Apples’

Strategies Jump as high as possible/duck as low as possible Focus on the height at which the bricks moved. Estimate the time from the instant each brick appeared. Estimate how much time we needed to react. Focus on the instant moment we must move to avoid the collision Other strategy – Write which. No answer No strategy

Answers

%

24 13 4 11 12 0 10 6

30 16.25 5 13.75 15 0 12.5 7.5

306

Kynigos et al.

described their actions (20.96%) and the percentage of the category ‘other strategy’ (19.35%) remained quite significant. The results of the students’ answers, related to the strategies they would implement if they played again, give us indirect information, as we have seen, about the way they perceived the ‘hidden’ concepts, their cooperation, as well as their image as persons in the game (cognitive and psychological beings) and as members of a team that competed with the same goal. 6.1.3 Results Related to the Scientific Concepts Although from the students’ answers regarding the strategies they implemented, we can come to indirect conclusions related to the way they perceived the ‘hidden’ scientific concepts, we also present the students’ answers to the questions regarding directly the scientific concepts of the game. 6.1.3.1 Movement of the Bricks Most of the students perceived the bricks’ movement as changing (Table 13.2) with some explanations of that answer being ‘they move continuously and everywhere,’ ‘other times up, other times down’ or ‘speed changes.’ Students who answered that the bricks make a linear movement they argued that ‘time is proportional to distance,’ ‘they continuously move at the same speed,’ ‘depending on the level, the bricks move faster or slower but at a steady speed’ or ‘to be easier.’ They gave answers that were sometimes contradictory to each other, for example, they discussed a smooth straight movement and at the same time as explaining that that the brick moved in a straight line because its speed changed (i.e., it accelerated). To the question, which of the magnitudes changes when the player tries to avoid each brick, the students chose variable magnitudes as shown in Table 13.3. Specifically, we point out some significant percentages like 20.15% table 13.2  The type of movement made by them in the game ‘Apples’

Type of movement Accelerating Linear Changing Other movement No answer

Answers

%

9 6 21 4 7

19.15 12.77 44.68 8.51 14.89

307

Augmented Playgrounds

table 13.3  The physical magnitudes which change when the player plays the game ‘Apples’

Magnitude Your speed The speed at which the brick moves The reaction time The movement time of the brick The movement time of the brick from the moment you make the decision to move in order to avoid it. The height of your jump The height at which the brick moves The distance that the brick covers No answer

Answers

%

27 21 23 11 7

20.15 15.67 17.16 8.20 5.22

15 11 8 11

11.19 8.20 5.97 8.20

of the students’ answers refer to the magnitude of speed at which they themselves moved, 17.16% refer to the response time, 15.67% refer to the speed at which the brick was moving, and 11.19% refer to the height of the jump. If we combine these answers, we will see that approximately 36% refer to the magnitude of speed, 30% to time, and 25% to distance. If we combine these answers, depending on whether they give importance to physical magnitudes regarding themselves or to physical magnitudes regarding the brick, we will notice that 48% refer to magnitudes regarding themselves and 43% to those regarding the brick. 6.1.4

Results Related to the Interactions among the Students and between the Students and the Game To the question which movements/gestures they themselves or their co-players used to communicate with each other, their answers vary, as shown in Table 13.4. Specifically, they used adverbs of place (up, down, left, etc.) that show the orientation of the movement (7.84%); words that denote parts of the body (arms, legs, head; 19.6%); words, nouns or verbs, that denote an action or a command to their co-player (push, jump, etc.; 7.84%); while the enormous majority of the students gave no answer (62.74%). From the movements/gestures that the players made in order to communicate with each other, we notice that none of them refers directly to the concepts of speed, time, height, or distance. We could say that approximately 6% refer indirectly to height (jump, duck, up, down), 4% to position (left, right). Of course, 6% swore or pushed because their

308

Kynigos et al.

table 13.4  Movements/gestures among students in the game ‘Apples’

Communicating through movements/gestures Up, down Left, right Swear words Arms Legs Head Pushing Jump, duck No movement/gesture No answer

Answers

%

2 2 1 6 2 2 2 1 1 32

3.92 3.92 1.96 11.76 3.92 3.92 3.92 1.96 1.96 62.74

co-player hindered them from executing the moves they wanted to make and were related to spatial criteria. From their communication through movements/ gestures, we could also draw evidence related to collaboration or to the design principles of this game as in the case of verbal communication. For example, the adverbs of place or the ‘lively’ behaviours (13.72%) are related to spatial characteristics or to the design principles of the game, the movements of the parts of the body to the embodied interaction (19.6%) and the imperative verbs or expressions to the kind of collaboration and the players’ role in the team (1.96%). 6.1.5 Aggregate Results for the Game ‘Apples’ We recorded and analysed the research data we collected for the game ‘Apples,’ during and after the few minutes played by the students, according to 4 axes, which are: the strategies they implemented, the ‘hidden concepts,’ their role in the game as member of a team, and the interaction that took place. Regarding the strategies, the results showed that it is easier for the students to duck as low as possible than to jump as high as possible. The percentages regarding their answers in relation to the concepts of time and height are similar, while the majority of the students refer to magnitudes that concern themselves. Regarding the hidden concepts, to the question which of the magnitudes change when the player tries to avoid each brick, 36% refer to the magnitude of speed, 30% to time, and 25% to height. If we combine the answers depending on the importance they give to physical magnitudes concerning themselves or to physical magnitudes concerning the brick, we notice that the percentages are similar, 48% and 43% respectively.

Augmented Playgrounds

309

This game urged students to combine the physical movements that they perform in space with the digital ones performed by the brick in one common context. This led them to apply and explore scientific concepts, such as speed and collision of two objects, in an experiential way. In addition, the game setting seems to have the potential of promoting students’ in-game discussions on these concepts as a strategy to win the game. However, during this short period of time, the verbal interactions between players related to the ‘hidden’ concepts of the game were quite limited. 6.2 Case Study 2: ‘Wobble Board’ Game In the presented research, there were 49 students engaged in this game, 18 girls and 31 boys. Twelve of them were attending Elementary school while 37 of them were Junior High School students. The ‘Wobble Board’ game (Kynigos et al., 2010) involved the interaction between the composite weight on a 5x5 meter floor and a virtual board balancing on its centre (Figure 13.3). Up to twelve players need to collaboratively move on the floor so as to move the virtual board accordingly; the goal is to displace a number of balls so that they drop into a number of fixed holes on the virtual floor. In order to do that they need to negotiate how they will move on the board. The level of difficulty of the game can be adjusted based on the age and skill of the players.

figure 13.3 The ‘Wobble Board’ installation

310

Kynigos et al.

6.2.1 The Physics and Engineering Concepts behind ‘Wooble Board’ In this game, students have to think about the forces they act on, how they affect the balance of the floor, and what happens if the students are moved and placed in different positions. The ideas of forces, balance, weight, location, and direction are embedded in the game which has various parameters such as simulated friction of the balls on the virtual floor, etc. Students are challenged to approach abstract concepts through physical experiences in which they participate. Through their body movements, they communicate expressions which enclose conceptual content. 6.2.2 Results Concerning Strategies To the question ‘if you could play again, which strategies will you implement?’ the strategies with high rates were the same as before while the rates were a little bit different (Table 13.5). Hence, the strategies with the highest rates are ‘We were separated into 4 corners and we were going in each direction’ (33.75%) and ‘We were separated into two groups each of which was going in each direction’ (21.25%). To the question, in case that collaboration is obligatory, which two strategies would you most likely apply; the strategies that scored the highest rates are still the same (Table 13.6). In particular, the strategy ‘We were divided into 4 corners and we were going in each direction’ (27.5%) and ‘We were divided into two groups each of which was going in each direction’ (22.5%). 6.2.2.1 Observations from Notes and Videos The tutors that participated in the study commented on the strategies and moves of the players on the ‘Wobble Board’ game and everyone underlined table 13.5  The most often applied strategies if you replay the ‘Wobble Board’ game

Strategies We were going all together in each direction We were separated into 4 corners and we were going in each direction Everyone went alone in each direction We were separated into two groups each of which was going in each direction Everyone took one step at a time Another strategy No answer

Answers

%

10 27

12.5 33.75

4 17

5 21.25

15 6 1

18.75 7.5 1.25

311

Augmented Playgrounds table 13.6  The kind of movement performed by the player in the ‘Wobble Board’ game

Kind of movement

Answers %

Accelerating Linear Changing No answer

6 7 35 3

11.76 13.72 68.62 5.88

the need for teamwork and collaboration in this game. Specifically, a tutor says that three different strategies were observed: – Each player is free to run everywhere on the real floor and all of them scream almost anarchically about what the next correct move will be. – Players gather in the middle of the real floor and with small group movements cause small slopes in the virtual floor. – A combination of the previous two. Some players gathered in the middle of the real floor making small moves, while others run more freely at the edges in order to cause a faster tilt change. The coordination of the moves is done by everyone, but mainly the players in the centre coordinate the ‘satellites’ players. The first strategy has a strong element of individualism and is the least profitable. It is expected to be implemented by younger children as it is the least complex. The second and third are effective but the third is more complex and faster. Both of them rely on teamwork. Finally, another tutor emphasizes that initially, in this game the team should get together. We have more than three active players and so there has to be a lot of communication to make them collaborate. It is good for players to be set up symmetrically on the platform according to their number and also have someone to coordinate their movements so they are not chaotic. They do not need abrupt movements due to the sensitivity of the floor (especially at the corners) and to the fact that there is a short time lag between a player’s movement and its depiction on the virtual floor. This comment indicates possible restrictions (time lag, sensitivity) of settings like this that should be taken into consideration in students’ engagement. 6.2.3 Result Regarding the Scientific Concepts In their questionnaires, students also described the kind of movement they performed when they played the game. Their answers are shown in Table 13.6. As shown in the table, the majority of the students, 68.62% responded that

312

Kynigos et al.

they perform a changing motion, 11.76% accelerating motion, and 13.72% linear motion. Students have described the kind of marble’s movement as accelerating (52%), changing (28%) and linear (4%). The ‘no answer’ category scored 16% (Table 13.7). In the question which magnitudes change when the player is playing the game, the ‘your position’ answer was the most common with 20.25% followed by ‘the speed at which the ball moves’ 17.08% and ‘your speed’ 15.82% (Table 13.8). If we synthesize the answers according to the concepts, we see the presence of speed is 33%, position 27%, force 12%, time 11.39%, and angle 11.39%. If we synthesize the responses according to whether they refer to themselves in the physical world or the digital representation, we see that 47.46% of the students’ answers refer to sizes referring to themselves while 48.75% to the ball table 13.7  The kind of ball’s movement

Kind of movement

Answers

%

Accelerating Linear Changing Another movement No answer

14 2 26 0 8

28 4 52 0 16

table 13.8  The physical magnitudes which change when the player plays the ‘Wobble Board’ game

Variables Your speed The speed at which the ball moves Your reaction time Your position The angle of the inclined virtual plane The sum of the forces exerted on the inclined virtual plane The position of the hole on the inclined virtual plane The distance traversed by the ball No answer

Answers

%

25 27 18 32 18 19 11 2 6

15.82 17.08 11.39 20.25 11.39 12.02 6.96 1.26 3.79

313

Augmented Playgrounds

or the floor. This again indicates a distributed perception of students to both the physical world (how I as a person move) and the digital representation. 6.2.4

Results Related to the Interactions among the Students and between the Students and the Game In the question which parts of their body they use and why during the game, half of the students answered that they used their legs (Table 13.9). A smaller percent answered the hands (18.96%) or all (6.89%). To the question, which of the movements you made was the most important to score the most points and why, the plurality of students didn’t answer (36.73%), 14.28% answered that their movements were organized when they divided into groups, and 16.33% answered that they took steps or walked, as shown in Table 13.10. This game favors collaboration, which is also reflected in student answers about strategies. The most applied strategy during playing the ‘Wobble Board’ game was to split the team into the four corners of the floor and to move in table 13.9  Most used part of the players’ body during the game

Part of the body legs all hands No answer

Answers

%

29 4 11 14

50 6.89 18.96 24.13

table 13.10  The most important movements during the game

Movements Organized movements when we divided into groups all none Steps, walk jump diagonal Hands’ movement No answer

Answers

%

7 3 3 8 4 3 3 18

14.28 6.12 6.12 16.33 8.16 6.12 6.12 36.73

314

Kynigos et al.

the necessary direction. This is a quite complex collaboration strategy that requires communication and coordination between the four sub-teams. Most of the participating tutors also emphasize the presence of teamwork and collaboration in this game. In particular, they report that three different strategies have been observed: (1) each player is free to run everywhere on the floor, (2) players are gathered in the middle of the floor and with small group movements cause correspondingly small tilt variations on the floor, (3) a combination of the two previous ones, some players gathered in the middle of the floor to do small moves, and others run more freely at the edges in order to cause a fast tilt variation. The first strategy has a strong element of individualism and is the least profitable. The second and the third are based on teamwork and are just as effective. The movements used by the players in their communication relate to the orientation of movement (48.79%) and the members of their body performing these movements (15.6%). Regarding the scientific concepts behind this embodied communication, we could say that approximately 34.5% refer indirectly to the position while 14% to the shift (emphasis on change of direction). Regarding meaning generation about scientific concepts, students described the type of movement they perform as well as the marble as changing to 68.62% and 52% respectively. They believe that the variables which change, are ‘their position’ (20.25%), ‘the speed of the marble’ (17.08%), and ‘their speed’ (15.82%). If we synthesize the answers according to the concepts we will see that speed and position are the most popular with rates of 33% and 27%, respectively. Below are: the power (12%), the time (11.39%) and the angle (11.39%). 6.3 Case Study 3: The ‘Air Hockey’ Game The total number of students who participated in the hockey game is 25, 15 boys and 10 girls, with 64% of the students (or 16 students) attending Primary School, 12% (or 3 students) Junior High School, and 24% (or 6 students) Vocational School. In the ‘Air Hockey’ game, players kick a virtual hockey puck on the floor – a stadium in which the game is projected (Figure 13.4). Foot movement is detected by special infra-red sensors, causing the hockey puck to move. The puck’s movement is influenced by the angle with which the shadow of a body member ‘hits’ the hockey puck and the speed of that movement. The team’s goal is to score as many goals as possible. The game becomes more complicated when more hockey pucks appear and students have to calculate angles of reflection very quickly, estimate speeds and develop defense and attack strategies. During the game, players have to collaborate rapidly to fill in the gaps in the hockey field. The players act like air-hockey mallets and the movement of each

Augmented Playgrounds

315

figure 13.4 The field of air-hockey project on the floor and players interact with it with body movements

player affects the condition of the puck. They need to find how they can score as well as how to avoid a goal. Interactions between team members are necessary in order for the team to achieve many ‘virtual goals.’ This game can be played by two to four players due to the dimensions of the court unless one has the role of referee or coach. Each player will have the role of ‘shooter’ and ‘goalkeeper’ at the same time. The role of each player is decided by the team. For example, a player may play in the position of a goalkeeper and another in the position of ‘shooter.’ However, each player may have the virtual hockey puck under his/her own responsibility and must lead it through the ‘virtual goals.’ During the game, the team may decide to change the roles of the players. Consequently, all players are potential shooters, goalkeepers, referees and coaches. 6.3.1 The Physics Concepts in ‘Air Hockey’ In this game, the pupils will experience the conflict phenomenon. This phenomenon involves important concepts for inquiry by the students during the activity, including but not limited to: bodily control, speed before impact, speed after impact, angle of reflection, mass, time, position change, momentum, internal forces, external forces, mechanical energy, kinetic energy, dynamic energy, impulse, and many other principles of physics. The conflict here however is that even though intuitions regarding physical impact are evoked, the real cause of puck displacement is the connection of the shadow of a body part with the virtual puck. Students tend to assume a literal ‘kick’ will do the job not

316

Kynigos et al.

realizing that puck displacement and ‘kick’ can be achieved with any part of the body provided it creates a shadow touching the puck. 6.3.2 Results Concerning Strategies The strategies that were applied most frequently when playing ‘Air Hockey’ are described in Table 13.11. As can be seen from the table, the strategy that was applied most often is that ‘someone was sitting at the goalpost and the others were kicking the hockey puck with their feet playing in front’ (39.58%). Students also claimed to have applied the following strategies: ‘we kicked the hockey puck with our feet, playing all in one line’ (20.84%), ‘someone was sitting at the goalpost and the others were playing in front of him by tapping the hockey puck with their hands’ (20.84%), ‘We waved our hands to hit the hockey puck playing everyone in a straight line’ (16.67%). table 13.11  Strategies most often applied in ‘Air Hockey’

Strategies We kicked the hockey puck with our feet, playing all in one line. Someone was sitting at the goalpost and the others were kicking the hockey puck with their feet, playing in front of him We waved our hands to hit the hockey puck, playing everyone in a straight line Someone was sitting at the goalpost and the others were playing in front of him by hitting the hockey puck with their hands Other strategy No strategy

Answers

%

10

20.84

19

39.58

8

16.67

10

20.84

0 1

0 2.08

The fact that they were all together in the same physical space without using any device, helped students to develop strategies which had a strong element of collaboration. We can see that most of their strategies are a combination of body movement (wave hands, kick, hit, etc.) with cooperative game tactics (someone was sitting at the goalpost, we were playing all in one line, etc.). 6.3.3

Result Regarding Concepts around Player’s and Hockey Puck’s Motion Students described the type of movement that the hockey puck performs as changing (48%), linear (12%) and accelerating (8%). The ‘no answer’ category accounted for the same percentage as before (28%; Table 13.12).

317

Augmented Playgrounds table 13.12  Hockey puck’ s movement

Type of movement Accelerating Linear Changing Other motion No answer

Answers

%

2 3 12 1 7

8 12 48 4 28

table 13.13  Sizes that change when the player plays the ‘Air Hockey’ game

Size Your speed Hockey puck’s speed Time of reaction Time of hockey puck’s movement Collision angle between the hockey puck and the ‘wall’ The angle that is left open by the other team Collision angle between you and the hockey puck Distance No answer

Answers

%

8 3 6 5 2 5 4 2 7

19.04 7.14 14.28 11.90 4.76 11.90 9.52 4.76 16.67

When asked which of the magnitudes vary when the player plays the ‘Air Hockey’ game, most of the students answered ‘your speed’ followed by the answers ‘time of reaction’ (14.28%), ‘the angle that is uncovered by the other team’ (11.9%) and ‘hockey puck’s movement time’ (11.90%), as shown in Table 13.13. If we revise the answers according to the concepts we will see that 26.18% of the students refer to the speed, 26.18% to the time, 26.08% to the time and 4.7% to the distance. In addition, we will observe that 54.74% of the students’ answers refer to magnitudes concerning themselves while 28.56% to magnitudes related to the virtual movement of the hockey puck. 6.3.4

Results Related to the Interactions among the Students and between the Students and the Game In the question of how you shared the roles in the group during the game, the majority of the students responded that they all decided together after

318

Kynigos et al.

dialogue each time (53.84%), as shown in Table 13.14. The percentage of students who did not respond (26.92%) is quite important. In the question of what movements they or their teammates used to communicate with each other, the overwhelming majority did not answer (58,06%), as shown in Table 13.15. Foot movements accounted for 12.88%, hand movements 9.66% while movements related to the layout and one blocking the other in the game (pushing, slips) accounted for 6.44%. In the question about which parts of their body they use and why during the game, the students reported a significant percentage of the legs (34.37%) and the hands (25%), as shown in Table 13.16. The percentage of respondents who did not respond was 37.5%. It is interesting that even if this hockey game is usually played using hands in the real world, students used whole-body table 13.14  Players’ roles in ‘Air Hockey’

Role

Answers

%

14 3 1 1 0 7

53.84 11.53 3.84 3.84 0 26.92

You all decided together after communication each time Someone from the team decided Someone outside the team decided Everyone decided on his/her own Other No answer

table 13.15  Players’ movements for the communication between team members

Communication through motions applause hands feet eyes push hits No move No answer

Answers

%

1 3 4 2 1 1 1 18

3.22 9.66 12.88 6.44 3.22 3.22 3.22 58.06

319

Augmented Playgrounds table 13.16  Body members which are used by players

Body members Feet Hands None No answer

Answers

%

11 8 1 12

34.37 25 3.12 37.5

movements in the augmented setting. In addition, according to the observations, sometimes it seemed that they combined game movements from different sports like hockey, soccer, and volley. The strategies that were applied more often when playing the ‘hockey’ game are that someone was sitting at the end and the others kicking the hockey puck with their feet (39.58%), kicking the hockey puck with their feet, while playing all in a straight line, (20.84%), someone was sitting at the finish and the others were playing in front of him, tapping the hockey puck with their hands (20.84%), waving their hands to strike the hockey puck, playing in a straight line (16.67%). If we synthesize the answers, we will notice that 60.42% of students’ strategies involve interaction with the hockey puck either with their feet or with their hands. The experiential experiences of children are strong and are expressed with the strategies they apply (someone was sitting at the goalpost, kicking the hockey puck with feet, etc.). Regarding team roles during the game, 53.84% of the students said they decided altogether after discussion, which indicates the presence of collaboration within the team. However, they don’t seem to recognize the use of body movements in order to communicate with each other. The most common scientific concepts that students engaged with were speed, collision angle and time. For engagement with game’s ‘hidden’ concepts, it is important for children to understand what the sensors are sensing and that there is nothing strange on the floor. It is important to encourage them to ‘hit’ the iconic hockey puck off the legs by hand or with other parts of their body and to guide them to create meaning about body impact and behaviour according to the original conditions (such as relationship angle of incidence with reflection angle, speed before and after impact). In addition, friction plays an important role, which is perceived by the tutors as they have access to the structure of the game through the game software and can change its value.

320 7

Kynigos et al.

Discussion

The games in the study embedded three kinds of concepts – Those related to physical bodily movement such as ducking and jumping in ‘Apples,’ moving about in ‘Wobble Board’ and moving hands, legs and body in ‘Air Hockey.’ – Those simulating physical phenomena, such as the balls rolling and colliding on the ‘Wobble Board.’ – Those simulating virtual phenomena with arbitrary rules such as a brick flying through the air horizontally or in a mathematically defined trajectory in the ‘Apples’ game or a ‘puck’ colliding with a shadow created by infra-red sensors but simulating a Newtonian collision. Our principal question was: what kinds of meanings do children invoke and what intuitions may come into play and be questioned as they communicate and interact amongst themselves and with the games. The presented results concern (a) strategies formed by the children during the game, (b) interactions via words and actions, (c) concepts embedded in the games and (d) collaboration. The analysis indicated that children formed different strategies to approach their interaction with each game influenced principally by their full-body actions. So, their strategies were based on intuitions concerning their own movement and sense of touch even though objects such as the brick and the puck were virtual. Their interactions were playful and simple. In many cases they perceived body motion as a natural infrastructural way to interact with the games without this turning their attention away from the concepts and the need to negotiate about them. They seemed to directly connect their movement, gesture, and communication with the concepts which they perceived as embedded in the games. Their invoking of concepts in their strategy however was not consistent. In ‘Apples,’ they did not seem to consider that the brick could be affected by gravity and air friction, affording it properties of a virtual object arbitrarily governed by Newtonian rules regarding only speed, time, and position. In ‘Air Hockey,’ they assumed Newtonian concepts of physical collision using their feet, mistakenly thinking for instance that placing the foot on a puck path would stop it (Anastopoulou, Smyrnaiou, & Kynigos, 2011). The intuitions regarding the behaviour of a virtual brick seemed more open to question than those regarding collision with a virtual ‘puck.’ We also observed students using different kinds of gestures, depending on their purpose, which included: non-verbal gestures with no apparent communication

Augmented Playgrounds

321

purpose, non-verbal gestures with apparent communication purpose, verbal gestures in order to communicate with others, and verbal gestures with no apparent communication with others purpose (Knapp, 1972). In addition, we found that ‘movement-based play often demands different kinds of communication and presuppose interactions between different communication modes’ (Grønbæk, Iversen, Kortbek, Nielsen, & Aagaard, 2007) and different types of metaphors such as orientational, which are based on spatial orientation like up-down, leftright, etc. (Lakoff & Johnson, 1980). However, during their verbal interactions, they didn’t refer to the embedded concepts, but only when they were questioned afterwards. Then, they thought again and gave scientific explanations. Another significant finding is that most of the children, in all games, were influenced by their digital image (either as a shadow or as a digital graph) represented on the augmented space. Our understanding of human learning shifts from a narrow ‘in-the-mind’ focus to a broader ‘person-in-the-world’ focus (Chee, 2007) and the concepts of embodiment, embeddedness, and experience dominate. Students, as embodied persons in the world, possess a clear sense of being, of location in a socio-material world in space-time. There is a sense of self and personhood and a sense that one is. This sense of being, as argued by Heidegger (1962), reflects an entity that ‘shows up’ within the context of practical engagement. The augmented playground settings allowed for natural access to the game structure which can be quite beneficial for the development of collaborative dialogue and argumentation. The fact that students could play a game together in the same physical space, without the use of any device, promoted collaboration and dialogue between them. The embodied interactions, both among students and between students and the game interface, played a significant role in expressing ideas and generating meanings about the relevant scientific concepts. Through processes of argument, explanation, and discussion about their gestures and body movements, students got engaged with the ‘hidden’ concepts of the games and they developed their critical thinking by recognizing good and bad gameplay strategies. It seems that the use of augmented collaborative games in the learning process can enable the creation of innovative and dynamic learning environments in which the students can make up their own concepts, build their own representations and discover their own ways of expression via participation procedures in activities of a collaborative character and of personal interest. This is a field where further research is needed in terms of both the design and the pedagogical principles that these settings should follow.

322

Kynigos et al.

References Anastopoulou, S., Smyrnaiou, Z., & Kynigos, C. (2011). Bringing intuitions of natural and virtual interactions into conflict: The POLYMECHANON experience. In Ε. Efthimiou, G. Kouroupetroglou, & C. Vogler (Eds.), Proceedings of the GW 2011: The 9th International Gesture Workshop Gesture in Embodied Communication and Human-Computer Interaction (pp. 68–71). Cham: Springer. Antle, A., Kynigos, C., Lyons, L., Marshall, P., Moher, T., & Roussou, M. (2009, June 8–13). Manifesting embodiment: Designer’s variations on a theme. In Proceedings: Computer supported collaborative learning practices (pp. 15–17). Rhodes, Greece. Beth, E. W., & Piaget, J. (1974). Mathematical epistemology and psychology. Dordrecht: Springer. Burton, L. (1999). Why is Intuition so important to mathematics but missing from mathematics education? For the Learning of Mathematics, 19(3), 27–32. Retrieved from http://www.jstor.org/stable/40248307 Chee Y. S. (2007). Embodiment, embeddedness, and experience: Game-based learning and construction of identity. Research and Practice in Technology Enhanced Learning, 2(1), 3–30. Dede, C. (2010). Comparing frameworks for 21st century skills. In J. Bellanca & R. Brandt (Eds.), 21st century skills: Rethinking how students learn (pp. 51–75). Bloomington, IN: Solution Tree Press. diSessa, A. (1983). Phenomenology and the evolution of intuition. In D. Gentner & A. Stevens (Eds.), Mental models (pp. 15–33). Mahwah: NJ: Erlbaum Associates. diSessa, A. A. (1988). Knowledge in pieces. In G. Forman & P. Pufall (Eds.), Constructivism in the computer age (pp. 49–70). Hillsdale, NJ: Lawrence Erlbaum. diSessa, A. (2013). Processing inaccurate information. In D. N. Rapp & J. L. G. Braasch (Eds.) Processing inaccurate information: Theoretical and applied perspectives from cognitive science and the educational sciences (pp. 279–297). Cambridge, MA: MIT Press. Fischbein, E (1987). Intuition in science and mathematics: An educational approach. Dordrecht: Reidel. Grønbæk, K., Iversen, O. S., Kortbek, K. J., Nielsen, K., & Aagaard, L. (2007). IGameFloor: A platform for co-located collaborative games. In Proceedings of the International Conference on advances in computer entertainment technology (pp. 64–71). New York, NY: ACM. Heidegger, M. (1962). Being and time (J. Macquarrie & E. Robinson, Trans.). New York, NY: Harper & Row. Kaufmann, H. (2003). Collaborative augmented reality in education. In Proceedings of Imagina Conference, 2003. Monaco: Imagina.

Augmented Playgrounds

323

Knapp, M. L. (1972). Nonverbal communication in human interaction. New York, NY: Holt, Rinehart & Winston. Kynigos, C., Smyrnaiou, Z., & Roussou, M. (2010). Exploring the generation of meanings in mathematics and science with collaborative full-body games. In Proceedings of the 9th International Conference on Interaction Design and Children, Barcelona, Spain (pp. 222–225). Lakoff, G., & Johnson, M. (1980). Metaphors we live by. Chicago, IL: University of Chicago Press. Lakoff, G., & Núñez, R. (2000). Where mathematics comes from. New York, NY: Basic Books. Lee, K. (2012). Augmented reality in education and training. TechTrends, 56(2), 13–21. Linn, M. C., Chang, H.-Y., Chiu, J., Zhang, H., & McElhaney, K. (2010). Can desirable difficulties overcome deceptive clarity in scientific visualizations? In A. Benjamin (Ed.), Successful remembering and successful forgetting: A festschrift in Honor of Robert A. Bjork (pp. 239–262). New York, NY: Routledge. McCarthy, J., & Wright, P. (2004). Technology as experience. Interactions, 11(5), 42–43. Nemirovsky, R., Tierney, C., & Wright, T. (1998). Body motion and graphing. Cognition and Instruction, 16(2), 119–172. Papert, S. (1980). Mindstorms: Children, computers, and powerful ideas. New York, NY: Basic Books. Parnafes, O. (2007). What does “fast” mean? Understanding the physical world through computational representations. Journal of the Learning Sciences, 16, 415–450. Resnick, M. (1998). Technologies for lifelong kindergarten. Educational Technology Research and Development, 46(4), 43–55. Rhodes, M., & Wellman, H. (2013). Constructing a new theory from old ideas and new evidence. Cognitive Science, 37(3), 592–604. Rogoff, B. (1993). Children’s guided participation and participatory appropriation in sociocultural activity. In R. H. Wozniak & K. W. Fischer (Eds.), Development in context: Acting and thinking in specific environments (pp. 121–153). New York, NY: Psychology Press. Salen, K., & Zimmerman, E. (2003). Rules of play: Fundamentals of game design. Cambridge, MA: MIT Press. Sherin, B. (2001). How students understand physics equations. Cognition and Instruction, 19(4), 479–541. Smith, J. P., diSessa, A. A., & Roschelle, J. (1993). Misconceptions reconceived: A constructivist analysis of knowledge in transition. Journal of the Learning Sciences, 3(2), 115–163. Tall, D. (1991). Intuition and rigor: The role of visualization in the calculus. Visualization in Mathematics, 19, 105–119.

324

Kynigos et al.

Wilder, R. L. (1984). The role of Intuition. In D. M. Cambpell & J. C. Higgins (Eds.), Mathematics, people, problems, results (Vol. II, pp. 37–45). Belmont, CA: Wandsworth International. Wu, H. K., Lee, S. W. Y., Chang, H. Y., & Liang, J. C. (2013). Current status, opportunities and challenges of augmented reality in education. Computers & Education, 62, 41–49. Zaree, Q. (2010). Role of intuition in mathematics education (Unpublished Master’s dissertation). Fars University, Iran.

CHAPTER 14

Augmented Reality in Mathematics Education: The Case of GeoGebra AR Melanie Tomaschko and Markus Hohenwarter

Abstract Augmented Reality enables users to perceive the real world extended by virtual elements. This chapter presents possibilities for exploiting the potential of augmented reality in learning and teaching mathematics. As an example, the novel mobile application GeoGebra Augmented Reality (AR) for iPhones and iPads is presented. GeoGebra AR renders 3D graphs and objects in real world environments and literally joins the real world with the abstract world of mathematics. In this way, the application allows exploring 3D math objects virtually placed in learners’ environments, while they can walk around them and observe them from different perspectives. Additionally, guided activities are offered that lead users to discover math in the real world by taking screenshots from different perspectives. Furthermore, suggestions for possible future development of the GeoGebra AR app are given.

Keywords augmented reality – GeoGebra – mathematics education – mobile devices

1

Introduction

The integration of Information and Communication Technology (ICT) in school practices is increasingly becoming popular because it can enrich students’ learning in various ways. Especially for high-quality mathematics education, the literature indicates that the integration of technology in educational settings is indispensable (Burrill, 2011; NCTM, 2008; Pimm & Johnston-Wilder, 2005). Current technological innovations such as wireless mobile devices and virtual or augmented reality provide additional opportunities for developing novel teaching and learning environments. © koninklijke brill nv, leideN, 2020 | DOI: 10.1163/9789004408845_014

326

Tomaschko and Hohenwarter

Augmented Reality (AR) enables users to perceive the real world while it is extended by virtual elements (Azuma, 1997). Unlike Virtual Reality (VR), where the real world is completely replaced by a synthetic environment, AR facilitates users to see the real world augmented by a virtual overlay (Bower, Howe, McCredie, Robinson, & Grover, 2014). Hereby users can interact in real-time with real and virtual objects, such as text, images, videos, and 2D or 3D models that are coexisting in the same space. Even 20 years ago, AR was being applied in medical, military, industrial, and entertainment activities (Azuma, 1997). Nowadays, there exists a variety of initiatives for the integration of AR technologies in education, but these are yet to have significant effects on school practices. This may be due to the fact that until recently, the use of AR in classrooms entailed the requirement of expensive and cumbersome equipment, such as head-mounted displays. Nowadays, because of the evolving technologies, AR experiences can be delivered through common mobile devices such as smartphones and tablets. In this way, AR can provide additional benefits to conventional mobile learning that facilitates the access of learning resources anytime and anywhere. The integration of AR technology has been highly recommended by researchers as a potential pedagogical approach to support and enhance learning and teaching (Bower et al., 2014; Dunleavy & Dede, 2014; Hwang, 2014; Wu, Lee, Chang, & Liang, 2013), such as: – Bridging formal and informal learning – Collaborative learning – Constructivist learning – Context-aware ubiquitous learning – Enquiry-based learning – Games-based learning – Situated learning Due to the augmentation of reality, new ways of teaching and learning are provided. In recent years, these new learning environments in school practices supported by AR technologies have been extensively researched. These study findings show that students’ intrinsic motivation and interest could be increased. AR also can exert a positive influence on cognitive learning (Sotiriou & Bogner, 2008), and improve creativity, critical analysis, and students’ learning outcomes (Bower et al., 2014). In particular for mathematics education, AR provides benefits in visualization, manipulation, and genuine contexts (Estapa & Nadolny, 2015). Kerawalla, Luckin, Seljeflot, and Woolard (2006) indicate that AR provides possibilities to motivate and engage students in exploring virtual objects from a variety of perspectives. As described by Quintero, Salinas, González-Mendívil, and Ramírez (2015) mathematics learning often requires skills to construct mental models

Augmented Reality in Mathematics Education

327

of abstract mathematical concepts that are drawn or orally described by teachers. Through real-time interaction with virtual 3D math objects, students could be supported in this ability. Even though students can develop those skills and knowledge in other technology-enhanced learning environments, AR facilitates it in a more effective way (Dünser, Walker, Horner, & Bentall, 2012; Shelton & Hedley, 2004). One reason could be because the interaction with the AR technology is more natural (Bujak et al., 2013). Students can walk around the virtual objects to change the perspective and also move closer or farther away from the objects in order to change the scale. Moreover, the physical movements could help learners to operate on, measure, and manipulate 3D objects in order to understand spatial relationships (Bujak et al., 2013). Further advantages result from the interactive content displayed by AR technologies because it is controllable by learners themselves (Bujak et al., 2013). Since students’ preferences vary, AR technologies allow different perspectives on the mathematical content. In this way learners can choose whether they prefer to examine the content as a static image or walk around the virtual objects to explore them from many angles. The AR application presented in this chapter tries to take advantage of the possibilities of placing virtual 3D math objects into real world environments, allowing them to be explored from any angle. In particular, the presented application allows the modeling of 3D graphs and additionally includes several examples of 3D math objects that can be placed on tables, floors, or any other flat surface, while users can walk around them and observe them. Additionally, guided activities are offered that lead users to discover math in the real world by taking screenshots from different perspectives.

2

GeoGebra Augmented Reality Application

GeoGebra (2018) provides a set of dynamic mathematics learning tools, developed by an international team of open source software developers and researchers. Currently, GeoGebra offers, among others, the following features and applications: – Graphing Calculator for plotting functions and solving equations. – Geometry app for creating interactive geometric constructions. – 3D Graphing Calculator for graphing functions, surfaces, and geometric solids in 3D. – Spreadsheet for working with data and statistical concepts. – Computer Algebra System for symbolic computations with equations and fractions.

328

Tomaschko and Hohenwarter

– Probability Calculator for calculating and graphing probability distributions. In order to exploit the potential of AR in mathematics education, GeoGebra has developed a novel AR mobile application for the iOS operating system. In this manner GeoGebra offers new opportunities for enhancing mathematics learning and teaching environments by AR experiences. In September 2017, the first version of GeoGebra’s Augmented Reality application was released. This AR app has mainly been developed to complement and enrich mathematics education in both classroom settings and outside of school. The app allows exploring computer generated math objects placed in real world environments (see Figure 14.1). This should engage and motivate learners to explore and interact with mathematical objects, resulting in deeper understanding of the content. We know from math education research that many students have problems connecting a picture on a 2D screen with a 3D math object (Pittalis, Mousoulides, & Christou, 2009). Even when rotating the on-screen image, many still have a hard time to see anything 3D, as all we really give them is just a 2D image. The great benefit of augmented reality is that it solves the 3D navigation problem in a very intuitive way. Math objects can be put into students’ known environment and they can walk around the 3D objects to look at them from all sides.

figure 14.1 Exploring virtual 3D content using GeoGebra AR

Augmented Reality in Mathematics Education

329

2.1 The Application After starting the GeoGebra AR application, the user interface displays the camera’s view of the device. In order to help users to get started, an information text is shown at the top of the view, introducing the first steps. This is important because users should always be informed about the current state of the system, as well as to give cues about their options for interaction. The application offers a suggested 3D graph that can be placed in the real world. Therefore users are prompted to scan the environment for horizontal planes to place the virtual 3D graph. For this reason, a yellow dashed square is shown on the screen that changes size and orientation in order to indicate scene depth (see Figure 14.2a). After the app has detected a plane in the camera’s scene view, the information text shows the notification to tap anywhere on the screen to place the 3D math object on the currently detected surface. In order to allow users to focus on the displayed graph, the background is automatically dimmed (see Figure 14.2b). Additional settings allow users to completely darken the background or to see the camera’s view. After the virtual object is inserted, users can walk around the 3D graph and explore it from any perspective. The application further allows users to manipulate existing equations and see the results in real time to explore certain characteristics of the 3D graph. Additionally, it is also possible to put a second graph in the same view so they can be compared easily (see Figure 14.2c). All of the students’ observations can then be documented by screenshots. This allows them to build a collection of math objects in AR as an assignment. (a)

(b)

(c)

figure 14.2 (a) Plane detection. (b) Placed object. (c) Two graphs

330

Tomaschko and Hohenwarter

Finally, there are many other shapes to explore in addition to the free-input graphs. Tapping the menu button at the top left corner opens a menu that allows the selection from different predefined mathematical objects. After choosing one of those elements, it is placed in the real world and a specific mathematical task (discussed in Section 3.1) is displayed at the bottom of the view (see Figure 14.3). Those instructions are directly integrated within the application’s view, because otherwise, if the instructional information is presented elsewhere, students’ attention may be divided. In this way, the learning effects of the students could be increased (Bujak et al., 2013). After choosing, for example, “Ruled surface” from the app’s provided set of predefined 3D objects, the object is displayed in the real world and the integrated instruction asks the user to create different screenshots from specific perspectives. While doing so, users can walk around the objects to freely explore them from any perspective. Additionally, familiar gestures such as one-finger pan, pinchzoom, or two-finger rotation can be used to position and orient the GeoGebra 3D objects.

figure 14.3 Predefined 3D math object and corresponding mathematical task

Augmented Reality in Mathematics Education

331

2.2 Technical Background In general, two types of AR applications can be distinguished: marker-based and markerless AR applications. Marker-based AR applications recognize predefined markers in the real world and attach specific virtual elements to them. Those markers can be QR codes, images, or 2D or 3D objects. All of the predefined markers have to be stored in advance, either directly on the used device or online in some kind of cloud storage. In contrast, markerless AR apps, including GeoGebra AR, do not require any markers; the virtual content is directly visualized and placed within the real world. Therefore, the real world is scanned for horizontal or vertical planes and surfaces that are suitable for the placement of virtual objects. This has the advantage that the application can be used at any place without the need of any reference points. For the development of GeoGebra AR a common library from Apple is used that supports the implementation of markerless AR apps. In June 2017, Apple introduced the new operating system iOS 11 as part of the Worldwide Developers Conference (WWDC). The OS includes a new framework called ARKit (2018) that allows app developers to add augmented reality experiences to mobile applications. Thereby, no additional hardware resources are required, the only requirement is an iPhone or iPad with at least an A9 processor. The ARKit uses visual input from the camera and motion sensors of the device in order to place 2D or 3D objects into the real-time view of the device’s camera. To make those virtual elements appear like they are part of the real world, the ARKit library uses visual-inertial odometry. This technology allows the combination of the device’s motion and advanced computer vision techniques to create a highly accurate model of the position and motion of the device with six degrees of freedom (6DoF – rotation and translation along the three axes, x, y, and z) without the need of any configuration, as well as automatic estimation of the position, scale, and rotation of the placed virtual elements while the device is moved. In this way, users experience virtual 3D objects as if they were part of the real world. In addition, light estimation is used to light virtual objects based on the lighting of the scene to further improve the illusion that the virtual content is coexisting with the real world. For displaying 3D content within the application, Apple provides a 3D graphics framework, called SceneKit1 allowing the creation and animation of 3D objects. This is done by a high-performance rendering engine that enables the import, manipulation, and rendering of 3D assets. In order to display 3D GeoGebra models within the application, the 3D models are specified using the Collada (2017) standard. Collada is a specification for a file format (.dae) of interactive 3D models that allows exchange of digital assets between different 3D software applications. It is an XML-based schema that specifies a

332

Tomaschko and Hohenwarter

namespace and database scheme adopted by ISO (2017). These files can automatically be converted by Apple’s development environment, Xcode, to render them with SceneKit. Because of the integrated GeoGebra library, further opportunities for direct user input are provided. Users can enter 3D functions in the form of that are displayed as surfaces augmenting the real world. Using the GeoGebra iOS library enables converting the user input in a way that can be rendered with the GeoGebra AR application directly. All of the 3D SceneKit content is displayed as augmentation of the camera’s view. This is done by the ARKit library that places the virtual objects on the screen according to the real world. This is automatically handled through motion tracking and image processing, resulting in creating great AR experiences. Additionally, developers have the opportunity to enable plane detection that allows the positioning of virtual content in relation to real world surfaces. Through automatic plane detection, the virtual objects can be placed corresponding to anchors, i.e., real world surfaces. This provides an automatic update of the content relative to the position and extent of the anchor plane. 2.3 Augmented Reality SDKs For GeoGebra to be used widely in schools, it is important that it is not limited to the iOS platform, but can also support different mobile devices and a variety of operating systems. This section presents different AR software development kits (SDKs) that could be used for the development of GeoGebra AR for the Android operating system. Only SDKs that support the implementation of markerless AR apps are considered. 2.3.1 ARCore Google has released an SDK called ARCore (2017) that allows the development of mobile applications with AR features for the Android operating system. This library is available for different development environments like Android Studio, Unity, Unreal, or Web. The main features of ARCore are motion tracking, environmental understanding, and light estimation. Google uses concurrent odometry and mapping (COM) for motion tracking. Therefore the camera is used to detect feature points within the real world that are used in combination with inertial measurements (IMU) to compute position and orientation of the device. In order to make virtual objects appear to be placed in the real world, the scene camera is matched to the pose camera which allows the correct perspective on a virtual object. Environmental understanding is used to detect horizontal surfaces within the real world to place virtual objects on them. Technically, ARCore scans the

Augmented Reality in Mathematics Education

333

real world for clusters of feature points to detect those planes. However, the documentation notes that planes without texture are hard to detect. In order to improve AR experiences, ARCore includes light estimation. In this way, virtual objects appear under the same lighting condition as real objects within the real world. ARCore does not directly support loading or rendering of 3D assets, however a framework is offered that allows loading of predefined 3D models. Additionally, OpenGL ES can be used to display custom virtual objects within the real world. 2.3.2 Vuforia Vuforia (2017) can be used to add AR experiences to mobile applications for the platforms Android, iOS, UWP, and smart glasses, and can be used in combination with Unity. It enables different features such as recognition and tracking of images and objects. In order to build an AR application to place 3D math objects into the real world, Vuforia provides a feature called “Ground Plane.” It allows the detection and tracking of horizontal surfaces, on which virtual content can be placed. Therefore anchor points in the real world can be used. In the background, 6DoF is used to track and update displayed virtual objects. The native API of Vuforia does not support the loading or rendering of predefined 3D models; it relies on OpenGL. For the use of GeoGebra 3D math objects, a third party library needs to be used or a custom 3D format loader needs to be implemented. Vuforia is specific in that it can also be used in combination with ARKit and ARCore. In this way, it is possible to support more devices than ARKit or ARCore would do and additionally it also takes care of ARKit or ARCore features that could improve AR experiences. 2.3.3 Wikitude Wikitude (2017) is an AR SDK for the operating systems Android, iOS, and smart glasses. Additionally, extensions for Unity, Cordova, Titanium, and Xamarin are provided. The SDK offers multiple features such as plane detection, image recognition, and object tracking. For the implementation of the GeoGebra AR application, instant tracking is one of the most important features. This technique allows placement of objects into the real world in such a way that it appears as part of the real world. For this purpose a SLAM (Simultaneous Localization and Mapping) based solution has been developed. Their algorithm works in two phases. The first one is the initialization state. In this state, it is necessary to manually specify the current height of the used device and the origin of the tracking procedure.

334

Tomaschko and Hohenwarter

The next state is the actual tracking state that can be started after the user has confirmed the origin tracking position. For displaying 3D models, wikitude supports internal and external rendering. For internal rendering, OpenGL ES and Metal (iOS only) can be used. For external rendering, it is only possible to load 3D assets from wikitude 3D format files (.wt3). However, wikitude also provides a 3D encoder for converting FBX (.fbx) files to the required wikitude 3D format. 2.3.4 ARToolKit The ARToolKit (2017) is freely available as a compiled SDK for the major platforms Android, iOS, Linux, Mac OS X, Windows, and smart glasses, and also provides a plugin for Unity. The SDK supports square markers, 2D barcode makers, and natural feature tracking. For calculating the position of the device relative to the markers, computer vision techniques are used. The current version of ARToolKit 5.3.2 does not support any feature like plane detection. In order to display virtual 3D objects, markers are used. However, with ARToolKit 6 (currently available as a beta version), a new image tracker is introduced that can also be used for plane detection. For loading 3D models, ARToolKit supports static Wavefront (.obj) files as well as the rendering of OpenGL ES and OpenSceneGraph. 2.3.5 MAXST MAXST (2017) can be obtained for free, however, the freely available version includes watermarks and can only be used for non-profit AR projects. It supports the operating systems iOS, Android, Mac OS, Windows, and a plugin for Unity. The main features include image tracker, instant tracker, visual SLAM, object tracker, and QR or Barcode scanner. The instant tracker is able to detect horizontal planes in the real world that allows placing and tracking of virtual objects in relation to the detected horizontal surface. The underlying algorithm is based on VIO that even supports the detection of featureless planes. 3D models can be rendered using OpenGL ES. 2.3.6 Kudan Kudan (2018) offers native AR SDKs for Android and iOS, as well as a plugin for Unity. This engine is available with a free (with watermarks) or commercial license. The SDK supports markerless and marker-based applications. In order to place and track virtual objects in the real world, a custom SLAM-based tracking solution has been developed. For determining the position of the device,

Augmented Reality in Mathematics Education

335

the environment is scanned for feature points that can be tracked while the device is moved. This allows the correct placing and tracking of virtual objects. 3D models that should be rendered with KudanAR have to confirm the .armodel file format, a custom format for fast loading, and an optimized file size. Additionally, Kudan’s ARToolkit offers a 3D model converter tool for converting existing 3D models from FBX, OBJ, or DAE format to the internally used .armodel file format. 2.3.7 Comparison Google ARCore and ARToolKit are the only SDKs that are available for free. ARToolKit is even an open source project. Wikitude, Kudan, and MAXST offer a free trial version with full feature support, however the camera view includes a watermark. Additionally, MAXST explicitly mentions that the free SDK can only be used for non-profit AR projects. Vuforia also offers a free version of the SDK, however it can only be used as Unity plugin. For the native iOS and Android development, a commercial license has to be obtained. An important feature of the GeoGebra AR app is the rendering of 3D models. Kudan AR SDK is the only one that directly supports Collada file formats in combination with a 3D model converter. All of the other SDKs provide a possibility for internal rendering of 3D models using OpenGL. ARCore, Wikitude, and ARToolKit allow additional rendering of 3D assets of a special file format.

3

School Practices

In order to demonstrate GeoGebra AR’s potential in school practices, this section presents different examples for math classes. As GeoGebra AR allows displaying predefined 3D math objects as well as rendering user input of any 3D graph, this section is split into “Exploring” and “Free User Input.” Section 3.1 describes the included 3D math objects and guided activities that are offered by the GeoGebra AR app, whereas Section 3.2 describes possible sessions where custom 3D objects are created by students. 3.1 Exploring GeoGebra AR provides a set of predefined geometrical 3D math objects that can be placed on tables, floors, or any other flat surface in the real world and explored by students. Additionally, for each predefined object, GeoGebra AR offers specific tasks to be done by students while the 3D math object is investigated. All of the guided activities should engage students in discovering math in the real world by taking screenshots of underlying geometry and internal

336

Tomaschko and Hohenwarter

structures. This section presents different examples of the GeoGebra AR application that were developed together with math teachers. 3.1.1 Basic Solids The first example, basic solids, can be used to introduce basic functionalities of GeoGebra AR. It was deliberately kept simple in order to avoid cognitive overload. Students learn how to use simple features of the application, such as detecting planes and placing virtual 3D objects, and get a first impression of exploring virtual 3D math objects in the real world from any perspective while walking around them.

figure 14.4 Taking screenshots from different objects

In addition to the introduction of the use of GeoGebra AR, students can explore and classify different geometric solids. After choosing “Basic solids” from the list of predefined 3D math objects, a set of different solids such as spheres, pyramids, cones, cylinders, cubes, prisms, and tetrahedrons are placed into the real world (see Figure 14.4). The included guided activity asks students to take different screenshots of specific objects. In this way, this example allows users to compare characteristics of different objects and to explore further properties of specific solids. While investigating the different 3D math objects, students are supposed to classify, name, and distinguish different solids according to their specific characteristics. 3.1.2 Penrose Triangle A Penrose triangle appears to be a solid object, however it is an impossible object: as a two-dimensional representation (see Figure 14.5) the Penrose

Augmented Reality in Mathematics Education

337

figure 14.5  Penrose triangle

triangle appears logical and only evades comprehension when viewed from different perspectives in a 3D space. The included Penrose triangle example displays a 3D object that should be placed in a way so that it gives the illusion of a connected triangle (see Figure 14.6). Students can then take a screenshot of this illusion. While trying to get the correct perspective in order to see a solid 3D object, students can walk around the object and look at it from any perspective. This example allows experimental exploration of an impossible object, perspective illusions, and optical illusions in an interactive way in order to foster spatial abilities, especially in the field of spatial orientation.

figure 14.6 Constructing the illusion of a closed triangle

338

Tomaschko and Hohenwarter

3.1.3 Sierpinski Pyramid The Sierpinski Triangle is a fractal based on an equilateral triangle that is recursively subdivided into smaller equilateral triangles. The Sierpinski Pyramid is an analogue of the Sierpinski triangle in 3D space and based on a regular tetrahedron. After the Sierpinski pyramid is placed in the students’ environment, they are asked to take several screenshots of the same pattern at different fractal levels (see Figure 14.7). In this way, students are encouraged to explore fractal patterns and properties in an intuitive and interactive way. 3.1.4 Football Another great example is the 3D model of a real object, a football (see Figure 14.8). This example requires students to reflect and explore how math applies

figure 14.7 Sierpinski pyramid

figure 14.8 Football

Augmented Reality in Mathematics Education

339

in real life by investigating real world math connections. The provided activity asks students how a football can be constructed from a regular icosahedron. Before they answer the questions, they can explore the different characteristics of an icosahedron and the football. 3.1.5 Klein Bottle The Klein bottle is an example for a non-orientable surface, without an “inside” or “outside.” Considering a Klein bottle as a mathematical object in a threedimensional space, it intersects itself. In the case of the example of a Klein bottle, heading down into the bottle allows students to experience its topological properties (see Figure 14.9). GeoGebra AR lets you walk around the Klein bottle and even get inside it.

figure 14.9 Klein bottle

3.2 Free User Input In addition to the provided predefined set of 3D math objects, GeoGebra AR further allows the free input of 3D functions. This allows users to call any function into being, such that students can explore how it behaves right in front of their eyes. Using two surface functions even allows users to model 3D solids. The following subsections present examples of how GeoGebra AR can be used in school practices in order to virtually model, explore, and analyze 3D functions and objects. 3.2.1 Exploring 3D Function Graphs In order to get users started in exploring 3D function graphs using GeoGebra AR, the app provides already a suggested equation that can be placed in the real world. After finding a proper plane, the graph of the 3D function is placed and displayed at the detected surface. The example shows a paraboloid (see Figure 14.10a). Students can now explore it from the front or any side. Looking from above shows that a cross section of this graph from above results in a circle.

340 (a)

Tomaschko and Hohenwarter (b)

(c)

figure 14.10 (a) Paraboloid. (b) Changing equations. (c) Two graphs in one view

In order to explore further properties of paraboloids, students can also change the equation to and again walk around the object and explore it from different perspectives (see Figure 14.10b). Teachers can now give further inputs for students that allow them to interactively investigate different paraboloids. In addition, it is also possible to put a second graph in the same view so they can be compared easily (see Figure 14.10c). All of the students’ observations can be documented by screenshots. This enables them to take pictures of the graphs and build a collection of different characteristics of paraboloids as an assignment. 3.2.2 Modeling Real-World Objects In order to get started in modelling 3D solids, students are first introduced to the characteristics of surfaces of revolution. For this introduction the interactive GeoGebra activity created by Brzezinski2 (2018b) is perfectly suitable. Students are gradually introduced to surfaces of revolution formed by rotating the graph of a function about the x-axis. The activity starts with a short demo video demonstrating the development of such a surface. Afterwards students are encouraged to model their own solids using an embedded GeoGebra 3D applet (see Figure 14.11). In the next step, Brzezinski introduces GeoGebra AR’s notation of functions of the form and the necessity of defining a surface of revolution as the connection of two surfaces. The activity guides students especially in finding and formulating equations on their own. After students have successfully completed this activity, they can start to model and explore different surfaces of revolution using GeoGebra AR. Students can further benefit from the exceptional opportunities that GeoGebra AR offers by modelling real-world objects. Brzezinski (2018a)

Augmented Reality in Mathematics Education

341

figure 14.11 Extract from the interactive GeoGebra Book (from Brzezinski, 2018)

presents numerous application possibilities for the use of GeoGebra AR in active, student-centered learning environments. In particular the author presents multiple examples of how GeoGebra AR can be used in school practices in order to virtually model, explore, and analyze every-day, 3D objects (see Figure 14.12).

figure 14.12 Modelling every-day, 3D objects with GeoGebra AR (from Brzezinski,

2018a)

342 4

Tomaschko and Hohenwarter

Future Work

The AR application presented in this chapter is a first prototype implementation and thus has a lot of potential for further improvements. This section presents possible future development directions of the GeoGebra AR application. 4.1 User Generated 3D Models Although the use of AR technologies in education seems promising, Kerawalla et al. (2006) indicate that teachers would like to have more control over the content that is visualized through AR. The GeoGebra AR application could be extended to allow the integration of 3D models that are created by importing models from other apps. In this way, a possibility would be provided that allows the integration of user generated content. This could be done by extending the existing GeoGebra iOS library and GeoGebra AR application. In this way it would be possible to use the mathematical kernel of GeoGebra to load 3D objects that are created with the web, desktop, or Android version of GeoGebra’s 3D Graphing Calculator. Internally the kernel would be used to parse the GeoGebra file and to convert the 3D objects contained therein in a way that can be rendered within the AR application. 4.2 Measurement of Real World Objects ARKit enables a possibility to detect 3D coordinates in the real world. If a user taps on the screen of the device, the 2D coordinates of the screen, device orientation, and camera projection are used to place a ray within the scene. Any point that is placed on this ray, e.g., intersection points of the ray and surfaces from the real world, can be returned by the ARKit. In this way, it is possible to determine 3D coordinates of points that are placed in the real world. Using this feature could enable opportunities to measure the distance between certain points and to calculate the length of certain segments. This feature would allow users to gain knowledge about objects that are immediately related to the real world. Moreover, this feature would enable opportunities for dynamic geometry. It could be used to perform 2D constructions on any plane in the real world. 4.3 Dynamic Interactions with 3D Models Currently, the GeoGebra AR app supports only the manipulation of equations of 3D functions. But as indicated by the literature, the interactive manipulation and exploration of 3D math objects could promote inquiry-based learning

343

Augmented Reality in Mathematics Education

figure 14.13  Draft of an interactive GeoGebra AR app

(Kerawalla et al., 2006). For this reason, a possibility for dynamic manipulation of all 3D models could be offered. As already known from the other GeoGebra math applications, the mathematical objects could directly be manipulated by users, for instance by dragging points. Previous research has confirmed the effectiveness of interactive interaction and manipulation of mathematical objects (Roschelle et al., 2010). Therefore, it would be a great opportunity to extend GeoGebra AR in a way to allow dynamic interactions with 3D models through the use of sliders or moveable points. Figure 14.13 shows a draft for the interactive interaction and manipulation of virtual 3D objects placed in the real world. A similar approach is presented by Kaufmann and Schmalstieg (2002). They have developed an AR system for mathematics and geometry

344

Tomaschko and Hohenwarter

education. The presented application system allows users to construct basic elements such as points, lines, and basic solids. Furthermore, the system supports features such as intersections, normal lines, symmetry operations, and measurements. As shown by their studies, the AR system can help users to improve spatial abilities and maximize transfer of learning. However, this approach requires additional hardware resources that may not be available in schools. Through the extension of GeoGebra AR, this could be achieved, as students only require their mobile device without any additional hardware.

5

Conclusion

Augmented Reality enables users to perceive the real world augmented by computer-generated virtual objects. Enhancing teaching and learning environments by AR experiences offers great potential to improve and enhance students’ learning. This chapter introduced GeoGebra AR, an application that allows the creation of novel learning environments in math education. In particular, GeoGebra AR provides possibilities to place virtual 3D math objects into the real world and to explore them from any perspective. Since the application has been developed for the iOS operating system, possible opportunities for the development of an AR application for the Android operating system were presented. Furthermore, suggestions for future improvements of the GeoGebra AR app were given.

Notes 1 SceneKit, URL: https://developer.apple.com/scenekit/, last accessed December 28, 2017. 2 Brzezinski is an independent math-ed consultant, math teacher, and accredited GeoGebra trainer/author.

References ARCore. (2017). Official website. Retrieved from https://developers.google.com/ar/ ARKit. (2018). Official website. Retrieved from https://developer.apple.com/arkit/ ARToolKit. (2017). Official website. Retrieved from https://artoolkit.org/ Azuma, R. (1997). A survey of augmented reality. Presence: Teleoperators and Virtual Environments, 6(4), 355–385.

Augmented Reality in Mathematics Education

345

Bower, M., Howe, C., McCredie, N., Robinson, A., & Grover, D. (2014). Augmented reality in education – Cases, places and potentials. Educational Media International, 51(1), 1–15. Brzezinski, T. (2018a). Augmented reality: Ideas for student explorations. Retrieved from https://www.geogebra.org/m/RKYFdQJy Brzezinski, T. (2018b). Rotating about X-AXIS: Creating surfaces of revolution in GGB AR. Retrieved from https://www.geogebra.org/m/kPNmmHgj Bujak, K. R., Radu, I., Catrambone, R., MacIntyre, B., Zheng, R., & Golubski, G. (2013). A psychological perspective on augmented reality in the mathematics classroom. Computers and Education, 68, 536–544. Burrill, G. (2011). ICT in the United States: Where we are today and a possibility for tomorrow. In A. Oldknow & C. Knights (Eds.), Mathematics education with digital technology (pp. 12–22). London & New York, NY: Continuum International Publishing Group. Collada. (2017). Retrieved from https://www.khronos.org/collada/ Dunleavy, M., & Dede, C. (2014). Augmented reality teaching and learning. In J. Spector, M. Merrill, J. Elen, & M. Bishop (Eds.), Handbook of research on educational communications and technology (pp. 735–745). New York, NY: Springer. Dünser, A., Walker, L., Horner, H., & Bentall, D. (2012). Creating interactive physics education books with augmented reality. In V. Farrell, G. Farrell, C. Chua, W. Huang, R. Vasa, & C. Woodward (Eds.), Proceedings of the 24th Australian Computer-Human Interaction Conference – OzCHI ’12 (pp. 107–114). Estapa, A., & Nadolny, L. (2015). The effect of an augmented reality enhanced mathematics lesson on student achievement and motivation. Journal of STEM Education, 16(3), 40–49. GeoGebra. (2018). Official website. Retrieved from https://www.geogebra.org/ Hwang, G.-J. (2014). Definition, framework and research issues of smart learning environments – A context-aware ubiquitous learning perspective. Smart Learning Environments, 1(1), 1–14. ISO. (2017). Retrieved from https://www.iso.org/standard/59902.html Kaufmann, H., & Schmalstieg, D. (2002). Mathematics and geometry education with collaborative augmented reality. In SIGGRAPH 2002 Conference Abstracts and Applications (pp. 37–41). New York, NY: ACM. Kerawalla, L., Luckin, R., Seljeflot, S., & Woolard, A. (2006). “Making it real”: Exploring the potential of augmented reality for teaching primary school science. Virtual Reality, 10(3–4), 163–174. Kudan. (2018). Official website. Retrieved from https://www.kudan.eu/kudan-sdkfeatures/ MAXST. (2017). Official website. Retrieved from http://maxst.com/

346

Tomaschko and Hohenwarter

NCTM. (2008). The role of technology in the teaching and learning of mathematics. National Council of Teachers of Mathematics. Retrieved from http://mathforum.org/ kb/servlet/JiveServlet/download/204-1745521-6225738-478563/att1.html Pimm, D., & Johnston-Wilder, S. (2005). Technology, mathematics and secondary schools: A brief UK historical perspective. In S. Johnston-Wilder & D. Pimm (Eds.), Teaching secondary mathematics with ICT (pp. 3–17). Maidenhead: Open University Press. Pittalis, M., Mousoulides, N., & Christou, C. (2009). Students’ 3D geometry thinking profiles. Proceedings of CERME, 816–825. Quintero, E., Salinas, P., González-Mendívil, E., & Ramírez, H. (2015). Augmented reality app for calculus: A proposal for the development of spatial visualization. In E. G. Mendívil, P. G. Ramírez Flores, J. M. Gutiérrez, & E. Ginters (Eds.), International Conference on virtual and augmented reality in education (pp. 301–305). Elsevier B.V. Roschelle, J., Shechtman, N., Tatar, D., Hegedus, S., Hopkins, B., Empson, S., … Gallagher, L. P. (2010). Integration of technology, curriculum, and professional development for advancing middle school mathematics: Three large-scale studies. American Educational Research Journal, 47(4), 833–878. Shelton, B. E., & Hedley, N. R. (2004). Exploring a cognitive basis for learning spatial relationships with augmented reality. Technology, Instruction, Cognition and Learning, 1(4), 323–357. Sotiriou, S., & Bogner, F. X. (2008). Visualizing the invisible: Augmented reality as an innovative science education scheme. Advanced Science Letters, 1(1), 114–122. Vuforia. (2017). Official website. Retrieved from https://vuforia.com/ Wikitude. (2017). Official website. Retrieved from https://www.wikitude.com/ Wu, H. K., Lee, S. W. Y., Chang, H. Y., & Liang, J. C. (2013). Current status, opportunities and challenges of augmented reality in education. Computers and Education, 62, 41–49.

CHAPTER 15

Automatically Augmented Reality with GeoGebra Francisco Botana, Zoltán Kovács, Álvaro Martínez-Sevilla and Tomás Recio

Abstract In this chapter we describe a scenario in which automated reasoning and augmented reality are interrelated, yielding an Automatically Augmented Reality (in short: AAR). This is exemplified in the context of “math promenades”, in which GeoGebra’s recently implemented features for the automated manipulation of geometric facts are the basis for the automated development of mathematical layers on top of monuments’ photographic images, captured by smartphones as we walk along these cultural or touristic attractions, as a way to highlight their geometric properties for mathematical education or dissemination purposes.

Keywords automated reasoning – elementary geometry – augmented reality – GeoGebra

1

Introduction

Augmented reality (AR) is entering into the educational world (cf. Restivo, Chouzal, Rodrigues, Menezes, & Lopes, 2014) regarding, in particular, mathematics education (see, for instance, Figueiredo, 2015; Yingprayoon, 2015). A most recent example in this direction is the launching, on September 2017, of a GeoGebra Augmented Reality app,1 allowing the user to use a smartphone to visualize math 3D objects, generated by GeoGebra (GG), on any surface he/she likes and to walk around or inside these 3D objects, while taking screenshots from different perspectives. It is this combination of real scenes and computer-modelled images, and the ability to interact with virtual objects while still in the real world that is characteristic of augmented reality, and distinguishes it from virtual reality, the latter being understood as the mere embedding of the user in a digital, interactive, 3D environment. © koninklijke brill nv, leideN, 2020 | DOI: 10.1163/9789004408845_015

348

Botana et al.

GeoGebra has recently added automated reasoning features (proving, discovering) concerning geometric statements (cf. Botana et al., 2015). Roughly speaking, this means that a GeoGebra user can make a geometric construction, starting from some free points arbitrarily placed by the user, and then he/she can ask GeoGebra to confirm or deny the truth about some observed relation among different elements in the construction, say, whether certain three points P, Q, R, that are part of the figure, are always collinear for any other figure that would be the result of applying the same construction but starting with different placements of the free points. The automated reasoning tools within GeoGebra will then reply by declaring that such alignment is just circumstantial (i.e., that it will disappear if we drag some of the elements in the figure) or, on the contrary, that it is always true for whatever position of the free elements in the given figure, or that it is almost always true, except for some degenerate cases, such as the coincidence of the three vertices of a given triangle, etc. It is in this sense that we say that GeoGebra is now able to prove or disprove the validity of a geometric statement. See Section 3 for some graphic examples of the current performance of Automated Reasoning Tools in GeoGebra. Moreover, if GeoGebra answers that the statement is generally false, the user can invoke another automated reasoning tool in GeoGebra and thus to ask the program to discover the necessary and sufficient conditions for the proposed thesis to become true. That is, let us assume we are given some hypotheses and some thesis, but the thesis does not follow from the hypotheses. Then we want to automatically discover how to add more hypotheses (in some clever way) so that the given thesis now follows the extended set of hypotheses. Say, the user can ask GeoGebra for precise information about where to place some free point X in the figure, so that a certain triple of points P, Q, R, that have been constructed depending on X, will become aligned. Then GeoGebra will output (algebraically and visually) a geometric locus for X that is necessary (but perhaps not sufficient) for the alignment to hold. Next, the user needs to reformulate the given construction, adding the requirement that X lies in the given locus, and then ask again – using the proving features of GeoGebra – to verify if, under this new constraint for X, it holds that always P, Q, and R are aligned. See Section 3 for an example and further details. It must be remarked that in all cases GeoGebra’s answer is the result of a symbolic algorithm – performed with Giac, GeoGebra’s embedded Computer Algebra System, see – after translating into algebraic equations the different elements of the figure and the proposed statement. That is, GeoGebra relies on the exact verification of the correctness of the observed property. It is not based on a probabilistic or numerical, approximate, approach. It is in this

Automatically Augmented Reality with GeoGebra

349

sense that we consider that GeoGebra is able now to rigorously prove or discover geometric propositions. In this chapter, we will aim to describe a scenario in which both tools are merged: automated reasoning and augmented reality, for educational purposes. That is, we will be talking about Automatically Augmented Reality (AAR). What is this about? It involves several facets, that we will discuss below. Thus, most Augmented Reality mathematical applications, such as the GeoGebra AR app, deal with exploring math objects placed on real scenes (e.g., place a Klein bottle over your table – see Figure 15.1). We will pursue a different direction: exploring real scenes by overlaying mathematical information that can help students understand reality more deeply. This can be achieved using the geo-positioning features of current mobile smartphones, or through image recognition techniques, to recognize different landmarks that have been previously added to a landmark database which contains both variables that help identify the landmark, and extra mathematical information about the landmark, such as with the medieval bridge in Granada in Figure 15.2. This approach could take advantage of different smartphone devices (camera, GPS, compass, accelerometer, proximity sensor, thermometer, etc.) to explore this augmented reality context. A detailed description of this approach appears in Section 2, together with its potential application to the development of mathematical competencies for real life and for the didactical utilization of Mathematical Walks, such as the ones described in Martínez-Sevilla and Recio (2017), Martínez-Sevilla (2017), that use GeoGebra as an auxiliary tool.

figure 15.1 Klein bottle visualized through the GeoGebra AR app

350

Botana et al.

figure 15.2 Reconstruction, through a GeoGebra layer, of the ruins of a medieval bridge in Granada (from Martínez-Sevilla, 2017)

But the more ambitious goal of our proposal is, again, two-fold. On the one hand, we assume GeoGebra will, in the near future, be able to automatically convert the images of reality captured on a smartphone into a standard GeoGebra input. This could be done by implementing some techniques for identifying lines, circles, etc. in a given image, such as the ones described in Chen, Song, and Wang (2015) for use in a context quite similar to ours, based on the Hough transform.2 The technical issues of digital image processing go beyond the scope of this chapter. Once the visual representation of reality has been turned into a precise GeoGebra input, we aim to help achieving the automatic discovery of the existing mathematics lying behind this geometric version of reality by automatically and systematically deploying GeoGebra reasoning tools over the input, as described in Section 3. This is the task that we would like to be performed, in the near future, by the Automated Geometer module to be implemented in GeoGebra, as described in Section 4 of this chapter. Finally, Section 5 reflects on the educational changes, advantages, and dangers that could follow the implementation and popularization of our envisioned AAR scenario.

2

GeoGebra Mediation between Mathematics and Reality

AR has arrived to stay in general educational settings and, in particular, in mathematics education. Yingprayoon (2015), discusses the use of the AR

Automatically Augmented Reality with GeoGebra

351

markers app in mathematics education and Figueiredo (2015) uses both AR markers and markerless AR apps. In all these references, and in many others, the idea behind the educational use of AR is to facilitate the learning process by enhancing the student visualization of mathematical objects. Also, AR is used to increase student motivation, as AR invites inquiry-based approaches. Nevertheless, in all these references, the actual reality is just a neutral setting for abstract objects, so, for example, the Klein bottle in Figure 15.1 remains the same, no matter whether it is placed on a desk, on a courtyard floor, or on a bench (as in Figure 15.1). In some other mathematics AR apps, such as those translating handwritten formulae to typewritten ones, we can say that AR is just an interface between two identical mathematical realities, represented in different forms. In all these AR examples we have two parts, a maths layer and a reality layer, and there is little dependence between them. As we have outlined above, our proposal is to link mathematics to reality in such a way that the first item actually strictly depends upon the second one. In fact, in our approach, we will add, as in the standard AR apps, a mathematical layer over reality. But in our approach, the layer is developed by computing mathematical properties of the captured image of reality. Thus, if we change the real input, that would mean we will also have to modify the mathematics augmenting our understanding of such reality. We think that establishing a tight interplay between the two actors, math and reality, facilitates the solving of mathematical problems dealing with a concrete reality, it is more adequate as a didactical tool, and it contributes to motivate the student. GeoGebra has become, for its versatility, technical soundness, and popularity, the most suitable mathematical software for implementing the kind of AR tools we are thinking of. Let us briefly describe what could be some characteristics of a pair of GG-based apps aiming to interact with reality in a substantial way. In the first one (see Figure 15.3, attempting to analyse the barycentre of the sculpture “El Instante Preciso”, by Guillermo Pérez Villalta, in the Granada City Hall Square) the mathematical layer will have to be modified following the changes of the reality it refers to, so, for instance, if we look at a different sculpture, then our automated augmented reality device should display a new math layer helping finding the sculpture’s centre of gravity. Notice this dependence goes far beyond the obvious modification of the display according to the changes of the user point of view and in that sense we could say the interdependent mathematics-reality we are building is, comparatively, very weakly user-perspective dependent (thus, in some exaggerated sense we could

352

Botana et al.

figure 15.3 Dependent maths and reality. Independent maths and user viewpoint (from Martínez-Sevilla, 2017)

talk about independency), although the visualization of the synchronous math and reality layer, of course, will have to follow user’s perspective. In the second kind of apps (see Figure 15.6 for one prospective example), the mathematical layer that is constructed over the reality strictly depends on the user point of view, or on the location of the mobile device that runs the app. Obviously, changing the user position will produce a change in the different measurements and intermediate computations performed by these apps towards finding the height of a certain building, even if the final result is constant. Thus, the mathematical layer of these AR apps will depend upon the relative position of the analysed reality with respect to the observer’s point of view. Here, the AR apps involve mathematics that are both dependent on reality and on the user point of view. Next, let us describe two large areas of potential didactical application for these two types of GeoGebra-based AR apps we have been envisioning, namely, what we are calling mathematical walks and real-life mathematics.

Automatically Augmented Reality with GeoGebra

353

2.1 GeoGebra Mathematical Walks Mathematical Walks are routes through real-world settings that allow working out different mathematical aspects of real-world objects and locations: studying geometrical shapes, elaborating models of some natural phenomena, establishing measures of different objects and doing approximate mathematics with these data, etc. Such routes can be followed through a diversity of landscapes, from urban environments to natural settings, including mountains, beaches, forests and geologically significant sites. Bearing in mind all these possibilities, we think that the most fertile walks for our purposes could be the ones we are calling Mathematical Walks through Ancient Monuments, because the involved mathematics are surprisingly so rich and elaborated. The pedagogical value of such trails along Ancient Monuments is not restricted to the field of mathematical sciences, it can also promote historical and artistic knowledge (Martínez-Sevilla & Recio, 2017), in a true STEM collaborative approach (see Restivo et al., 2014) for different arguments in this regard). For Mathematical Walks, both GeoGebra and the AR app would be available through a mobile phone. In this context, roughly speaking, we think of two techniques to superpose a mathematical layer onto the screen capture of a mobile device camera. The most common one is by means of geo-positioning and compass, where, once the user has reached a certain location and its GPS coordinates have matched some previously recorded ones, by pointing the camera towards some associated prescribed direction, the mobile device automatically downloads a GeoGebra file with the corresponding mathematical layer for the artistic monument that should be envisioned in that precise place and position (see Figure 15.4). One important source of the necessary mathematical layers for this approach could come from the European Erasmus + Project MoMaTre (http://www.momatre.eu), which attempts to provide a database of mathematical trails. The project is conceived in a collaborative manner, with several researchers’ and math teachers’ societies of different European countries, enterprises, and institutions. The associated content for the mathematical walks is being elaborated in the form of PDF documents, with derived didactical activities, that could be geo-positioned in different locations all over Europe. The other technique we are thinking of for these Math Walks through Ancient Monuments is image-recognition based. The geometry of some object in reality (e.g., a monumental facade, a frontal view, or a window arch) should be identified or defined using pattern recognition (e.g., through Hough transform, as mentioned earlier) without the need of geo-positioning or markers. This would give more flexibility to superpose a mathematical layer with the

354

Botana et al.

figure 15.4 Mathematical layer over a renaissance palace facade (Martínez-Sevilla, 2017)

GeoGebra interpretation of the given object, but it will require also more computational power, something not far from the capabilities of the newest smartphones. Of course, a combination of these two techniques is also conceivable: perhaps, when scanning a car – as displayed in Figure 15.5 – the system uses only image recognition, but when scanning some other more permanent object (say, a building), it can both use image recognition and also check to see if there are any matching entries in a “mathematical walks” database. Moreover, as described in Section 4, GeoGebra could be able to mechanically recognize geometrical relations over this mathematical scheme associated to the real object, producing, in this automatic way, a richer Augmented Reality output. In summary, we think that GeoGebra is perfectly positioned to implement, in the near future, some apps dealing with AR in the context of mathematical walks. In fact, GG is already present in mobile phones and by including some new features into GeoGebra, such as those related to 3D mobile cameras and to geo-positioning, GeoGebra could be able to download and show layers concerning mathematical walks already done by others. Moreover, GeoGebra

Automatically Augmented Reality with GeoGebra

355

figure 15.5 Measuring a distant object with a mobile app (Source: Smart Distance Pro app)

could be enriched with some specific tools to help along math walks, such as including a menu with items and buttons for the analysis of ratios, arches, curves, tessellations, etc. The complete development of AR GeoGebra apps could turn GG into an even more powerful tool for STEM education, as well as for scientific dissemination. It could be a reference tool for educational projects and conferences or flipped classroom techniques. We think that Mathematical Walks, increasingly including more immersive 4.0 technology (i.e., generating a three-dimensional image which appears to surround the user),3 could have a thrilling future in mathematics education. And GeoGebra deserves a leading role in this respect. 2.2 GeoGebra and Real-Life Mathematics (GG_RLM) Adding images – for instance, captured through a mobile phone – to the GeoGebra screen is, currently, the only possibility for incorporating elements of the surrounding reality in GeoGebra. But most of the power of the many other devices and sensors included in smartphones remains totally unexplored for use with GG. Devices such as high-resolution cameras (to be used for measuring distances, etc., beyond the traditional solitary task concerning the capture of images), GPS, magnetic field sensors, accelerometers, gyroscopes, proximity sensors, microphones, pedometers, pulsometers, barometers, thermometers, and others are now incorporated into high-end mobiles, which could be connected to GG. These devices have been already used by numerous apps, allowing a wide range of exchange with reality, for instance, through linear, surface, and volumetric measurements. This is usually done with different marginal errors, although with more than acceptable precision. In the near future, the connection of GG with these and other similar sensors would make GG even a more versatile app, able to deal interactively with real-life

356

Botana et al.

problems and to become an effective tool for other fields of interest, beyond mathematics. For example, in physics, biology, geology, geography, archaeology, agriculture, engineering, or architecture or, speaking generally, fields in which problems arise with some initial component or starting data that have to do with the measurement or integration of aspects of the surrounding reality. Thus, in this future we are envisioning, we could use GG to interactively determine the height of a tree in front of us, the surface of the shade that it casts during a certain day, or the measurement of a terrain’s surface on a mountain side with different average slopes per stretches, etc. It would be undoubtedly a crucial educational contribution for the achievement of mathematical competency and, thus, for the connection of mathematics to many other disciplines, making mathematics more contextual through GG as a tool that helps in this direction in an easy and intuitive way. Among the menus and tools that could be added to GG to deal with this particular approach, we could think about providing: – Menu of linear and surface measurements – Caliper tool, with a detailed precision (mm) for certain objects of size smaller (to be able to keep mm precision) than that of the mobile’s screen. – Measuring tape tool, with a precision of cm, obtained through the linear accelerometer. – Rangefinder tool, with a precision of dm for large distances, achieved by means of the camera and the gyroscope (for the measurement of angles) and elementary trigonometry to calculate straight-line distances to a distant point. – Land surface tool, using GPS tracking to measure boundaries of a surface of considerable size. – Protractor Menu – Inclinometer – Angular measurement tool between two objects, by means of the phone’s gyroscope. – Measurement tool for inaccessible points (with an indication of the error margin). It could be carried out by merging several options. One of them is using the rangefinder mode over short distances (tens to hundreds of meters). Another could be the angular measurement mode, involving some calculation using the sine or the cosine theorem over the inaccessible distance. – Sextant tool, which performs the simulation of a classic sextant. To this tool, a small telephoto lens accessory for the camera of the device could be added (with the main function of optical field delimitation). Nowadays there are several apps that carry out some of these functions (or similar).

Automatically Augmented Reality with GeoGebra

357

So, what would be the advantage of having these tools available in GeoGebra? Such integration would allow GG performance beyond a few standard linear, surface, or angular measurements. In this new context, the users could deal with a wide range of more sophisticated problems which require some added mathematics, for example, as sketched above, concerning the measurement of distances to inaccessible points. Doubtlessly, in this scenario, developers and users will easily discover multiple real-life problems and relevant didactical applications. As a first example, let us consider the measurement of a tower by using the trigonometric calculation by differences. That is, we assume, that it is not possible to know the distance from the user position to the base of the tower. The following image (Figure 15.6) shows what a GG screenshot in this situation could be like. It would incorporate measures obtained by the GeoGebra Real Life Mathematics (GG_RLM) app (shown in black), a geometric approach to the problem and the final solution (shown in red). A second example of the use of such an app could be the following: Assume we want to calculate the height of a virtual horseshoe arch being reconstructed from a small, partial remnant of it (see Figures 15.2 and 15.7; In fact, this is a real problem, which arises in the context of (Martínez-Sevilla, 2017). Then, we could proceed by measuring the cord AB (in yellow, Figure 15.7) of the remaining part of the arch, using the Pythagorean Theorem, after determining the

figure 15.6 Imagined screen of a GG_RLM app task, computing the height of a building

358

Botana et al.

figure 15.7 Imagined screen of GG_RLM app, computing the height of a horseshoe arch

distances FA and FB. Then, GG_RLM can proceed with a reconstruction of the whole arch geometrically, with its centre C and radius R (blue colour, 4.1 m) yielding the total height of the arch, CL-H, of 5.74 m. This solution can also be interpreted in an historic-artistic context: the elevation (CL-C), 1.64 m., of the centre of the arch over the basis line, is about 2/5 (40%) of the length of the radius R, as was usual for the caliphal architecture in Spain (8th to 10th centuries), though the construction of this particular arch over the Darro river in Granada, down the Alhambra hill, is dated on the 11th century. Summarizing, the previous paragraphs presented two early attempts to realize the potential applications of augmented reality apps obtained by connecting GeoGebra with some smartphone sensors. In the above examples, we have just used sensors capable of linear, surface or angular measuring. These are by far the most immediate sensors for developing mathematical tasks, allowing us to deal with different math variables, or even with some physical ones, like velocity. The possibilities for connecting new sensors with GeoGebra are countless. We can think of magnetic field sensors, proximity sensors or other specially designed ones, like laser range-finder sensors. In the latter case, this kind of very specialized sensor could be added by the industry just on some models oriented to engineering or mathematics, thus providing specialized tools of great precision and flexibility for solving problems in these fields. This proposal merges GeoGebra with measuring sensors, giving us a deeply interrelated Math, Reality, and User-Viewpoint tool. In this way, we could end

Automatically Augmented Reality with GeoGebra

359

up displaying on a smartphone or notepad, a mathematical model of this reality, accompanied by the effective solution of the concerned problem: the digital counterpart for the old-fashioned student’s paper and pencil notebook!

3

GeoGebra Automated Reasoning Tools

As mentioned in the Introduction, since 2015, GeoGebra has included features allowing the automatic derivation, verification, and discovery of geometric statements. Here, by “automated derivation of geometry statements” we refer to a tool that can be asked to provide information about the existence of some relation verified by selected elements within a geometric construction. For instance, given a free point A and three points B, C, D on a given line, consider E, F, G, the midpoints of segments AB, AC and AD and ask GeoGebra about them.4 Then, the automatic derivation tool (here, the Relation command, see Figure 15.8) should output the numerical verification of some property holding among them for the concrete positions of the input data A, B, C, D, such as the alignment of E, F and G. On the other hand, the “automated discovery of geometric statements” refers to tools able to find, in an algorithmic style (i.e., not by merely searching a collection of precomputed examples), complementary, necessary, hypotheses for the truth of a conjectured geometric statement. For example, given an arbitrary triangle ABC and an arbitrary point D, let D’, D’’, D’’’, be the symmetric images of D with respect to the sides of the triangle (taken from Recio & Vélez, 1999). Then D’, D’’, D’’’ are collinear. Obviously, this conjecture is false but

figure 15.8 The automatic derivation tool gives the numerical verification of some property holding among them for the concrete positions of the input data A, B, C, D, such as the alignment of E, F and G

360

Botana et al.

figure 15.9 The automatic discovery algorithm should be able to output the necessary (and sufficient) location for D in order to have the collinearity of D’, D’’, D’’’

… the automatic discovery algorithm should be able to output the necessary (and sufficient) location for D in order to have the collinearity of D’, D’’, D’’’ (see Figure 15.9). Finally, in this context, when we mention “automated proving of geometry statements” we mean algorithms that accept as input a geometric statement, such as this one taken from Howson and Wilson (1986): “If two lines are drawn from one vertex of a square to the midpoints of the two non-adjacent sides, then they divide the diagonal into three equal segments” (see Figure 15.10). Then, the algorithm translates the given geometric data into a collection of algebraic equations, performs some computational algebraic geometry techniques, with exact handling of the involved quantities (i.e., not using floating point numbers), and outputs a mathematically rigorous (not approximate or based upon a probabilistic case analysis) yes/no answer on the truth of the given statement. That is, the algorithm performs a mathematical proof internally (it is not readable by human users). In all cases, we have to start by drawing a geometric construction in GeoGebra. Then we can do some visual investigation, by using the Relation tool to compare objects and to obtain relations among them, or by using the Locus tool to get the trace of a point, as if it would have been dragged subject to the constraints imposed by the given construction. These methods are well known

Automatically Augmented Reality with GeoGebra

361

figure 15.10 If two lines are drawn from one vertex of a square to the midpoints of the two non-adjacent sides, then they divide the diagonal into three equal segments

by the GeoGebra community and well documented at the GeoGebra Materials web (https://www.geogebra.org/materials/). But these methods are basically visual or numerical, so their output is valid only on the specific construction with concrete coordinates assigned to the free elements, so Relation and Locus answers do not apply to general statements. Yet, a merely visual conclusion in a specific instance can provide some hints about properties that could hold in general. In fact, the More option (see Figure 15.3) of the Relation output can be then used to re-compute the results symbolically, establishing – through the internal call to the Prove and ProveDetails commands – the rigorous truth, the general validity (or failure) of the conclusion that appears to hold in the specific case displayed by GeoGebra. Likewise, the LocusEquation command can refine the result of the Locus command by displaying the algebraic equation of the graphical output. An algebraic equation that can be, then, interpreted by the user as providing new, complementary hypotheses for the general validity of the constraint that motivates the locus, yielding conjectural statements such as “If point D lies on the circumcircle of a triangle A, B, C, then the symmetrical images of D with respect to the sides of the triangle, D’, D’’, D’’’, will be collinear” (see Figure 15.9), which could then be checked through the Prove command.

362

Botana et al.

See Botana et al. (2015) for a more detailed description of the context, motivation, and implementation on GeoGebra reasoning tools, as well as Kovács, Recio, and Vélez (2018) for a complete tutorial on its use; moreover, we refer to Recio and Vélez (1999) for a technical analysis of the involved mathematics and references.

4

GeoGebra as an Automated Geometer

In this section, we will assume that lines, circles, etc. in a given image can be algorithmically identified and GeoGebra will be capable of interpreting photos or screenshots of reality and translating them into a precise internal data structure. On the other hand, as described in the Introduction, here we will emphasize describing how GeoGebra can automatically obtain geometric information from this input that mathematically represents the real object in the image. In order to provide a simple example, we will refer to a parquet floor that is captured by a smartphone camera, as seen in Figure 15.11, and which has had some of its geometrical features identified as GeoGebra objects. Here we assume that the detected input is restricted to ten points: C, D, E, F, G, H, I, J, K, and L, and to 13 segments, from f to s. For a further simplification, we will focus only on 6 points and 7 segments, namely C, D, E, H, I, J, and f, g, j, k, n, p, and q. By comparing these numerical objects, we learn that some of the quantities are approximately equal. For example, the lengths of segments f and g differ by

figure 15.11 Imaginary detection of reality with an internal representation of a parquet floor being photographed by a smartphone and translated into GeoGebra

Automatically Augmented Reality with GeoGebra

363

about 3%, and j and k differ by about 6%; finally, n, p, and q are approximately equal. Also, we notice that the points C, D, and E look collinear, and the same property can be observed for points H, I, and J. Also, the segments f and g seem to be parallel to the lines j and k, and the same holds for the lines n, p, and q. Clearly, these pieces of information are usually inaccurate, perhaps because of imprecise measurements or by minor variations in the actual parquet pieces. But the user has the possibility to collect more than one sample of the reality by simply continuing capturing some other shots, and for a second or a third sample the above observations can be presumably refined. That is, we can assume that some “ideal”5 properties of the reality can be faithfully detected in some consecutive steps; perhaps the user would be able to set the reasonable level of imprecision (say, 3%) for the measures of a particular object being analyzed by GeoGebra. Now GeoGebra could label the collected properties, in our case we could have decided that the following facts are taken as holding over the mathematical model of that parquet piece, as represented in GeoGebra: p1: “f=g”, p2: “j=k”, p3: “n=p”, p4: “n=q”, p5: “C, D and E are collinear”, p6: “H, I and J are collinear”, p7: “f and j are parallel”, p8: “n and p are parallel”, p9: “n and q are parallel”. Here we remark that this list should be kept as short as possible, because, for example, the unlisted property “f and k are parallel” clearly follows from p6 and p7. As a next step, conjectures will be built by using these properties. Hopefully a subset of these properties could be enough to mathematically conclude all of them, in the simplest case only one property could be needed. To test if this is the case here, let us consider, for example, the conjecture {p1, p2} → {p5}, which is illustrated in Figure 15.12.

figure 15.12 Internal analysis of a conjecture in GeoGebra

364

Botana et al.

Clearly, this conjecture is false. The conjunction of both hypotheses, properties p1 and p2, is not sufficient to make the thesis property p5 hold in general. After some combinations of the given properties, we might identify {p1, p2, p5, p6, p7, p8} as the main set of hypotheses, and we will see that all of the remaining properties follow from them. Indeed, for example, we can check if p3 actually is a logical consequence (i.e., not just in this particular Figure 15.11) valid on all geometric constructions verifying {p1, p2, p5, p6, p7, p8}. To have the correct geometrical setup, GeoGebra needs to perform a sophisticated algorithm producing a rigorous Euclidean construction of a figure that can be used to verify (or disprove) the conjecture. In Figure 15.13 we have used the fact that the combination of p1 and p5 implies that point D can be taken as the midpoint of the free points C and E. Also, after creating the perpendicular lines t and u, the point J has been created by reflecting the free point H across the intersection point I of t and u, etc. More details about this construction can be obtained from the Algebra column in Figure 15.13: for instance, it describes that points C and E are taken as two freely given points to start the construction process; then point D is built as the midpoint of C and E, then f is Segment(C,D), etc. In the same column, the fact that H is written with a blue color means, in GeoGebra that, again, it is a freely given point; the 7th command line in that column, t=Line(H,f ), means that line t is built as the line parallel to f through H, etc. The figure does not display the labels for segment j or k, but, according to the notation in Figure 15.11, is clear that j is Segment(H, I) and k is Segment(I,J).The parallelism of f and j (property p7) is an obvious consequence, in the figure, of the construction process, since j is on line t and t is parallel to the segment f by definition, etc.

figure 15.13 Internal analysis of another conjecture in GeoGebra

Automatically Augmented Reality with GeoGebra

365

figure 15.14 GeoGebra confirmation of statement

Once this purely geometric Figure 15.14 has been built, verifying all the given hypotheses {p1, p2, p5, p6, p7, p8}, we can try to deduce if necessarily p3 also holds here. Then, we can use the Relation tool (Figure 15.9) to show that GeoGebra is already capable of determining the truth of the conjecture {p1, p2, p5, p6, p7, p8} → {p3}, by clicking on the segments n and p after selecting the Relation tool (see also Section 3). Actually, the property p8 is also listed in the window, but this is a trivial deduction for us, since p8 is listed among the hypotheses. The obtained information could also be communicated to the user in a more practical way, in an automated augmented reality application. Namely, the user might be informed on the truth of the statement {p1, p2, p5, p6, p7, p8} → {p3, p4, p9}, without having to do anything, that is, in a completely automatically way. Of course, this simple theorem {p1, p2, p5, p6, p7, p8} → {p3}, is not the only one that can be deduced. A much simpler one is, for example, {p3, p8} → {p7}. Surprisingly, the hypotheses for this theorem also implies f=j, a property that is not listed among the observed ones. The reason why f=j has not been collected as a basic fact could be that the photographed image is a 2-dimensional representation of a 3-dimensional reality and, sometimes, setting some properties correctly requires a conversion step that should correct the input by a projective transformation first! To summarize, we can sketch up a possible algorithm for the Automated Geometer, as follows:

366

Botana et al.

1.

Collect some consecutive photographs of the reality and (after a projective transformation) convert them to 2- or 3-dimensional GeoGebra constructions. 2. Collect common properties of the GeoGebra constructions including similar lengths, collinearity, parallelism and perpendicularity (for the moment; obviously, this list can be enlarged to other GeoGebra items). 3. Label the collected properties as p1, p2, …, pN. Let P={p1, p2, …,pN}. 4. Create a subset S of P and check for every p in P\S whether S→ {p}. Communicate the obtained theorems to the user. 5. On demand, continue with step 4 by choosing a different subset. We remark that GeoGebra ART is designed to work with 2D objects. Therefore, it should be either extended to 3D, or the translated objects must be analyzed in a planar model. Finally, we would like to mention that, at this initial stage, we regard parquet floors as some basic, usual, objects for making indoor experiments with GeoGebra Automatically Augmented Reality. In general, many rectangular or grid-like objects could be also observed including doors, windows, simple tables and chairs, or tiles. For outdoor experiments, simple houses, fences, walls and stairs are expected to be the basic samples.

5

Conclusions

In this chapter, we have presented some proposals concerning the development of Augmented Reality issues within GeoGebra, in two different ways. First, by connecting GeoGebra with some of the many sensors already present in mobile devices, thus enriching the capabilities of GeoGebra as a tool for mathematically analyzing reality, and for mathematically augmenting reality, as regarded through the eyes of these devices. Imagine you look to a monumental door through this future GeoGebra AAR app and you will perceive both the real door and the added fact that its design involves the golden ratio. Two different sorts of apps, concerning Mathematical Walks and Real-Life competency, have been described and illustrated by some examples. The second approach relies on the establishment of a trio of links: the link between reality and GG input, the link between GG input and the Automated Geometer, and the link between the Automated Geometer and AAR. The first step, to be developed along the lines of some previous work in the same direction, (Chen et al., 2015), is related to the first link and attempts to convert an image of reality captured by a device’s camera into a GeoGebra input that can be examined mathematically. Then, establishing the second link involves

Automatically Augmented Reality with GeoGebra

367

analyzing such reality by automatically using GeoGebra automated reasoning tools (see Section 3), as described in Section 4. Finally, the result of this automatically generated knowledge should be integrated in the mathematical layer over the screen, thus augmenting the device’s presentation of reality. It is hard to evaluate and to predict what could be the impact, in the educational world, of having at hand such an automated geometer, automatically augmenting reality, a kind of geometric calculator that operates by itself by merely responding to some visual inputs, without requiring the user to formulate questions. Finally, we should emphasize that in our proposal we focus on describing the near future availability, for the student, of a tool with (in practice) a kind of unlimited geometric knowledge; but we do not have a clear picture about how to use it. Perhaps the most reasonable panorama could be to consider GeoGebra’s Automatically Augmented Reality as a kind of guide that could allow the teacher and the student to improve their journey into the surrounding geometric reality, enriching their activities towards improving the acquisition of mathematical competencies. An important consideration on the potential impact of these tools in the future of mathematics education, is the current availability of GeoGebra (GG) over computers, notebooks, and smartphones, disseminated all over the world6 and reaching millions of users. In this regard, our proposal does not want to be just a visionary science fiction disquisition, but an urgent warning call to the community of math teachers and math education researchers, to help us in addressing the pros and cons for the educational community of the availability of such GeoGebra-based AAR app, and thus guiding the development of this potentially powerful and challenging instrument.

Notes 1 2 3 4

https://itunes.apple.com/us/app/geogebra-augmented-reality/id1276964610?mt=8 https://en.wikipedia.org/wiki/Hough_transform https://en.wikipedia.org/wiki/Industry_4.0 See https://www.emis.de/data/projects/reference-levels/EMS_RQ_BUNDLE_ENGLISH.pdf, Reference question No. 046. 5 Ideal: i.e. the mathematical object that is close to representing reality, as a rectangle is the “ideal” mathematical representation of the frame of a picture, albeit the real frame can have boundaries that are not, strictly speaking, parallel or perpendicular with infinite precision. 6 See https://www.geogebra.org/partners#material/Ez5ZSdDg

368

Botana et al.

References Botana, F., Hohenwarter, M., Jani Čic, P., Kovács, Z., Petrovic, I., & Weitzhofer, S. (2015). Automated theorem proving in GeoGebra: Current achievements. Journal of Automated Reasoning, 55(1), 39–59. Chen, X., Song, D., & Wang, D. (2015). Automated generation of geometric theorems from images of diagrams. Annals of Mathematics and Artificial Intelligence, 74(3–4), 333–358. Figueiredo, M. (2015, June 24–27). Teaching mathematics with augmented reality. 12th International Conference on Technology in Mathematics Teaching, Faro, Portugal. Retrieved from http://w3.ualg.pt/~mfiguei/papers_pdf/15_ICMT12.pdf Howson, G., & Wilson, B. (1986). ICMI study: School mathematics in the 1990’s. Cambridge: Cambridge University Press. Kovács, Z., Recio, T., & Vélez, M. P. (2018). GeoGebra automated reasoning tools: A tutorial. Retrieved from https://github.com/kovzol/gg-art-doc/blob/master/pdf/ english.pdf Martínez-Sevilla, Á. (Ed.). (2017). Paseos matemáticos por Granada: Un estudio entre arte, ciencia e historia. Ed. Universidad de Granada. Martínez-Sevilla, Á., & Recio, T. (2017). GeoGebra y realidad aumentada en la divulgación de la matemática. III Foro Iberoamericano de Divulgación y Cultura Científica (Córdoba, Spain, 2017). Retrieved from http://www.oei.es/historico/ divulgacioncientifica/IMG/pdf/orales-3.pdf Recio, T., & Vélez, M. P. (1999). Automatic discovery of theorems in elementary geometry. Journal of Automated Reasoning, 23, 63–82. doi:10.1023/A:1006135322108 Restivo, T., Chouzal, F., Rodrigues, J., Menezes, P., & Lopes, J. B. (2014). Augmented reality to improve STEM motivation. In Proceedings of Global Engineering Education Conference (EDUCON), 2014 (pp. 803–806). New York, NY: IEEE. Yingprayoon, J. (2015). Teaching mathematics using augmented reality. In Proceedings of the 20th Asian Technology Conference in Mathematics (pp. 384–391). Leshan, China.

CHAPTER 16

Augmented Reality in Museums and Cultural Heritage Settings Georgios Papaioannou

Abstract This chapter reviews augmented reality applications with emphasis on cultural heritage settings, particularly museums. It presents the history and the evolution of augmented reality programmes and applications in museums and cultural heritage institutions. Examples of good and poor practices are examined. In presenting these examples, we discuss issues related to the use of augmented reality in museum environments and within museum educational programmes. We conclude with thoughts on future research and development.

Keywords museum – big data – visitors – libraries

1

Introduction It was so cool!!! The little girl on the painting got alive and talked to us about her life. (student’s comment) … magical!!! (student’s comment) Thanks to the app, my class and I spent time on paintings we wouldn’t spot otherwise. I was very satisfied with the app’s content for final-year primary-school students, and found it useful and interesting. I would have it differently for students in smaller grades. In my opinion, students solidly learnt new information thanks to the app. They may have heard exactly the same words from a tour-guide but their interest would not have been the same. They would get tired and bored. (teacher’s comment)

© koninklijke brill nv, leideN, 2020 | DOI: 10.1163/9789004408845_016

370

Papaioannou

The rest of the exhibition seemed pointless and completely unattractive. This is a pity. (teacher’s comment) The aforementioned comments were responses to a survey on students’ and teachers’ views after using the “Five Paintings talking to children …” Augmented Reality (AR) application in the Corfu Art Gallery, Greece, in 2017 (Papaioannou & Paschou, 2017). It is indicative of the impact of museum and cultural heritage augmented reality apps to students, teachers and visitors. Starting from the real thing, which can be anything from an image and a museum object to a masterpiece of art and a monument, visitors’ mobile devices get triggered and gain access to rich media digital content including images, videos, sounds, 2D and 3D graphics, animations, multimedia applications, and games. The one and only real object becomes the trigger for endless related digital material and narratives. Opportunities are unlimited and inexhaustible. Pitfalls are too. This chapter offers a view to augmented reality (AR) applications and impact in museums and cultural heritage settings by discussing specific examples of recent AR applications in those contexts. The aim is to critically address good and poor practices towards informing decision making processes on introducing, developing and maintaining (or not) AR applications in museum and cultural heritage settings. The chapter concludes with reflections, recommendations and expectations regarding the future of augmented reality in museums and cultural heritage institutions.

2

A Brief History

Augmented reality (AR) has a long history of experiments and applications in multiple fields, from the industry to medicine, from the military to education (Koutromanos, Sofos, & Avraamidou, 2015; Schmalstieg & Höllerer, 2016). One of its many definitions is “a technology that allows overlaid digital content into our view of the real world through the camera of a Smartphone or a tablet. Video, audio, 2D and 3D images, web and text are just some examples of the type of content that can be overlaid on our perception of the real world” (Villarejo, González-Reverté, Miralbell, & Gomis, 2014). Smartphones and tablets are common devices for AR applications. In recent years, AR has been an attractive, useful and increasingly popular medium for museums and cultural heritage organisations (American Alliance of Museums (AAM) & Museum Association (MA) Members, 2012).

Augmented Reality in Museums and Cultural Heritage Settings

371

Before the now-popular smartphones and tablets, other media were used to introduce augmented reality into museum spaces. AR practices started with audio content and devices. According to Loïc Tallon, the first Ambulatories Lectures were set in 1952 in the Stedelijk Museum in Amsterdam via shortwave radio broadcasting, still analog, in Dutch, French, English, or German, transmitted inside a loop (Tallon, 2008). As it was broadcasted from a single tape, everyone had the same information at the same time. In the late 1970s, the itinerant exhibition “Treasures of Tutankhamon” in the USA used audio tapes and portable cassette players to provide a guided tour of eight stops. It was a huge success with over three million visitors/users (Tallon, 2008). This technology enabled visitors/users to follow the guided tour at their own pace. Since then, the appearance of multimedia devices at the end of the 20th century has broadened the possibilities for interactives and interactive visits (FIlippini-Fantoni & Bowen, 2008). In the 21st century, augmented reality has taken the form of audio/video enhanced tours using smartphone applications triggered by QR codes and/or the objects themselves via the mobile device’s camera. AR applications usually encompass a Global Positioning System (GPS) and a compass, so that the holders’ position be identified and routes be suggested, recorded and followed. Tours are available inside museums as well as outside, in open-air spaces, such as archaeological sites and historical places. Real information is supported by endless layers of additional digital information, offering tools and opportunities for engagement, interactivity, education, and gaming. Interactive guided AR tours have been spread all over the world in the form of AR apps. To mention but a few characteristic examples, in both the British Museum and the Louvre Museum, a visitor/user can move freely in the galleries by using applications with Global Positioning System (GPS) tracking, interactive maps, 2D or 3D reconstructions of masterpieces with zoom in/out and rotation options, thematic guided tours and audio comments (British Museum, n.d.; Musée du Louvre, n.d.). This is the most common use of augmented reality in museum and cultural heritage institutions. Other museums and heritage institutions have used augmented reality to attract visitors by narratives through serious games (Koutromanos & Styliaras, 2015), like the “Spy in the city” game launched by the Spy Museum in Washington DC (MuseumMobile, 2009; Spy Museum, n.d.). Augmented reality has also been used for suggested reconstructions of monuments, buildings of architectural importance, and/or archaeological/historical sites. The aim has been to give a better perception of how they might have looked at a certain time in the past. Moreover, there have been attempts to superimpose a digitised

372

Papaioannou

old photograph or other digital graphic over the image of a real building or object seen through the mobile device’s camera (Museum of London – Street Museum, n.d.; Vlahakis et al., 2002; Younes et al., 2017). Note that augmented reality endeavours in museums and cultural heritage settings have been limited so far to mobile handheld devices running visual- and audio-oriented applications. There are very few AR applications that engage other sensory modalities (see Sections 3.3 and 3.4) and are to be further explored in the future (Naimark, 2018).

3

Discussing Examples of AR Practices

In this section, we will address five AR examples in museums. The examples were chosen for their variety of museum type (big museums to modest ones, national, regional or municipal), the diversity of their exploration, and the documentation available. This survey is not exhaustive. 3.1

Skin & Bones, Smithsonian’s National Museum of Natural History, Washington DC, USA ( January 2015 to Present) Skin & Bones is a free app launched in January 2015 at the Smithsonian’s National Museum of Natural History in Washington DC, USA. The app relates to the “Bone Hall” at the Smithsonian’s National Museum of Natural History and addresses issues the curators were facing in engaging visitors in an active experiment in the “Bone Hall.” The “Bone Hall” was set up in 1965 with a historical collection coming from the late 19th century. The presentation of exhibits has not changed since 1965, and the public was just crossing the five exhibition rooms with little consideration for the display. The precise labels were appropriate in stating taxonomic terms, but inappropriate in making the larger public interested and engaged. As this is a historical collection, Robert Costello and Diana Marques (museum staff) chose to offer visitors a new way of exploring the display without distorting the old Victorian narrative (Costello & Marques, 2015; CPNAS, 2015; Marques & Costello, 2015; SmithsonianMobile, 2013). The project received a grant from Booz Allen Hamilton (Ding, 2017). The free AR app can easily be downloaded before the visit. The museum also offers free Wi-Fi, so that the app can be downloaded at the museum before starting. Users get access to a 2D map and 3D navigation through QuickTime Virtual Reality panoramas, in which they can spot the app’s thirteen (13) skeletons out of the three hundred (300) skeletons of the collection and exhibition in the “Bone Hall.” The app provides a five-icon menu: (a) Animal life, (b) Meet the scientist, (c) Big Idea, (d) Activity, and (d) Skeleton Works. Animal life gives

Augmented Reality in Museums and Cultural Heritage Settings

373

access to short videos of animals in their natural context. Meet the scientist introduces visitors to past and present scientists at the Smithsonian’s. Big Idea offers a video on selected scientific issues related to the animal. Activity turns the mobile smart device into a haptic accessory reproducing an action of the animal, e.g. tapping the screen to imitate the sound and behaviour of a woodpecker (Marques & Costello, 2015; SmithsonianMobile, 2013). Last but not least, the Skeleton works icon menu is associated with AR symbols for ten (10) of the thirteen (13) skeletons. These symbols are next to those skeletons and trigger static or dynamic 3D digital models of the respective animals (Smithsonian’s National Museum of Natural History, 2015). Educators have written short texts for all digital models. Actors recited these texts inside the “Bone Hall.” Sound creators added soundscapes to match respective ecosystems. Graphic designers took soundscapes and texts/narratives into account and created models and animations. Video artists supported the text and the soundscape by special features, such as the movements of the hyoid bone and tongue of the Pileated Woodpecker when catching insect or the venom flowing through the fangs of the Diamondback Rattlesnake (Marques & Costello, 2015). Augmented reality elements are triggered when the visitor films the featured skeleton with their device. Between March and August 2015, a visitor study was conducted. Visitors were given an interviewed by researchers using open-ended questions. The feedback was very positive. iPad with the app installed and were let free to explore the Bone Hall on their own. They were then Testimonies were published by Marques and Costello (2015) with visitors’ comments explaining how AR makes the bones more real: “information … a lot deeper that the written part,” “… it has a dynamic that makes the animal more real for them [the kids],” “[the swordfish] looked real.” Also, it was noted that visitors would spend about 14 minutes in the Bone Hall with the app, as opposed to a minute and a half without it (Ding, 2017). Skin & Bones seems to have met the stakeholders’ and the public’s expectations. The museum has a feedback scheme, which is supported by visitor studies research. This scheme is expected to further develop and enhance the app. On the continuous development and improvement front, free Wi-Fi needs to be secured throughout the building and visitors need to be informed on the specifications of app, before they decide to download and use it. Note also that the app relates to only thirteen (13) skeletons, making the rest of the exhibition practically invisible to visitors, something that has also been noted in other similar settings (see teacher’s comment above, Papaioannou & Paschou, 2017). Also, the app has been designed for the general public, leaving room for development for targeted audiences, such as school students, young adults, experts, etc.

374 3.2

Papaioannou

ArtLens Gallery and App, Cleveland Museum of Art, Cleveland, USA (16 June 2016 to Present) The Cleveland Museum of Art was founded in 1913 “for the benefit of all the people forever.” Entrance is free. In their constant effort to include the community in its totality and engage the public, the museum launched the ArtLens Gallery progressively between June 2016 and September 2017. The ArtLens Gallery replaced the StudioPlay, an engaging museum space active from December 2012 to March 2017 (DeBonis, 2012b). The gallery has been designed to attract a new public intimidated by classical museum displays as well as to engage visitors via embodied experiences and let them explore artworks in a different way. The new gallery focused on body motions and emotions. A number of rooms compose the ArtLens Gallery, all with an emphasis on digital engagement, including the ArtLens Exhibition, the ArtLens Studio, and the ArtLens Wall (MuseWeb Foundation, 2018). The ArtLens Exhibition displays twenty (20) real works of art and hundreds of digitised objects addressed by fourteen (14) serious games divided in four themes: (i) Gesture and Emotion, (ii) Symbols, (iii) Purpose, and (iv) Composition. In addition, two stations offer other features. The first one is a gaze tracker, following the eyes of the visitor and their points of focus on an artwork in comparison to the artist’s intent in their composition (Hirsh, 2017c). The second employs a machine learning system with a facial-recognition software to show visitors their emotional response to artworks from the collection. ArtLens Studio is a dedicated space for edutainment based on augmented reality (AR) technologies featuring time-of-flight depth cameras, customized C++ software and real-time graphics on screens. Amongst other possibilities, visitors can experience pottery making on a wheel. Visitors’ movements are tracked by a camera shaping the digital clay. Another feature uses motiontracking technology to digitally realise a painting with the participant’s arms and hands as brushes. Each gesture’s intensity determines the digital result, from splatter to concentrated colour (Hirsh, 2017b; Moore, 2015). The free ArtLens App, available on iOS and Android, enables participants to keep their artworks by docking their devices on many stations. Visitors can download the artwork to their own mobile devices or borrow one from the museum entrance desk. The ArtLens App provides augmented reality generated information and services beyond the ArtLens Gallery. The users may pinpoint artworks or themselves on the interactive map. Beacons increase the accuracy of the localisation. The ArtLens App also offers thematic guided tours with multimedia content, but also the possibility to customise a tour picking favourite artworks from the ArtLens Wall or directly on the app. An

Augmented Reality in Museums and Cultural Heritage Settings

375

image-recognition software identifies some of the two-dimensional artworks of the museum. Its contents are activated when scanning the artworks (DeBonis, 2012a; Ding, 2017; Hirsh, 2017a). The ArtWall displays digital images all artworks in view at the museum. The ArtLens Gallery and App have won multiple awards. A large team of researchers collect feedback (Kmiers, 2013) and constantly integrate it on the App (Ding, 2017). According to the analytics, visitors enjoy the app, they engage more with the artworks and spend seven times longer with every artwork in the ArtLens Gallery (76 seconds as opposed to 15; Armitage, 2018). Augmented reality in this case has been well and successfully integrated into the museum’s overall exhibition and engagement plan with the audience. It is part of a digital project of many digital and interactive parts, including the Exhibition with serious games and activities, the Studio with kinaesthetic activities, and the ArtWall of digital images. The supporting team of researchers develops the app based on collected feedback from visitors, securing sustainability of audience interest and visitors/users satisfaction. In the future, the app can be expanded to address targeted audiences, such as school students, artists, experts, etc. 3.3

Perfumes of China, Cernuschi Museum, Paris, France (9 March–26 August 2018) The Cernuschi Museum is dedicated to the ancient arts of China, Japan, Korea, and Vietnam. It was built around the private collection of Henri Cernuschi and is one of the fourteen museums directly run by Paris’ municipality. The exhibition Perfumes of China explores different aspects of Chinese culture on the use of perfume and incense in a chronological journey of two millennia. Incense was essential for religious rituals, helping medication and perfuming the house. The Cernuschi Museum has been collaborating with the Shanghai Museum featuring 110 objects displayed in five rooms. Curators decided to move beyond depictions and take visitors to an olfactory journey via augmented reality. Fréderic Obringer, an academic specialising in Chinese perfumes, and François Demachy, a Dior perfumer, have collaborated to recreate some of the perfumes of ancient China (Musée Cernuschi, 2018; Paris Musées, 2018a). Obringer selected four ancient recipes from Chinese texts (Paris Musées, 2018c). Demachy translated the ancient components into modern essences and recreated the recipes (Paris Musées, 2018b). Moreover, Demachy was challenged to create a new fragrance inspired by the evolution of the scents during time. Five interactive kiosks were created, one in each room. Each kiosk has a digital touch screen. On the screen, all the ingredients of a Chinese perfume

376

Papaioannou

recipe are identified by their name and a graphic. When pressed, animations and information are generated, and on the top of the screen an animation invites visitors to press and trigger a perfume (Anamnesia, n.d.; Musée Cernuschi, 2018). The multimedia device and contents have been realised by Anamnesia, a creative studio near Strasbourg, France. A quick survey of the museum’s stakeholders’ blogs and magazines shows the good reception by the public. Its specificity and innovative approach explain the public’s enthusiasm as well as the surprise provoked by the smell of strong remote scents (Bougault, 2018; La Fabrique de l’Histoire, n.d.; Maguer, 2018). This is a special case of augmented reality in museums and cultural heritage, since it engages not only the senses of vision and hearing, but also that of smell. Moreover, this case does not involve handheld devices but touch screens connected to perfume-generating devices. It deviates from the digital aspects of augmented reality applications (there are no digital smells yet!), but it follows the concept of augmenting the text with non-text layers of information, such as digital images and generated smells. Possibilities here are to be explored. 3.4

“Essential Is Invisible to the Eyes” Exhibition at the Musée de l’Ardenne, Charleville-Mézières, France (9 June–16 September 2018) The Municipal Museum Musée de l’Ardenne is situated in Charleville-Mézière in East France. It hosts archaeological, ethnographic and art collections mainly of local and regional origin. The museum goes back to the 19th century, but it was inaugurated in 1994 in its current form (Tourneux, 1994). The exhibition Essential is invisible to the eyes (a title borrowed from a quotation of Saint-Exupéry’s The Little Prince) uses 3D printing models of twelve (12) paintings of the Musée de l’Ardenne, so that visually impaired visitors can have access to them. It is the outcome of the Smart’Art project, which was initiated in 2016 after a call for proposals by the Champagne-Ardenne region to support and develop social inclusion. A fruitful multi-pa extra funding from Charleville-Mézières council. Full, high-relief, and bas-relief 3D digital models of sculptures were generated from the selected paintings (“Actualités,” n.d.; France 3 Grand Est, 2018) with the help of eight visually impaired 25–65 year-old persons from the Michel Flandre Institut. They have been asked to define what provokes emotion and have actively collaborated to the creation of the touch strip. The digital interpretative models were elaborated by Remy Closset from the Valentin Haüy Association (an association that supports blind and partially-sighted persons and their families) (Association Valentin Haüy, 2018). Remy Closset is an architect who became visually impaired and works with the IFTS to realise architectural models (Diot, 2018). 3D printings were undertaken at the FabLab

Augmented Reality in Museums and Cultural Heritage Settings

377

Smart Material, some of them necessitating almost 500 hours of work (“Actualités,” n.d.; Blanchardon, 2018; France 3 Grand Est, 2018). High quality 3D printers were used (drill of 1mm thick; France 3 Grand Est, 2018). An audio guide with audio descriptions completed the tour. This project worked towards the creation of a haptic augmented reality app based upon the digital 3D models for the painting-based sculptures. It adds to attempts that investigate the haptic sense and multisensory experiences in museums and the sense of touch’s contribution to the pleasure felt in front of an art piece (Brewster, 2005; Dima, Hurcombe, & Wright, 2014; Fondation Orange, n.d.-b, n.d.-a; Vi, Ablart, Gatti, Velasco, & Obrist, 2017). The aim of the Smart’Art project was to translate emotions from the visual to the tactile sense (Lan Hing Ting, Di Loreto, & Champigny, 2017). Even if it was primarily designed for a targeted audience, it will certainly enhance the experience of all visitors. The exhibition will travel all around France (Blanchardon, 2018). Feedback is expected. 3.5 Corfu Old Fortress App, Corfu, Greece (2015 to Present) The Corfu Old Fortress augmented reality app was launched in 2015 to provide a tour of the Old Fortress in the UNESCO-listed Old Town of Corfu, Greece, based upon user-generated customised interactive storytelling scenarios (Deliyannis & Papaioannou, 2016). The AR app used the AURASMA platform and was based on single trigger-image recognition from certain points to generate content. It was developed without funding as an initiative of the Ionian University, Corfu, Greece, to explore augmented reality in cultural settings. It was developed as a result of the fruitful collaboration of the Museology Lab of the Department of Archives, Library Science, and Museology, and the Interactive Arts Lab, Department of Audio and Visual Arts, Ionian University. The app supports an eleven-location/stop tour in the Corfu Old Fortress. Users are asked to choose a profile/channel at the beginning of their tour. Available user profiles are linked to different content pools. Current profiles include ‘experts’ (emphasis on detailed and analytical historical and cultural information), ‘mainstream visitors/tourists’ (emphasis on standard-guide informative presentations), and 9–11 year-old school students (emphasis on edutainment, discovery learning and serious game-oriented presentations). Content has been developed after expert consultations and pilot studies. Every location/stop relates to an information point where users/visitors can access customised content after trigger-image recognition. At each location/stop, users/visitors are offered alternative navigation options and routes. The Corfu Old Fortress App has been tested by undergraduate and postgraduate students, and received positive feedback. It became known by word of

378

Papaioannou

mouth and some people in Corfu asked for more details and used it too. There was no provision for promotion and dissemination. At present, the Corfu Old Fortress App still exists but it is not in use. It needs updating and a host for its preservation, maintenance, and promotion.

4

Concluding Remarks and the Future

Augmented Reality (AR) applications in cultural heritage and museums have developed a lot in the last few decades, taking advantage of the ubiquity of Wi-Fi, mobile and smartphone technologies in everyday life. Examples have shown successes and failures, which can serve as starting points for future development. AR technologies serve museums as tools for interpretation and accessibility of extra, available and on-demand information, since they offer platforms of enhanced and targeted information retrieval, adapted and adjusted to visitors’ individual needs and requirements. AR technologies also add value to museums’ educational programming via discovery learning, kinaesthetic learning, and learning-by-doing and serious games learning. Moreover, AR applications support and encourage engagement by promoting observation, triggering imagination, and provoking conversations among visitors. AR applications prolong visitors’ stay in the museum and make visits more fruitful and fun. To secure the above, museums and cultural heritage institutions need to be aware of the following issues and practicalities. An AR endeavour can only be successful if the museum can support it. Funds and sustainability need to be secured, as well as Wi-Fi and a user-friendly format with minimal storage space requirements and download times. Also, an AR endeavour should address an exhibition requirement and relevant visitors’ needs. Front-end curatorial research and visitor studies can help towards this, while formative evaluation processes would secure that the qualities of undertaking are met during its period of development. A marketing strategy is necessary for promotion and to secure awareness and successful dissemination, while continuous feedback from visitors/AR users, together with visitor studies research programmes, will address users’ satisfaction and help towards necessary future adjustments. Current and future applications need to abide by intellectual property, privacy, and ethics in the world of big personal data, following the GDPR compliance and other local, national, and international regulations. Taking the above into account, current and future AR attempts and research should focus on addressing museum and cultural heritage AR policies and strategies, updated by continuous feedback from stakeholders, so that they

Augmented Reality in Museums and Cultural Heritage Settings

379

provide solutions and assure a smooth, successful integration of AR technologies into museums’ operations, aims, objectives, and outcomes.

References Actualités. (n.d.). Retrieved June 10, 2018, from http://fablab.ifts.net/actualite.php?id= American Alliance of Museums (AAM) & Museum Association (MA) Members (2012). Mobile in museums study. Arlington: American Alliance of Museums. Anamnesia. (n.d.). Parfums de Chine. Retrieved June 27, 2018, from http://anamnesia.com/fr/reference/parfums-de-chine/ Armitage, M. (2018). ARTLENS gallery. Retrieved June 29, 2018, from https://mw18.mwconf.org/glami/artlens-gallery/ Association Valentin Haüy. (2018, June 12). “L’essentiel est invisible pour les yeux,” une exposition conçue par, pour et avec les déficients visuels. Retrieved June 29, 2018, from https://www.avh.asso.fr/fr/lessentiel-est-invisible-pour-les-yeux-une-expositionconcue-par-pour-et-avec-les-deficients-visuels Blanchardon, M. (2018, February 17). Des tableaux en 3D pour les malvoyants au musée de l’Ardenne – Le Parisien. Retrieved June 28, 2018, from http://www.leparisien.fr/ culture-loisirs/des-tableaux-en-3d-pour-les-malvoyants-au-musee-del-ardenne-17-02-2018-7564113.php?utm_content=bufferb301e&utm_ medium=social&utm_source=twitter.com&utm_campaign=buffer Bougault, V. (2018, June 8). Le parfum des dieux au Musée Cernuschi. Retrieved June 28, 2018, from https://www.connaissancedesarts.com/non-classe/le-parfumdes-dieux-au-musee-cernuschi-1196896/ Brewster, S. (2005). The impact of haptic ‘touching’ technology on cultural applications. In J. Hemsley, V. Cappellini, & G. Stanke (Eds.), Digital applications for cultural and heritage institutions (pp. 273–284). Aldershot: Ashgate. British Museum. (n.d.). Audio guides. Retrieved from http://www.britishmuseum.org/ visiting/planning_your_visit/audio_guides.aspx?lang=en Costello, R., & Marques, D. (2015). Skin & Bones app for the Bone Hall. Retrieved June 7, 2018, from https://mw2015.museumsandtheweb.com/bow/skin-bones-app-for-thebone-hall/ CPNAS. (2015). Robert Costello and Diana Marques: Skin and Bones Mobile App. Retrieved from https://www.youtube.com/watch?v=ERBtaWQdd7Y DeBonis, J. (2012a, November 16). ArtLens App [Text]. Retrieved June 7, 2018, from http://www.clevelandart.org/artlens-gallery/artlens-app DeBonis, J. (2012b, November 16). ARTLENS gallery [Text]. Retrieved July 4, 2018, from http://www.clevelandart.org/artlens-gallery/about

380

Papaioannou

Deliyannis, I., & Papaioannou, G. (2016), Augmented reality edutainment systems for open-space archaeological environments: The case of the Old Fortress, Corfu, Greece. In I. Deliyannis, P. Kostagiolas, & Ch. Banou (Ed.), Experimental multimedia systems for interactivity and strategic innovation (pp. 307–323). Hershey, PA: IGI Global. Dima, M., Hurcombe, L., & Wright, M. (2014). Touching the past: Haptic augmented reality for museum artefacts. In Virtual, augmented and mixed reality. Applications of virtual and augmented reality (pp. 3–14). Cham: Springer. https://doi.org/ 10.1007/978-3-319-07464-1_1 Ding, M. (2017). Augmented reality in museums. In The augmented museum: Essays on opportunities and uses of augmented reality in museums (pp. 1–15). Pittsburgh, PA: Carnegie Mellon University/ETC Press. Diot, N. (2018). La 3D révolutionne l’accessibilité à l’art – 4 juillet 2018. Le Journal Des Arts, 505(6 juillet-6 septembre), 8. FIlippini-Fantoni, S., & Bowen, J. P. (2008). Mobile multimedia: Reflections from ten years of practice. In L. Tallon & K. Walker (Eds.), Digital technologies and the museum experience handheld guides and other media (pp. 79–96). Lanham, MD: AltaMira Press. Fondation Orange. (n.d.-a). le parcours tactile dans le nouveau département des Arts de l’Islam du Musée du Louvre. Retrieved from https://www.youtube.com/ watch?v=3Nqt83n4whU Fondation Orange. (n.d.-b). L’élaboration et la réalisation d’images tactiles pour les Arts de l’Islam au Louvre. Retrieved from https://www.youtube.com/watch?v=wcHfj2SdjHo France 3 Grand Est. (2018). Charleville-Mézières: Des tableaux en 3D pour les malvoyants. Retrieved from https://www.youtube.com/watch?v=RqRmYJYU01U Hirsh, E. (2017a). ArtLens App – MW17: Museums and the web 2017. Retrieved June 7, 2018, from https://mw17.mwconf.org/glami/artlens-app/ Hirsh, E. (2017b). ArtLens Studio – MW17: Museums and the web 2017. Retrieved June 29, 2018, from https://mw17.mwconf.org/glami/artlens-studio/ Hirsh, E. (2017c, June 19). ArtLens exhibition [Text]. Retrieved July 4, 2018, from http://www.clevelandart.org/artlens-gallery/artlens-exhibition Kmiers. (2013, February 20). Awards/Collaborators [Text]. Retrieved July 4, 2018, from http://www.clevelandart.org/artlens-gallery/collaborators Koutromanos, G., Sofos, A., & Avraamidou, L. (2015). The use of augmented reality games in education: A review of the literature. Educational Media International Journal, 52(4), 253–271. doi:10.1080/09523987.2015.1125988 Koutromanos, G., & Styliaras, G. (2015). The buildings speak about our city: A location based augmented reality game. In Information, Intelligence, Systems and Applications (IISA), 2015 6th International Conference (pp. 1–6). doi:10.1109/IISA.2015.7388031

Augmented Reality in Museums and Cultural Heritage Settings

381

La Fabrique de l’Histoire. (n.d.). Histoire des odeurs et des parfums (2/4) : Parfums de Chine, la culture de l’encens au temps des empereurs. Retrieved from https://www.franceculture.fr/emissions/la-fabrique-de-lhistoire/histoire-desodeurs-et-des-parfums-24-parfums-de-chine-la-culture-de-lencens-au-temps-desempereurs Lan Hing Ting, K., Di Loreto, I., & Champigny, F. (2017). Projet SmartArt – Livrable de la Phase 1 (Research Report). UTT-Université de Technologie de Troyes. Retrieved from https://hal.archives-ouvertes.fr/hal-01577661 Maguer, S. L. (2018). Compte-rendu d’exposition : « Parfums de Chine », Musée Cernuschi, Paris [Billet]. Retrieved June 28, 2018, from https://eem.hypotheses.org/415 Marques, D., & Costello, R. (2015). Skin & bones: An artistic repair of a science exhibition by a mobile app. MIDAS. Museus e estudosinterdisciplinares, 5. https://doi.org/ 10.4000/midas.933 Moore, C. (2015, December 9). ArtLens studio [Text]. Retrieved July 4, 2018, from http://www.clevelandart.org/artlens-gallery/artlens-studio Musée Cernuschi. (2018, March 14). Voyage olfactif au Musée Cernuschi ! Retrieved June 10, 2018, from http://www.cernuschi.paris.fr/fr/voyage-olfactif-au-musee-cernuschi Musée du Louvre. (n.d.). Audioguides du Louvre : Nintendo 3DSTMXL et application mobile. Retrieved from https://www.youtube.com/watch?v=kABy4wOV36Q Museum of London – Street Museum. (n.d.). Retrieved June 26, 2018, from https://www.museumoflondon.org.uk/Resources/app/Dickens_webpage/ home.html MuseumMobile. (2009, August 19). Spy in the City: The GPS game of Washington, DC : MuseumMobile. Retrieved from http://museummobile.info/archives/240 MuseWeb Foundation. (2018). ARTLENS gallery. Technology. Retrieved from https://www.slideshare.net/MuseWeb/artlens-gallery Naimark, M. (2018). VR/AR fundamentals — 3 Other senses (Touch, Smell, Taste, Mind). Retrieved from https://medium.com/@michaelnaimark/vr-ar-fundamentals3-other-senses-haptic-smell-taste-mind-e6d101d752da Papaioannou, G., & Paschou, S. (2017), Five paintings talking to children: Producing and evaluating a digital educational augmented reality app in Corfu Art Gallery, Greece. Researching Digital Cultural Heritage International Conference, Manchester, UK. Retrieved from http://www.manchester.ac.uk/digitalheritageconference/#digherit age17 Paris Musées. (2018a). Exhibition Perfumes of China | Eric Lefebvre, Director of Cernuschi Museum | Cernuschi Museum. Retrieved from https://www.youtube.com/ watch?v=ZKH_H0FYgQM Paris Musées. (2018b). Exposition Parfums de Chine | François Demachy, parfumeur créateur de Dior | Musée Cernuschi. Retrieved from https://www.youtube.com/ watch?v=BkbitG9dRk0

382

Papaioannou

Paris Musées. (2018c). Exposition Parfums de Chine | Frédéric Obringer, Spécialiste parfums | Musée Cernuschi. Retrieved from https://www.youtube.com/ watch?v=nXzDMSXq-GM Schmalstieg, D., & Höllerer, T. (2016). A brief history of augmented reality. In Augmented reality: Principles and practice. Addison-Wesley Professional. Retrieved from http://proquest.safaribooksonline.com/book/web-development/ usability/9780133153217/chapter-1dot-introduction-to-augmented-reality/ ch01lev1sec2_html SmithsonianMobile. (2013). 19 Skin and Bones | Smithsonian Natural History Museum. Retrieved from https://www.youtube.com/watch?v=S7KZ91oTBik Smithsonian’s National Museum of Natural History. (2015). Skin & bones: Anhinga augmented reality experience. Retrieved from https://www.youtube.com/ watch?v=vPl_9af1r4Y Spy Museum. (n.d.). Spy in the city. Retrieved from https://www.spymuseum.org/ exhibition-experiences/interactive-spy-experiences/spy-in-the-city/ Tallon, L. (2008). Introduction: Mobile, digital and personal. In L. Tallon & K. Walker (Eds.), Digital technologies and the museum experience. Handheld guides and other media (pp. xii–xxv). Lanham, MD: AltaMira Press. Tourneux, A. (1994). Musée de l’Ardenne. La revue du Louvre, 5–6, 14–15. Vi, C. T., Ablart, D., Gatti, E., Velasco, C., & Obrist, M. (2017). Not just seeing, but also feeling art: Mid-air haptic experiences integrated in a multisensory art exhibition. International Journal of Human-Computer Studies, 108, 1–14. https://doi.org/10.1016/ j.ijhcs.2017.06.004 Villarejo, L., González-Reverté, F., Miralbell, O., & Gomis, J. M. (2014). Introducing augmented reality in cultural heritage studies. eLC Research Paper Series, 8, 6–14. Vlahakis, V., Ioannidis, M., Karigiannis, J., Tsotros, M., Gounaris, M., Stricker, D., … Almeida, L. (2002). Archeoguide: An augmented reality guide for archaeological sites. IEEE Computer Graphics and Applications, 22(5), 52–60. https://doi.org/10.1109/ MCG.2002.1028726 Younes, G., Kahil, R., Jallad, M., Asmar, D., Elhajj, I., Turkiyyah, G., & Al-Harithy, H. (2017). Virtual and augmented reality for rich interaction with cultural heritage sites: A case study from the Roman Theater at Byblos. Digital Applications in Archaeology and Cultural Heritage, 5, 1–9. https://doi.org/10.1016/j.daach.2017.03.002

CHAPTER 17

Applications of Augmented Reality Apps in Teaching Technical Skills Courses Lilla Korenova, Maria Kožuchová, Jiří Dostál and Zsolt Lavicza

Abstract Teaching technical skills courses is an integral element in most educational systems, however, the skills students need to acquire have been changing radically over the past 100 years. This change continues, as technology increasingly penetrates everyday life, demanding to acquire new skills. In many countries, technical education starts in primary school and the basics of technical literacy are taught to pupils aged 6–14. In the Czech Republic and Slovakia, future primary teachers are required to develop novel teaching approaches to develop technical literacy and skills for pupils aged 6–14 in a rapidly evolving digital world. This chapter aims to highlight the possibilities of using extended and augmented reality applications in technical education for the preparation of future teachers.

Keywords primary level – technical education – curriculum reform – conception of education – effectivity – implementation

1

Technical Education at Elementary Schools and in the Preparation of Future Teachers

Technical skills, as an important part of human culture, have always been closely linked to the creative work of humans. People were, are, and will be the main initiator of any technological innovations and changes that enter into their professional and private lives. Technical skills affect our attitudes, values, mental and physical health, behaviours, and lifestyles. EU countries (including the Czech Republic and Slovakia) must also pay close attention to technical education as a result of recent educational policy changes. While © koninklijke brill nv, leideN, 2020 | DOI: 10.1163/9789004408845_017

384

Korenova et al.

creating actual educational programs, they follow the recommendations from the Council of Europe (2000), which sets out the main strategic objectives: to develop such educational programs to support the European Union to become a more dynamic and competitive economy in the world. This goal initiated the preparation, creation, and approval of the Education Work Programme by the Ministers of the Member States of the European Union, responsible for education and training. From the point of view of technical education, it was necessary to increase the interest of pupils in the study of natural sciences and technical fields; develop their scientific and technical competences; ensure access for all pupils to information and communication technologies; improve the training of teachers for science and technology education; and establish a link between the world of work and research. Europe faces a series of social and economic challenges, overcoming them is critical if it is to preserve our values and way of life, keep competing in the global economic race, and secure peace, stability, and prosperity for our children, as well as to preserve an inhabitable planet. STEM (Science, Technology, Engineering, and Mathematics) education has a crucial role in achieving such aims. In recent years, the number of people working in STEM occupations has grown by 12% – three times more than the overall employment in the EU. STEM occupations now account for 7% of all jobs across the Union, and the demand for STEM competences keeps growing (European Commission, 2017). However, in several parts of Europe, employers have difficulties hiring people with the right STEM skills, particularly ICT (Information Communication Technologies) professionals. And, unfortunately, the latest PISA data showed that more than one in five 15-year olds in Europe are functionally illiterate in reading, maths, and science (OECD, 2015). Schools are on the frontline in addressing this skills shortage. By offering students the opportunity to learn about STEM subjects, they open up career opportunities in existing and newly emerging sectors. But businesses do not only need people with sophisticated STEM knowledge, they also need creative thinkers and good communicators – graduates who can work with others and solve problems. In his State of the Union address, Commission President JeanClaude Juncker called for a more united, stronger, and more democratic Union: “We have to build more cohesive, resilient and fair societies and economies. And we have to build them with young people. These young people need the right skills and attitudes – including at least basic STEM proficiency, as well as cultural awareness, an entrepreneurial mind-set and a willingness to engage” (Juncker, 2015). These skills need to be developed from an early age. At school, we must motivate children to learn maths and science and we must help them to imagine working in these fields.

Applications of Augmented Reality Apps

385

The starting point for technical education at elementary schools in European Union countries was motivated by the rapid development of new technologies and the integration of information technology into all areas of human life. In most EU countries, primary education includes materials through which pupils get a basic orientation in the field of technology. Target requirements are included in four areas: 1. Teach students to produce technical products: – gain experience in field of imaging, measuring, reading technical drawings, and learning technical symbols and characters, – acquire the basic knowledge and skills necessary to perform certain activities, – gain experience from the processing of technical materials (wood, plastics, metals, textiles, and other technical materials). 2. Teach students to use technology-related tools: – acquire the basics of the use of technology-related tools, – gain experience in handling electrical appliances at a suitable age. 3. Teach students to form their own opinions about the possibilities of technology and its impact on nature and society: – awareness of how the technology affects our lives in various life situations (at home, school, during travel, sports, hospital, etc.), – balanced perception of technology (technology can help, but also endanger health and life), – developing moral awareness and action in relation to the use of technology. 4. Teach students to use digital technologies: – students are required to become users of technologies and develop creative skills working in such environments. Technical education is an essential part of basic education in Slovakia and the Czech Republic. Learning technical and technological skills in primary school is the first opportunity for occupational choice for a pupil, providing the chance to learn about the most common technological tools and basic procedures for working with different materials. The focus of technical education is related to the effort to take into account the scientific, technical, and economic potential of the country. As part of the curriculum in both countries, it offers opportunities to engage students with new technology-related skills. Before proceeding further, we will try to define the concept of technical education as several authors approach the concept differently. According to the European Schoolnet (EUN), skills in science, technology, engineering, and mathematics (STEM) are becoming an increasingly important for basic literacy in today’s knowledge economy (European Schoolnet,

386

Korenova et al.

2017). Europe needs one million additional researchers by 2020 to keep its economic growth potential. Science education can no longer be viewed as only an elite training for future scientists or engineers. It is clear that only science-aware citizens can make informed decisions and engage in dialogue on science-driven societal issues. As stated in the recent Report of the European Commission (EC) – Science Education for Responsible Citizenship, knowledge of and about science are integral to preparing people to be actively engaged and responsible citizens, creative and innovative, able to work collaboratively, and fully aware of and conversant with the complex challenges facing society.

2

What Is STEM?

The acronym STEM refers to the academic disciplines of Science, Technology, Engineering, and Mathematics. STEM is concerned with the development of scientific, technological, and mathematical insights, concepts and practices and how to use and apply them in practice in order to solve complex questions or real-life challenges. Therefore, within the broader education context, STEM implies bringing together the different components of the acronym to identify social and scientific challenges in a coherent manner, solving them in an inquiry-based manner and communicating about them. STEM also implies that the various disciplines are taught in the best possible way (STEM Framework, 2016). STEM or science only, has often been seen as something separate from all other subjects or disciplines in education, disconnected from people’s lives beyond school, but it has been proven that science influences all parts of our lives and our decision-making processes. Along with language and artistic literacy, knowledge of science and mathematics is the basis for personal accomplishment and responsible citizenship, social and economic development and a benchmark of innovation, entrepreneurship, and competitiveness in our global world. Therefore STEM education ensures that concepts and practices important in these areas are understood and applied in an interdisciplinary manner. This shows that these concepts and practices are founded on STEM principles and ideas that can be applied from various angles. STEM is oriented towards innovation: it responds to current challenges and looks for innovative and creative solutions through the interconnected STEM components. Technical education, in our context related to STEM teaching, is a managed process that allows students to learn knowledge of current technology and the

Applications of Augmented Reality Apps

387

processes necessary to solve technical problems. In this context, two levels of technical education need to be distinguished: a) technical education within general education, b) specialized technical education for vocational training. In this chapter, we will focus on technical education in general education and its relation to STEM teaching.

3

The State Education and Training Program in the Slovak Republic from the Point of View of Educational Policy

The State Education and Training Program in the Slovak Republic (ŠVP, 2008) has significantly integrated trends from other EU countries. It is structurally comparable to similar documents from the countries of the European Union. The main program goals of primary education are to develop pupils’ key competencies (as a combination of knowledge, skills, experience, and opinions) at a level that is personally achievable. The most recent edition of the Education Program (IŠV, 2014) also respected the government’s program statement, which resulted in the enhancement of mathematics, computer science, science, and work studies, which was reflected in increased school hours for those subjects. The set of basic scientific and technical competences consists of cognitive, motoric, social, social-moral, and information-communication competencies that ensure that individuals “participate in the use of technology as an element of culture.” The Education Program in the Slovak Republic (IŠV, 2014) defines: General (universal) competencies: 1. ability to solve the problem, the ability to apply creative ideas in their work, 2. ability to take responsibility, ability to be self-sufficient, ability to evaluate and express own opinion, 3. ability to self-identify and self-assess with respect to their own professional orientation, 4. ability to respond flexibly to changes in the labour market in an effort to apply the best. Cognitive competencies: 1. ability to perceive technical skills as part of human culture, to perceive technology as the driving force of the development of society and to realize that technology is an instrument in the hands of human, and that usage of technology will always be decided by a person,

388

Korenova et al.

2. ability to authentically and objectively identify the surrounding world, to master the basics of research and design activities, 3. ability to learn the principles of design, construction and other technological activities, 4. ability to acquire knowledge of science (especially natural and technical) independently and to know how to use that knowledge in the production of technical products, 5. ability to acquire basic entrepreneurial skills, 6. awareness of the constant changes and development of science and technology, 7. recognition of the need for lifelong learning in regard to the enormous growth in technology. Social competencies: 1. ability to establish social contact, 2. ability to work with different social subjects, 3. ability to express own identity, 4. ability to accept and play different social roles. Social-moral competencies: 1. ability to assume an attitude to usage of technology (pupils oriented towards humanitarian use); 2. ability to respect moral rules and conventions in their own actions and to be responsible for the quality of the outcome of the work. Psychomotor competencies: 1. ability to realize adequate technical and engineering activities in the processing of technical materials, 2. ability to safely use and safely dispose of the technology products, 3. ability to perform technical experiments and research activities, 4. ability to responsibly access and use digital technologies. Information and communication competencies: 1. ability to build a positive relationship with digital technologies whose use supports lifelong learning, personal advancement, and productivity, 2. ability to use digital technologies to improve learning and to increase skills, 3. ability to acquire digital skills to search, evaluate, and collect information from multiple sources, 4. ability to use digital technologies to develop real-life resolution strategies,

389

Applications of Augmented Reality Apps

5. ability to learn basic ways of remote communication, 6. ability to learn the basics of technical communication (technical symbols and technical graphics) that have a universal (global) character. (Vláda SR, 2012) All of these competencies also apply to STEM education and connect school with practical skills in life and employment. It enables pupils and their parents to recognize their professional orientation correctly and in a timely manner, and to achieve a harmonious and integral development of the personality of a young person so that his or her abilities and talents can be best applied in real life. With the current demands of scientific and technical literacy, it is difficult to define the basic facts, concepts, and processes that represent technical advancements. These terms are “social representatives of reality” and they are variable and temporal. It is well known that many facts, concepts, and processes in the field of technology are quickly changing. One of the most significant benefits of social constructivism, which we use during our research of technical literacy, is to draw attention to socio-cultural conditionality and thus to the relativity of concepts in the fields of STEM and related technologies (Kostrub, 2017). But for the development phase of its existence, technical concepts and processes are the best explanations that science has for a given section of scientific knowledge at the time. The content of technical education at primary schools in Slovakia consists of the following areas (Table 17.1): The first group is related to the understanding of the importance of technology in society. Although the content framework is created, pupils acquire the necessary competence in the whole content of education, because the human use of technology is emphasized throughout the entire content. The second group is focused on materials and technologies, which, until recently, was the main focus of technical education. Currently, teaching methods based on the principles of scientific work are used in this area of learning: experimentation and scientific research. Remarkable is the third group – transportation, which table 17.1  Main areas of technical education in SVK

Society and Technology

Production

Transfer

- impact of technology on human life

- materials - means of - technology transport - energy - mobility sources

Construction

Technology at home

planning, designing technical devices, instruments and and realization of their use structural models (buildings, bridges...)

390

Korenova et al.

is about development of mobility, as well as the development of constructive skills. The fourth group is program-oriented to develop scientific research in the field of technology. The last group is focused on the acquisition of adequate skills in the usage of technical equipment. The established education system respects the main areas of technology to understand basic technical issues, and in addition respects all three lines of development of scientific and technical literacy (attitude, content, and process). 3.1

Preparing Primary Education Teachers for Technology-Oriented Teaching The master’s degree program in Teaching Primary Education at the Faculty of Education of the Comenius University in Bratislava follows the bachelor’s study program of Pre-school and Elementary Pedagogy, which focuses on teaching at nursery schools and educators in school children’s clubs. The bachelor’s degree provides a theoretical basis for addressing pedagogical and educational issues at the primary level of education.

4

Technical Education in a Digital Environment

As discussed earlier, we live in an environment where we absolutely need computers, Internet, mobile phones, and other digital assets to be connected to this digital environment. Today, pupils in primary and secondary schools were born into this environment, and therefore it is their natural world. The digital environment in schools could also enable learning. Tablets, smartphones, smartwatches, and similar new digital devices have become affordable in recent years, along with mobile Internet, and they are attractive for pupils. The term m-learning (for “mobile learning”), as well as for the term e-learning are becoming common in education and defined differently. In our case, we consider m-learning as a form of teaching where mobile devices are used. New technologies require necessary changes in education. Effective learning with technology is difficult because digital technologies bring new variables into an already complicated learning system. The Technological Pedagogical Content Knowledge framework describes the possibilities of effective learning with technologies, based on liberal and open cooperation among technologies, pedagogy, and content of education. The use of the TPACK framework requires understanding of technologies in a context where specific pedagogical goals determine the choice of technologies and the possibility of changing their purpose to the needs of learning content (Mishra, 2009).

Applications of Augmented Reality Apps

391

We utilise this framework for our research projects. A new field developing in technology that enables augmentation of the real world is the field of Augmented Reality (AR), which is becoming increasingly applied in education.

5

AR applications for STEM Education

AR can be closely connected to STEM education and utilised in various ways. In STEM education, technologies can play a dual role in classrooms: 1. Digital technologies in the teaching-learning process form an important part of the educational content of STEM education. Children and students ought to get familiar in the classroom with the technologies that surround them in real life. 2. Digital technologies are a means for new teaching methods and forms. Interactive whiteboards, PCs, NBs, tablets, and smartphones can be an effective means for innovative educational forms and methods, such as constructivist learning. Therefore, mobile technologies and the AR technology have a double dimension in STEM education (especially in technical education). Mobile applications based on AR can help students to understand new knowledge, concepts, and relationships from the field of science and technology. They can be divided into several types. In the following section, we present a classification of AR applications for STEM education and include examples. We have selected mainly freely available applications and demos. We do not present any applications requiring special VR and AR hardware beyond a smartphone or tablet. The number of AR applications has been increasing rapidly during the past years. 5.1 Benefits of Digital Technologies in STEM Education Digital technologies can provide symbolic, graphical, and dynamic representations of STEM systems. Digital technologies provide opportunities for students to (Davies, 2013): – develop a formal model of some aspect of the world, supported by the feedback from the behaviour of their model; – explore the behaviour of simulations of STEM systems, and carry out safe and efficient virtual experiments and inquiry projects both in the classroom or at home online; – experience virtual decision-making scenarios that give them a better appreciation of the power and limitations of science and mathematics;

392

Korenova et al.

– use virtual learning interfaces that keep them engaged and on-task, becoming self-motivated, rather than externally driven, and thereby developing the skills of reflective independent learning; – discover and explain the world around them using digital tools to plan their project, gather and analyse the data, and represent their results in different forms of textual, graphical, statistical, mathematical, and dynamic representation; – experience collaborative learning through an orchestrated online process of negotiating the representation of a model, or document, or plan, or inquiry, and sharing and discussing their ideas; – learn from digital games that have the potential to support and guide independent practice in problem solving and high-level conceptual thinking in science and maths; – benefit from automated formative assessment that teachers can also use to track students’ needs and progress; Open online resources and tools for STEM subjects create new challenges for teachers: they need time to develop an appreciation of what these resources provide before they can use them to help their students develop more independent learning skills (Davies, 2013). Digital technologies can have a transformational potential for STEM education because they foster sensori-motor experiences and interaction with the environment, providing opportunities for students to explore and examine invisible phenomena or abstract ideas in concrete ways. Psychologists and philosophers, such as Vygotsky, Luria, and Leontiev have long argued for the important role of sensori-motor interaction with the world for cognitive development (e.g., Clark & Chalmers, 1998), and the role of external tools in shaping activity and mediating cognition. In the past decade there has been an increased recognition that learners from a very young age engage in the informal and untutored use of digital technologies, interpreting animation and visualization in sophisticated interactive games, and even construction of their own artefacts using simple design and production tools. Young learners have frequent access to digital technologies in the form of desktop, internet, or mobile applications. This is important for STEM education, because it is mathematics and computational thinking that invisibly drive the tools that pupils grow up with (Confrey et al., 2010; Kafai, 1995; Rosasa et al., 2003). Technology can model approaches to learning, ways of interacting with peers, adults or children, and most importantly, it can model through visual and audio information how adults and children can engage in meaningful discourse about science and the world around them. In addition, technology can be a tool for educators to use for their own learning, providing access to resources, professional development and to examples of how developmentally

Applications of Augmented Reality Apps

393

appropriate STEM concepts and activities can be introduced to children in ways that will expand their sphere of experience (Pasnik & Hupert, 2016). This also applies to AR technologies and in this chapter we make an attempt to classify them into different categories. As regards STEM education, technologies play a dual role in classrooms: 1. Digital technologies in the teaching-learning process form an important part of the educational content. Technologies as such are part of the content of STEM education, which means that children and students get familiar in the classroom with the technologies that surround them in real life. 2. Digital technologies are used for new teaching methods and applications. Interactive whiteboards, PCs, Notebooks, tablets, and smartphones can be effective for innovative education and pedagogical methods, such as constructivist learning. Therefore, mobile technologies and the AR technology have a double roles in STEM education (especially in technical education). Mobile applications based on AR can help students understand new knowledge, concepts, and relationships from the field of science and technology. They can be divided into several types. In the following section we present the classification of the AR applications for STEM education, including examples. We have selected mainly applications and demos that are freely available. We do not present any applications requiring VR and AR hardware beyond a smartphone and tablet, such as special glasses, etc. The amount of AR applications has been increasing exponentially. Even during the time before this chapter was completed, the number available in GooglePlay and AppStore has approximately doubled.

6

Classification of AR Applications According to Their Use in STEM Education

6.1 AR Can Model How Technological and Industrial Hardware Works Applications can visualize technology devices, engines, and technological processes in industry and manufacturing and this is valuable for developing technical literacy for both younger pupils and students of specialized subjects. These AR applications can replace physical models, which are often very expensive or otherwise difficult to acquire. For example, some of the following applications can be used in this category: 6.1.1 Car Engine – AR With the help of this application (Figure 17.1), students can explore car engines. Interactive 3D models are functional and allow better understanding of how

394

Korenova et al.

figure 17.1 Car Engine AR app (https://play.google.com/store/apps/ details?id=com.magicsw.carenginear)

combustion engines work. Users have the opportunity to see an engine from different angles and track its operation in different settings (they may vary the number of revolutions, pistons, and shafts, and may watch only a certain part of an engine, etc.). The application can be downloaded for free and utilised in teaching and learning of physics and technical subjects related to engines. Students can discover virtual engines in the real world, which is probably a motivating factor for learning. In teaching, this app will replace visualization, but it will also help in the process of understanding the phenomenon and context. 6.1.2 How Does an Airport Work? AR The application (Figure 17.2) is a complement to the educational book of the same name. By using it, the reader learns how an airport works. Students can experiment with different features introduced in airports and then it can be presented in a book format when the projects are completed. This app is a typical example of a textbook that is supplemented by an AR. Such textbooks are likely to be preferred in the future. We recommend for the age category: high school and college. 6.1.3 Steam Museum AR The application enables visitors (including virtual visitors) to the London Museum to take a tour of the exhibits, including their virtual functionality, in the London Museum of Water & Steam near Kew Bridge on the Thames in West

Applications of Augmented Reality Apps

figure 17.2 How does an airport work? AR app (https://play.google.com/store/apps/ details?id=com.books2ar.airportsAR)

figure 17.3 Steam Museum AR app (https://play.google.com/store/apps/ details?id=com.thinqdigitalmedia.android.steammuseumar)

395

396

Korenova et al.

London, England. The Museum is centred on a collection of steam engines dating from 1820 to 1910 (Figure 17.3). This application makes students explore various machines and put them into historical perspectives and connect the concepts to STEM subjects. Students can understand the functioning of steam engines in terms of technology, but they can also compare these machines to each other. It is only in the virtual world – and it is motivating for students. We recommend this app for high school and college students 6.1.4 Tideway – Tunnelworks AR The application explains the project for the construction of the tunnel under the Thames in London. The Thames Tideway Tunnel is the biggest infrastructure project for a sewer system in Great Britain (Figure 17.4). The application illustrates and concurrently advertises the project, being a suitable source of information also for students in STEM education. We recommend this app for high school and college students. 6.1.5 Spacecraft 3D The application was developed in collaboration with NASA and it simulates the work and communication with the devices that are used for space exploration

figure 17.4 Tunnelworks AR app (https://play.google.com/store/apps/ details?id=com.tenalps.tbm)

Applications of Augmented Reality Apps

397

(Figure 17.5). This application can enable students to explore state-of-the-art space technology and connect it to concepts learned in schools. The application can be used by students, for example, in designing space exploration projects, as well as visualizing work on another planet. We recommend the app for primary, high school, and college students.

figure 17.5 Spacecraft 3D AR app (https://play.google.com/store/apps/ details?id=gov.nasa.jpl.spacecraft3D)

6.2 AR Applications that Assist in Interior and Exterior Design AR applications have become a favourite option for companies selling interior and exterior household products. The student can select the virtual form of a product from a paper or virtual catalogue and place it into an image of a real home, house, apartment, or garden. Such a visualization is both beneficial for the student and a good commercial ploy for the company selling the products. However, these applications can be used in schools to develop students spatial abilities and enable measurement in space. Some examples of these applications: 6.2.1 IKEA Catalogue The application enables the user to download the IKEA catalogue and other IKEA publications. The student can then use the AR function to place selected

398

Korenova et al.

furniture directly to a real room and create his or her own design (Figure 17.6). The application can be used in teaching to develop spatial skills and imagination, which are important in learning mathematics and geometry. In addition, it also contributes to a better understanding of the materials from which furniture is made. This is important for the development of students’ technical literacy. We recommend it for primary, high school, and college.

figure 17.6 Katalog IKEA app (https://play.google.com/store/apps/ details?id=com.ikea.catalogue.android)

6.2.2 AR for Samsung Appliances Using the application, students can select and place Samsung products from the catalogue (refrigerators, washing machines, TVs, etc.) in their own households (Figure 17.7). In addition to the visualization of a product from the outside and from the inside (for example in the case of a fridge, you can open the door and look inside), the student has access to all the technical parameters of a product. The application can be utilized (like the AR IKEA application) in developing spatial imagination. The possibility of virtual housekeeping is such a competence that students will need in their future lives and should therefore be addressed in the context of teaching as well. We recommend it for all ages.

Applications of Augmented Reality Apps

399

figure 17.7 House in AR app (https://play.google.com/store/apps/ details?id=com.etezo.HouseConcept)

6.2.3 ARki: A-R Architecture With this application it is possible to visualize some architecturally interesting buildings (Figure 17.8). It is motivating for pupils if they can present interesting buildings as they look in their real world. This application also develops students’ spatial imagination. They can use it, for example, in teaching and learning projects for the construction of interesting buildings, both in terms of construction, architecture, and culture. We recommend it for students of all ages. 6.3 AR Applications in Advertisements Based on our experience, applications with AR are usually developed for the sake of advertising. But there are also applications that have been developed using printed catalogues, brochures, or product packaging, and also applications that do not need any markers at all. By means of markers on product packaging, it is, for example, possible to retrieve important information, instructions for use or the price of a product or the final product after its assembly can be visualized (e.g., LEGO). 6.3.1 Bike 3D Configurator This application enables students to choose a customized configuration of a bike, while being able to see the selected type of a bike from each side and in

400

Korenova et al.

figure 17.8 ARki: A-R Architecture app (https://play.google.com/store/apps/ details?id=com.darfdesign.arki)

a variety of colours (Figure 17.9). Students can change frames, pedals, brakes, seats, tires, derailleurs and the look of the bicycle. This application can be used to develop technical literacy. It is also suitable for developing their competencies for future life. We recommend this app for students of all ages. 6.3.2 LEGO 3D Catalogue The application enables students to visualize fully assembled LEGO constructions (Figure 17.10). It is interactive and contains short video stories, which are very attractive for children. At the same time, the application also enables viewing the manufacturer’s virtual catalogue. With this app, children can meet the most important AR function in industrial production. Using virtual reality, we move on to build LEGO, just like repairing machinery in the industry. It is advisable for the teacher to draw attention to this subject and to discuss it. We recommend this app for students of all ages. 6.4 AR Applications as User Manuals or Product Repair Instructions AR applications, most frequently used in connection with “smart glasses,” are also used in industry, for example as a virtual guide to the repair or installation of equipment. Manufacturers are increasingly using applications with VR and AR instead of printed user manuals or instructions for minor repairs or maintenance of their products.

Applications of Augmented Reality Apps

figure 17.9 Bike 3D Configurator app (https://play.google.com/store/apps/ details?id=com.Elementals.Bike3DConfigurator)

figure 17.10 LEGO 3D catalogue AR app (https://play.google.com/store/apps/ details?id=com.lego.catalogue.global)

401

402

Korenova et al.

6.4.1 Augmented Repair The application is the AR version of a user manual (for example for a coffee machine), together with instructions for minor repairs, which students can handle on their own (Figure 17.11). Contrary to the application, product manuals are usually long and hard to understand. In addition, the application also enables users to visualize the procedure. The teacher can discuss with pupils how similar applications could be designed to repair or maintain household appliances. It is a suitable application for developing their digital and technical literacy. We recommend We recommend this app for students of all ages.

figure 17.11 Augmented repair AR app (https://play.google.com/store/apps/ details?id=de.reflekt.enterprise.awedemo)

6.5 Programming, Creation of 3D Objects There could be a dozen more applications used to create various 3D objects that are subsequently placed in the AR environment, in particular those applications that can be used by school-age children. In addition to using AR as a motivating factor, this could also encourage their creativity. So far, we have managed to find only one such application. We hope that there will be more of them in the future. 6.5.1 Assemblr – Create 3D Models Assemblr is an application that enables creation of 3D structures as AR objects (Figure 17.12). It contains a collection of basic geometric figures and predefined

Applications of Augmented Reality Apps

403

figure 17.12 Assemblr – Create 3D Models AR app (https://play.google.com/store/apps/ details?id=com.octagonstudio.assemblr)

3D elements (which the developer adds to the collection gradually), and students can easily create their own 3D objects such as their own robot. They can then use this object as an object for AR. The application has the following modes: Build, 3D viewer, and AR. The application offers the possibility of cooperating through the internet, watching and commenting on the creations of others, and communicating within the social network of application users. This app is useful for developing students’ creativity. Not only can they can use the AR but also create their own AR environment. We recommend this app for high school and college students. 6.6 Games – Educational Games We have included AR applications that are designed for young children or primary school pupils and combine AR with playful activities, being able to positively influence their technical literacy. 6.6.1 ThomasAR World This application is a complement to Thomas & Friends, the storybook for children, published on the occasion of the 70th anniversary of the story. The book is provided with a free AR application, in which Thomas and other favourite characters come to life in an interactive 3D reality (Figure 17.13). Children can “pilot” a helicopter or “drive” a train in their own room. The pictures in the book are

404

Korenova et al.

figure 17.13 ThomasAR World AR app (https://play.google.com/store/apps/ details?id=com.redfrog.thomasarworld)

used as markers. The book, including the AR application, is suitable for enhancing technical literacy skills of pre-school children and primary school pupils. 6.7 STEM Scholarly Journals Scholarly journals dealing with STEM issues can target not only the professional community but they can also serve as an excellent supplement tool for developing technical literacy in schools. Prestigious magazines use pictures in their printed version so that they could offer the reader better visualization and interaction with newly released innovations and presented technologies. Following is an example: 6.7.1 HBR Augmented Reality The HBR AR application (Figure 17.14) is a supplement to the Harvard Business Review magazine. In its November-December 2017 issue, the HBR magazine published the “Why Every Organization Needs an Augmented Reality Strategy” article by Michael Porter and James Heppelmann. They offer this article free of charge and it can be printed directly from the application, including markers. Watch an animated X-ray view of a hydraulic power unit, see step-by-step instructions for a repair, and control a robot arm with your touchscreen. We recommend this app for high school and college students.

Applications of Augmented Reality Apps

405

figure 17.14 HBR AR app (https://play.google.com/store/apps/details?id=com.ptc.hbrar)

6.8 Aids for Simple Technical Handling – Measuring Devices Measuring tools hidden in AR applications have also become popular recently. They enable users to measure the length, area, volume, angle, and so on. These applications are very useful for mathematics and physics lessons. Developers acknowledge that the accuracy of the measurement is not yet perfect, but they continue working on increasing the applications’ accuracy. 6.8.1 ARuler – AR Ruler App The application (Figure 17.15) has the following tools: Ruler (measurement of linear dimensions in mm, cm, m, inches, feet, yards), Distance (from the camera of a device to a fixed point in real surroundings), Protractor (measures angles in real surroundings), Area and Perimeter (calculates the area and perimeter of polygons), Volume (enables measurement of the size of 3D objects using a designated base), Route (calculates the length of a trajectory), Height (measures the height of an object with respect to a detected surface), Layout (creates the layout of a marked object and exports it into the PDF format). These apps are well suited to developing digital literacy and mathematics to estimate distance, content, and body volume in real life. Students can first estimate and then measure with the app data they would otherwise not be able to measure (for example, the height of buildings or trees). It is a good tool for discovering and creative learning and learning. We recommend this app for students of all ages.

406

Korenova et al.

figure 17.15 ARuler – AR Ruler app (https://play.google.com/store/apps/ details?id=com.grymala.aruler)

figure 17.16 Prime Ruler AR app (https://play.google.com/store/apps/ details?id=com.grymala.photoruler)

Applications of Augmented Reality Apps

407

6.8.2 Laser Levelling Device This app (Figure 17.16) is equipped with: laser pointer, regular 3-mode spirit level/bubble level (libella), and clinometer (inclinometer) for measuring vertical angles and determining the exact horizontal plane. It enables measurement of the angle or inclination of objects (including remote ones) such as roofs, buildings, columns, mountains, trees, etc. Using this application is the same as the application ARuler. 6.9 Hiking – Navigation AR technology has greatly improved the functionality and use of navigation applications. When a route, direction, goal, and other data necessary for orientation in the space are additionally drawn into a real image acquired by means of a smartphone camera, it is very useful. Sygic was the first company to introduce AR into its navigation application in 2017. Since then, a large number of commercial and free navigation applications have appeared on the market. We present some of them: 6.9.1 AR GPS Drive/Walk Navigation This is a free app. The driver is guided directly by a virtual path from the camera preview video that is intuitive and easy to understand. While using this system, the driver does not need to map the path and the road. The driver can watch the real-time camera preview navigation screen to get driving conditions without impacting driving safety. It is motivating for students if they can use similar applications in their school as their parents in real life. Because children cannot yet chose, this app can also be used as a visual aid, for example, by school bus. The application can also be used for walks. Then they can discuss with the teacher the usefulness of such applications. We recommend this app for students of all ages. 6.8.2 AR GPS Compass Map 3D Pro This application combines a magnetic field sensor, accelerometer, and gyroscope to accurately determine direction. In addition to the compass features, it also enables users to measure the height of large objects as well. Using this application is the same as the application AR GPS.

7

Discussion

In this chapter, we highlighted some theoretical considerations for using technology in technical education, especially in Slovakia, as well as offered examples of AR technologies that could be utilised in technical education. It is likely

408

Korenova et al.

that AR will be part of technical education in schools and curricula at both primary and lower-secondary levels. The uses of AR falls within the field of technical education, which is typically provided in the form of a separate subject in Slovakia and the Czech Republic, or in the worldwide common form of STEM. If we analyse the prescribed curriculum that is implemented in schools in both specified countries, we do not find any teaching content dedicated to augmented reality yet, but there are efforts to integrate AR into its curriculum. Implementations where teachers introduce AR applications to their pupils and teach them to use such applications in practice are rare, but our teacher programmes are starting to utilise these applications. We hope that we can contribute to the development of this field in the near future.

References Clark, A., & Chalmers, D. J. (1998). The extended mind. Analysis, 58(1), 7–19. European Commission. (2017, October 19). Why STEM subjects and democratic citizenship go together. CESAER Annual Conference, Budapest. Retrieved from https://ec.europa.eu/commission/commissioners/2014-2019/navracsics/ announcements/why-stem-subjects-and-democratic-citizenship-go-together_en European Schoolnet. (2017). STEM education. Retrieved from http://www.eun.org/ focus-areas/stem IŠV. (2014). Inovovaný štátny vzdelávací program pre 1. stupeň ZŠ. 2014. Bratislava: MŠVVaŠ SR. Retrieved from http://www.statpedu.sk/sk/svp/inovovany-statnyvzdelavaci-program/inovovany-svp-1.stupen-zs/ Juncker, J. C. (2015). Completing Europe’s economic and monetary union. Retrieved from https://ec.europa.eu/commission/sites/beta-political/files/5-presidents-report_ en.pdf Kostrub, D. (2017). 3x meraj, potom rež, len si žiakov neporež! Interpretatívne skúmanie vyučovania matematiky [Look before you leap, just don’t jump on your students. Interpretative examination of teaching mathematics]. Pedagogická Revue, 64(1), 103–124. OECD. (2015). Students, computers and learning: Making the connection, PISA. Paris: OECD Publishing. http://dx.doi.org/10.1787/9789264239555-en Pasnik, S., & Hupert, N. (2016). Early STEM learning and the roles of technologies. Waltham, MA: Education Development Center, Inc.

Applications of Augmented Reality Apps

409

STEM Framework for Flemish Schools: Principles and Objectives. (2016). Retrieved from http://onderwijs.vlaanderen.be/sites/default/files/atoms/files/STEMkader%20%28Engels%29.pdf ŠVP. (2008). Štátny vzdelávací program pre 1. stupeň základnej školy v Slovenskej republike. Bratislava: MŠVVaŠSR. Retrieved from www.statpedu.sk/files/articles/ dokumenty/statny-vzdelavaci.../isced1_spu_uprava.pdf

CHAPTER 18

Devices for Virtual and Augmented Reality Robert Bohdal

Abstract Augmented reality offers unique opportunities for connecting the real world with the digital one through a variety of technologies. Many of these can be found in mobile phones and tablets. This chapter provides an overview of the most commonly used devices used in virtual and augmented reality (VR/ AR). We will describe the history, the principle of functioning and the possibilities offered by these devices. We will also mention devices that are still under development as prototypes. The most commonly used technologies undoubtedly include LCD and OLED display devices. Nowadays, 3D displays are being developed that use many different technologies. Smart glasses will certainly be used more often in the future. Many input/output devices provide communication with VR/AR devices. These include the touchscreen, microphone, and speaker, as well as increasingly used haptic devices. This chapter will enable readers to better understand the new technologies used in VR/AR devices.

Keywords virtual reality devices – augmented reality devices – technology – displays

1

Introduction

With the advent of new, better and more affordable technologies, particularly miniature cameras, position sensors, and head-up displays, the interest in virtual reality (VR) has returned after many years. Augmented reality (AR) is now available on mobile device platforms (smartphones and tablets) and has become attractive due to a huge number of new applications, for example, the very popular Pokemon GO. At present, there are many manufacturers of devices for virtual and augmented reality. Among the best-known devices are undoubtedly Oculus Rift and Google Glass. © koninklijke brill nv, leiden, 2020 | doi: 10.1163/9789004408845_018

Devices for Virtual and Augmented Reality

411

Current tablets and smartphones offer enough computational power to be used for VR and AR. For a full VR experience, however, it is also good to use other VR/AR devices such as a head-mounted display and data gloves or other haptic devices. Current screens usually display only 2D images. However, prototypes of imaging devices have been created that can also display 3D images. Today, stereoscopic and autostereoscopic displays serve as a compromise to this technology of the future. The aim of our chapter is to show that even though the current technologies used in VR/AR devices are complicated, they use relatively simple principles. Many people who use these devices want to know how they work. This chapter can also be used by teachers to help students to answer technical questions about VR/AR devices and to encourage interest in studying technical sciences as technology is developing very quickly nowadays and new devices are invented. We would like to briefly and simply explain how the most commonly used VR/ AR devices work so that a reader without technical education can understand their functionality and at the same time get an overview of what devices are used. Every progress begins with some discovery or invention with its subsequent expansion into ordinary life. The same is true for VR/AR devices. Therefore, the first part of the work is focused on their history. At the same time, the reader can compare the difference between current devices and their first prototypes. The second part describes the most commonly used technologies in displays – LCD and OLED. Although OLED displays provide better images than a LCD, their development is still in early stages and their costs are high. The third part of our chapter describes devices which can display 3D images. Most of them, except for stereoscopic displays, are just prototypes or are inaccessible to a common user. The fourth part is dedicated to increasingly available head-mounted and optical head-worn displays, which are used not only in games but also in many industries. The fifth part describes touch panels, image sensors, and position sensors, which are mainly used in mobile devices. The sixth part briefly describes haptic devices that can simulate touch with objects in virtual reality. Many of them are commercially available, but the price of the most sophisticated ones is very high. The seventh part describes the principle of loudspeaker and microphone, which are used not only for communication between people and virtual assistants, also for controlling other devices by voice commands.

2

History

The history of AR devices is associated with the history of VR devices because the vast majority of these devices are used in both areas.

412

Bohdal

The first example we can find comes from the beginning of the 40s and was a part of the British Royal Air Force aircraft. In this airplane, a head-up display was used in which the radar image was projected onto a translucent glass placed in front of the cockpit of the aircraft.3 Thanks to this solution, pilots were not disturbed by the bright light of the radar display during night battles. The idea of a head-up display is also used today in mobile navigation software – the image is shown on the car’s front window (Wikipedia, 2018). Between 1957 and 1962, Morton Heilig invented and built the first VR machine called “Sensorama” in the form of a human sized cabinet. Sensorama2 not only projected a movie on a small projector, but, together with other devices, it stimulated other senses such as hearing (stereo sound), touch (vibrations and wind) and even smell (aromas; Boas, 2013). In 1968, Ivan Sutherland constructed the first VR system “Sword of Damocles,” using a head-mounted display that included head tracking and see-through optics. The device was heavy, so it was on an arm fixed to the laboratory ceiling. Since the performance of computers at that time was very low, the virtual reality scene consisted of simple wireframe models (Schmalstieg & Hollerer, 2016). Myron Krueger and his colleagues created an artificial reality lab called “Videoplace” in the mid-70s. Videoplace used projectors, video cameras, and special purpose hardware to display real-world silhouettes in an interactive virtual environment. Even though users were in different rooms, they could communicate through this virtual environment and also manipulate objects in this virtual environment. Computer advances at the end of the 80s allowed Krueger and his collaborators to experiment with many concepts of human interaction with computer-generated objects and scenes (Krueger, Gionfriddo, & Hinrichsen, 1985). In the 90s, AR was not only the domain of computer graphics scientists, it expanded to other areas – medicine, military technology, art, and others. In 1992, Caudell and Mizell introduced the concept of augmented reality for the first time. Both scientists collaborated on the development of a system that helped optimize and simplify the work of aircraft workers by using headmounted displays to avoid the necessity of looking at extensive drawings and schemes (Caudell & Mizell, 1992). In the same year, Louise Rosenberg created the first immersive augmented reality system in which the user controlled a remote robot’s arms by moving the arms of an exoskeleton. At the same time, the user sensed limitations in his movement by haptic sensor feedback. The system also used computer-generated objects (fields and guides) that assisted the user in performing real physical tasks (Rosenberg, 1993). In 1994, Julie Martin composed the first AR theatre performance, “Dancing in Cyberspace,” in which the artists danced around virtual objects on a real stage. In the same

Devices for Virtual and Augmented Reality

413

year at North Carolina University in Chapel Hill, scientists presented a medical AR application allowing the doctor to observe the fetus in a pregnant patient. As part of a football match broadcast on September 27, 1998, Sportvision displayed a yellow line that enabled fans watching the game to see the exact moment when the ball passed through the end zone. Various additional markers and information are currently used in almost every sport broadcast. In addition, a computer-generated map containing runways with existing obstacles was displayed during a test flight of NASA prototype X-38 rescue aircraft in 1999 (iGreet, 2017). Until 1999, no AR software was available outside of specialized research laboratories. This situation changed when Kato and Billinghurst released the first open-source AR platform called ARToolKit. ARToolKit uses video-tracking to complement a computer-generated object to an image taken by the webcam (Kato & Billinghurst, 1999). This library is still used in various areas of AR. After 2010, especially with the arrival of smartphones and tablets, AR became available for the general public. AR has also come into the automotive industry, not only because of the wide use of car navigation but also as a helper for service technicians. In 2013, Google introduced its Google Glass, and Microsoft introduced HoloLens a few years later. These smart glasses can display various content on an integrated display. Since then, AR has become very popular and widely used, although we often do not realize we are using it.

3

Technologies of Displays

Currently only two technologies are used in VR/AR, namely LCD (liquid crystal display) and OLED (organic light emitting diode). In the past, for a long time, displays used CRT (cathode ray tube) technology that was replaced by LCD technology at the end of the 20th century. OLED technology has expanded massively since 2003, when Kodak introduced a digital camera with a 2.2-inch screen. A few years later, in 2008, Sony XEL-1, the first commercially available TV using OLED technology, was launched. Most types of current displays are composed of coloured pixels. Each pixel is composed of three RGB (Red, Green, Blue) subpixels. Some technologies also use a fourth subpixel for another colour component, usually white or yellow. 3.1 LCD Screens LCD displays have the advantage of lower energy consumption compared to CRT displays, and, in addition, they allow the production of flat displays with thickness of only a few millimetres.

414

Bohdal

figure 18.1 Scheme of the LCD screen

The history of LCD screens began in 1888 when Friedrich Rienitzer discovered a natural liquid crystal in cholesterol obtained from carrots. In 1962, Richard Wiliams discovered interesting electro-optical properties of the liquid crystal and two years later George Heilmeier created the first liquid crystal display. Since 1970, liquid crystal displays have been used in displays for calculators and wrist watches. Colour LCD screens have been used in TVs, monitors, and projectors since the 1980s (Bellis, 2017). The LCD (see Figure 18.1) is formed by a thin layer of liquid crystals (1) which is located between two polarizing filters (2, 3). The liquid crystal changes its ordered structure under the influence of the electrical field and determines the amount of light (8) passing through. The LCD screen further includes a light source, as the liquid crystals do not produce the light directly. As a source, neon tubes were first used, but they were very impractical due to their shape and size. Today’s monitors use LEDs that evenly cover either the entire back surface of the panel or are only located on the edge of the display. The light they produce is diffused across the back of the display through a light guide. LCDs also contain electrodes (4, 5) that affect individual pixels of the display. The principle of operation of the LCD screen is that the light generated by the LEDs first passes through the polarizing filter. The polarized light further passes through a layer of liquid crystals arranged in a helix shape, which gradually turn the plane of polarization of the passing light by 90 degrees. Finally, this light passes through a second polarizing filter that is perpendicularly rotated to the first polarizing filter. When voltage is applied to the internal electrodes, an electric field is formed between them, causing the liquid crystal molecules to “straighten” and stop the light passing through. Since the polarization plane of the output filter is perpendicular to the first filter, “unrotated” polarized light will not pass through it, and the pixel will appear as “black.” The

Devices for Virtual and Augmented Reality

415

colour of the points is achieved by adding a colour filter with a RGB array of colour microprobes, which is usually located just under the LCD panel’s protective layer. Lightness – colour intensity is achieved by changing the electrical voltage that affects the degree of rotation/straightening of the spiral of liquid crystals (Tyson, 2000). The development of LCD panels has gone through a relatively long path, gradually eliminating individual deficiencies. At first, arrays of transistors (active matrix) began to be used instead of a simple grid of electrodes (passive matrix; see Figure 18.2). This technology was named thin film transistor (TFT), and it allowed controlling the intensity of each pixel separately and displaying the image at once, which solved the problem of slow image display. Furthermore, a better arrangement of the structure of the liquid crystals (Vertical Alignment) in individual subpixels increased the contrast and improved colour rendering even when viewed from the sides. Finally, in-plane switching technology (IPS) achieved not only better colour reproduction but also a wider viewing angle, allowing viewers to see the image clearly even from very sharp angles. Later, better colour rendering was ensured by using backlighting with colour LEDs. At present, quantum dot technology is the most advanced, and the screen no longer contains a colour filter that blocks a portion of the light, and therefore the display is brighter and consumes less energy. In more expensive displays, the contrast is improved by using many small LEDs covering the entire back of the display, each of which can be turned on and off independently. LCD panels can be found in almost every consumer electronics device – watches, calculators, smartphones, tablets, auto navigation, PC monitors, TVs, projectors, and head-mounted displays. 3.2 OLED Screens OLED technology is currently one of the most advanced. Like the LCD, this technology also allows the production of flat displays, but with a thickness of far less than 1 millimetre. These displays can be both flexible and transparent.

figure 18.2 Passive vs. active matrix

416

Bohdal

This technology still has one drawback: burning out of pixels, a gradual dimming of the intensity of the individual subpixel light. Unlike the LCD, the image on the OLED is much more saturated, more faithful to dark colours, has superior contrast, lower power consumption, wide viewing angle in both horizontal and vertical direction, and faster display of pixels. Although electroluminescent materials have been known since the early 1950s, the first practical use of OLED was made possible by Ching Tang and Steven Slyke, who in 1987 built the first prototype of the classic two-layered structure that is used today. As the beginning of the real development of OLED technology, we can consider the first use of lighting polymers by Cambridge Display Technology in 1996. The first displays with this technology were expected in 2001, but due to various problems, the first OLED TV came to market only in 2008 (Gaspar & Polikarpov, 2015). OLED (see Figure 18.3) consists of several layers. On the upper side there is a protective layer, under which there is a metal cathode (1) that feeds the emission layer (2). The conductive layer (3) serves for transmission of electron holes and is controlled by the transparent anode (5). On the bottom there is a layer of solid material (6), which ensures that the display has a solid shape. When voltage is applied to the cathode and the anode, electrons begin to pass through the emission layer towards the anode. On the opposite side, the anode takes electrons off, creating electron holes (places with missing electrons) in the conductive layer. The emission layer thus receives a negative charge and the conductive layer gets a positive charge. Since “positive holes” are much more mobile than electrons, they skip the boundary between the layers and get from the conductive layer into the emission layer. When an electron moves into a hole in the emission layer, electron recombination occurs and the organic material emits photons. Since each subpixel contains a different

figure 18.3 Scheme of the OLED screen

Devices for Virtual and Augmented Reality

417

chemical composition of organic polymers, the emitted photon has a green, blue or red colour. The light intensity of the individual subpixel is influenced by the magnitude of the voltage. In contrast to the LCD, the black colour is actually black as the OLED does not need backlighting (Freudenrich, 2005). Like the LCD, OLED technologies have gone through development. In simpler displays, a passive matrix grid is still used instead of active matrix grid due to a lower production price. Another possibility to lower the cost is to use only one kind of organic polymer that emits only one colour, for example, white. The colour of the subpixels is then achieved by adding a colour filter, as with some LCDs. Super AMOLED is currently the most advanced OLED display. Compared to the older types, it has a higher contrast, greater saturation of pixel’s colour and smaller power consumption. OLED displays, despite their advantages, have not yet massively expanded. They are most commonly used in displays of high-end mobile phones, cameras, and smaller displays in shorter-lifetime devices. Greater expansion for large display formats used in TVs and PC monitors is prevented not only by the higher cost but also the lifetime of organic polymers.

4

Devices Showing 3D Images

There are many kinds of devices displaying 3D images that can be used in VR. At present, stereoscopic devices are used most often. Besides stereoscopic devices, there are also volumetric (spatial), holographic, and pseudo-3D devices. Humans need two eyes to correctly perceive depth, with each eye seeing a slightly different image due to the distance between the eyes. The human brain can then, based on small differences in these images, perceive correct depth. When sensing depth, people also use the fact remote objects are smaller than those that are closer. 4.1 Stereoscopic Displays The basis of stereoscopic displays is to display a different image for each eye. In general, we can divide them into two types. The first type requires glasses for depth observation, while for the second type – autostereoscopic displays, the glasses are not needed. However, both types require only one image to be observed with each eye. There are many ways to achieve this. In head-mounted displays, both images are so close to the eyes that each eye can only see one image at a time. In classic stereoscopic displays, including 3D TVs, the images are separated by additional glasses. In autostereoscopy, the parallax barrier is the most commonly used technique for image separation.

418

Bohdal

figure 18.4 Brewster stereoscope (CC BY-SA 4.0 – Alessandro Nassiri, Museo Nazionale della Scienza e della Tecnologia Leonardo da Vinci, Milano, https://commons.wikimedia.org/wiki/File:IGB_006055_Visore_stereoscopico_ portatile_Museo_scienza_e_tecnologia_Milano.jpg)

Although the first stereoscope (a device for seeing a stereoscopic image) was created by British inventor Charles Wheatstone in 1838, practical usage of stereoscopy was only possible in 1849 when British scientist David Brewster used two lenses to separate two images (see Figure 18.4; Zone, 2007). For small displays like those used in head-mounted displays (HMDs), there is no problem in using one small display for each eye. However, for large screens, two images must be displayed on one display. The separation of these two images is allowed by special glasses. So-called active glasses that incorporate electronics to actively hide the image passing through the glasses are standardly used. There are also passive glasses without electronics that are used, especially in 3D cinemas. Technologically different solutions use autostereoscopic displays that do not need glasses, and separation of images for the right and left eye is provided by the display itself. 4.2 Autostereoscopic Displays Even though autostereoscopic displays have the advantage that the user does not need to use special glasses, they can only view the display from a certain distance. In addition, almost all autostereoscopic displays have a “horizontal parallax-only” (HPO) restriction which means that the user sees different and correct images for the right and left eye only in horizontal direction. With multi-view autostereoscopic displays, it is even possible to view the image from multiple sides. There are many technologies to ensure that only the right

Devices for Virtual and Augmented Reality

419

image reaches correct eye. They can use a parallax barrier, or bending, reflection, or diffraction of light (Geng, 2013). 4.2.1 Displays Using Parallax Barrier Stereoscopic images using the parallax barrier have been known since 1902 when American inventor Frederic Ives made it possible to observe a stereoscopic image without glasses (Ives, 1902). This technology uses a parallax barrier, i.e., mask of vertical stripes placed at a specified distance in front of the display. These stripes hide those parts of the image that should not be visible from specific locations in the area where the eyes of the observer are (see Figure 18.5). Each eye of the observer sees different vertical stripes of the image on the display because each second stripe is occluded by mask stripes. As a parallax barrier, a separate LCD panel can also be used to dynamically create vertical black bars so that the mask matches the current observer’s position. Displays with a parallax barrier have several drawbacks. The most important is reduction of brightness and resolution in half (Peterka et al., 2008). 4.2.2 Displays Using Time Aperture At the end of the 1990s, the University of Cambridge presented a multi-view autostereoscopic display based on high-speed CRT display and ferroelectric panel of liquid crystals (FELC) (see Figure 18.6). The CRT screen displayed images at high speed, each image for one view and one eye. For the image to enter the correct eye, a set of lenses was used, while one FELC cell was translucent and transmitted a portion of the light cone

figure 18.5 The parallax barrier principle

420

Bohdal

figure 18.6 Display with a time aperture

emitted by the CRT display. The remaining FELC cells were dark at that time and blocked the remaining part of the light cone so that the same image did not reach the second eye. The CRT screen quickly displayed individual images, and the FELC synchronously with the CRT display worked as a necessary barrier to get the image to the right place in each viewing area (Dodgson, 2005). In a newer version of this technology, the CRT display was replaced by high-speed LCD. These displays have an advantage of displaying a full-resolution image, but at the cost of slower image display (Geng, 2013). 4.2.3 Displays Using Light Bending (Refraction) The basis of these displays is a panel consisting of vertical stripes of cylindrical lenses – lenticulars (see Figure 18.7). These lenses play the same role as the parallax barrier, and as they are translucent, there is no loss of brightness. Stripes of the lenses are precisely aligned with vertical pixel columns on the LCD panel located in the focal plane of the lenses (Hong et al., 2011). Lenticular displays also reduce the image resolution. For multi-view displays, the resolution decrement is more than double. This disadvantage is partially reduced by slanted arrangement of stripes of the lenses that decompose the reduction of resolution into the vertical direction (Berkel, Parker, & Franklin, 1996). Another interesting solution is the use of a second panel of liquid crystals which are rotated by the electric field to bend the light and hence replace the lenticulars. This display can be easily changed from 2D to 3D and vice versa (Huang, Chen, Shen, & Huang, 2010). Because of the limited number of views that multi-view 3D displays can display, there is a discontinuous change of the view when an observer looks at a virtual object from different angles. However, when observing a real object, there is no discontinuous change. This problem has been solved with super-multi-view

Devices for Virtual and Augmented Reality

421

figure 18.7 The lenticular display principle

and adaptive technology. Super-multi-view 3D displays provide up to 256 views and adaptive 3D displays track eye movement and distance from the display. Another problem in classic 3D displays is that it causes eye fatigue when looking at a stereoscopic image. Because of the convergence of the human eye, the observer perceives the correct depth of virtual objects, but the accommodation makes the eyes always focused only on the screen (Geng, 2013). 4.3 Volumetric Displays While stereoscopic displays display the image on a two-dimensional area, volumetric 3D screens display 3D images in real 3D space. Each voxel1 of the 3D image has a true position in the three-dimensional space of the display, and it emits light from that position in multiple directions, which enables seeing a real 3D object image. Volumetric 3D displays give the viewer a realistic view of the depth of 3D objects, and there is no accommodation-convergence conflict. On the other hand, their biggest drawback is their volume, which makes it impossible to produce large volumetric displays. Depending on the type, we can divide the displays into displays with static screens and swept volume planes. In both cases, the voxel can emit light directly or the voxel can be illuminated by an external light source, for example with a laser beam (Geng, 2013). 4.3.1 Volumetric Displays with Static Screens These displays create an image without moving parts in the volume of the display.

422

Bohdal

In 1997, American researcher Elizabeth Downing presented a volumetric display that used infrared laser beams. The two rays emerging from the lasers are directed using mirrors to the desired point inside the display. At the point of their intersection, the two “infrared” photons are absorbed, and consequently visible light is emitted. This process is repeated very quickly for all voxels that should be “lit.” The interior of such a display is either glass with fluoro-zirconium or a gas that includes rubidium (Downing, 1997). At the beginning of 2018, engineers at Brigham Young University showed a fascinating way of displaying coloured lights in the air. Using a photophoretic trap, it is possible to create a full-colour 3D graphic display anywhere in open space visible from any angle. Captured particles are simultaneously deflected and illuminated by lasers, which produces a three-dimensional image with a large colour scale and fine detail (Smalley et al., 2018). The concept of a 3D display consisting of colour LEDs is relatively simple. It is a 3D grid in which three-colour LEDs are evenly placed. At present, there is a commercial manufacturer that offers a 3D LED cube with a resolution of 32×32×32 voxels. Since the diodes are not translucent, there must be considerable spacing between them so that they do not hide each other. As a result, it is not possible to achieve a satisfactory resolution of the display when using LED elements with the current sizes. A 3D display using optical fibres makes it possible to achieve a somewhat higher resolution than the technology using LEDs. Optical fibres are used similar to transparent electrical conductors in LCD panels, but they transmit light instead of electricity. The system of optical fibres is embedded in a translucent resin with a mixture of organic pigments. In a quiet state, the cells are transparent, but when they are affected by light coming from the end of the optical fibres, they fluoresce (MacFarlane, 1994). Another option is to use several layers of panels that display threedimensional image sections. They can be transparent OLED panels or liquid crystal panels acting as two-state diffusers. Voxels of the LC panel are optically transparent when they are not influenced by the electric field. However, if a cell (voxel) of the panel is influenced by an electric field, it behaves like a diffuser and diffuses the light from the cell towards the observer. Images of the individual cuts are generated by the projector which is located on the back of the 3D display. The high-speed projector and individual LC panels are synchronized to each other, which allows displaying 2D cuts of virtual 3D objects on individual LC panels to create a true three-dimensional image (see Figure 18.8). A volumetric 3D image system based on this technology and with 20 LC panels has been sold since 2014 by LightSpace Technologies (LightSpace3D, 2014).

Devices for Virtual and Augmented Reality

423

figure 18.8 Example of a multi-layered 3D display with LC panels

However, displays using this technology have low brightness due to light absorption by many panels and a short amount of time in which the individual voxels can shine. The benefits of a 3D volumetric display using static voxel elements include their conceptual simplicity as they do not contain any moving parts. However, without using a rotating area that creates a 3D volume from an array of 2D elements, a huge number of voxels must be placed into the 3D cube, which is currently beyond our capabilities. 4.3.2 Volumetric Displays with Rotating Surfaces This type of 3D displays relies on the inertia of image perception by humans. If planar cuts of a 3D object are displayed quickly after each other, the human brain perceives them as one 3D image. These cuts may not actually be planar, depending on the type of the rotating surface in the 3D display. The rotating surface may have a rectangular, circular, or helical shape. As the surface rotates at a high speed, it stops being visible to the observer. The observer can walk freely around the 3D display and look at the three-dimensional image of the virtual object from any angle. The first volumetric display with a rotating surface used a CRT monitor as the source of the image, and it was built in 1958 (Hirsch, 1961). Later, in the 1960s, a display containing a rotating surface covered by a luminophore was introduced. Voxels on this surface were illuminated with an electron beam, similarly as in CRT screens. In 1979, a 3D display was developed, the rotating surface of which consisted of many high-speed LED elements. The resolution of these displays is influenced not only by the number and density of the illuminating elements, but also by the speed of rotation of the surface and by how long the individual points shine (Geng, 2013).

424

Bohdal

At present, a laser beam or a high-speed projector is used as the image source. An example of a 3D display using laser-scanning was introduced in 1997 by Soltan and colleagues. This display uses multiple laser beams to draw coloured points on the surface of a rotating helix. An acoustic-optical unit deflects the laser beam flashes onto the surface of the quickly rotating helix so that it seems that each voxel glows from a specific place in the 3D space (Soltan, Lasher, Dahlke, Acantilado, & McDonald, 1997). Laser scanning 3D displays produce an image on the rotating surface by the laser beam which is deflected to a specific point using a time-accurate synchronized mechanism. Since the image is rendered by scanning, only one point can be displayed at a given moment. All active points must be illuminated one by one using a single laser beam. The time required to draw a point limits the number of voxels that can be used in the display. If there is a need to increase the number of voxels, more laser beams can be used, but this makes the display more complicated, which results in a higher price of the display (Geng, 2013). An example of a commercially available Voxon VX1 3D display is in Figure 18.9 on the right. A high-speed projector can be used instead of a laser beam. The projector is synchronized with the quickly rotating plate and displays the previously calculated cuts of the 3D object (Favalora, 2005).

figure 18.9 The principle of the volumetric display with a rotating surface (photo courtesy of Voxon Photonics)

4.4 Holographic Displays A true 3D image is much more than just a set of shining points on a 3D display. Light is made up of a huge number of electromagnetic waves that affect each other. If we could “artificially create” such a wave and direct it to the eyes of the observer, an ideal 3D display would be created. Such a display would be something like a virtual window into a real 3D scene. Of all devices displaying 3D images, holographic displays are the closest to this ideal.

Devices for Virtual and Augmented Reality

425

4.4.1 Electro-Holographic Displays Scientists from the Massachusetts Institute of Technology built the first holographic display called Mark I in 1992. Their first prototype displayed an image of size 2.5cm × 2.5cm × 2.5cm with 192 rows and 15° field of view. Later, in 2014, the scientists introduced Mark IV that included significantly cheaper components and displayed an image with a higher resolution. This display is still horizontal parallax-only, displays a maximum of 468 rows and has a small field of view (Jolly, Smalley, Barabas, & Bove, 2014). Creating an image in an electro-holographic display begins with a calculation of the optical modulation characteristics for every point of a 3D image of the object. These characteristics later modulate the laser beam in the display. It is necessary to process a huge amount of data – while a usual 2D display has a pixel size of about 100 micrometres, the holographic display uses interference patterns with size less than 0.5 micrometre. In a holographic display, we can see objects like in a real environment. The device screen behaves like a hologram and all points on its surface emit light waves in different directions with different characteristics, similarly as when looking through the normal window (Lacko, 2016). In the holographic display Mark II (see Figure 18.10), a laser beam is modulated in an acousto-optic modulator (AOM) which changes its original properties as if it was affected by a real hologram. The AOM is controlled by the computer and receives data about individual “digital” interference patterns. The modulated beam is further focused by the lens and then directed to the vertical scanner. The beam then passes through a splitter and goes to one of the

figure 18.10 The electro-hologram principle

426

Bohdal

six horizontally connected scanners. These scanners deflect the beam to get the light onto a particular point of the display. Further, the beam is reflected by the beam splitter onto the magnifying lens and eventually falls on the diffuser where the resulting 3D image can be observed. The whole image is created by gradual rendering of all points (St-Hilaire, Benton, Lucente, & Hubel, 1992). Computational holography calculates interference patterns numerically, simulating light propagation, or it combines data obtained from multiple cameras that capture the real object from many views. Computer generated holograms must process a huge number of interference patterns. The first electro-holographic display Mark I needed many minutes of calculation for each static image. Holographic display by SeeReal company uses eye-tracking to calculate the direction of the light wave propagation reflected from virtual objects only for the places where the eyes of the observer are located. This approach greatly reduces the amount of information that needs to be processed and subsequently displayed, allowing the observer to move freely in front of the screen and to view the modified image in real-time with sufficient resolution and wide field of view (SeeReal Technologies, 2018). The holography principle itself causes the holographic display to process and then display a huge amount of information. This alone poses a huge technical challenge, and it is not possible to predict when the first commercial holographic displays will be available at an affordable price. 4.5 Pseudo-3D Displays As 3D imaging becomes more appealing to the media and the public, many devices displaying a virtual image often have the name “holographic.” Examples include the “hologram” of Jessica Yellin, broadcast on CNN during US presidential elections in 2008 (YouTube, 2008). Similarly, “hologram” is sometime used to refer to a projected image on glass surface or semi-transparent film (see Figure 18.11). Additionally, the term “hologram” is also used for an image projected onto fog or an image that creates graphic patterns on drops of flowing water, which could be seen by visitors of the World Expo in Spain in 2008 (Geng, 2013). Some pseudo-3D displays have even become an interesting commercial product, for example, Kono-mo Hypervsn™ that uses fast-moving arms with miniature colour LEDs (Hypervsn™, 2018). However, these “holograms” are not based on the principles of multi-view stereoscopic, volumetric, or electro-holographic displays. There are also technologies that cannot be clearly categorized into any of the previous categories. Examples include commercially successful products from the American company called Zebra Imaging, whose products are used by the US military research agency DARPA. Zebra Imaging produces not only

Devices for Virtual and Augmented Reality

427

figure 18.11 An example of virtual image projected on semi-transparent film

large-format holograms, but also multi-view “autostereoscopic” displays that use elements called “hogels” that distribute light into multiple views.

5

Head-Mounted and Optical Head-Worn Displays

In the category of head-mounted and optical head-worn displays that can be used in VR/AR, we can already find a relatively wide range of commercially available devices. The best known are Oculus Rift, Google Glass and Microsoft HoloLens (Figure 18.12). Their practical use is relatively extensive. They can be used in the entertainment industry, computer games, in various scientific and

figure 18.12 Google Glass (left, CC-BY-SA-3.0, Tim Reckmann, https://commons.wikimedia.org/wiki/File:Google_Glass_Main.jpg) and Microsoft HoloLens (right, CC-BY-SA-4.0, Ramadhanakbr, https://commons.wikimedia.org/wiki/File:Ramahololens.jpg)

428

Bohdal

technical disciplines, in medicine, in teaching, and later we will probably use them in all areas of our life. Greater use is hindered by a high price and not quite satisfactory technical parameters, such as the field of view (FOV), the resolution, the sharpness or the contrast of the displayed virtual image, and, finally, the transparency of the optical system. Researchers have been developing technologies to support interactive 3D visualization in head-mounted displays (HMD) and optical head-worn displays (OHWD) for over five decades. In 1968, a leading American computer graphics expert Ivan Sutherland presented his first device with a head-mounted display at Harvard University. In the mid-nineties, Microvision developed a retinal scanner that displayed a raster image directly onto the eye retina (Rolland & Cakmakci, 2009). In the last decade, scientists have made great progress in constructing HMD. In 2013, Google introduced Google Glass, and a few years later, Microsoft introduced HoloLens. Currently there are more than a dozen commercial manufacturers of these devices. Their size and weight have gradually diminished, the field of view and image resolution have increased, and some of them have become affordable for ordinary people. These devices can be divided into two types, depending on whether they are transparent like sunglasses or opaque, like glasses used in virtual reality. As part of the group of transparent displays, we can also include head-up display technology, which is currently used in aircraft and some cars. Using this technology, it is possible to display a virtual image (e.g., navigation data) on the surface of the car’s windscreen. 5.1 Head-Mounted Displays (HMD) Devices with an opaque optical system worn by the user on the head have been developed for several decades. In the last five years, however, they have greatly expanded. The best-known product is Oculus Rift (see Figure 18.13), which is mainly used in virtual reality and computer games. The first HMD displays were very bulky because they used small CRT displays. Currently, miniature LCD or OLED panels are used. Apart from the displays, we can also find optics needed to prevent the eye from focusing on the screen that is just a few centimetres away from the eye. Since there are two displays, one for each eye, the user sees a stereoscopic image. HMD displays additionally have head-motion trackers and stereo headphones, and some of them can also have eye-motion trackers. All of this helps to create a fully-fledged experience during playing games or using other virtual reality applications. 5.2 Optical Head-Worn Displays (OHWD) Optical head-worn displays allow not only display of a virtual image, they also allow the user to see the surrounding environment. Using this technology, it is

Devices for Virtual and Augmented Reality

429

figure 18.13 Oculus Rift (CC-BY-SA-4.0, Samwalton9, https://commons.wikimedia.org/wiki/ File:Oculus_CV1_Back.jpg)

possible to see real-world images together with virtual images through transparent optics. Some devices display an image for only one eye, but some of them display a stereoscopic image for both eyes. The basic element of these displays is an optical combiner that combines virtual and real images into one. Even though the first display with this technology appeared at the end of the 1990s, there is always something that needs to be improved. Despite the considerable technological advances in optics, these displays do not offer sufficient image resolution along with a large field of view (more than 90°). The critical element of these displays is not an image-generating device (a display or a micro-projector), but an optical combiner that has the task of joining the light coming from the viewer’s surroundings with the virtual image and directing it through the optics into the eye of the observer. The simplest principle of how these devices work can be seen in Figure 18.14. The image is created by a small display, which is usually located outside of the field of view. This image is directed through the optical system to an optical combiner, which transfers the image from the viewer’s surroundings along with the virtual image into the eye of the observer (Cakmakci & Rolland, 2006). The device that creates the virtual image is a small LCD or OLED panel or a micro-projector. To make the projectors as small as possible, only one DMD (digital micromirror device) chip or one LC or LCoS (liquid crystal on silicon) panel is used instead of the three needed for each RGB colour component. The full colour image is then created by displaying three images sequentially in rapid succession, each in one RGB colour component. Due to the imperfection of image perception by the human, it is possible to see the resulting image with a wide range of colours. As the light source, it uses three RGB light diodes, or

430

Bohdal

figure 18.14  The optical HMD operation principle

the light from a white LED is decomposed into the RGB colours by a dichroic mirror (Hazra, 2015). The created virtual image then goes through a complex optical system. First, we need the optical system to redirect the image created outside the field of view to the eye. Second, we need the virtual image to be displayed at a greater distance from the eye, so the eye does not have to focus on a very short distance. And finally, we need to make the small picture from the micro-display enlarged to the desired size. In the optical system, there are not only lenses and mirrors as in early devices. In some devices, we can find spherical or planar semi-transparent mirrors (see Figure 18.15) that serve as the optical combiner. However, these standard optical combiners are relatively large and absorb more than 50% of the light. Free-form optical prisms are slightly better with transparency, but they are still large. Therefore, modern and smaller OHWD use either polarized beam combiners, as in Google Glass, or flat holographic waveguides (see Figure 18.16), which can be found in Microsoft HoloLens smart glasses. However, their small size currently does not provide a very wide field of view. Google Glass has a field of view of only 13°, Microsoft HoloLens is better as it has 35° field of view (Guttag, 2016). OHWDs are not just “bigger glasses,” they contain many other electronic components. In addition to the aforementioned optical elements and the micro-display or micro-projector, they often contain one or more cameras, a microphone, head-movement sensors, speakers and, of course, a small

Devices for Virtual and Augmented Reality

431

figure 18.15 Planar, spherical and free-form optical combiner

figure 18.16 Optical combiner with holographic optical elements and waveguide plate

computer that drives it all. Finally, there is also a rechargeable battery that supplies the necessary power to the device. Some models also have GPS and even a small touchpad. It is quite natural that these devices are often called “smart glasses.” They can be connected to a laptop or smartphone via Bluetooth, Wi-Fi, or micro-USB. Cameras are needed not only for video recording, but also when the user controls their own smart-glasses by hand gestures or when they are used in augmented reality applications for object recognition. Some of them are also controlled by sound commands via the built-in microphone, or by the fingers on the touch panel, just like in a laptop (Warren, 2016). The use of OHWD can be truly versatile but they are currently not widely used due to their high price. Some models cost more than $3,000. But if you own a smartphone with at least a Full HD display, you can buy a VR headset for a few dozen dollars (see Figure 18.17). The headset is a simple “holder” with touch panel and with the necessary optical system, into which you can easily insert a smartphone.

432

Bohdal

figure 18.17 Headset Gear VR for Samsung Galaxy smartphones (CC BY 2.0, Maurizio Pesce, https://commons.wikimedia.org/wiki/File:Samsung_Gear_VR_ (15247457825).jpg)

6

Input Devices

There are many input devices that we use on smartphones, tablets, and similar devices to play some AR games. The most common input device is the touch panel. We often do not even realize that we communicate using other input devices. Even when unlocking a smartphone or some laptops, we use the fingerprint scanner or built-in camera to capture the face of the user. In mobile devices, we can also find a GPS (Global Positioning System) module that can be used to determine geographic position, an accelerometer to determine the current acceleration, a gyroscope for finding the change of rotation, a compass for finding the north, and a light sensor that detects the intensity of the surrounding light. The proximity sensor is suitable for measuring short distances and serves, for example, for turning off the display during the phone call. Moreover, we can find an infrared 3D scanner or stereo camera with two lenses to get a fully-fledged 3D image in some smartphones. Location and motion sensors are useful not only for navigation and orientation in space, they also allow playing games that use the rotation of the display for control. Accelerometers can even be used to detect earthquakes or trigger the airbag in the car. The production of microelectromechanical accelerometers began in 1993 (SCME, 2017), and nowadays the accelerometer, along with the gyroscope, is so small that it can fit on one small chip. Data gloves are also often used in the VR. Using data gloves, the software can determine

Devices for Virtual and Augmented Reality

433

the location of the user as well as the location of his fingers and allow them to interact with virtual objects. 6.1 Touch Panel Touch panels have become common in the era of smartphones and tablets. There are three basic systems that are used to detect touch: resistive, capacitive, and wave-acoustic. They are mostly used as a replacement for the computer mouse. The basis of resistive touch displays is formed by two conductive layers separated by an insulator, one of which is conductive and without resistance. The second one is constantly powered by electric current and has a specific resistance. Upon touching, the two layers are joined together and the electrical voltage reduced by the resistance appears on the conductive layer depending on where the layers are joined. According to the amount of the electrical voltage, it is then possible to determine the x or y distance from the edge of the touch surface. In capacitive touch panels, a layer that can hold an electrical charge is at the top. When a user touches the panel with their finger, a portion of the charge is transferred to the user and the charge on the capacitive layer is reduced. This decrease is measured in circuits located in each corner of the touch panel. From relative differences of charges in each corner of the panel, the resulting touch position can be determined. Capacitive technology has the advantage that only 10% of the intensity is lost when the light passes through the panel while resistive panels absorbs up to 25% of the light. Panels with resistive technology are more susceptible to damage, but unlike capacitive panels, they also react to the touch of a non-conductive object. Acoustic wave touch panel technology uses two transducers; one transmitter (1, 2) and one receiver (3, 4), positioned along the x-axis and y-axis of the touch panel (see Figure 18.18). The sound waves propagating from the transmitter are scattered throughout the display layer using an array of reflectors (5). Using another array (6), they are then directed to the receiver transducer. When the touch interrupts the flow of sound waves, this interruption is detected by the device’s processor from the signal on the receiver. Since the acoustic wave principle does not require any conductive layers, the touch panel does not absorb any light (HowStuffWorks.com, 2001). 6.2 Image Sensor Image sensors are mainly used in cameras. The light sensor converts the photon energy into electricity. The more photons fall on the sensor cell, the more electricity appears in the sensor and the brighter the point will be in the resulting

434

Bohdal

figure 18.18 The principle of an acoustic-wave touch panel

image. The voltage magnitudes in each cell are sent to an analogue-to-digital converter where they are converted to a numerical value. From the numeric values of each sensor cell, the camera processor calculates the values of each pixel of the resulting image. More than three cells are reserved for each pixel of the scanned image. Since the human eye is more sensitive to green colour, more cells are used for the green light (see Figure 18.19). The resulting image is then calculated from the cell values by interpolation.

figure 18.19  Example of an array of subpixels of the CCD sensor

State-of-the-art image sensors are not just classic 2D light sensors. A third dimension is needed to accurately detect the movement of an object in space. As a 3D image sensor, two classic 2D light sensors separated by several centimetres can be used. From the two images that the sensors provide, the depth of each pixel is calculated by simple mathematical relations. The popular Kinect device uses a different technology (see Figure 18.20). The infrared projector illuminates the space with a structural pattern consisting of

Devices for Virtual and Augmented Reality

435

figure 18.20 Microsoft Kinect, the old (public domain, Evan-Amos, https://commons.wikimedia.org/wiki/Category:Kinect#/media/ File:Xbox-360-Kinect-Standalone.png) and the new version (public domain, Evan-Amos, https://commons.wikimedia.org/wiki/Category:Kinect#/media/ File:Xbox-One-Kinect.jpg)

many bright circles. The infrared camera scans the reflected structural pattern and determines the distance of the individual structural points based on the pattern deformation changed by reflection from objects, using the pre-calibrated map of the distance of the individual structural points. The sensor in Microsoft Kinect v2.0 uses Time-Of-Flight technology, which indirectly measures time during which the pulse transmitted from the laser projector is reflected from the target surface and returned to the image sensor. This is repeated very quickly for each 3D point and a resulting depth map is obtained (Lau, 2013). In addition to the infrared projector and sensor, Kinect has also a standard camera and several microphones. Although it is a relatively unique device and many people enjoyed it, Microsoft decided to stop manufacturing Kinect at the end of 2017. 6.3 GPS Receiver To determine position (including altitude), a GPS receiver must use a signal from four or more satellites. This signal, which propagates with the speed of light, contains highly accurate information about the current time and position of the satellites. Based on the time of transmission and reception of each satellite signal, the receiver calculates the distance to each satellite. Subsequently, a sphere is created for each satellite. Each sphere is centred in the position of the satellite and has radius equal to the distance of the satellite to the GPS receiver. Since the receiver is located at the point of intersection of all the spheres, its position can be calculated by the centre of this intersection. 6.4 Accelerometer An accelerometers measure acceleration in a given direction. Capacitive accelerometers are currently used the most often. To measure the acceleration, change in capacity on miniature capacitors is used. A movable part (2)

436

Bohdal

is attached using springs (1) to the base, on which fixed plates (3) are located (see Figure 18.21 on the left). When accelerating, the movable part moves in the opposite direction to the direction of acceleration and the capacity c1 and c2 between the fixed and movable plates changes (see Figure 18.21 on the right). The value of the change in capacity then determines the magnitude of the acceleration (Bernstein, 2003). To determine the local change and orientation of the device in 3D space, three accelerometers are needed, one for each coordinate axis.

figure 18.21  The principle of the accelerometer

6.5 Gyroscope Using the gyroscope, it is possible to measure rotation around one of the coordinate axes. The gyroscope uses the Coriolis force to measure angular velocity (see Figure 18.22). If angular velocity (4) is applied to the moving object (1) in the direction of the vector (2), the object begins to deviate from the original direction due to the influence of the Coriolis force (3). Similarly, as in the case of the accelerometer, this movement causes a change in capacity that is proportional to the magnitude of the applied angular velocity (Bernstein, 2003).

figure 18.22 The effect of the Coriolis force on a moving object

In the gyroscope, the movable part (2) vibrates very quickly in the driving direction (4). However, if the gyroscope starts to rotate around axis (5), the vibrating part begins to deviate in a direction perpendicular to the direction of vibration, resulting in a change of capacity c1 and c2 (see Figure 18.23 on the right).

Devices for Virtual and Augmented Reality

437

figure 18.23 The principle of the gyroscope

6.6 Compass The compass uses the Hall effect, which causes a potential difference between the electrodes of the semiconductor plate. If the current passes through the semiconductor plate, the electrons pass from the cathode to the anode. However, if the plate is affected by the magnetic field of the Earth, their direct flow is disturbed, and the electrons begin to deflect to one side of the plate. As a result, it is possible to observe the potential difference, i.e., the voltage between the opposite sides of the plate, which depends on the strength of the magnetic field and its direction.

7

Haptic Devices

Haptic devices (HDs) allow manipulation of virtual objects by touch. They are usually divided into tactile, kinaesthetic, and hybrid. Using tactile haptic devices, we can feel the texture of the surface of a virtual object, its roughness or smoothness, temperature, surface tension, and other characteristics. Kinaesthetic HD serve for interaction with the object as such, allowing us to perceive not only its overall shape and weight, but also the force it exerts on us, to determine or change its position, orientation, and others. Hybrid haptic devices are a combination of tactile and kinaesthetic and combine the capabilities of both types. In haptic devices, feedback is often used when manipulating virtual objects, i.e., response of the object to touch. The simplest form of feedback is vibration, which is often used when pushing a virtual (software) button on a tablet or mobile. Another example of feedback is the use of mechanical forces that moves the arms of the haptic device (Bermejo & Hui, 2017). Haptic feedback has been used in the VR for quite some time. For example, a vibrating joystick was used in gaming machines in the 70s. 7.1 Tactile HD Tactile HDs are now among the most commonly used HDs. There are many technologies to create a feeling of touching a virtual object. For example,

438

Bohdal

vibration, as response to touch, can be made using a miniature motor with an eccentric rotating mass (see Figure 18.24) or a linear resonant actuator that uses an electromagnetic coil to move. It is also possible to use active surfaces to create an illusion of touching an object. Pin arrays technology simulates the surface and shape of the object by ejecting a large number of small pins. Similarly, there is a refreshable braille display for blind people, where small electromagnets are used to eject the pins. A field of small ultrasonic speakers is also used for a touch-free mid-air Ultrahaptics device. These speakers create sound waves that simulate the contact of the object with the skin of the hand (Bermejo & Hui, 2017).

figure 18.24  The principle of a vibrator using electric motor

7.2 Kinaesthetic HD In kinaesthetic HDs, the muscles of the hand and its movement are used to interact with a virtual object. Feedback is generated by mechanical forces that create variable resistance to hand movement. These forces are most often generated by electromechanical, pneumatic or hydraulic technology. According to the construction, we can divide the kinaesthetic HDs into two groups. The first group consists of manipulanda, for example, attached to a desk. The second category consists of grasp devices and exoskeletons that are attached to a certain part of the body, for example, to the hand (Bermejo & Hui, 2017). Manipulanda use a set of arms and electric motors. When pulling or exerting pressure on the end element, the force is transmitted by the arms to the rotors of the electric motors (see Figure 18.25 on the left). The electric motor will produce the necessary braking power depending on the resistance that a virtual object will produce when touching a hand or finger at a given point. Grasp devices and exoskeletons put variable pressure on individual fingers or other part of the body to simulate contact with the virtual object. Finger movement is transmitted, for example, using a system of wires to the electric motors. These electric motors produce the necessary mechanical force that reverses the movement of the wires.

Devices for Virtual and Augmented Reality

439

figure 18.25 The principle of how manipulandum works (left), commercial product Novint Falcon (right) (GNU Free Documentation License, Archimëa, https://commons.wikimedia.org/wiki/File:Novint_Falcon.jpg)

figure 18.26 HaptX glove (photo courtesy of HaptX)

An example of hybrid HD HaptX Glove from HaptX company, which combines kinaesthetic and tactile HDs, is in Figure 18.26. Many kinaesthetic HDs contain motion sensors so that the software can identify the position of end elements (e.g., fingers) when using them. Haptic devices are an invaluable addition to improving user experience in VR. They can be useful not only for playing games, but, for example, for teaching medicine, when screening is often done using touch. The widespread use of hybrid and kinaesthetic HDs is still hindered by a high price, which is several tens of thousands of dollars in the case of more expensive HDs.

8

Sound Devices

Sound is an important part of our communication. It is therefore understandable that sound is used in many devices. By using a soundtrack, the user

440

Bohdal

experience in VR can be greatly enhanced. Sound is also often used as feedback for a variety of software events such as mouse clicks or keystrokes. Historically, sound devices have been linked to Alexander Graham Bell, who in 1876 built the first telephone using a microphone and a speaker. At present, audio input is not only used for communication between people, but thanks to speech recognition, we can also communicate with a virtual assistant in a phone (Siri), laptop (Cortana), or a separate device such as Amazon Alexa. There are several technologies that are used in speakers and microphones. The most common are dynamic and piezoelectric, but there are many more. 8.1 Speaker A dynamic speaker (see Figure 18.27) produces sound by vibrating a diaphragm (5), which is attached by means of a flexible border (7) to the metal frame (4). The diaphragm is connected at the other end to a flexible ring (2), which allows free movement of the coil (3) around which there is a permanent magnet (1).

figure 18.27  The scheme of the speaker

The principle of the speaker is relatively simple. Alternating electric current (audio-signal) passes through the coil wire and creates a magnetic field, which is either attracted or repelled by the magnetic field of the permanent magnet, depending on the current polarity and magnitude of the input current. The movement of a coil influenced by a variable magnetic field is performed very quickly, many times per second. As the coil is connected to the diaphragm, the coil movement is transmitted to the membrane which makes the air in front of the speaker vibrate, which in turn generates sound waves (Harris, 2002). 8.2 Microphone The principle of a dynamic microphone (see Figure 18.28) that uses electromagnetic induction is exactly the opposite of the speaker. Sound waves cause

Devices for Virtual and Augmented Reality

441

figure 18.28  The scheme of the microphone

movement of the diaphragm (3) that is connected to the coil (2). As the coil moves in the magnetic field of the permanent magnet (1), electric current is generated. The electric current then passes through wires (4) to the audio device where it is further processed.

9

Conclusion

In this survey, we described the current state-of-the-art of multiple VR/AR devices, their principles of operation and the features they offer. Many devices are currently used for a full-featured user experience in VR. These devices integrate 2D displays, image sensors, motion and position sensors, audio devices, data gloves, haptic devices, and others. Many of them are accessible to a common user, but the most sophisticated ones, including kinaesthetic haptic devices utilizing many of the latest technologies, are unavailable due to their high price. However, some of these devices, especially smartphones and tablets, have been used in VR for several years now because they have affordable price and a large number of available AR applications. Since imaging devices play a major role in VR/AR, it is good to know how current 2D displays work, as well as technologies that will be used to display real 3D images in the near future. We hope this overview has helped readers not only understand how some of the described devices work, but we also hope it has allowed the readers to get an idea of the direction of development of VR/AR devices.

Acknowledgement The chapter was written with the support of the grant KEGA 012UK-4/2018 “The Concept of Constructionism and Augmented Reality in the Field of the Natural and Technical Sciences of the Primary Education (CEPENSAR).”

442

Bohdal

Notes 1 For details on the Sensorama machine, visit https://commons.wikimedia.org/wiki/ File:Sensorama-morton-heilig-virtualreality-headset.jpg 2 For details on the Mark IID gyroscopic gunsight, visit https://www.iwm.org.uk/ collections/item/object/205211647 3 Voxel is a 3D analogue of a two-dimensional pixel.

References Bellis, M. (2017). Liquid Crystal Display – LCD. Retrieved from https://www.thoughtco.com/liquid-crystal-display-history-lcd-1992078 Berkel, C., Parker, D., & Franklin, A. (1996). Multiview 3D LCD. In Stereoscopic displays and virtual reality systems III. 2653 (pp. 32–40). International Society for Optics and Photonics. Bermejo, C., & Hui, P. (2017). A survey on haptic technologies for mobile augmented reality. arXiv preprint arXiv:1709.00698. Retrieved from https://arxiv.org/abs/1709.00698 Bernstein, J. (2003). An overview of MEMS inertial sensing technology. Retrieved from https://www.sensorsmag.com/components/overview-mems-inertial-sensingtechnology Boas, Y. A. (2013). Overview of virtual reality technologies. Retrieved from https://pdfs.semanticscholar.org/4214/cb09e29795f5363e5e3b545750dce027b668.pdf Cakmakci, O., & Rolland, J. (2006). Head-worn displays: A review. Journal of Display Technology, 2(3), 199–216. Caudell, T., & Mizell, D. (1992). Augmented reality: An application of heads-up display technology to manual manufacturing processes. In Proceedings of the Twenty-Fifth Hawaii International Conference on System Sciences (Vol. 2, pp. 659–669). System Sciences, IEEE. Dodgson, N. (2005). Autostereoscopic 3D displays. Computer, 38(8), 31–36. Downing, E. A. (1997). US Patent No. 5,684,621. Washington, DC: U.S. Patent and Trademark Office. Favalora, G. (2005). Volumetric 3D displays and application infrastructure. Computer, 38(8), 37–44. Freudenrich, C. (2005). How OLEDs work. Retrieved from https://electronics.howstuffworks.com/oled.htm Gaspar, D., & Polikarpov, E. (2015). OLED fundamentals: Materials, devices, and processing of organic light-emitting diodes. Raton, FL: CRC Press. Geng, J. (2013). Three-dimensional display technologies. Advances in optics and photonics, 5, 456–535.

Devices for Virtual and Augmented Reality

443

Guttag, K. (2016). AR/MR optics for combining light for a see-through display (Part 1). Retrieved from https://www.kguttag.com/2016/10/21/armr-optics-for-combininglight-for-a-see-through-display-part-1/ Harris, T. (2002). How speakers work. Retrieved from https://electronics.howstuffworks.com/speaker3.htm Hazra, S. (2015). How are pico-projectors built? How do I build one? Retrieved from https://www.quora.com/How-are-pico-projectors-built-How-do-I-build-one Hirsch, M. (1961). US Patent No. 2,967,905. Washington, DC: U.S. Patent and Trademark Office. Hong, J., Kim, Y., Choi, H.-J., Hahn, J., Park, J.-H., Kim, H., et al. (2011). Three-dimensional display technologies of recent interest: principles, status, and issues. Applied Optics, 50(34), H87–H115. HowStuffWorks.com. (2001). How do touch-screen monitors know where you’re touching? Retrieved from https://computer.howstuffworks.com/question716.htm Huang, Y.-P., Chen, C.-W., Shen, T.-C., & Huang, J.-F. (2010). Autostereoscopic 3D display with scanning multi-electrode driven liquid crystal (MeD-LC) lens. 3D Research, 1(1), 39–42. Hypervsn™. (2018). Retrieved from https://kino-mo.com/ iGreet. (2017). Brief History of Augmented Reality. Retrieved from http://www.igreet.co/ brief-history-of-augmented-reality Ives, F. (1902). A novel stereogram. Journal of the Franklin Institute, 153(1), 51–52. Jolly, S., Smalley, D., Barabas, J., & Bove, M. (2014). Computational architecture for fullcolor holographic displays based on anisotropic leaky-mode modulators. Practical Holography XXVIII: Materials and Applications. 9006. International Society for Optics and Photonics. Kato, H., & Billinghurst, M. (1999). Marker tracking and HMD calibration for a videobased augmented reality conferencing system. Proceedings of the 2nd IEEE and ACM International Workshop on Augmented Reality ‘99 (pp. 85–94). IEEE. Krueger, M., Gionfriddo, T., & Hinrichsen, K. (1985). VIDEOPLACE—an artificial realit. ACM SIGCHI Bulletin, 16(4), 35–40. Lacko, Ľ. (2016). Ako funguje holografia. Retrieved from https://www.pcrevue.sk/a/ Ako-funguje-holografia Lau, D. (2013). The science behind Kinects or Kinect 1.0 versus 2.0. Retrieved from https://www.gamasutra.com/blogs/DanielLau/20131127/205820/The_Science_ Behind_Kinects_or_Kinect_10_versus_20.php LightSpace3D. (2014). Multi-planar 3D volumetric imaging – The LightSpace 3D volumetric display. Retrieved from http://www.lightspace3d.com/multiplanar-3d-volumetric-imaging.html MacFarlane, D. (1994). Volumetric three-dimensional display. Applied optics, 33(31), 7453–7457.

444

Bohdal

Peterka, T., Kooima, R., Sandin, D., Johnson, A., Leigh, J., & DeFanti, T. (2008). Advances in the dynallax solid-state dynamic parallax barrier autostereoscopic visualization display system. IEEE transactions on visualization and computer graphics, 14(3), 487–499. Rolland, J., & Cakmakci, O. (2009). Head-worn displays: The future through new eyes. Optics and Photonics News, 20(4), 20–27. Rosenberg, L. (1993). Virtual fixtures: Perceptual tools for telerobotic manipulation. In Virtual reality Annual International Symposium (pp. 76–82). IEEE. Schmalstieg, D., & Hollerer, T. (2016). Augmented reality: Principles and practice. Addison-Wesley Professional. SCME. (2017). Learning module: History of MEMS. Retrieved from https://nanohub.org/ resources/26535/download/App_Intro_PK10_PG.pdf SeeReal Technologies. (2018). Technology. Retrieved from https://www.seereal.com/ technology/ Smalley, D., Nygaard, E., Squire, K., Van Wagoner, J., Rasmussen, J., Gneiting, S., et al. (2018). A photophoretic-trap volumetric display. Nature, 553(7689), 486. Soltan, P., Lasher, M., Dahlke, W., Acantilado, N., & McDonald, M. (1997). Laser-projected 3D volumetric displays. In Cockpit displays IV: Flat panel displays for defense applications 3057 (pp. 496–507). International Society for Optics and Photonics. St-Hilaire, P., Benton, S., Lucente, M., & Hubel, P. (1992). Color images with the MIT holographic video display. Practical Holography VI. 1667 (pp. 73–85). International Society for Optics and Photonics. Tyson, J. (2000). How LCDs work. Retrieved from https://electronics.howstuffworks.com/lcd.htm Warren, T. (2016). Inside Microsoft’s HoloLens. Retrieved from https://www.theverge.com/2016/4/6/11376442/microsoft-hololensholograms-parts-teardown-photos-hands-on Wikipedia. (2018). Head-up display. Retrieved from https://en.wikipedia.org/wiki/ Head-up_display YouTube. (2008). CNN Hologram TV First. Retrieved from https://www.youtube.com/ watch?v=thOxW19vsTg Zone, R. (2007). Stereoscopic cinema and the origins of 3-D Film. Lexington, KY: University Press of Kentucky.

Index 2D 59, 60, 200, 201, 220, 230, 255, 326, 328, 331, 334, 342, 365, 366, 370–372, 411, 420, 422, 423, 425, 434, 441 3D xxxi, xxxii, xxxiv, 7, 16, 27, 29, 34, 36, 44, 59, 60, 90, 93, 94, 103, 115–117, 123–125, 128, 130–134, 137, 140, 150, 171, 172, 175, 188, 197, 209, 219, 220, 225, 230, 239, 254, 255, 282, 285, 326–344, 354, 365, 366, 370, 371–373, 377, 393, 395, 397, 399–403, 407, 411, 417–428, 432, 433, 436, 441 action research 6, 7, 9, 18, 20, 22, 37–53 adult education 66–68, 73, 273, 274 app 393–407 AR books xxx, 59–63, 68, 103, 104, 175, 284, 285, 394, 403, 404 AR Software Development Kits (SDKs)  332–335 AR technologies xix, xxx–xxxii, 59, 61, 62, 106, 107, 117, 118, 149, 150–152, 155, 169, 170, 172–174, 177, 179–181, 188–190, 195, 196, 207, 210, 212, 219, 274, 277, 278, 280, 286, 326, 327, 391, 393, 407 artificial intelligence 217–231 augmented playgrounds 295–321 augmented reading 67, 56–75, 209, 254 augmented reality xxx–xxxv, 3–22, 24–53, 59, 69, 80–99, 101–118, 123–144, 148–164, 168–190, 195–212, 217–231, 236–261, 267–288, 295–441 augmented teacher xxxi, 58, 63, 67–73 automated geometer xxxiv, 350, 362–367 automated reasoning xxxiv, 348, 349, 359–362, 367 automatically augmented reality xxxiv, 347–367 big data 373 coding 10, 11, 18, 19, 39, 42, 48, 84, 85, 94–96, 113, 178 collaborative xxxiv, 20, 21, 52, 60, 63, 68, 73, 102, 125, 127, 151, 199, 200, 206, 221, 225,

226, 228, 230, 295, 296, 299, 301, 309, 321, 326, 353, 386, 392 communities communities of practice 59, 63 online communities 63, 69, 90 competencies xxxi, xxxii, xxxiv, 4–6, 8, 20, 25, 29, 37, 38, 40, 46, 58, 64, 67, 68, 71, 83, 95, 127, 221, 224, 227, 230, 248, 267, 268, 274, 276, 281, 287, 288, 349, 356, 366, 367, 384, 387–389, 398, 400 constructionism 59, 63 constructivism 59, 63, 81, 82, 84, 85, 95, 177, 389 covariation 148, 152–153, 155–157, 159, 161–163 covariational reasoning xxxii, 148–164 design-based research xxxii, 60, 149–152, 163, 164 devices xxxv, 7, 37, 59, 103, 107, 108, 115, 118, 151, 174, 185, 187, 196, 225, 226, 243, 247, 252, 254, 260, 278, 279, 282–285, 299, 300, 316, 321, 325, 326, 329, 331–335, 342, 344, 349, 351–353, 355, 356, 366, 410–441 dialogic learning 59, 63 digital literacy xxx, xxxi, 5, 7–10, 11, 14–21, 44, 45, 65, 83, 93, 405 digital technologies xxix, 3–6, 8, 61, 101, 236–239, 249, 269, 385, 388, 390–392 diSessa, Andy 297 displays autostereoscopic displays 411, 417–421, 427 displays using time aperture 419–420 electro-holographic displays 425–426 Head-Worn Displays (Head-Mounted Displays (HMD)) 60, 326, 412, 415, 417, 418, 428 holographic displays 424–426 Optical Head-Worn Displays (OHWD) 411, 427–431 pseudo-3D displays 426–427 stereoscopic displays 411, 417, 421 volumetric displays 421–424

446 education biology education xxxii, 168–190 early childhood education 101–118 pre-primary education xxx, xxxi, 3–22, 24–53 primary education 80–99, 169, 175, 189, 190, 237, 385, 387, 390, 441 technical education xxxiv, xxxv, 383–408, 411 tertiary education xxxii, xxxiii, xxxv, 33, 195–212 eLearning 58, 203, 222, 224–227, 230, 238, 239, 269, 271, 283 European Commission 270, 386 gamification 187, 205–207 GeoGebra xxxiv, 258, 325–367 ICT-mediated instruction 70 Inclinometer 356, 407 Information and Communication Technology (ICT) 62, 65, 66, 70, 102, 182–184, 186, 201, 242, 275, 287, 325, 384 inquiry-based learning 268–270, 273, 278–280, 285, 286, 342 intelligent diagnostic platform 226–230 intelligent learning environments 226, 227, 230 intuitions xxxiv, 295–321 kindergarten xxx, xxxi, 7, 8, 10, 11, 35, 40, 51, 101, 103, 109, 185, 188 Koehler, Matthew 65, 274 laboratories xxxiii, 169, 170, 201, 222, 223, 227, 267–288, 412, 413 learning bridging formal and informal learning xxx, 196, 326 collaborative learning 20, 52, 73, 199, 296, 299, 392 constructivist learning 177, 189, 391, 393 context-aware ubiquitous learning 60 enquiry-based learning 326 games-based learning 403–404 situated learning 208, 326 literacy digital literacy xxx, xxxi, 5, 7–11, 14–21, 44, 45, 65, 83, 93, 405 natural literacy 24–53 reading literacy xxxi, 57, 58

index living book xxxi, 56–75 living library 58, 68–71, 75 medical diagnosis xxxiii, 217–231 medical education 218–222, 225–227, 230 medical training xxxiii, 217–231 Mishra, Punya 65, 274 mixed reality 225, 228–230, 268–270, 272, 275, 276–280, 282–284, 286–288, 296 mobile applications xxxiv, 175, 328, 331–333, 391–393 mobile devices 59, 103, 107, 108, 118, 174, 187, 196, 217, 223, 225, 226, 279, 284, 285, 325, 326, 332, 344, 352, 353, 366, 370–372, 374, 390, 410, 411, 432 models dynamic models 31–32, 34 interactive models 29, 31–33 multiliteracies 65 museum xxx, xxxiv, 107, 175, 199, 279, 285, 369–379, 394–396 Papert, Seymour 63 Piaget, Jean 34, 63, 102, 297 problem-based learning 67, 273, 286 professional development xxxi, xxxiii, 39, 58, 59, 63, 65, 66, 69–75, 98, 270, 275, 392 qualitative methodology 14, 18, 38, 39, 48, 86, 197 reality Augmented Reality (AR) xxx–xxxv, 3–22, 24–53, 59, 69, 80–99, 101–118, 123–144, 148–164, 168–190, 195–212, 217–231, 236–261, 267–288, 295–441 Mixed Reality (MR) 225, 228–230, 268–270, 272, 275, 276–280, 282–284, 286–288, 296 Virtual Reality (VR) 34–35, 125, 126, 141, 219, 282, 326, 347, 372, 400, 410–441 Rosenblatt, L. 64 screens 102, 124, 126, 127, 134, 136, 211, 282, 303, 305, 328–330, 332, 342, 353, 355–358, 367, 373–376, 404, 407, 411, 413–416, 418–421, 423, 425, 426, 428 Shulman, L. S. 65, 274

447

index simulations xxxiii, 25, 33, 60, 61, 73, 109, 125, 170, 200, 201, 210, 218–221, 230, 280, 286, 295–321, 356, 391 STEM (Science, Technology, Engineering and Mathematics) xxxii, xxxiii, 57, 149, 168, 169, 185, 187, 189, 267–288, 353, 355, 384–387, 389, 391–397, 404, 408 student student-centred 63, 273 student-directed 63 teachers in-service teacher 29, 66, 73, 269, 270, 272–275, 287 mathematics teachers 236–261 teacher-scaffolded 63, 273–274 theory contemporary literary theories and theories of literacy 59, 64–65 embodied theory 300–301

sociocultural theory 59, 63 Technological, Pedagogical, Content Knowledge (TPACK) 59, 65–66 transactional theory 64 tool angular measurement tool 356 Caliper tool 356 land surface tool 356 measurement tool for inaccessible points 356 measuring tape tool 356 rangefinder tool 356 sextant tool 356 virtual reality 34–35, 125, 126, 141, 219, 282, 326, 347, 372, 400, 410–441 virtual world xxxii, 124, 125, 127, 128, 131–135, 137–143, 217, 283, 396 visual-kinesthetic activities xxxii, 149 Vygotsky, Lev 392