The Digital Era of Learning: Novel Educational Strategies and Challenges for Teaching Students in the 21st Century 153618750X, 9781536187502

Citation preview

EDUCATION IN A COMPETITIVE AND GLOBALIZING WORLD

THE DIGITAL ERA OF LEARNING NOVEL EDUCATIONAL STRATEGIES AND CHALLENGES FOR TEACHING STUDENTS IN THE 21ST CENTURY

No part of this digital document may be reproduced, stored in a retrieval system or transmitted in any form or by any means. The publisher has taken reasonable care in the preparation of this digital document, but makes no expressed or implied warranty of any kind and assumes no responsibility for any errors or omissions. No liability is assumed for incidental or consequential damages in connection with or arising out of information contained herein. This digital document is sold with the clear understanding that the publisher is not engaged in rendering legal, medical or any other professional services.

EDUCATION IN A COMPETITIVE AND GLOBALIZING WORLD Additional books and e-books in this series can be found on Nova’s website under the Series tab.

EDUCATION IN A COMPETITIVE AND GLOBALIZING WORLD

THE DIGITAL ERA OF LEARNING NOVEL EDUCATIONAL STRATEGIES AND CHALLENGES FOR TEACHING STUDENTS IN THE 21ST CENTURY

CHRISTOPHER S. KEATOR, PHD EDITOR

Copyright © 2020 by Nova Science Publishers, Inc. All rights reserved. No part of this book may be reproduced, stored in a retrieval system or transmitted in any form or by any means: electronic, electrostatic, magnetic, tape, mechanical photocopying, recording or otherwise without the written permission of the Publisher. We have partnered with Copyright Clearance Center to make it easy for you to obtain permissions to reuse content from this publication. Simply navigate to this publication’s page on Nova’s website and locate the “Get Permission” button below the title description. This button is linked directly to the title’s permission page on copyright.com. Alternatively, you can visit copyright.com and search by title, ISBN, or ISSN. For further questions about using the service on copyright.com, please contact: Copyright Clearance Center Phone: +1-(978) 750-8400 Fax: +1-(978) 750-4470 E-mail: [email protected].

NOTICE TO THE READER The Publisher has taken reasonable care in the preparation of this book, but makes no expressed or implied warranty of any kind and assumes no responsibility for any errors or omissions. No liability is assumed for incidental or consequential damages in connection with or arising out of information contained in this book. The Publisher shall not be liable for any special, consequential, or exemplary damages resulting, in whole or in part, from the readers’ use of, or reliance upon, this material. Any parts of this book based on government reports are so indicated and copyright is claimed for those parts to the extent applicable to compilations of such works. Independent verification should be sought for any data, advice or recommendations contained in this book. In addition, no responsibility is assumed by the Publisher for any injury and/or damage to persons or property arising from any methods, products, instructions, ideas or otherwise contained in this publication. This publication is designed to provide accurate and authoritative information with regard to the subject matter covered herein. It is sold with the clear understanding that the Publisher is not engaged in rendering legal or any other professional services. If legal or any other expert assistance is required, the services of a competent person should be sought. FROM A DECLARATION OF PARTICIPANTS JOINTLY ADOPTED BY A COMMITTEE OF THE AMERICAN BAR ASSOCIATION AND A COMMITTEE OF PUBLISHERS. Additional color graphics may be available in the e-book version of this book.

Library of Congress Cataloging-in-Publication Data ISBN: 978-1-53618-H%RRN

Published by Nova Science Publishers, Inc. † New York

CONTENTS Preface

ix

Acknowledgments

xi

Part 1 Chapter 1

Chapter 2

Novel Teaching Strategies Developed to Help Today’s Student A Novel Two-Stage Model of Evaluation to Successfully Identify Areas of Weakness and Guide the Development of Learning Activities Designed to Improve Knowledge of Self-Directed Learning in Primary School-Aged Learners Penny Van Deur Mobile Learning Literature Review in Medical Education Heeyoung Han, Larry Hurtubise, Geraud Plantegenest, Carolyn R. Rohrer Vitek, Rahul Patwari, Cecile M. Foshee and Elissa R. Hall

1

3

41

vi Chapter 3

Chapter 4

Contents Novel Use of 3D Bone Models in the Anatomical Sciences Education of Millennial Students: A Description of the Process and an Assessment of the Printing Accuracy Yousef AbouHashem, Manisha R. Dayal, Stephane Savanah and Goran Štrkalj Explorations: Promoting the Use of Teamwork and Collaboration to Strengthen the Skills Needed for a Lifetime of Self-Directed Learning in the Modern US Healthcare System Christopher S. Keator

Chapter 5

Post-Exam Reviews in Medical Education Kenneth D. Royal, Jennifer A. Neel and Laura L. Nelson

Part 2

Challenges Facing Educators in Today’s Digital-Savvy Learning Environment

Chapter 6

Chapter 7

Chapter 8

Chapter 9

Researching the Best and Brightest: The Challenge of Researching Medical Learners Karen Hughes Miller, Lori Wilks Wagner and Erin Michelle Davis

79

101 157

171 173

Modern Challenges of Integrating Technology into Medical Education Vicki R. McKinney and Robert W. Rebar

203

Guessing in Multiple Choice Exams: Theory, Context and Detection Procedures Kenneth D. Royal

233

Challenges Identifying and Stimulating SelfDirected Learning in Publicly Funded Programs Carol Nash

259

Contents

vii

About the Editor

301

Index

303

PREFACE Students of the 21st century, typically those of the Millennial (also referred to as ‘Gen Y’) or Gen Z generations, were born into a digitally advanced world. Unlike in the 1960’s when the smallest computers occupied entire rooms at the National Aeronautics and Space Administration (NASA) complex, today’s digital landscape is smitten with the abundant use of modern laptops, tablets and smartphones. Modern computing technology has evolved due to the marriage with extremely powerful computing software, which collectively has resulted in the commonplace use of modern technology on a regular basis throughout all aspects of everyday life. This relatively unrestricted access to computers is coupled with an unfettered access to the internet, providing ‘users’ unlimited freedoms to search for boundless amounts of information. This constant stream of electronicallyaccessible information, the ‘digital highway’, has subsequently led to the creation of novel strategies to teach today’s students. Today’s students, or more aptly referred to as ‘modern learners’, are quite unique compared with previous students of the Baby Boomer or Gen X generations. Students of the Gen X generation were the first students to experience wide-spread access to computers during high school and undergraduate studies, whereas the majority of students from both the Gen Y and Gen Z generations have been literally bombarded with computer technology since birth. This access has created an ‘on-demand’ lifestyle that

x

Christopher S. Keator

relies on searchable databases, instant access to live-streaming events and the ability to communicate electronically (in various formats) from almost anywhere on the face of the planet. This on-demand lifestyle has permeated every facet of everyday life to the degree that many of these technologies are now incorporated routinely into all forms of business and science, and used throughout all levels (elementary, secondary and professional) of education. Thus, the constant use of modern technology – coupled with the on-demand lifestyle – has led to profound changes in learner expectations, resulting in the need for educators to develop new strategies and face unique challenges on a regular and often recurring basis. This book provides a detailed overview into those educational strategies and various challenges faced by today’s educators. It is conveniently divided into two parts. The first part includes chapters examining different strategies for teaching a wide variety of students covering multiple age groups. The second part includes chapters providing unique insights into some of the varied challenges facing today’s educators. The vast majority of strategies – and challenges – are focused on how the emerging technology of the early 21st century has resulted in profound influences for both learner and educator expectations and limitations, and how technology has opened up endless opportunities that will ultimately alter the modern educational landscape.

ACKNOWLEDGMENTS As the editor, I can honestly attest that this volume would be nothing without the contributions of the other authors and their shared desire to complete this project. During this process, several authors have endured personal hardships and all of us have witnessed firsthand the destruction caused by two of the most powerful Hurricanes (Harvey and Maria) on record in the Atlantic basin and the unfathomable destructive force of the fires that destroyed large swaths of the Australian wilderness. As if hurricanes and massive wildfires weren’t enough, all of mankind has been impacted in 2020 by a novel corona virus that spread so rapidly around the world that it forced entire countries to close their borders and shutter their economies. Thus, I’m deeply indebted to them for their patience on this journey, one that was unfortunately extended due to the Covid-19 pandemic that has forever altered our world. Sincere thanks to Penny, Heeyoung, Goran, Ken, Karen, Vicki and Carol. I thank my wife and children, who have always provided unwavering support in my constant drive to improve myself. Their love and upbeat attitudes make it easy for me to maintain my role as husband, father and teacher. My constantly overachieving children also motivate me to educate and inspire others, especially in my chosen passion of reproductive endocrinology. Thanks and love to Meg, Cass and Ty.

xii

Christopher S. Keator

I conclude with recognizing my past and current mentors. My doctoral advisor, who peacefully passed away at the age of 84 during this project, taught me the lifelong value of practicing the 5Ps (proper planning prevents poor performance) in all facets of life. My postdoctoral mentor instilled in me the mantra that if something is worth doing then give it 110% and do it to the extreme. My current faculty mentor has instilled in me the urge to write, and editing this book or contributing to other writing projects probably wouldn’t get finished without his encouragement. His prolific love of writing helped me realize that those of us who can write, should write; and that authors write best when they write often. Deepest respect and thanks to John, Ov and Bob.

PART 1: NOVEL TEACHING STRATEGIES DEVELOPED TO HELP TODAY’S STUDENT

In: The Digital Era of Learning Editor: Christopher S. Keator

ISBN: 978-1-53618-750-2 © 2020 Nova Science Publishers, Inc.

Chapter 1

A NOVEL TWO-STAGE MODEL OF EVALUATION TO SUCCESSFULLY IDENTIFY AREAS OF WEAKNESS AND GUIDE THE DEVELOPMENT OF LEARNING ACTIVITIES DESIGNED TO IMPROVE KNOWLEDGE OF SELF-DIRECTED LEARNING IN PRIMARY SCHOOL-AGED LEARNERS Penny Van Deur, PhD College of Education, Psychology and Social Work, Flinders University, Adelaide, South Australia, Australia

ABSTRACT Teachers are being urged to develop successful independent learners, but how might this be done, and how would teachers find out what their 

Corresponding Author’s Email: [email protected].

4

Penny Van Deur students know about self-directed learning? This chapter focuses on the use of an assessment to chart primary (elementary) students’ knowledge of self-directed learning (SDL). Design-Based Research methods structured this investigation of learners’ knowledge of SDL. First, there is a brief description of the development of an innovative approach to assess primary students’ knowledge of SDL. Self-Directed Learning is defined and a model of effective SDL in primary students is outlined. A two-part assessment of SDL was based on the model. In part one of the research, students’ ratings of their knowledge of SDL were collected before and after a short researcher-led teaching intervention to develop their knowledge of SDL. In the second part of the research, the framework for classroom development of SDL was matched with domains of a local curriculum document and used to guide collaborative planning of an intervention by two teachers with the aim of increasing students’ knowledge of SDL. Assessment showed small increases in students’ knowledge of SDL. Finally, there is an outline of teachers’ perspectives on developing SDL in their classrooms.

Keywords: self-directed learning, self-regulated learning, assessment of learning processes, sustainable intervention, teachers’ perspectives

INTRODUCTION Self-Directed Learning (SDL) is a priority in current Australian government policy, as evidenced by curriculum documents that describe the need for all students to be self-sufficient and successful learners (Reid, 2017; Australian Curriculum, Assessment and Reporting Authority, 2015; Ministerial Council for Education, Early Childhood Development, and Youth Affairs, 2008). The Australian Professional Standards for Teachers (Australian Institute for Teaching and School Leadership, 2011) emphasize the need for teachers to be able to better identify how learners prefer to learn (standard 1), be able to plan for and implement effective teaching and learning (standards 2 and 3) and be able to assess student learning (standard 5). This approach takes students’ knowledge, attitudes and interests as the starting point for learning situations and uses constructivist pedagogy (Phillips, 2000) where the teacher begins their teaching with what learners know about managing their learning.

A Novel Two-Stage Model of Evaluation …

5

Pintrich and Zusho (2002, cited in Nicol & Macfarlane-Dick, 2007) showed that all students were able to learn to become more self-regulating when teachers made learning processes explicit and gave them opportunities to practice self-regulation. In addition, Pintrich (2010) argued that the competencies, attitudes and beliefs necessary for self-regulated learning can be learned by most students and that helping students do this enables them to manage their own academic functioning (Pintrich, cited in Wolters, 2010). Another study extended Pintrich’s work by having fifth-grade teachers teach their students the processes and attitudes needed for effective SRL (Zumbrunn, Tadlock & Roberts, 2011). The study found that there were benefits for fifth-grade students’ academic work, motivation, and task engagement and that they became more successful learners, leading Zumbrunn et al., to argue that it is important for classroom curriculum and assessment systems to be organized to support and value autonomous inquiry. At this point it is important to clarify the terms self-regulated and selfdirected learning because they are often used inter-changeably to describe learning that is managed by learners. Self-regulated learning (SRL) is an integral aspect of SDL with its focus on the co-ordination in learning of selfprocesses (Schunk & Zimmerman, 1998). Importantly, Saks and Leijin (2014) argued that SDL includes SRL although it is not the same as SRL. They described SRL as a micro-level concept because of its involvement with processes used to execute a task. According to Saks and Leijin, a teacher might set a task and students would then be involved in selfregulation by monitoring, regulating, and controlling their cognition, motivation and behaviour. As learners carry out these processes they are guided and constrained by their goals and the contextual features of the environment. On the other hand, Saks and Leijin described SDL as operating at the macro-level because learners (rather than the teacher) plan the learning trajectory, decide what and how to study, diagnose learning needs, formulate learning goals, find resources for learning and monitor learning. In order to assume such agency, self-directed learners need to be able to self-regulate their learning processes.

6

Penny Van Deur

Hmelo-Silver (2004) outlined that self-directed learners need to develop the ability to identify gaps in their knowledge so they can set learning goals, identify what they need to learn more about, and plan and select the best strategies to find and learn new information. Then they need to independently determine when or if their learning goals have been accomplished. Importantly, Hmelo-Silver argued that self-directed learners need adequate support in school for this type of learning. When SDL is discussed in this way, it is clear that it requires a collaborative approach between the learner and school system, because it involves institutional support that both provides and promotes opportunities for students to take responsibility for their learning. As the teacher guides learners toward selfdirection they may take on the role of managing learning objectives, selecting resources and strategies and creating assessments to evaluate students’ learning outcomes (Knowles, 1975; Zimmerman & Lebeau, 2000). Learners are central to SDL because they are required to assume responsibility for directing their cognition and motivation and choosing effective learning strategies to achieve their specific learning goals. Selfdirected learners have an active involvement in the SDL process because they need to select and evaluate their learning materials (Loyens et al., 2008) and they need to have the freedom to choose what and how they will learn. Collectively, they need to fully understand this cycle of SDL to adequately assess their knowledge, which will permit them to identify their weaknesses and develop new learning goals, thereby continually challenging themselves to learn more. Learners need to have a well-developed knowledge base of SDL that they can draw on as they carry out the various processes that help them manage their learning. Treffinger (1993) emphasized that learners need to have a knowledge base of SDL in his description of SDL in elementary school students where he outlined SDL as the process of learning in which students ‘learn how to set goals, identify resources, develop learning activities, make decisions and evaluate ideas and create and share products’ (p.438). This description stresses that if elementary (primary) students are to carry out SDL, they need to be aware of what they do and do not understand about a topic. This awareness enables them to set learning goals, identify what they need to

A Novel Two-Stage Model of Evaluation …

7

learn more about, plan and select learning strategies and monitor if their goals have been met (Hmelo-Silver, 2004). Overall, SDL involves learners in controlling the direction of a learning project and exercising self-regulation as they manage their own resources to complete it. In this process learners need to be able to choose strategies to reach learning goals they have set for themselves (Loyens et al., 2008). Like Saks and Leijin (2014), Barrell (1995) combined perspectives of SRL and SDL in his outline of the teacher’s role as guiding learners to develop selfdirection and encouraging them to be aware of their beliefs about their own abilities to manage their learning. Importantly, Barrell stressed that learners have the power to choose to learn or not learn. As Barrell described it, learners can be taught specific strategies for setting goals for personal and instructional improvement and planning ways to achieve these goals. This approach emphasizes teaching learners to be metacognitive by making them aware of their thinking and learning processes so they can independently plan what to do, monitor their own progress, and critically evaluate the results of their learning. Teachers participate in this process by helping learners develop their self-direction and improve their personal efficacy. They do this by making learners aware of their personal beliefs about their abilities and helping them better understand the role they play in choosing what and how to learn. A sound knowledge base of SDL is essential if learners are to make self-assessments of their learning processes and their capacity to employ them to learn in a self-directed way. It is timely for teachers to pay attention to the processes of self-directed learning and consider how knowledge of SDL might be developed in learners as young those in primary (elementary) classrooms.

A Model of Self-Directed Learning The first task in generating a novel model for SDL was to develop a comprehensive definition. This process was completed by consulting research on school contexts (Rhine, 1998; Rutter & Maughan, 2002;

8

Penny Van Deur

Wigfield, Eccles & Rodriguez, 1998; Pintrich & DeGroot, 1990), adult selfdirected or autonomous learning (Knowles, 1975; Candy, 1991; Brockett & Hiemstra, 1991; Spencer & Jordan, 1999; Evensen, 2000; Hmelo-Silver, 2004) and self-directed learning in gifted students (Treffinger, 1975, 1993). These resources were then reviewed for similarities and key points that could be used to develop an outline useful for the targeted learners in elementary (primary) schools. After reviewing the relevant research literature, a novel outline of SDL was developed that includes a small number of very influential structured constructs from outside sources (termed ’external’) and a larger number of constructs that are determined by the learner (termed ‘internal’). The ‘external’ influences are predominantly determined by each school and include global aspects such as school level motivation, organization, strategies to support independent learning requiring SDL (with some teacher guidance). The learner or ‘internal’ constructs encompass individual motivation, learning strategies and perception of support for independent learning offered by the school. Central to the ‘internal’ characteristic, is knowledge of SDL that is built up as students engage in SDL and incorporate this knowledge into their metacognitive strategies to be used for autonomous learning. Metacognitive strategies of planning, checking and reflecting are internal-characteristics that students use as they manage their learning. External and internal influences, those generated typically by the larger school system and the individual learner, were the base for the outline of SDL that guided the development of a two-part assessment for SDL. The main psychological principles discussed in the literature on SDL and SRL were combined with descriptions of SDL in elementary students to become the foundation for the model of effective SDL in primary schools (shown in Figure 1). School (external) and learner (internal) influences are depicted in the model; the inter-relationships among self-regulating, metacognitive, and self-directing processes are highlighted, as these are required for students to take responsibility for their learning (Candy, 1991; Barrell, 1995; Schunk & Zimmerman, 1998; Bransford, Brown & Cocking, 1999; Evensen, 2000; Zimmerman & Lebeau, 2000; McCombs, 2001; Pintrich, 2003). The model

A Novel Two-Stage Model of Evaluation …

9

positions knowledge of SDL as part of SRL, an internal influence. An effective self-directed learner would self-regulate their learning by setting goals, accessing knowledge of SDL, using metacognitive strategies to plan, implement and monitor learning efforts toward goals, while generating selffeedback on strategies used. The learner’s ability to manage and carry out learning would be influenced by their personal characteristics and the school context in which learning occurs. Knowledge of SDL would be activated by the need to take responsibility for learning which happens when students carry out inquiry that requires them to choose metacognitive strategies based on their knowledge of SDL.

Figure 1. Model of effective Self-Directed Learning.

The model illustrates the interactions among the external environment (including guidance from teachers, school and student motivation) as well as internal characteristics (including personal characteristics and self-

10

Penny Van Deur

regulation). These internal self-aspects include affective concerns which derive from self-belief and lead to motivation, self-regulation in learning, and knowledge of SDL. Metacognition is an important consideration because it concerns awareness and control of personal resources. As such, metacognitive self-evaluations play an important role in developing learners’ beliefs in their self-regulating capacities to reach their goals. Teachers as an ‘external’ influence within the school context, play an important role in helping students discover how to manage themselves, carry out inquiry activities, assess their own progress and learn effectively. They can assist learners by helping them gain explicit knowledge of SDL, and then prompting them to use this knowledge to reflect on their learning as they learn autonomously. In this way teachers promote the practice of selfassessment where learners continually monitor their progress toward their learning goals. The model also highlights the contribution of external factors that are both behavioral (external learning strategies) and contextual (motivating factors in the school environment) and shows how those external factors overlap with internal factors that are both cognitive (knowledge and internal learning strategies) and attitudinal (motivational). In sum, this new description of SDL contributes to educational theory by including knowledge of SDL as part of self-regulated learning; an internal component of SDL. It highlights how aspects of SRL exert positive influences on effective SDL and emphasizes that knowledge of SDL is the base from which decisions about managing learning are made.

The Framework of SDL for Classroom and School Development of SDL The Framework for classroom development of SDL, shown in Table 1, was developed from the model of effective SDL to provide educators with guidance for planning teaching activities that aim to develop students’ knowledge of SDL.

A Novel Two-Stage Model of Evaluation …

11

Table 1. The framework for classroom development of SDL Internal and external influences (within and outside learner) Develop self-efficacy for learning, inquiry, planning, and seeking information Encourage collaboration where students have opportunities to work with peers Be able to seek help with difficulties in inquiry, to resolve mistakes, and acquire skills needed for inquiry Show persistence to overcome difficulties in learning and inquiry Make causal attributions to choosing appropriate resources, and trying hard Develop a positive attitude to SDL in terms of view of own ability to be self-directed, looking for information, feeling comfortable learning at school, and being aware of own strengths and weaknesses Be able to negotiate to work on topics of interest and organize how to work on inquiry topics Support of school facilities for SDL in terms of displaying students’ inquiry work around the school, availability of resources for inquiry, school climate that promotes a positive orientation to wanting to learn, ease with which students can move around the school and students’ access to Information Technology at school. Encourage parent involvement in students’ learning at home and at school Internal influences (within learner)-Self-Regulated Learning (SRL) Represent what is known about a topic at the beginning of learning, recognize own skills for the task, and clarify the purpose of inquiry topic Analyze the topic to identify what needs to be done, and the main knowledge goals Plan what to do, resources available, information needed, main inquiry activities, time needed, and specific skills needed to investigate a topic Monitor relevance of information for the topic, time being spent on inquiry, and resources related to the inquiry topic Review information being found for its relevance to the topic, and review strategies for suitability for the inquiry, reflect on learning achieved and learning processes used

The framework was developed to maintain congruence between assessment, objectives and instruction as recommended by Watkins and Biggs (2001), and Sternberg and Grigorenko (2002). It guided the development of activities that were taught to Year 5 students (aged 10-11 years of age in Australian primary schools) and the planning of activities by classroom teachers in an intervention study in the second part of the research. These activities aimed to develop learners’: self-efficacy for inquiry; collaboration with peers; help seeking behavior; persistence to overcome difficulties; ability to make causal attributions to controllable factors and have a positive attitude to being self-directed. They also aspired

12

Penny Van Deur

to build students’ capacity to build self-regulation in terms of leaners being able to represent what they know about a new topic, analyze tasks, plan learning, monitor information for its relevance to current learning tasks, and monitor time being spent and specific skills being used. Learners would be guided to be able to review strategies and information for their relevance to the inquiry being undertaken and to be able to reflect on their learning. The activities were also designed to assist students’ to be able to negotiate with their teachers to work on topics of interest to them. In addition, teachers were assisted to be able to evaluate whether school facilities supported inquiry– based learning. It also highlights to teachers that parents could be involved in classroom learning by sharing their expertise at school as well as by assisting their children to learn at home. An explicit teaching approach, based on the framework for classroom development of SDL, would emphasize clarity of explanation, scaffolding of new strategies and social support through discussion with other students. It focuses on scaffolding the different types of questions learners can ask themselves as they practice SDL, with the aim of moving them toward being able to use strategies to self-assess their own learning. Opportunity for inquiry is an important antecedent to SDL because SDL itself implies that students will have opportunities to carry out inquiry at school. When learners are involved in inquiry they need to have opportunities to take responsibility for their learning, be able to access resources in their school and work in flexible groupings rather than solely in classrooms led by teachers. Motivation is generally seen as a characteristic of students; however, Pintrich (2003) argued that it exists in schools. The framework for classroom development of SDL builds on Pintrich’s view by characterizing motivation as applying to both the context of a school and to learners. The framework identifies that inquiry in schools is supported by school level motivation, organization and strategies. These aspects informed the development of the Primary School Characteristics Inventory.

A Novel Two-Stage Model of Evaluation …

13

Design-Based Research (DBR) Design-based research methods were employed in this project because they allow for progressive refining of research in the light of experience in school contexts (Collins, Joseph & Bielaczyc, 2004). They emphasize the production of useable knowledge which can be transferred to curricula and fitted into the real work of teachers and learners (Kelly, 2004). Design-Based Research methods have been used previously to explore the creation of novel learning and teaching environments, examine various contextually-based developmental theories of learning and instruction, advance and consolidate design knowledge, and to evaluate educational innovations (The DesignBased Research Collective, 2003; Lewis, Perry & Murata, 2006). The methodology used to conduct DBR is consistent with having an alignment of theory, design, practice and measurement over time. The DBR approach highlights the importance of looking at multiple contextual and dependent variables. This approach was appropriate for the investigation described here because it involved research on educational tools and materials that was carried out in real settings (in this case in primary schools with teachers and students). Using DBR meant that the innovation was more likely to be adopted by practitioners (Design-Based Research Collective, 2003). The assessment of knowledge of SDL and explicit teaching about SDL in primary (elementary) schools fits well with the principles of DBR with its aim of applying insights gained to improve student motivation and learning outcomes (Dede, 2004). Design-based research (DBR) emphasizes model development and innovation, as evidenced by the primary goal to ‘inquire more broadly into the nature of learning in a complex system and to refine generative or predictive theories of learning.’ (The Design-Based Research Collective, 2003 p. 7). It is a coherent method of inquiry that bridges theoretical research with educational practice and is well-suited to the research described here that is directed at understanding learning and teaching processes (Kelly, 2003; The Design-Based Research Collective, 2003). Design experiments were utilized within this DBR investigation because they are useful for testing innovations and modifying theories on the basis

14

Penny Van Deur

of analysis and in relation to classroom learning environments (Cobb, 2000). Cobb, Confrey, diSessa, Lehrer and Schauble (2003) explain that design experiments are generally conducted to develop theories and are carried out in educational settings such as classrooms. In this context the analysis of student learning is tied to the environment in which the learning occurs, and feedback is used to inform the ongoing instructional design work being undertaken.

Survey Instruments and Data Collection This research takes an innovative approach to the assessment of knowledge of SDL by assessing it in two sections; at school and learner levels. At the school level, the Primary School Characteristics Inventory (PSCI) assesses school support for inquiry because this is a pre-requisite for SDL, and at the learner level, the Learning At School Questionnaire (LASQ) assesses students’ knowledge of SDL and its development following teaching interventions. Consistent with DBR, the innovation of a two-part assessment of SDL was devised, trialed and revised as required, based on experience in school contexts (Lewis, Perry & Murata, 2006). In this way both theory and practice were refined as recommended by Collins, Joseph and Bielaczyc (2004). The following three dependent variables were considered for this project: (1) learning variables of knowledge, (2) systemic variables of ease of adoption (defined by Collins, Joseph & Bielaczyc, 2004) and (3) climate variables of student engagement. Design experiments are useful for modelling how students learn and for testing a model empirically in order to use it to inform an instructional innovation (Lovett, 2000). Elements of design experiments were appropriate for this research because of the focus on classroom learning processes in which the social nature of learning is recognized (Lobato, 2003). A model was employed to guide the research from initial conceptualization to diffusion and adoption (Bannan-Ritland, 2003) of the two-part innovation by educators in schools. The model of effective SDL in primary

A Novel Two-Stage Model of Evaluation …

15

(elementary) students was formulated through processes of consulting, cooperating and asking questions, listening to answers and then incorporating background theory. Following this, data were collected and examined to assess the important features of the model so it could be used as the base for constructing an assessment of SDL (Sloane & Gorard, 2003). Sloane and Gorard (2003) stressed that it is more important to describe weaknesses in models, than to question the fine detail of each model fit. They outlined the ‘concept of failure’ (Sloane & Gorard, 2003 p. 31), which allows developers to anticipate how an innovation, such as the two-part assessment of SDL, could fail to perform as intended when used with students and teachers in school settings. Modifications to the assessment could be based on this information. Including elements of design experiments in this investigation allowed an examination of students’ learning that is situated in the social context of their regular classrooms. Theory was developed from the work on instructional design carried out in classrooms (Cobb, 2000) and this was then tested in practice. Another strength of this approach is that the researcher was able to work with practitioners to co-construct knowledge about an everyday problem in schools (Shavelson, Phillips, Towne & Feuer, 2003; Lewis, Perry & Murata, 2006). Understanding SDL through defining and modelling it supported construction of the framework of classroom development of SDL which guided the development of classroom activities to develop students’ knowledge of SDL. Qualitative and quantitative research techniques were employed in each aspect of this DBR investigation. Qualitative analysis was well suited for elucidating participants’ actual perspectives and interpretations (Maxwell, 2004) about SDL. Initially, perspectives were sought from primary school staff, parents and students, to contribute to the development of a model of SDL in primary-age (elementary) school children. The model (shown in Figure 1) clarifies the nature of effective SDL in primary students. Similarly, in order to develop an instrument to assess school support for inquiry, school staff were asked for their perspectives on what an inquiry school context would be like so a description could be developed of school influences that encourage inquiry and require learners to be self-directed. Quantitative

16

Penny Van Deur

methods (SPSS software, etc.) were used to analyze data from the instruments used to assess school context support for inquiry and students’ knowledge of SDL.

A Two-Part Assessment of Self-Directed Learning One part of the assessment of SDL is the Primary School Characteristics Inventory (PSCI) that was developed to allow school staff to distinguish aspects of their school context that support inquiry from areas that require further development. The PSCI statements (which were based on previously published research and teachers’ perspectives of school support for inquiry) represented five broad characteristics of a school’s support for inquiry. These characteristics included (1) the general ethos of the school, (2) the nature of classroom tasks, (3) the role of the teacher, (4) the role of students, and (5) the actual organization of the school (See Van Deur, 2018 for a full description of the instrument and its development through pre-testing, trialing and piloting). Through the processes of continuous quality improvement, the final instrument evolved to capture teachers’ ratings of their school’s support for inquiry. The final PSCI instrument is comprised of 50 items divided into three subdivisions: Motivation for student inquiry, Organizational structures that support inquiry, and Structures supporting inquiry strategies in school. Table 2 shows examples of statements in each sub-scale of the PSCI. While the Primary School Characteristics Inventory’ (PSCI) is an important part of charting support for inquiry in primary schools, it forms one part of the assessment of knowledge of SDL in primary schools with the other part being the Learning At School Questionnaire (LASQ). The development of this questionnaire to assess primary students’ knowledge of SDL is described in Van Deur (2018). The Learning At School Questionnaire is comprised of 46 items in three sub-scales. Motivation (19 items) has statements about attitudes to SDL which contribute to dispositional orientation, which in turn influences behavior, as well as statements about self-efficacy, causal attributions, and persistence. The sub-

A Novel Two-Stage Model of Evaluation …

17

scale of Strategy (17 items) has statements about learning strategies that can be employed in SDL and Context (10 items) has statements about the perceived support available in the school environment for SDL needed to carry out inquiry tasks. Table 3 shows examples of statements in each subscale of the Learning At School Questionnaire. Table 2. Examples of statements in each sub-scale of the Primary School Characteristics Inventory (PSCI) No. 1 9 11 14 24 38

PSCI item The environment is warm and there are friendly interactions Results of inquiry activities are displayed around the school The physical organization of the school supports student collaboration Students have ready access to information communication technology Students are often encouraged to solve problems and work on real issues Students are involved in school decision making and planning

Sub-scale 1 1 2 2 3 3

Sub-scale 1= Motivation for student inquiry. Sub-scale 2=Organizational structures to support inquiry. Sub-scale 3= Structures to support inquiry strategies in school.

Table 3. Examples of statements in each sub-scale of the LASQ No. 1 8 20 28 37 41

LASQ item I know how to learn about topics that I am interested in When I get stuck on something I cannot do, I encourage myself to keep going When I start to learn about a new topic I always ask myself what it is all about I keep track of the time I am spending as I am finding out about my topic The teacher helps me to be clear about the topic I am trying to find out about At school there are different resources that I can use to find out about topics

Sub-scale Motivation Motivation Strategy Strategy Context Context

18

Penny Van Deur

Assessing Knowledge of Self-Directed Learning in Six Primary Schools The primary objective of the first part of the research was to assess knowledge of SDL in primary (elementary) schools, by using the two-part assessment of SDL (which was based on the model of effective SDL in primary schools). The specific aims were to (1) employ an assessment instrument to assess knowledge of SDL in primary students, and (2) develop objectives to teach primary students about SDL. As described in previous sections, the variables that identify school support for inquiry were pre-defined and included on the survey instrument; specific statements concerning these variables were evaluated by school staff, then evaluated post-hoc to identify aspects of the school that support inquiry and those that could be developed further. The second part of the assessment was developed to identify students’ initial knowledge of SDL and assess whether this knowledge changed as a result of lessons about SDL. A framework for classroom development of SDL, developed from the model of effective SDL, provided the base from which the author planned lesson activities designed to assist students develop their knowledge of SDL. This knowledge was assessed before, after and over time following a teaching intervention conducted by the author. The novel prototype (i.e., the two-part assessment of SDL) was used to assess the effectiveness of teaching students about SDL. The LASQ questionnaire was administered to 150 Year 5 students (average age 10 years 6 months at the first assessment) in six schools as a pre-test before students engaged in four class lessons across one school week to develop knowledge of SDL. At the end of the week of lessons (planned and taught by the author) students in each school were assessed on the LASQ as an immediate posttest, then two delayed post-tests were carried out at three months and six months. Thus, the students’ knowledge of SDL was assessed both before and over time after the intervention. The results of the first Learning At School Questionnaire were inspected to identify aspects highlighted as needing development and recorded on the Framework for school and classroom development of SDL. The author used

A Novel Two-Stage Model of Evaluation …

19

the framework as the base for planning classroom activities to further develop students’ knowledge of SDL. Regular classroom teachers did not teach the activities designed to promote SDL in this part of the research. 1. The lessons, designed to improve self-directed learning skills, involved students in discussing SDL strategies they could use to work on a problem that had an inquiry focus (however students had limited time to carry out investigations within the four classroom lessons). The lessons included discussions about the following strategies: 2. collaborating with other learners to develop methods to plan activities and strategies to approach a difficult problem; 3. adopting attitudes of perseverance when encountering difficulties; 4. developing behavioral skills that promote positive attitudes to SDL; 5. incorporating planning skills to work more efficiently on a topic; 6. outlining organizational strategies to improve time-management; 7. consulting with parents and utilizing parental expertise to work on problems; 8. negotiating with teachers to work on topics learners are most interested in investigating; 9. appraising the resources used to investigate topics; 10. evaluating methods that learners can use to check their work; 11. maintaining a positive attitude to SDL by being able to attribute lack of success in an inquiry to external concerns such as failing to choose the most helpful resources; and 12. refining the skills needed to search for information, evaluating the strategies used to filter information, and strategies that might be useful for reflection during and after learning.

RESULTS Each sub-scale of the LASQ (raw score) was converted to a Rasch scale (an interval scale, see Sheridan, Andrich & Luo, 1997 for more information),

20

Penny Van Deur

and the Rasch interval scale logits were then used to study changes in the students’ knowledge of SDL over four assessment times (pre, immediate post-test, one delayed post-test at three months and one delayed post-test at six months). Descriptive statistics were calculated for the Rasch scale scores (using SPSS, version 11.0 for Windows) on each LASQ sub-scale. Effect sizes were calculated (Coe, 2000) to indicate the size of the effect in each sub-scale between each assessment in students’ knowledge of SDL. The mean scaled scores in logits for each LASQ sub-scale were calculated to depict changes over time in students’ knowledge of SDL. A principal components analysis was carried out on the Rasch scaled scores for each LASQ sub-scale across the four assessment times. The pattern of results on LASQ scores across the aggregated sample of 150 students showed that there was an influence on students’ knowledge of SDL following lessons where the author taught them about SDL. There were significant gains in scores (p = < 0.05) for both Motivation and Strategy. This result suggested that even though the teaching intervention was short, it influenced positively students’ knowledge of SDL across most of one school year. The results of this first part of the research indicated that primary students’ knowledge of SDL could be developed through teaching about learning processes and strategies. This finding aligns with a constructivist view of learning where knowledge is made rather than found (Phillips, 2000). In the second part of the research the two-part assessment provided the starting point for assessing students’ knowledge of SDL, then two classroom teachers were guided in collaborative planning of classroom activities to develop their students’ knowledge of SDL.

Guiding Teachers to Develop Primary Students’ Self-Directed Learning Knowledge The primary objective of the second part of the research was for: teachers to use the results of the two-part assessment of students’ knowledge of SDL (PSCI and LASQ) to identify changes that could be made to their

A Novel Two-Stage Model of Evaluation …

21

school environment to better support inquiry that requires self-directed learning; and to identify aspects of students’ knowledge of SDL that could be developed. The framework for school and classroom development of SDL was used to help teachers collaboratively plan modifications to their school environment for the classroom development of SDL; these modifications were designed to support inquiry and develop activities to build students’ knowledge of SDL processes. The specific aims of this second part of the research were to: (1) use the assessment of SDL to identify aspects of a school context to be modified to support inquiry, and aspects of students’ SDL knowledge that could be developed; (2) identify the effectiveness of guiding teachers to plan collaboratively teaching activities to develop students’ knowledge of SDL, and (3) collect the perspectives of teachers on the effectiveness of the assessment and teaching students about SDL.

METHODS The participants in the second part of the research were two Year 5 teachers and two classes (sixty Year 5 students with 30 students in each class) at one school. Both Year 5 teachers completed the Primary School Characteristics Inventory (PSCI) to identify aspects of their school context that could be modified in order to support student inquiry. Teachers were asked to provide their response on a 5-point Likert scale (never, rarely, sometimes, often, always) to statements on the PSCI designed to determine if the teachers thought the characteristic was observed in the school. For all items that teachers rated below 3 (‘sometimes’ or lower), the researcher discussed relevant modifications to the school context that would further develop and improve these aspects. Thirty-two Year 5 students (average age 11 years at the time of the first assessment) from both year 5 classes assented to complete the Learning At School Questionnaire (LASQ) on two occasions over one school term (pretest and immediate post-test) following lessons and activities collaboratively planned by their teachers and the researcher to develop knowledge of SDL.

22

Penny Van Deur

Participants responded to statements on the LASQ by circling one of three response categories (Disagree scored 0, Unsure scored 1, Agree scored 2). The lower third of ratings scores (below a mean of 1.36, Unsure and below) were identified and provided the basis for teachers to collaboratively plan teaching activities to develop students’ knowledge of self-directed learning. The results of the first questionnaire, completed before the lessons about SDL were prepared and implemented, were compared with the second questionnaire. This comparison was needed to identify the specific changes and/or further development in students’ knowledge of SDL in response to activities planned by their regular teachers. In addition, each teacher participated in an individual interview in the final week of the research where their views were sought about the effectiveness of the PSCI for helping them modify their school environment, and the effectiveness of the LASQ for assessing students’ knowledge of SDL.

A Collaboratively-Planned Intervention The collaboratively planned intervention was preceded by the author recording on the framework for classroom development of SDL areas needing further development as identified by teachers on the PSCI and learners on the LASQ. These areas were the base from which the researcher and teachers collaboratively planned activities designed to develop students’ knowledge of SDL. This unique process saw the creation of novel learning activities designed to help learners develop further their knowledge of SDL that could be used on inquiry activities. Participating students completed the LASQ post-assessment (Time 2) six weeks after the LASQ pre-assessment (Time 1), and following the intervention implemented by their regular classroom teachers. The data for the 46 LASQ items at Time 1 and Time 2 were entered into the software program SPSS (version 11.0 for Windows) and mean scores and standard deviations were calculated. Paired T-tests were conducted to determine if higher or lower mean scores at Time 2 were significant.

A Novel Two-Stage Model of Evaluation …

23

In order to assist collaborative discussions with the teachers, the author referred to the South Australian curriculum framework Teaching for Effective Learning (SATfEL, 2010), and used this document as the base for teachers to plan activities to develop their students’ knowledge of SDL. This curriculum support document was used because it was already available to the teachers in their school and this availability was likely to promote the sustainability of the intervention they planned. This customization was intended to make the intervention more effective for teachers and students in the school as recommended by Yeager et al., (2016). The author worked in collaboration with the teachers to develop activities that would assist students to develop their knowledge of SDL. For example, the SATfEL Question wall (p 51) helped guide teachers to develop strategies to teach students to judge how useful specific strategies were for finding information. As part of the Question wall strategy students were encouraged to ask themselves ‘Where am I heading? What else might I need to know? How could I find information another way?’ The teachers decided how they encouraged students to use questions beginning with the stems: ‘Why, What, Where, Who, How, Which, When, If’ to help them reflect on the usefulness of information finding strategies they were using. They decided to show their students how to use Learning Log sentence starters (SATfEL, p 52) to help them reflect on strategies by asking: What happened….? How do I feel about it…? What did I learn…? Upon completion of these activities the teachers prompted students to write short summaries describing their personal views on the effectiveness of any strategies they used to find information.

Findings The results of the second part of the research are discussed in three sections: (i) the effectiveness of the two-part assessment of SDL for identifying school context support for inquiry, and aspects of students’ knowledge of SDL that could be developed; (ii) using the framework for school and classroom development of SDL to guide school and classroom

24

Penny Van Deur

level changes to support student inquiry, and as the base for teachers to plan classroom activities to develop students’ knowledge of SDL, and (iii) perspectives of teachers on the value of the two-part assessment of SDL for helping them modify their school environment, and plan classroom activities to develop students’ knowledge of SDL.

The Effectiveness of the Two-Part Assessment of SDL for Identifying School Context Support for Inquiry, and Aspects of Students’ Knowledge of SDL That Could Be Developed The teachers completed the Primary School Characteristics Inventory (50 items) at the beginning of a school term (Time 1) in order to identify their ratings of support for inquiry in the school context. Both teachers gave ratings of 3 (sometimes) to items about: displaying students’ inquiry work around the school; giving students a voice in deciding the purposes of their learning; learning together using group and peer structures in the classroom; students negotiating learning with their teachers, and students having a voice in organizing how they will learn. The author recorded these aspects on the relevant sections of the Framework for classroom development of SDL and discussed them with both teachers. During discussion the teachers were encouraged to find places around the recently refurbished school where students’ work could be displayed, talk with students about topics they would like to investigate, and discuss with them how they would like to organize their learning. Thirty-two students agreed to complete the Learning At School Questionnaire to identify their knowledge of SDL. Six weeks later they again completed the LASQ to identify whether their knowledge of SDL had changed after they engaged in activities collaboratively planned by the researcher and their teachers to address aspects of SDL given low mean ratings (below 1) and recorded on the framework for classroom development of SDL. Prior to the intervention, the LASQ results identified the following areas (i.e., lower third of mean ratings) of student concern:  

being able to select and evaluate resources liking independent learning

A Novel Two-Stage Model of Evaluation …        

25

seeing many learning activities as involving self-directed learning being able to evaluate strategies used to learn having time at school to find information being able to keep track of time as inquiry is carried out having a say in topics to be worked on in the classroom believing parents could be involved in school learning believing that information technology could be used at school when it is needed. believing they can check that they had found information they were looking for

The author worked with teachers to develop teaching strategies to help students view SDL as being involved in many school activities; develop and evaluate learning strategies; use school time well during inquiry; develop some strategies to be able to negotiate with the teacher topics they would like to work on; encourage greater involvement of parents in school activities, and find opportunities to allow students to use information technology when appropriate in the classroom. Table 4 shows the comparison of student concerns (lower third of mean ratings) identified on the LASQ at Time 1 and Time 2, demonstrating the changes attributed to the short intervention. Standard deviation scores are also shown. Time 2 mean scores for all LASQ items indicated that most of the items with higher mean ratings were associated with activities collaboratively planned as part of the teaching intervention. The mostly higher mean ratings at Time 2 suggest that students had increased their knowledge of SDL following a short intervention, and that the intervention helped learners develop this knowledge through novel activities planned and implemented by their regular teachers. This adds support to the view that independent learning can be developed in most students (Wolters, 2010) when they have opportunities to learn about it explicitly (Nicol & Macfarlane-Dick, 2007; Zumbrunn, Tadlock & Roberts, 2011). However, the increased mean ratings at time 2 were not statistically significant as determined by paired t-test calculations, which could be due to the short period of the intervention and the small participating sample of 32 students.

Motivation If I have not been able to find out about a topic it was because I did not choose the right resources to help me I like working on topics at school in a self-directed way I think that it would be good to work on most school learning in a self-directed way When I am at school I have plenty of time to find the information I need for a topic I think self-directed learning is only about project work Strategies I choose the exact skills I will need to use to find out about a topic I keep track of the time I am spending as I am finding out about my topic When I have finished I look back to check that I have found the information I wanted When I have finished I think about the strategies I used to find information to see if they were the best ones I could have used When I start a new topic I think about whether I have the skills I need to use so that I can learn about the topic I plan to get help with any special skills I will need to find out about my topic Context At school I am able to have a say about choosing the topics I am interested to work on At school I am able to have a say about the topics that we will study in the classroom At school I can use information technology whenever I need to At school parents have a chance to show students the things they know a lot about 0.86 1.29 1.10

1.35 1.19

1.35

1.19 1.38 0.90

1.34 1.06 1.31 0.96 1.22

Mean Time 1 1.18

0.86 0.90 0.84

0.70 0.83

0.75

0.87 0.80 0.90

0.78 0.66 0.78 0.78 0.66

0.78

SD

*Table shows item mean scores below average mean (1.36-Unsure) across LASQ at Time 1 and corresponding mean ratings at Time 2. N Time 1 = 32 students N Time 2 =32 students.

42 45 46

36 38

35

28 32 33

16 17 18 19 26

Item No. 11

Table 4. Mean and standard deviation scores for LASQ Time 1 and Time 2*

0.92 1.41 1.07

1.41 1.24

1.41

1.45 1.56 0.98

1.38 1.10 1.38 1.08 1.32

Mean Time 2 1.20

0.84 0.83 0.86

0.72 0.77

0.75

0.79 0.72 0.90

0.74 0.73 0.76 0.81 0.70

0.80

SD

A Novel Two-Stage Model of Evaluation …

27

Using the Framework for School and Classroom Development of SDL to Guide School and Classroom Level Changes to Support Student Inquiry, and as the Base for Teachers to Plan Classroom Activities to Develop Students’ Knowledge of SDL In order to increase sustainability of the strategies discussed with teachers, the items of the LASQ were matched with domains of the curriculum document South Australian Teaching for Effective Learning (SATfEL, 2010) which is readily available in South Australian schools. This proved valuable for collaborative planning of teaching strategies to develop students’ knowledge of SDL. Perspectives of Teachers on the Value of the Two-Part Assessment of SDL and the Framework for Development of Classroom SDL for Helping Them to Modify Their School Environment, and Plan Classroom Activities to Develop Students’ Knowledge of SDL Individual interviews were conducted with both teachers who expressed their enthusiasm for the SATfEL-based activities that had been discussed and used in the collaborative planning sessions. The teachers commented that they would continue to use the activities to help students find information. Both teachers commented that they valued being able to work together and that this collaboration had enabled them to modify the classroom set-up by pulling back the room divider so that both classes could work together regularly. Other organizational changes implemented to support independent learning in their classrooms (Zumbrunn et al., 2011) involved sharing the planning and implementation of curriculum units and placing an increased emphasis on the process of working in groups. It should be noted that teachers’ comments showed some resistance to giving students choice in their learning. They indicated that they were concerned that students would not be able to make good learning choices and would not use their time well. This reluctance of both teachers to allow students to assume control of their learning indicates that more work needs to be done to help them involve students in negotiating and carrying out their own learning. This is important because students need to be able to make decisions about what they would like to study, and they need to have

28

Penny Van Deur

opportunities to practice self-regulation (Nicol & Macfarlane-Dick, 2007) as they investigate topics they have determined. The teachers were asked to describe the strategies they implemented and any other strategies they intended to try. In relation to helping students to see many learning activities as involving SDL, one teacher commented that she gave students more choice in mathematics activities and observed that they responded well to this approach. She found that students enjoyed making decisions as a group, and further commented that since the intervention, students were more comfortable talking with her about topics they would like to investigate. The teachers described how they had begun working together and had introduced a star rating of resources to help students select and evaluate resources for their learning. They both commented that the star rating system had proved useful for helping students select appropriate resources to use. They discussed how involving students in activities designed to help them find information had resulted in students using group strategies to check whether information was relevant to their topic. The teachers observed that some students were able to extract information from resources by using dot points, whereas some students were copying information verbatim as written. The teachers supported the concept that students retained information better when they used their own words to take notes but commented that, even after the intervention, this information-gathering strategy was still not clear to some students. During the interview discussion about letting students determine topics they would be interested to investigate, both teachers revealed that they chose the groups students would work in. They expressed concern about letting students choose their own groups explaining that they were worried that students would interpret this strategy as meaning they could do anything they liked. However, they commented that they had begun to allow students to choose their own topics and observed that students did not realize that this meant they were ‘having their say’. The teachers stated that they did not find it difficult to plan and teach activities designed to develop students’ knowledge about SDL, and said this aspect was implemented successfully due to the support provided by the author in the collaborative planning

A Novel Two-Stage Model of Evaluation …

29

sessions. An unexpected problem identified by one of the teachers was the difficulty of finding a place in the recently refurbished school to exhibit student work. The teachers were asked to comment on any aspects of the research project they found helpful and one teacher commented: ‘I thought it was good to direct the students toward using each other as resources.’ The other teacher said, ‘Some started to think laterally… They were thinking about how they research rather than what their endproduct is … thinking about the process of how to get there.’

These comments indicated that students were becoming more aware of the process of carrying out inquiry. They suggest that the teachers were actively promoting collaboration among their students with students recognizing that classmates could act as resources for each other when learning about certain topics. Both aspects further support the utility of the collaboratively planned intervention and highlight that teachers can promote development of SDL in primary-aged learners. The teachers explained that they had modified the classroom environment by retracting the room-divider and working with both classes in blocks of time, ‘If there were 60 kids working on Light or doing whatever they were doing, they were all on task and one teacher could float around and back up what the other teacher was doing’. This comment showed that the teachers had changed the way they worked with their classes and that they believed this change was assisting students to be more independent in their learning. Overall, the interviews with two teachers in one school indicated that they had:  

increased their awareness of aspects of SDL outlined in the framework; developed an understanding that SDL involves students having choice in their learning and that students can learn to negotiate with teachers about some of their classroom learning;

30

Penny Van Deur      

had come to value students learning how to select their own learning resources; identified time as a possible restriction that could impede students finding information; begun to see that students could learn ways to use their time well when working in a self-directed way; begun to see that parents could be involved in school learning and use their expertize to assist students’ learning; begun to understand that students could be encouraged and allowed to use information technology as part of their SDL, begun to develop strategies to support students to reflect on their learning.

Interviews with the teachers indicated that they were changing the way they perceived learning and developing strategies to help their students learn more effectively. This indicates that these points are strengths for promoting SDL in primary school learners. The findings of the second part of the research indicate that the two-part assessment to identify aspects of a school context that might be modified to support inquiry and identify areas of students’ knowledge of SDL (an important component of SRL) could be employed to assist classroom teachers develop students’ knowledge of SDL.

DISCUSSION AND CONCLUSION The research reported here responded to calls by various organizations for teachers to develop successful independent learners in Australian schools (Reid, 2017; ACARA, accessed 2015; MCEETYA, 2008). This research considered how classroom and school environments can support the development of knowledge of SDL and the skills required to promote student-driven inquiry. This relatively new focus on SDL raises the question of whether teachers can effectively help students learn the skills required to

A Novel Two-Stage Model of Evaluation …

31

become independent learners. The research described herein addressed this challenge, using Design-based research (DBR) methods to consider what primary (elementary) learners know about self-directed learning (SDL). The nature of SDL was examined and a model was developed to describe effective SDL in primary-age students. Based on this model, an assessment was developed to identify students’ knowledge of SDL, and the complementary PSCI assessment instrument was developed to determine the institutional (including teaching) support for inquiry offered in primary (elementary) schools. The LASQ assessment allowed an examination to be made of students’ knowledge of SDL and chart its development over time following a teaching intervention lead by the researcher (first part of the research) and then by classroom teachers in the second part of the research. Collectively, the model of effective SDL was the basis for the framework for classroom development of SDL, which was employed by the researcher in part one of the research and by the researcher and two teachers in part two of the research. The framework guided the design of collaboratively-planned teaching activities to further improve the skills required for SDL and independent learning. The second part of the research showed that the assessment could be linked with a local curriculum document; an important way to increase sustainability of classroom teachers’ efforts to develop students’ knowledge of SDL. Interviews carried out with two participating teachers in the second part of the research confirmed that they believed that the two-part assessment of SDL helped them think about how the school context could support student inquiry and assisted them to identify areas they could work on in the classroom to develop students’ knowledge of SDL. The first part of the research used design thinking (Razzouk & Shute, 2012) to create a model of effective self-directed learning. In the second part of the research an assessment of SDL in primary schools, based on the model, was tested in a school (Lewis, Perry & Murata, 2006). The two-part assessment was used innovatively to identify school and classroom level changes teachers could make to support student inquiry (pre-assessment). Teachers engaged in collaborative discussions to plan teaching activities aimed at developing students’ knowledge of SDL. They assessed (post-

32

Penny Van Deur

assessment) students to identify those areas of SDL that had improved after a short (6 week) intervention, and those areas still requiring additional improvement. Consistent with design thinking, teachers’ perspectives were sought about the effectiveness of the assessment for charting the development of students’ knowledge of SDL. The results of both parts of the research indicate that process knowledge of SDL can be made explicit and that assessments were able to chart students’ development of this knowledge. Although the second part of the research showed only small gains in students’ knowledge of SDL after a brief intervention where teachers modified their classroom environments and taught students about SDL, teachers’ comments suggested that the twopart assessment was effective for charting the development of students’ knowledge of SDL. This indicates that the intervention could be scaled up in a wider sample of schools. A strong emphasis on self-directed learning disrupts the assumption that teachers need to manage students’ learning activities (Bjork & MacfarlaneDick, 2007). Previous research studies have found that students who were taught about independent learning processes, such as self-regulated learning, showed improved learning outcomes (Zumbrunn et al., 2011) Further to this, Bjork, Dunlovsky and Kornell (2013) argued that students can take responsibility for their learning if they are taught how to do it. The second part of this research supports these findings by showing that a short intervention, planned and implemented by classroom teachers, improved knowledge of SDL in primary (elementary) learners. They were becoming more cognizant of setting goals, monitoring progress toward accomplishing them, and reflecting on their learning processes. This finding highlights that explicit teaching about learning processes is not inconsistent with a curriculum based on constructivism. Knowledge of SDL is described here as an essential pre-requisite for inquiry because students need to be able to draw on it to manage their learning behaviour when they are being self-directed in their learning. Applying the ‘concept of failure’ to consider how the innovation could fail when implemented in school contexts reveals that assessing SDL and teaching about it would not help develop self-directed learners if authentic

A Novel Two-Stage Model of Evaluation …

33

opportunities to carry out inquiry were not available to students. Student-led inquiry is essential to SDL because, by its very nature, it requires students to draw on their knowledge of SDL as they take responsibility for managing their own learning.

Implications The results of the intervention-phase (second part) of this research were limited by the small participating sample of students (32 students) and therefore, the quantitative data should be interpreted cautiously. It is worth noting that teachers noted obvious qualitative improvements brought about by a simple collaboratively-planned 6-week intervention tailored to specifically address deficiencies identified by the teachers and students in their classes. Thus, the results of the second part of the research suggest that an assessment of knowledge of SDL could be integrated into learning about SDL, because this ‘constructive assessment’ is focused on the process (how) of learning. It could bring knowledge of SDL to students’ attention and scaffold their ability to reflect on and develop it by enhancing their metacognitive awareness. The framework for development of classroom SDL could guide ‘constructive pedagogy’ (Phillips, 2000) and become the foundation for developing, in primary (elementary) schools, effective selfdirected learners who are equipped to negotiate, manage and control their own learning. Recognizing that learners develop process knowledge in schools has implications for teaching and assessment, largely because primary school teachers may not be accustomed to carrying out explicit discussions about process skills and assessing this type of knowledge. Showing that knowledge of SDL can be assessed and developed in younger learners indicates that greater recognition could be given to students constructing process knowledge (an essential aspect of constructivism) because there is a danger that ‘self’ or ‘internal’ processes could be overlooked by teachers and not regarded as teachable. Teachers should not assume that all students will develop the skills required for SDL on their own; the findings of this

34

Penny Van Deur

research indicate that it is beneficial to teach students about SDL as early as primary (elementary) school. While teachers’ perspectives about developing SDL in their classrooms were collected in this research via individual interviews, those of students were not. Thus, future research will seek students’ perspectives on activities their teachers have implemented to develop their knowledge of SDL and capacity to carry it out. The assessment of knowledge of SDL in primary schools has implications for learning, when SDL is regarded as a longitudinal process for learning (i.e., using the skills to acquire knowledge) that can be described, assessed and developed by teachers in classrooms. It could be expected that students who have high scores for knowledge of SDL would also be more likely to demonstrate behaviors associated with SDL because they would be able to use their knowledge about Motivation, Strategy and Context resources to take responsibility for their own learning. This assertion will be investigated in future studies by using larger samples of primary school students in various school contexts. The implication for schools of assessing knowledge of SDL is that recognition is given to the need to ensure school contexts support inquiry. There are implications for suggesting that teachers need to be aware of students’ knowledge of SDL and integrate assessment of it into independent learning activities. There are implications for pedagogy that involve assessing students’ knowledge of SDL and using a framework to plan ways to develop it. The implications for a curriculum designed to promote teaching of the skills required for SDL involve giving explicit attention to developing inquiry and process knowledge about SDL so that the curriculum provides opportunities for students to engage in inquiry where they have the opportunity to take responsibility for their own learning. For policy makers, the implication of assessing knowledge of SDL is that it promotes the value of developing process knowledge in schools. As such, inquiry, SDL and the development of process knowledge about SDL should be emphasized in policy and curriculum documents.

A Novel Two-Stage Model of Evaluation …

35

ACKNOWLEDGMENT This work was supported by Flinders University under a Faculty Small Research Project Grant (# 20506326).

REFERENCES Australian Curriculum, Assessment and Reporting Authority (ACARA). (2015). The Australian curriculum. http://www.acara.au/curriculum/ curriculum.html. Australian Institute for Teaching and School Leadership (AITSL) National Professional Standards for Teachers. http://www.aitsl.edu.au/ australian-professional-standards-for-teachers/standards/list (2011). Bannan-Ritland, B. The role of design in research: The integrative learning design framework. Educational Researcher (2003) 32(1):21-24. Barrell, J. Critical issue: working toward student self-direction and personal efficacy as educational goals. Available [online]: http://www.ncrel.org/ sdrs/areas/issues/learning/lr200.htm (1995). Bjork, R., Dunlovsky J. & Kornell, N. Self-Regulated Learning: Beliefs, techniques, and illusions. Annual Review of Psychology (2013) 64:417444. Bransford, J. D. Brown, A. L. & Cocking, R. R. (Eds.) How people learn: Brain, mind, experience and school. Washington. D C: National Academy Press; 1999. Brockett, R. & Hiemstra, R. Self-direction in adult learning: Perspectives on theory, research and practice. London: Routledge; 1991. Candy, P. C. Self-direction for lifelong learning: A comprehensive guide to theory and practice. San Francisco: Jossey-Bass; 1991. Cobb, P. Supporting improvement of learning and teaching in social and institutional context. in S. M. Carver & D. Klahr. (Eds.), Cognition and instruction: Twenty five years of progress. (pp. 455-478). Mahwah, New Jersey: Lawrence Erlbaum Associates; 2000.

36

Penny Van Deur

Cobb, P., Confrey, J., di Sessa, A., Lehrer, R., & Schauble, L. Design Experiments. Educational Researcher (2003) 32(1):9-13. Coe, R., What is an ‘effect size’? Available [Online]: http://www.cemcentre. org/ebeuk/research/effectsize/ESbrief.htm. Accessed 25-9-2010. Collins, J., Joseph, D. & Bielaczyc, K. Design research: Theoretical and methodological issues. Special Issue of The Journal of Learning Sciences (2004) 13(1):15-42. Dede, C. If design-based research is the answer, what is the question? A commentary on Collins, Joseph, and Fishman, Marx, Blumenthal, Krajcik, and Soloway. JLS Special issue on design-based research. The Journal of the Learning Sciences (2004) 131(1):105-114. Evensen, D. H. Observing self-directed learners in a problem-based learning context: Two case studies. In D. H. Evensen & C. E. Hmelo (Eds.), Problem–based learning: A research perspective on learning interactions. (pp. 263-298). Mahwah, New Jersey: Lawrence Erlbaum Associates Inc.; 2000. Hmelo-Silver, C. E. Problem-based Learning: What and how do students learn? Educational Psychology Review (2004) 16(3):235-266. Keeves, J. P. & Alagumalai, S. New Approaches to Measurement. In G. N. Masters & J. P. Keeves (Eds), Advances in measurement in educational research and assessment. (pp. 23-42) New York: Pergamon; 1999. Kelly, (E.) A. Design research in education: Yes, but is it methodological? Special issue of The Journal of the Learning Sciences (2004) 13(1):115128. Knowles, M. Self-directed learning: A guide for learners and teachers. Chicago: Association Press and Follett Publishing Company; 1975. Lewis, C., Perry, R. & Murata, Aki. How should research contribute to instructional improvement? The case of lesson study. Educational Researcher (2006) 35(3):3-14. Lobato, J. How design experiments can Inform a rethinking of transfer and vice versa. In Educational Researcher (2003) 32(1):17-20. Lovett, M. A collaborative convergence on studying reasoning processes: A case study in statistics. In S. M. Carver & D. Klahr. (Eds.), Cognition

A Novel Two-Stage Model of Evaluation …

37

and instruction: Twenty-five years of progress. (pp. 347-384). Mahwah, New Jersey: Lawrence Erlbaum Associates; 2001. Loyens, S. M., Magda, J. & Rikers, R. M. J. P. Self-Directed learning in Problem-Based learning and its relationships with Self-Regulated Learning. Educational Psychology Review (2008) 20:411-427. Maxwell, J. A. Causal explanation, qualitative research and scientific Inquiry in education. In Educational Researcher (2004) 33(2):3-11. McCombs, B. L. Self-regulated learning and academic achievement: A phenomenological view. In B. Zimmerman & D. Schunk, (Eds.), Selfregulated learning and academic achievement. (2nd ed.) (pp. 51-82). Mahwah, New Jersey: Lawrence Erlbaum Associates; 2001. Ministerial Council for Education, Early Childhood Development, and Youth Affairs (MCEETYA). Melbourne Declaration on Educational Goals for Young Australians. Website: http://www.curriculum.edu.au/ verve/resources/National_Declaration_on_the_Educational_Goals_for _Young_Australians.pdf. 2008. Nicol, D. & Macfarlane-Dick, D. Formative Assessment and self-regulated learning: a model and seven principles of good feedback practice, Studies in Higher Education (2007) 31(2):199-218. Phillips, D. C. (Ed.) Constructivism in education: Opinions and second opinions on controversial issues. (pp. 6-18). Chicago, Illinois: The University of Chicago Press; 2000. Pintrich, P. R. A motivational science perspective on the role of motivation in learning and teaching contexts. In Journal of Educational Psychology (2003) 95(4):667-686. Pintrich, P. & DeGroot, E. Motivational and self-regulated learning components of classroom academic performance. Journal of Educational Psychology (1990) 82(1):33-40. Razzouk, R. & Shute, V. What is design thinking and why is it important? Review of Educational Research (2012) 82(3):330-348. Reid, A. Public Education in South Australia https://www.decd.sa.gov.au/ sites/g/files/net691/f/public-education-in-south-australia-statement.pdf 2017.

38

Penny Van Deur

Rhine, S. The role of research and teachers’ knowledge base in professional development. Educational Researcher (1998) June-July: 27-31. Rutter, M., & Maughan, B. School Effectiveness Findings 1979-2002. Journal of School Psychology (2002) 40(6):451-475. Saks, K. & Leijin, A. Distinguishing self-directed learning and selfregulated learning and measuring them in the e-learning context. Social and Behavioral Sciences (2014) 112:190-198. Schunk, D. H. & Zimmerman, B. J. Conclusions and future directions for academic interventions. D. H. Schunk & B. J. Zimmerman (Eds). Selfregulated learning: From teaching to self-reflective practice. (pp. 226232). New York: The Guilford Press; 1998. Shavelson, R., Phillips, D., Towne, L. & Feuer, M. On the science of education design studies. Educational Researcher (2003) 32(1):25-28. Sheridan, B., Andrich, D. & Luo, G. RUMM. Rasch Unidimensional Measurement Models. Perth: Murdoch University; 1996. Sloane, F. C. & Gorard, S. Exploring modelling aspects of design experiments. Educational Researcher (2003) 32(1):29-31. South Australian Department of Education and Children’s Services. South Australian Teaching for Effective Learning framework. Adelaide, South Australia, Lane Print & Post; 2010. Spencer, J. A. & Jordan, R. K. Learner-centred approaches in medical education. BMJ (1999) 318(8):1280-1283. SPSS. SPSS for Windows, Version 11.0. Chicago, Illinois: SPSS Inc.; 2001. Sternberg, R. J. Making school reform work: A mineralogical theory of school modifiability. Bloomington, Indiana: Phi Delta Kappa Educational Foundation; 2000. Sternberg, R. J. & Grigorenko, E. L. The theory of successful intelligence as a basis for gifted education. Gifted Child Quarterly (2002) 46(4):265277. The Design-Based Research Collective. Design-Based Research: An emerging paradigm for educational inquiry. In Educational Researcher (2003) 32(1):5-8. Treffinger, D. J. Teaching for self-directed learning: A priority for the gifted and talented. The Gifted Child Quarterly (1975) XIX(1):46-59.

A Novel Two-Stage Model of Evaluation …

39

Treffinger, D. J. Fostering effective independent learning through individualized programming. In J. S. Renzulli (Ed.), Systems and models for developing programs for the gifted and talented. (pp.429-460). Australia: Hawker Brownlow; 1993. Van Deur, P. Managing Self-Directed Learning in Primary School Education: Emerging Research and Opportunities IGI Global, Hershey, USA; 2018. Watkins, D. A. & Biggs, J. B. Insights into teaching the Chinese learner. In D. A. Watkins. & J. B. Biggs, (Eds.), Teaching the Chinese learner: Psychological and pedagogical perspectives. (pp. 277-301). Hong Kong, China: Comparative Education Research Centre & Melbourne, Victoria: The Australian Council for Educational Research Ltd; 2001. Wigfield, A., Eccles, J. S. & Rodriguez, D. The development of children’s motivation in school contexts. In D. P. Pearson & A. Iran-Nejad. Review of Research in Education. (pp. 73-118). Washington, DC: American Educational Research Association; 1998. Wolters, C. Self-regulated learning and the 21st century competencies. Universidad de Houston: Department of Educational Psychology. Consultado en: http://www. hewlett. Org/uploads/Self_Regulated_ Learning__21st_Century_Competencies.pdf. 2010. Yeager, D., Romero, C., Pauneska, D., Hulleman, C., Schneider, B., Hinojosa, C., Lee, H., O’Brien, J., Greene, D., Walton, G. & Dweck, C. Using Design thinking to improve psychological interventions: The case of the growth mindset during the transition to high school. Journal of Educational Psychology (2016) 108(3):374-391. Zimmerman, B. J., & Lebeau, R. BA commentary on self-directed learning. In D. H. Evensen & C. E. Hmelo (Eds.), Problem–based learning: A research perspective on learning interactions. (pp. 299-314). Mahwah, New Jersey: Lawrence Erlbaum Associates Inc; 2000. Zumbrun, S., Tadlock, J. & Roberts, E. Encouraging self-regulated learning in the classroom: A review of the literature. Metropolitan Educational Research Consortium, Virginia Commonwealth University; 2011.

In: The Digital Era of Learning Editor: Christopher S. Keator

ISBN: 978-1-53618-750-2 © 2020 Nova Science Publishers, Inc.

Chapter 2

MOBILE LEARNING LITERATURE REVIEW IN MEDICAL EDUCATION Heeyoung Han1,*, PhD, Larry Hurtubise2, Geraud Plantegenest3, Carolyn R. Rohrer Vitek4, EdD, Rahul Patwari5, MD, Cecile M. Foshee6, PhD and Elissa R. Hall4, EdD 1

*

Southern Illinois University School of Medicine, Springfield, Illinois, US 2 Ohio State University, Columbus, Ohio, US 3 Spectrum Health Medical Group, Grand Rapids, Michigan, US 4 Mayo Clinic College of Medicine and Science, Mayo Clinic, Rochester, Minnesota, US 5 Rush University, Chicago, Illinois, US 6 Cleveland Clinic, Cleveland, Ohio, US

Corresponding Author’s Email: [email protected].

42

Heeyoung Han, Larry Hurtubise, Geraud Plantegenest et al.

ABSTRACT Mobile learning has become a new learning modality, yet most practices of designing and developing mobile learning in medical education are often atheoretical with neither solid conceptual basis nor evidence-based instructional strategies. This paper aims to provide a literature review that investigates mobile learning instructional design strategies that were found to be effective in medical education. We conducted a literature search from 2010 to December 2015. An initial search yielded 1,196 articles, which were narrowed down to 28 papers for analysis based on exclusion and inclusion criteria. Adopting a qualitative synthesis methodology, we used an iterative data analysis approach using an inductive and deductive process to identify themes. Studies were from various disciplines: Emergency medicine, surgery, pediatrics, nursing, pharmacology. The qualitative synthesis of literature suggested six themes. (1) Timeliness capturing learning experience and assessing performance: The focus of this mobile learning process is on capturing learning activities and providing feedback on the spot. (2) Pushed reminders for knowledge learning and behavior change: Using simple text messages were found to be effective and can be extended to a more sophisticated personalized scaffolding. (3) Visual modeling for performance just in time: This instructional strategy often involves reducing extraneous cognitive load on the spot, providing worked examples right before performing a critical task, and visualizing learning object in three dimensional presentations. (4) Facilitating collaborative informal learning: The underlying concept to support this mobile learning pedagogy is informal work-based learning and social learning process. (5) Mixed evidence of multimedia: There was no consistent evidence that multimedia such as videos is more effective than traditional media for knowledge learning. (6) Simulated immersive clinical experience: The underlying premise is that mobile simulations and gaming provide virtual immersive clinical experiences with opportunities for repeated practice at their fingertips without risk. These six themes are not exclusive, but combined to create optimized effective mobile learning experiences. These study findings provide that mobile learning is a pedagogical process enriched by context-sensitivity of being (location) and doing (activity) of a learner or a teacher. It is a comprehensive process to create meaningful instructional and learning interactions in consideration of where learners and teachers will be and what they will be doing. Simple transferring decontextualized traditional curriculum activities via a mobile device can have limited effectiveness. Future studies could focus on further applicability to various contexts in medical education.

Mobile Learning Literature Review in Medical Education

43

MOBILE LEARNING LITERATURE REVIEW IN MEDICAL EDUCATION The influx of mobile technologies such as smartphones and tablet computers has profoundly influenced people’s daily activities. As mobile devices continue to diminish in size, their computing capacity is expanding at unprecedented rates providing users with increased on-the-go access, powerful information processing capabilities, and social opportunities. This mobility and instant connectivity phenomenon changes not only how people interact and share information but also presents educational opportunities across fields and academic levels—hence the coining of the term mobile learning.

INTRODUCTION For the last decade, the concept of mobile learning has been discussed and defined from various perspectives. From the perspective of technical attributes, mobile learning can be defined as learning that makes use of mobile technologies (Sharples, 2000). This techno-centric perspective focuses on the physical aspect of technologies; that is, the hardware, the software, and specific characteristics of mobile devices. This view has been challenged and has evolved to include a broader understanding of the technological phenomenon with an increased focus on the nature of ubiquitous learning embracing seamless learning context (Toh et al. 2013; El-Hussein and Cronje, 2010; Traxler, 2007). In this view, mobile learning is defined as a meaningful learning process “that occurs when learners have access to information anytime and anywhere via mobile technologies to perform authentic activities in the context of their learning” (p. 77, Martin and Ertzberger, 2013). The notion of ubiquitous mobile learning  learning anywhere and anytime  indicates a pragmatic understanding of learning experiences especially in an informal format that happens beyond classroom settings and resides in daily activities (Merriam and Bierema, 2014). The

44

Heeyoung Han, Larry Hurtubise, Geraud Plantegenest et al.

majority of adults engage in hundreds of hours of informal learning (Caffarella and Merriam, 2012); therefore, a unique aspect of mobile learning is that it is a “personal, contextual, and situated” learning process, especially emphasizing individuals’ informal and unstructured experiences (Traxler, 2007). Recognizing the disruptive nature of mobile learning regardless of location and time, it is imperative to understand what design strategies and approaches have been found to be effective in mobile learning. With the rapidly increasing interest in mobile learning, many papers have been published in the field. Most literature focused on effectiveness, system design, technology adoptions and satisfaction studies (Wu et al. 2012; Gwo-Jen and Chin-Chung, 2011). Publications in mobile learning has increased significantly in 2010 and the number of publications between 2006 and 2010 is nearly four times that of 2001-2005 (Hwang and Tsai, 2011). Hwang and Wu (2015) reviewed literature published from 2008 to 2012 regarding students’ learning performance and found that 61% of the selected literature did not measure students’ learning achievements. Among those with learning outcome data, 83% of the studies reported positive learning effects of mobile learning. However, few literature review studies have discussed what instructional design aspects were utilized in the outcome studies. Most discussions have been in K-12 or higher education settings (Wu et al. 2012; Hwang and Tsai, 2011; Hwang and Wu, 2015). Mobile learning has become a new learning modality, yet most practices of designing and developing mobile learning in medical education are often lack of solid conceptual frameworks and are devoid of evidence-based instructional strategies (Sandars, 2012). Moreover, most studies were conducted in K-12 or undergraduate settings with most being grounded in social cognitive and constructivism perspectives (Hwang and Tsai, 2011; Wang, Liu and Hwang, 2017). It is important to examine the medical education literature and identify the mobile learning development process and the evidence it provides to support current practices. For example, mobile learning designers must be mindful of the potential information overload imposed by these technologies and work diligently to reduce cognitive load (Sweller, van Merrienboer and Paas, 1998). These designers must also be well versed in multimedia principles (Mayer, 2005), pedagogy

Mobile Learning Literature Review in Medical Education

45

as it relates to technology (JISC, 2017), and general principles about how people learn (Council NR, 2000). Therefore, this paper aims to review mobile learning literature in medical education to understand what instructional design strategies and approaches have been utilized in studies examining learning outcomes.

METHODS The literature search focused on empirical studies of mobile learning interventions with learning outcomes such as knowledge, skills, affective domain, and behavioral changes. Neither conceptual, technology adoption, nor perspective papers were included since they did not provide evidence of learning outcomes, therefore are not in the scope of the current study. A search in the Web of Knowledge (Web of Science, MEDLINE, SciELO Citation Index, Bilogical Abstract, BIOSIS Citation Index, Current Contents Connect, Data Citation Index, Derwent Innovation Index, Inspec, Zoological Record) from 2010 to December 2015 using the keywords “Mobile Learning,” “MLearning,” “Mobile Devices,” “Tablet computer,” “Smartphone,” “Medical Education,” “Nursing,” “Health Care Education,” “Patient Education,” “Patient Care,” “Medicine,” and “Pharmacy” resulted in 1,196 articles. After an initial title review, 764 articles were removed because of irrelevance of the topics. The abstracts of 432 articles were reviewed for criteria as previously described and 56 articles were selected for full review. Two hundred fifty results were excluded because the topics were irrelevant. Sixty-six results were excluded because they were conceptual or opinion papers. Fifty-one results were excluded because the topics related to technology readiness or were simple satisfaction papers without pedagogical descriptions. Nine results were excluded because they were literature reviews, not empirical studies. Among the 56 articles for full paper review, 37 results were quantitative outcome studies and nineteen results were qualitative or descriptive studies. After full review of the 56 articles from medical education literature, as seen in Figure 1, 28 were selected for the final analysis.

46

Heeyoung Han, Larry Hurtubise, Geraud Plantegenest et al.

We used a qualitative synthesis methodology to analyze the selected 28 articles. Quantitative meta-analysis can be informative, yet can be limited in providing specifics of instructional design elements of mobile learning process. Qualitative analysis of instructional strategies adopted for mobile learning and their evidence can provide a rich and deep understanding of effective mobile learning processes. Qualitative synthesis is a “methodology whereby study findings are systematically interpreted through a series of expert judgments to represent the meaning of the collected work” (p.253) (Bearman and Dawson, 2013). During the process, we used an iterative data analysis approach applying an inductive and deductive process to identify themes. We created an Excel spreadsheet to depict study design and findings using the population, intervention, comparison intervention, and outcome (PICO) framework (Cook and West, 2012). Additionally, we included other categories such as conceptual framework/learning strategies, research topic/question, key findings, and other design specifics, as we are interested in detailed instructional design approaches as well as learning outcomes. Summarized and coded quantitative information and qualitative comments were included in the spreadsheet. Based on the preliminary information in the spreadsheet, we extracted several categories of learning strategies that were repetitively found in the articles. In order to analyze findings in-depth, we went back to each paper and reread the interventions sections discussed for a specific category. We constantly compared the papers. By this constant comparison using inductive and deductive processes, we identified several themes. The analysis focused on identifying the instructional design components in relation to effectiveness in mobile learning. Our research team has extensive background and experience in integrating educational technology into medical education curriculum. In the analysis, we made judgments based on our pedagogical understanding of educational technology, mobile technology, and curriculum development. Our professional perspectives and expertise were instrumental in analyzing the literature to extract themes.

Mobile Learning Literature Review in Medical Education

47

Figure 1. Inclusion and exclusion process.

RESULTS As seen in Table 1 (at the end of this chapter), out of the selected 28 studies (n = 2,861), 8 were from undergraduate medical education, 7 from graduate medical education or continuing medical education, 13 from nursing or other healthcare education, and 4 from patient education. Twelve were randomized controlled studies, 9 were quasi-experimental studies, and 7 were descriptive evaluation or qualitative studies. Disciplines included emergency medicine, surgery, military trauma, pediatrics, OBGYN, forensic medicine, nursing, neurology, outpatient transplant service, pharmacology, and physiotherapy. Learning outcomes included knowledge (16 studies), procedural skills (5 studies), behavior change (6 studies) and attitude (3 studies). Mobile technology features included video, short message service (SMS),

48

Heeyoung Han, Larry Hurtubise, Geraud Plantegenest et al.

assessment forms, videos, and basic mobile features. Seventeen out of 21 experimental and quasi-experimental studies found significant effects of mobile technology on learning outcomes including knowledge, skills, and behaviors.

Figure 2. Six themes of mobile learning strategies in medical education.

The iterative review process focused on synthesizing instructional strategies, contextual elements and pedagogical design considerations, which are represented in the six themes: 1. Timeliness capturing learning experience and assessing performance, 2. Pushed reminders for knowledge learning and behavior change, 3. Visual modeling for performance just in time, 4. Facilitating collaborative informal learning, 5. Mixed evidence of multimedia, and 6. Simulated immersive clinical experience. These six themes are not exclusive (see Figure 2), instead these themes are interrelated and often combined to create optimized mobile learning experiences.

Mobile Learning Literature Review in Medical Education

49

Timeliness Capturing Learning Experience and Assessing Performance Timeliness of capturing learning experience and providing feedback was one of the central mobile learning strategies discussed in the medical education literature (Wagner et al. 2014; Renner et al. 2014; Dearnley et al. 2013; Ferenchick et al. 2013). The focus of this mobile learning process is on capturing learning activities and providing feedback on the spot. Narrowing the time gap by using mobile learning process between the moment when learning occurs and the moment when recording and assessing occur was the central idea to improve a validity of assessment (Dearnley et al. 2013). In the study of Renner et al. (2014), mobile learning technology was utilized to facilitate reflective process through documenting reflections of neurological staff including physicians and nurses. The system sent users reminders to reflect and help them document and capture their tasks for discussion with their mentor. This process aimed to capture daily activities for formative evaluation and informal learning. Given that reflections or assessment can change over time, the central focus of this mobile learning process was to capture the activity or thoughts at the time and place where performance and activity occur. This intervention was designed to facilitate reflections with a mentor as well as individual reflections. Wagner et al. (2014) also reported the effects of timely feedback on validity of performance assessment. Criticizing a conventional operative performance assessment where faculty provide a general impressions of resident’s operative performance, Wagner et al. proposed a mobile based performance assessment form to enhance assessment validity. When the workflow involved the resident initiating the feedback capturing process, faculty could provide valid feedback and assessment right after their observation of learner’s performance. Their mobile assessment system was supported by comprehensive analytics of residents’ progress longitudinally, which allowed both residents and faculty to monitor progress over time. Their results found that mobile reports provided greater sensitivity than the traditional assessment system, as it included a greater range of operation-

50

Heeyoung Han, Larry Hurtubise, Geraud Plantegenest et al.

specific parameters and detailed feedback. Ferenchick et al. (2013) also found mobile technology was helpful to capture and document teachers’ direct observation of student’s clinical skills. Most faculty spent less than 10 minutes to provide feedback from direct observation. However, they found no statistically significant correlation of students’ clinical evaluation grades and end-of-3rd-year OSCE scores. These findings may represent the need for further research on what exactly performance observations capture. Dearnley et al. (2013) reported a descriptive study to explore students’ perspectives on performance feedback forms. In their analysis of focus group data, they identified that clinical learners’ getting feedback from multiple sources supported their clinical practice and performance. With a learner-initiated feedback process, learners can receive feedback from multiple professionals including service users, nurses, staff, and peers who observe learners’ performance. They also discussed that a mobile assessment mode can be helpful especially when much of the feedback giving happens in hallways without access to a PC or internet. However, they also reported that there can be a concern of reliability and validity, that some may feel uncomfortable giving negative comments while the learner is present. Time pressure, workload, and organizational culture allowing using mobile devices were some areas to further explore. The mere use of mobile technology for self-assessment did not show any effectiveness in learning outcome in the literature. In the study by Ortega et al. (2011) with nursing students, the mobile self-assessment did not show a statistically significant difference in learning achievement compared to those without. The study findings indicated that using mobile devices for assessment without any benefit of timeliness of assessment for learning activities may provide limited effects.

Pushed Reminders for Knowledge Learning and Behavior Change Pushed information or reminders using simple text messages were found to be effective in enhancing medical learners’ knowledge acquisition and

Mobile Learning Literature Review in Medical Education

51

retention (Chuang and Tsao, 2013; Diedhiou et al. 2015; Walter et al. 2014; Alipour, Jannat and Hosseini, 2014). In a study in nursing education, students provided with SMS learning resources two times a day showed higher scores in a medical knowledge test than those without SMS observed at one week, two weeks, and four weeks after the intervention (Chuang and Tsao, 2013). Simple text messaging was an effective on-the-job training process for healthcare learners for their continuing education without disrupting their patient care work routine (Diedhiou et al. 2015; Alipour, Jannat and Hosseini, 2014). Most studies attributed the positive effect to spaced education principle and feasibility of mobile learning process to support the pedagogy (Chuang and Tsao, 2013; Diedhiou et al. 2015). The positive effect of simple reminders was also observed in patient education. Forman et al. (2014) conducted an observational study where they evaluated the effects of pushed notifications on patients’ participation in cardiac rehab. Patients received daily task reminders of meds and exercise and 90% of the patient completed at least one daily task and 70% of patients complied with activity prompts at least 3 days a week. In the study of smoking cessation, Whittaker et al. (2011) did not find group difference between those with video messages and personalized text messages for role modeling and those with only video messages to facilitate their smoking cessation. Pushed general reminder video messages had the same effects as the group with role model videos as well. Continuous abstinence at 6 months was 26.4% and 27.6% respectively. A simple reminder approach can be extended to a more sophisticated scaffolding strategy that is a pedagogical process to guide learners to achieve a learning goal. Wu et al. (2012) investigated mobile learning system providing guided questions to help nursing students practice the standard operating processes of respiratory assessment. Adopting a context-aware mobile technology that detects learner’s location and expected activities in learning environments, they designed the mobile learning process to guide the students to observe symptoms and differentiate patient diseases based on their findings. Their results showed the mobile learning process with personalized scaffolding had positive effect on students’ learning achievement and clinical skill performance.

52

Heeyoung Han, Larry Hurtubise, Geraud Plantegenest et al.

Visual Modeling for Performance Just in Time Visual modeling for performance just in time is another central instructional approach for effective mobile learning. This instructional strategy often involves reducing extraneous cognitive load on the spot, providing worked examples right before performing a critical task, and visualizing learning object in three dimensional presentations. Hawkes et al. (2013) investigated the effect of short visual instruction right before performing newborn intubation and found that this approach was effective for improving both knowledge and procedural skills for novice learners. Medical students performed better with patient encounters in an emergency department setting when they had immediate access to an instructional video right before seeing a patient in the emergency room (Tews et al. 2011). A similar effect was found in another study. According to Noguera et al. (2013), when second year students in physical therapy had access to two- or three-dimensional images of anatomy on their mobile devices during a practice session, their knowledge gain was higher than those who did not have such learning environments. With resources on their mobile devices, the students were able to visualize specific anatomy of knee, ankle, and pelvic zone as they practiced physical therapy. This allowed the participants to deeply engage in learning how physical therapy works in relation to a specific anatomy structure. Visualization in mobile learning extends to virtual reality. According to De Oliveira et al. (2013), when medical students experienced virtual reality training of airway on their handheld devices, they performed better in fiberoptic intubation using a high-fidelity simulation manikin than those who did not have virtual reality training. However, the effect of visual modeling seems to be limited to novice learners. In the study of Davis et al. (2012), residents and medical students viewed a three-minute video right before performing a chest tube insertion task. The study showed that the effect of the short video was not significant with residents but medical students. This appears to indicate that the visual modeling approach may be more effective for novice learners.

Mobile Learning Literature Review in Medical Education

53

Facilitating Collaborative Informal Learning Mobile technology was instrumental to facilitate collaboration (Renner et al. 2014; Davis, 2014; Davern et al. 2014; Wu et al. 2013; Fahlman, 2014). The underlying concept to support this mobile learning pedagogy is informal work-based learning and social learning process (Fahlman, 2014). Wu et al. (2013) reported positive effects of mobile learning, in a group collaborative investigation on nursing students’ home visit preparation, comparing mobile learning process to traditional paper-based learning process. They attributed the positive effect to the convenience and interactivity of mobile technology to share resources, update information, communicate real-time for feedback, and utilize map for home visit between nursing educators and learners, as well as among learners. In a descriptive study, Davies et al. (2014) found that mobile learning can be effective in capturing student-generated learning artifacts and sharing their views, which facilitate the collaborative group learning process. In a study of Davern et al. (2014), medical students created and uploaded exam practice questions via their mobile devices. This became a question bank, which they later used for daily exam practice. Students’ use of their collectively gathered practice question bank has a positive correlation with their exam performance.

Mixed Evidence of Multimedia Multimedia in mobile learning has been providing mixed results. There was no consistent evidence that multimedia, such as videos, is more effective than traditional media for knowledge learning. In the study of Stirling and Birt (2014), students with e-books on their mobile devices had a higher increase in their knowledge in anatomy than those with traditional instruction. Using an e-Book, the students interacted, and manipulated 3D models using physical gestures of rotation and zoom as well as annotated or non-annotated views to allow for formative learning and self-assessment. The students were supportive of the e-book as an adjunct resource to traditional methods but did not want it as the primary practical session.

54

Heeyoung Han, Larry Hurtubise, Geraud Plantegenest et al.

Trinh et al. (2014) studied educational videos delivered by a mobile device to educate patients about skin cancer who had had transplant surgery. The patient education topics included skin cancer prevalence, detection, risk, treatment and potential complications in the transplant population. A twominute short video was more effective than a traditional paper pamphlet for patient education in the topics. However, contrast to their study, Velasco et al. (2015) found no difference between video and paper instructional materials in educating health professionals in their knowledge of the treatment of childhood asthma. Both groups had educational benefits from both media.

Simulated Immersive Clinical Experience A few studies reported positive effects of using simulations and serious gaming approaches (Renner et al. 2014; De Oliveira et al. 2013; Evans et al. 2015). The underlying premise is that mobile simulations and gaming provide virtual immersive clinical experiences with opportunities for repeated practice at their fingertips without risk. Evans et al. (2015) developed a serious mobile game program about sepsis patient care and found positive effect on knowledge learning for both medical students and residents. The program was designed based on gamification principles including competition, rewards, and enjoyments to promote knowledge acquisition and retention. Additionally, there were many instructionally sound elements in the mobile game: Immersive clinical patient care scenarios, opportunities to practice to make decisions, testing, and treatment anytime anywhere, ability to observe the consequences of their decisions, individualized immediate feedback by a doctor, virtual character, and providing tips and resource materials from literature. De Oliveira et al. (2013) also reported statistically positive effects of mobile virtual airway simulation on performance. In their study, medical students experienced virtual reality training of airway on their handheld devices. The mobile learning process allowed the students to mimic hand movements for the performance of fiberoptic skills by using built-in accelerometers to duplicate

Mobile Learning Literature Review in Medical Education

55

the twisting of a fiberscope and rotating the device. With this mobile learning process, learners were able to be immersed in the clinical practice as many times as they needed. After the mobile simulation, they performed better in fiberoptic intubation using a high-fidelity simulation manikin than those who did not have the mobile virtual reality training.

DISCUSSION Findings of the current study suggest several instructional design frameworks of mobile learning. These design approaches are identified and discussed with an educational belief that mobile learning should go beyond a simple techno-centric view that introducing a mobile device technology to instructional activities would automatically provide positive pedagogical effects. As seen in the literature review of this paper, utilizing mobile technology should be perceived as a technological process to design authentic learning experiences (Han, Resch and Kovach, 2013). Thoughtful design of learning experience considering person, time, and place where learning experiences occur is the key for a meaningful mobile learning process. Mobile learning is a pedagogical process enriched by context-sensitivity of being (location) and doing (activity) of a learner or a teacher. It is a comprehensive process to create meaningful instructional and learning interactions in consideration of where learners and teachers will be and what they will be doing. Simple transferring decontextualized traditional curriculum activities via a mobile device can have limited effectiveness. The mobile leaning pedagogical question is not only about making instructions or guidance available for just-in-time learning, but also about creating an expert presence for a specific individual that facilitates context-based learning and performance in an authentic situation. Mobile learning literature in medical education has some differences as well as similarities to K-12 mobile learning literature. Consistent to our study, Baran (2014) reviewed mobile learning literature in teacher education and found that mobile learning is reported as beneficial for pre-service

56

Heeyoung Han, Larry Hurtubise, Geraud Plantegenest et al.

teacher education. Several advantages of mobile learning that Baran identified for pre-service teacher education include promoting reflection inaction, providing timely access to resources, and allowing participation in knowledge production and sharing regarding teaching practices. However, the design of assessment in mobile learning in medical education was somewhat different from K-12 literature. In mobile learning literature in K12, the focus of assessment was on providing corrective feedback to the students on the fly for their immediate remediation in a situation such as museum or lab observations (Chen and Huang, 2012; Chu et al. 2010; deMarcos et al. 2010; Hwang and Chang, 2011; Shi, Kuo and Liu, 2012). In this literature review focusing on healthcare, most studies investigated mobile learning as a process of capturing learning activities and recording feedback on the fly after direct observation since timely feedback is a way to improve feedback validity (Wagner et al, 2014; Dearnley et al. 2013; Ferenchick et al. 2013). Moreover, the mobile technology assessment allowed capturing learner performance from multiple stakeholders in healthcare. While seen in literature of medical education, location-based intelligence has been discussed more rigorously in K-12 literature (Martin and Ertzberger, 2013; Chen and Huang, 2012; de Jong, Specht and Koper, 2010; Hsiao et al. 2010; Hwang et al. 2011; Liu, Tan and Chu, 2009; Shih, Chuang and Hwang, 2010). Location-based intelligence is a systematic process to detect each learner’s location-related situation and to provide learning tasks, resource and support. Location aware technology such as radio frequency identification (RFID) and global positioning system (GPS) detects each learner’s location and expected tasks and activities (Chen and Huang, 2012; Chu et al. 2010; Hsiao et al. 2010; Hwang et al. 2011; Liu, Tan and Chu, 2009; Hwang et al. 2009; Liu and Chu, 2010). Knowing a learner’s context, the system can send an individual learner related questions, useful resources, and corrective feedback as detected. Contextualized location-based learning resource support was effective in foreign language learning (de Jong, Specht and Koper, 2010), museum learning (Chen and Huang, 2012), outdoor ecological learning (Hsiao et al. 2010; Liu, Tan and Chu, 2009), personal computer-assembling course

Mobile Learning Literature Review in Medical Education

57

(Hwang et al. 2011), and social science learning (Shih, Chuang and Hwang, 2010). In the study of de Jong, Specht, and Koper (2010), providing location-filtered related learning content significantly improved adult students’ knowledge gain compared with a control group. The students with the automatic location filtered contextualized intervention needed fewer actions to complete learning objects than the object filtered group. The studies suggest that inappropriate and unnecessary information navigation tends to result in aimless wandering in real places therefore leading to an ineffective learning process. Mobile learning environments providing individual learners with related resources in time in the right location can facilitate active and deep learning. Noting the significance of seamless learning in mobile ubiquitous learning, Toh et al. (2013) identified different learning spaces characterized by a combination of two factors, physical setting and learning process: Planned learning in classroom, planned learning outside of school, emergent learning outside of school, and emergent learning in classroom. Criticizing the techno-centric methodological issues, they investigated students’ mobile learning experiences in each space through longitudinal and multi-sites ethnographies. Adopting a socio-cultural perspective of Rogoff’s transformation of participation, they analyzed the role of mobile technology in learners’ cultural engagement and transformation. Considering their framework for healthcare settings, most research reviewed in this paper was conducted either in formal education or on planned patient care settings. It is necessary to expand research areas to investigate how mobile technology can work organically in healthcare providers’ informal learning in communities beyond formal planned educational settings. As clinical learning is full of idiosyncratic, informal, and opportunistic learning, mobile learning equipped with personalized flexibility and ubiquity can become a great opportunity for healthcare educators. This trend has been seen in K-12 mobile learning literature. Wang, Liu, and Hwang (2017) reviewed literature on mobile learning utilizing location-based system in K-12 museum learning published from 2009 to 2014. They found that an earlier trend focusing on individual cognitive learning in classrooms with less real-life social interactions has changed to collaborative and informal learning in various

58

Heeyoung Han, Larry Hurtubise, Geraud Plantegenest et al.

socio-cultural contexts due to context-awareness technology that allows learners multiple forms of interactions. El-Hussein et al. (2010) noted the potential of mobile learning and also reminded traditional instructional designers of new problems that this new pedagogical mobile learning paradigm focusing on organic, dynamic, and informal learning may need different design mind-sets. Findings in this study may not represent all possible instructional strategies exhaustively but provided some of the potentials of mobile learning processes. Simply adopting traditional e-learning approaches may not necessarily lead to positive mobile learning experiences as mobile learning leans more toward informal, personal, context-oriented, instant, and opportunistic experiences rather than formal, structured, and media-rich traditional e-learning experiences (Traxler, 2007). More insightful, thoughtful, and innovative design approaches aligned with the mobile learning characteristics should be investigated and studied. There are several limitations in this study. The selection of the database, key words, and years may not include studies available in other databases and journals. Additionally, there may be a selection bias from unpublished studies showing no effects or benefits to mobile learning. Implementing new technology into a curriculum should have a theoretical basis that clearly supports pedagogical choices. These study findings provide the collective conceptual basis to guide the development of effective mobile learning in medical education. The six themes identified through the analysis could be summarized here to remind the reader of how mobile technology can support learning experiences. Future studies could focus on further applicability to various contexts in medical education.

CONFLICT OF INTEREST Funding: None. Ethical Approval: N/A. Disclaimer: None.

Chuang and Tsao (2013)

Nursing

Pharmacology Knowledge N = 106 Short message service (SMS)

Knowledge N = 113 Learning modules delivered on a mobile device

Videos delivered on a mobile device

Schulman Physician Military et al. Assistant(PA)/ trauma (2012) Nursing/ Medic/ Tech

N = 67

Outcomes/ Subjects Intervention Learning (Mobile domain Technology) Skills

Specialty domain

Tews et al. Undergraduate Emergency (2011) medical medicine education (UME)

Population

Comparison/ Key findings Research Design

Demonstrate RCT statistical equivalence between mobileand traditional lecture-based learning in military trauma setting

Effects of Randomized JTI access to reviewing mobile Controlled instructional videos on students’ Trial (RCT) videos improved patient case the learners’ presentation skill presentation skills.

Research topics/questions

No difference between mobile and traditional lectures. But short (10 min) mobile learning was equivalent to traditional didactic lectures (45 min) Information Evaluate the use of QuasiSMS intervention processing theory mobile SMS to experimental was effective to Providing enhance enhance students’ reminders to knowledge of knowledge review the medication of acquisition and content nursing students retention.

Conceptual framework / Instructional design features Just-in-time (JIT) learning Watching the instructional video before entering a patient room JIT learning

Table 1. Summary of included articles

Physical therapy

Graduate Medical Education (GME)/UME

Nursing

Noguera et al. (2013)

Davis et al. (2012)

Ortega et al. (2011)

Population

Skills

Scores improved, but not statistically significant.

Just-in-time short video was found to be effective to teach procedural skills.

Students with 3D m-learning tools showed higher learning outcome in knowledge than those with traditional 2D resources.

Comparison/ Key findings Research Design

Evaluate effects of RCT a 3D mobile application on learning outcome Access to 3D images of the anatomy while they are practicing manipulation Evaluate effects of Quasi a 3-minute mobile experimental learning video on chest tube insertion performance.

Research topics/questions

Just-in-time learning View the video on a mobile device immediately before chest tube insertion Providing self- Motivation Evaluate effects of Descriptive assessment mobile selfquestion on a assessment on mobile device learning.

N = 128 Videos on a mobile device

Knowledge N = 76

Conceptual framework / Instructional design features 3D Cognitive load, visualization of JIT learning anatomy on a mobile device

Outcomes/ Subjects Intervention Learning (Mobile domain Technology)

Vaccination, Knowledge N = 56 Peds, Diabetes management

Surgery

Anatomy

Specialty domain

Table 1. (Continued)

Nursing

Hawkes et GME al. (2013)

Wu et al. (2011)

Population

the NeoTube iPhone application

JIT learning. Learners were provided pre procedural instruction and quick reference through the app right before a knowledge test and an intubation procedure.

Conceptual framework / Instructional design features Expert system Cognitive load, (Artificial JIT Intelligence) on Guided a mobile device observation, with a location learning sensing guidance through technology MindTools

Subjects Intervention (Mobile Technology)

Knowledge N = 48

Outcomes/ Learning domain

Pediatrics Knowledge, N = 20 (Neonatal Skills Resuscitation)

Problem solving for patient care

Specialty domain

Develop and evaluate a clinical mobile learning system that provides learning guidance for nursing courses based on the repertory grid approach Evaluate the NeoTube app on knowledge and procedural skills acquisition.

Research topics/questions

QuasiImmediate mobile experimental access to instructional materials can help learners acquire knowledge and procedural skills, reducing the time to successfully intubate.

QuasiKnowledge experimental learning has improved while cognitive loads decreased with the mobile expert system.

Comparison/ Key findings Research Design

Breast cancer screening

Alipour, Jannat, and Hosseini (2014)

Nursing

Surgery (fibreoptic intubation)

Specialty domain

Oliveira et UME al. (2013)

Population

N = 20

Knowledge N = 60

Skills

Short message service (SMS)

Virtual reality simulation

Outcomes/ Subjects Intervention Learning (Mobile domain Technology)

54 instructional messages were sent to the learners for 17 days rather than traditional one day lectures.

Conceptual framework / Instructional design features Virtual immersion through 3D game design.

Table 1. (Continued)

Evaluate effects of RCT the mobile learning delivered as text messages by cell phone on knowledge retention.

The 3D virtual reality mobile app was more helpful in learners’ skill acquisition compared to traditional instruction methods. Mobile learning utilizing SMS provide the same learning outcome as traditional face to face teaching for continuing education in working nurses.

Comparison/ Key findings Research Design

Evaluated the 3D RCT virtual reality mobile app (iLarynx virtual reality software) for airway simulation on skill acquisition.

Research topics/questions

Life and behavioral science

Davies (2014)

Nursing

Pre-clinical learning

Specialty domain

Davern et UME al. (2014)

Population

Attitude

Knowledge

Outcomes/ Learning domain

N = 24

N = 154

Subjects

Conceptual framework / Instructional design features Collaboration Student-generated practice questions bank

Research topics/questions

Student-generated practice questions bank via mobile devices positively impacted test performance. Overall positive perceptions on the mobile technology to facilitate group work including engagement in content, enhanced presentation skills and revisit to content even after instructional sessions.

Comparison/ Key findings Research Design

Evaluate effects Quasiof student experimental generated questions bank on students’ test performance. Accessibility Cloud-based easy Evaluate the use Descriptive to instructional access to of tablet in resource via a instructional tutorial group mobile device resources to support process on group (tablet). group collaborative engagement work.

Practice question on a mobile device

Intervention (Mobile Technology)

Cardiac rehab

Forman et al. (2014)

Patient education

Sepsis management

Specialty domain

Evans et al. UME/GME (2015)

Population

Behavior change

Knowledge

Outcomes/ Learning domain

N = 26

Health coach mobile app that providing task lists via texts and videos.

N = 156 Simulationbased serious game on a mobile device

Subjects Intervention (Mobile Technology)

Conceptual framework / Instructional design features Game pedagogy (Motivation, trial and error, immersion). Scaffolding Timely individualized feedback from an expert, virtual character. Problem-solving in complexity. Coaching and scaffolding to desired behaviors. Push notifications reminded patients to complete daily tasks (meds, exercise, feedback, surveys, edu). Personalized feedback through securing messaging.

Table 1. (Continued) Comparison/ Research Design

Evaluate the Descriptive feasibility and utility of a mobile application to help increase patient participation in cardiac rehab.

Evaluate the game’s Quasidissemination and experimental its impact on learners’ sepsisrelated knowledge, skills, and attitudes.

Research topics/questions

The participants completed on average 78% of daily tasks and had 42% lower visit cancelation rate than those who did not use the app.

Pre- and postgraduate learners with the experience of the serious game improved their performance in a written test in sepsis management.

Key findings

Patient education

Nursing

Gilliam et al. (2014)

Gomez et al. (2014)

Population

Outcomes/ Learning domain Knowledge

Anatomy, Knowledge, physiology, skills pharmacology, and placing peripheral vesicle catheters

OBGYN (Long-acting reversible contraceptive (LARC))

Specialty domain

N = 52

N = 60

Subjects

Intervention (Mobile Technology) Contraceptive counseling resource on a mobile device used for 15 minutes in a waiting room. Context awareness technology through sensoring technology Ubiquitous learning. Situated learning. Scaffolding. Context awareness. JIT learning Developed system tracking individual student’s profiles including learning activities and providing contextualized guidance by detecting type of clinical practice, state of pending activities, and needed resources.

Conceptual framework / Instructional design features Human-centered design for planned behaviors. Utilized waiting-room time to educate patients Video testimonials

Comparison/ Research Design RCT

Evaluate the RCT context awareness mobile technology on clinical skills learning.

Evaluate effects of the mobile learning task on patient knowledge in LARC

Research topics/questions

Guided scaffolding using mobile context awareness technology can improve knowledge and skills in nursing students.

The counseling app use improved patient knowledge acquisition and interest in LARC.

Key findings

Staff development

UME

Renner et al. (2014)

Stirling and Birt (2013)

Population

Anatomy

Neurology

Specialty domain

Knowledge

Attitude

Outcomes/ Learning domain

N = 71

N = 167 (In 2010). N = 210 (In 2012)

Subjects

eBook on a mobile device using 3D multimedia technology

Mobile app to facilitate reflecting, documenting, and discussing on difficult situations with patients.

Intervention (Mobile Technology)

Conceptual framework / Instructional design features Guided reflection. Collaborative social learning. The app is designed to capture participants’ reflection data and provide support for collaborative reflections. Multimedia interactive learning. Cognitive load theory. Simulation modeling. Formative learning and assessment.

Table 1. (Continued)

Evaluate effects of interactive multimedia eBook on anatomy learning.

Investigated the effects of a mobile app to support collaborative reflections on actual reflective behaviors and job satisfaction.

Research topics/questions

QuasiThe group with the experimental app showed more positive attitude toward collaborative reflection. Positive relationship between reflection and job satisfaction. RCT Multimedia eBook itself did not show statistically significant effects. Combining multimedia eBook with active experiment in the lab demonstrated improvement in knowledge acquisition overall.

Comparison/ Key findings Research Design

Knowledge

N = 66

Educational videos on a mobile device

N = 100 Instructional videos on a mobile device

Outcomes/ Subjects Intervention Learning domain (Mobile Technology)

Outpatient Knowledge transplant service

Velasco et GME/ Pediatrics al. (2015) Continuing Medical Education (CME)/ Nursing

Trinh et al. Patient (2014) education

Population Specialty domain

JIT learning. Multimedia learning.

Conceptual framework / Instructional design features Video-based multimedia learning.

Evaluate effects of RCT using digital media on a mobile device on knowledge on inhalation therapy for asthma patients compared to using traditional written instructional resources.

Participants scored significantly higher in skin cancer knowledge acquisition when watching the video compared to traditional pamphlet education. No statistically significant differences in learning between two techniques in all professional groups.

Comparison/ Key findings Research Design

Evaluate the impact of RCT 2 minute mobile video during follow-up care on patient knowledge acquisition and satisfaction

Research topics/questions

Internal medicine

Walter et al. (2014)

GME

Surgery

Wagner et GME al. (2014)

Population Specialty domain

Knowledge

N = 30

Behavioral N = 37 change (feedback validity through timeliness)

Conceptual framework / Instructional design features Performance assessment. Real-time casebased feedback

Pushed Spaced instructional learning. modules and questions to a mobile device.

Mobile-based feedback system with cloud-based technology

Outcomes/ Subjects Intervention Learning domain (Mobile Technology)

Table 1. (Continued)

Evaluate mobile-based RCT spaced learning modules and questions on knowledge acquisition.

Mobile-based realtime feedback form offers timely performance feedback, residents received more detailed feedback, resident skill levels were reflected with greater sensitivity than traditional assessment approaches. Mobile-based spaced learning group performed better on knowledge test than the group without.

Comparison/ Key findings Research Design

Investigate the Quasiassessment validity of experimental. using a mobile webbased feedback form compared to the end of rotation performance assessment.

Research topics/questions

Public health Knowledge N = 69

Nursing

N = 61

Wu et al. (2013)

Behavioral change

Smoking cessation

Cloud-based technology on a mobile device

Instructional videos and personalized text messages on a mobile device

Subjects Intervention (Mobile Technology)

Whittaker Patient et al. education (2011)

Outcomes/ Learning domain

Specialty domain

Population

Collaborative learning. Cognitive load theory. Active learning.

Conceptual framework / Instructional design features Social cognitive theory. Role modeling. Motivation. Personalized timely coaching.

No statistically significant effect of the personalized video messaging intervention on smoking cessation compared with simple general health video messages via mobile phone. Investigated effects QuasiLearners using of the use of cloud- experimental Google plus based collaborative performed better in technology group investigation (Google plus) on a than learners using mobile device on papers without a knowledge in significant increase patient home visit of cognitive load. preparation.

Comparison/ Key findings Research Design

Evaluate the RCT effects of a multimedia mobile phone intervention for smoking cessation.

Research topics/questions

Interprofessional education

Dearnley et al. (2013)

Occupational therapy, nursing, medicine, radiology, social work, and audiology,

Emergency medicine

Specialty domain

Alegria, UME Boscardin, and Poncelet (2014)

Population

Behavioral change

Attitude/ Motivation

Outcomes/ Learning domain

N = 85

N = 15

Subjects

Mobile apps to collect and record feedback

Information access via a mobile device (tablet)

Intervention (Mobile Technology)

Practice Based Learning. JIT learning. Feedback

Conceptual framework / Instructional design features Self-regulated Learning. Clinical Learning.

Table 1. (Continued) Comparison/ Research Design

Evaluate multiQualitative modal performance feedback tools (paper, web, and mobile) from the student perspective

Describe how Qualitative students utilize a tablet to support their longitudinal clerkship learning.

Research topics/questions

Students utilized the mobile technology to support their selfdirected learning through accessing to resources and utilizing downtime. Regardless of mode of delivery, assessment processes should seamlessly be aligned to the expected practice outcomes within the curriculum.

Key findings

Nursing

Ferenchick UME and Solomon (2013)

Fahlman (2014)

Population

Medicine

Clinical problem solving (situations, procedures, and/or treatments)

Specialty domain

Behavioral change

Behavioral change

Outcomes/ Learning domain

N = 516

N = 10

Subjects

Mobile app for documenting observation and assessment

Mobile technology to access resources

Intervention (Mobile Technology)

Practice Based Learning. Direct observation. Performance assessment.

Conceptual framework / Instructional design features Informal learning. Incidental learning. JIT learning. Work-based learning. Self-directed learning. Socio-cultural constructivism.

Comparison/ Research Design

Explore the feasibility of the mobile app for directly observing and assessing students’ clinical skills

Descriptive

Explore the impact Qualitative of registered nurses’ mobile use for individual and collaborative informal workbased learning

Research topics/questions

The mobile app is feasible and effective in capturing discrete students’ clinical performance.

Mobile technology was proactively used by the participants, who see the value of the technology as an effective informal learning process for knowledge and skills acquisition.

Key findings

72

Heeyoung Han, Larry Hurtubise, Geraud Plantegenest et al.

REFERENCES Alipour S, Jannat F, Hosseini L. Teaching Breast Cancer Screening via Text Messages as Part of Continuing Education for Working Nurses: A Casecontrol Study. Asian Pac J Cancer P (2014) 15(14):5607-5609. Baran E. A Review of Research on Mobile Learning in Teacher Education. Educational Technology & Society (2014) 17(4):17-32. Bearman M, Dawson P. Qualitative synthesis and systematic review in health professions education. Medical Education (2013) 47(3):252-260. Caffarella RS, Merriam SB. Learning in adulthood: A comprehensive guide. San Francisco, CA: Jossey-Bass; 2012. Chen CC, Huang TC. Learning in a u-Museum: Developing a context-aware ubiquitous learning environment. Computers & Education (2012) 59(3):873-883. Chu HC, Hwang GJ, Tsai CC, et al. A two-tier test approach to developing location-aware mobile learning systems for natural science courses. Computers & Education (2010) 55(4):1618-1627. Chuang YH, Tsao CW. Enhancing nursing students’ medication knowledge: The effect of learning materials delivered by short message service. Computers & Education (2013) 61:168-175. Cook DA, West CP. Conducting systematic reviews in medical education: a stepwise approach. Medical Education (2012) 46(10):943-952. Council NR. How People Learn: Brain, Mind, Experience, and School: Expanded Edition; 2000. Davern MS, Sandsmark E, Schwan J, et al. Mobile learning: students’ novel approach to practice problems. Medical Education (2014) 48(11):11061107. Davies M. Using the Apple iPad to facilitate student-led group work and seminar presentation. Nurse Education in Practice (2014) 14(4):363367. Davis JS, Garcia GD, Wyckoff MM, et al. Use of mobile learning module improves skills in chest tube insertion. J Surg Res (2012) 177(1):21-26.

Mobile Learning Literature Review in Medical Education

73

de Jong T, Specht M, Koper R. A Study of Contextualised Mobile Information Delivery for Language Learning. Educational Technology & Society (2010) 13(3):110-125. de Marcos Ortega L, Barchino Plata R, Jimenez Rodriguez ML, et al. Using m-learning on nursing courses to improve learning. Computers, informatics, nursing: CIN (2011) 29(5):311-317. De Oliveira GS, Jr., Glassenberg R, Chang R, et al. Virtual airway simulation to improve dexterity among novices performing fibreoptic intubation. Anaesthesia (2013) 68(10):1053-1058. de-Marcos L, Ramon Hilera J, Barchino R, et al. An experiment for improving students performance in secondary and tertiary education by means of m-learning auto-assessment. Computers & Education (2010) 55(3):1069-1079. Dearnley CA, Taylor JD, Laxton JC, et al. The student experience of piloting multi-modal performance feedback tools in health and social care practice (work)-based settings. Assess Eval High Edu (2013) 38(4):436450. Diedhiou A, Gilroy KE, Cox CM, et al. Successful mLearning Pilot in Senegal: Delivering Family Planning Refresher Training Using Interactive Voice Response and SMS. Global health, science and practice (2015) 3(2):305-321. El-Hussein MOM, Cronje JC. Defining Mobile Learning in the Higher Education Landscape. Educational Technology & Society (2010) 13(3):12-21. Evans KH, Dais W, Tsui J, et al. Septris: A Novel, Mobile, Online, Simulation Game That Improves Sepsis Recognition and Management. Academic Medicine (2015) 90(2):180-184. Fahlman D. Nurses’ Work-Based Mobile Learning. Mobile as MainstreamTowards Future Challenges in Mobile Learning, Mlearn (2014) 479:300-303. Ferenchick GS, Solomon D, Foreback J, et al. Mobile Technology for the Facilitation of Direct Observation and Assessment of Student Performance. Teaching and learning in medicine (2013) 25(4):292-299.

74

Heeyoung Han, Larry Hurtubise, Geraud Plantegenest et al.

Forman DE, LaFond K, Panch T, et al. Utility and Efficacy of a Smartphone Application to Enhance the Learning and Behavior Goals of Traditional Cardiac Rehabilitation A Feasibility Study. J Cardiopulm Rehabil (2014) 34(5):327-334. Gwo-Jen H, Chin-Chung T. Research trends in mobile and ubiquitous learning: a review of publications in selected journals from 2001 to 2010. British Journal of Educational Technology (2011) 42(4):E65E70. Han H, Resch DS, Kovach RA. Educational Technology in Medical Education. Teaching and learning in medicine (2013) 25:S39-S43. Hawkes CP, Walsh BH, Ryan CA, et al. Smartphone technology enhances newborn intubation knowledge and performance amongst paediatric trainees. Resuscitation (2013) 84(2):223-226. Hsiao HS, Lin CC, Feng RT, et al. Location Based Services for Outdoor Ecological Learning System: Design and Implementation. Educational Technology & Society (2010) 13(4):98-111. Hwang GJ, Chang HF. A formative assessment-based mobile learning approach to improving the learning attitudes and achievements of students. Computers & Education (2011) 56(4):1023-1031. Hwang GJ, Yang TC, Tsai CC, et al. A context-aware ubiquitous learning environment for conducting complex science experiments. Computers & Education (2009) 53(2):402-413. Hwang GJ, Tsai CC. Research trends in mobile and ubiquitous learning: a review of publications in selected journals from 2001 to 2010. British Journal of Educational Technology (2011) 42(4):E65-E70. Hwang GJ, Wu CH, Tseng JCR, et al. Development of a ubiquitous learning platform based on a real-time help-seeking mechanism. British Journal of Educational Technology (2011) 42(6):992-1002. Hwang GJ, Wu P. Applications, impacts and trends of mobile technologyenhanced learning: A review of 2008-2012 publications in selected SSCI journals. International Journal of Mobile Learning and Organisation (2015) 8(2):83-95. JISC. Effective practice in a digital age: A guide to technology-enhanced learning and teaching Available: http://www.jisc.ac.uk/media/

Mobile Learning Literature Review in Medical Education

75

documents/publications/effectivepracticedigitalage.pdf. Accessed June 3rd, 2017. Liu TY, Chu YL. Using ubiquitous games in an English listening and speaking course: Impact on learning outcomes and motivation. Computers & Education (2010) 55(2):630-643. Liu TY, Tan TH, Chu YL. Outdoor Natural Science Learning with an RFIDSupported Immersive Ubiquitous Learning Environment. Educational Technology & Society (2009) 12(4):161-175. Martin F, Ertzberger J. Here and now mobile learning: An experimental study on the use of mobile technology. Computers & Education (2013) 68:76-85. Mayer RE. Cognitive theory of multimedia learning. In Mayer RE, (Ed). The Cambridge Handbook of Multimedia Learning. NY: Cambridge University Press; 2005. Merriam SB, Bierema LL. Adult learning: Linking theory and practice. San Francisco, CA: Jossey-Bass; 2014. Noguera JM, Jimenez JJ, Catalina Osuna-Perez M. Development and evaluation of a 3D mobile application for learning manual therapy in the physiotherapy laboratory. Computers & Education (2013) 69:96-108. Renner B, Kimmerle J, Cavael D, et al. Web-Based Apps for Reflection: A Longitudinal Study with Hospital Staff. Journal of medical Internet research (2014) 16(3):168-176. Sandars J. Technology and the delivery of the curriculum of the future: Opportunities and challenges. Medical teacher (2012) 34(7):534-538. Sharples M. The design of personal mobile technologies for lifelong learning. Computers & Education (2000) 34(3-4):177-193. Shih J-L, Chuang C-W, Hwang G-J. An Inquiry-based Mobile Learning Approach to Enhancing Social Science Learning Effectiveness. Educational Technology & Society (2010) 13(4):50-62. Shih S-C, Kuo B-C, Liu Y-L. Adaptively Ubiquitous Learning in Campus Math Path. Educational Technology & Society (2012) 15(2):298-308. Stirling A, Birt J. An Enriched Multimedia eBook Application to Facilitate Learning of Anatomy. Anat Sci Educ (2014) 7(1):19-27.

76

Heeyoung Han, Larry Hurtubise, Geraud Plantegenest et al.

Sweller J, van Merrienboer JJG, Paas FGWC. Cognitive architecture and instructional design. Educ Psychol Rev (1998) 10(3):251-296. Tews M, Brennan K, Begaz T, et al. Medical student case presentation performance and perception when using mobile learning technology in the emergency department. Medical education online (2011) 16. Toh Y, So HJ, Seow P, et al. Seamless learning in the mobile age: a theoretical and methodological discussion on using cooperative inquiry to study digital kids on-the-move. Learning Media and Technology (2013) 38(3):301-318. Traxler J. Defining, discussing, and evaluating mobile learning: The moving finger writes and having writ.... Int Rev Res Open Dis (2007) 8(2):1-12. Trinh N, Novice K, Lekakh O, et al. Use of a Brief Educational Video Administered by a Portable Video Device to Improve Skin Cancer Knowledge in the Outpatient Transplant Population. Dermatol Surg (2014) 40(11):1233-1239. Velasco HF, Cabral CZ, Pinheiro PP, et al. Use of digital media for the education of health professionals in the treatment of childhood asthma. Jornal De Pediatria (2015) 91(2):183-188. Wagner JP, Chen DC, Donahue TR, et al. Assessment of resident operative performance using a real-time mobile Web system: preparing for the milestone age. J Surg Educ (2014) 71(6):e41-46. Walter RJ, Mallia AK, Massoumi A, et al. Mobile-centered spaced learning: A novel approach to graduate medical education in the digital era. Journal of General Internal Medicine (2014) 29:S526-S527. Wang HY, Liu GZ, Hwang GJ. Integrating socio-cultural contexts and location-based systems for ubiquitous language learning in museums: A state of the art review of 2009-2014. British Journal of Educational Technology (2017) 48(2):653-671. Whittaker R, Dorey E, Bramley D, et al. A Theory-Based Video Messaging Mobile Phone Intervention for Smoking Cessation: Randomized Controlled Trial. Journal of medical Internet research (2011) 13(1):5869.

Mobile Learning Literature Review in Medical Education

77

Wu PH, Hwang GJ, Su LH, et al. A Context-Aware Mobile Learning System for Supporting Cognitive Apprenticeships in Nursing Skills Training. Educational Technology & Society (2012) 15(1):223-236. Wu T-T, Huang S-H, Chung M-Y, et al. Group Investigation Learning with Google Plus for Public Health Nursing Practice Course. 2013 IEEE 13th International Conference on Advanced Learning Technologies (Icalt 2013):238-242. Wu WH, Wu YCJ, Chen CY, et al. Review of trends from mobile learning studies: A meta-analysis. Computers & Education (2012) 59(2):817827.

In: The Digital Era of Learning Editor: Christopher S. Keator

ISBN: 978-1-53618-750-2 © 2020 Nova Science Publishers, Inc.

Chapter 3

NOVEL USE OF 3D BONE MODELS IN THE ANATOMICAL SCIENCES EDUCATION OF MILLENNIAL STUDENTS: A DESCRIPTION OF THE PROCESS AND AN ASSESSMENT OF THE PRINTING ACCURACY Yousef AbouHashem1, Manisha R. Dayal2, Stephane Savanah3 and Goran Štrkalj4, 1

Faculty of Science and Engineering, Macquarie University, Sydney, NSW, Australia 2 School of Science and Health, Western Sydney University, Penrith, NSW, Australia 3 Faculty of Human Sciences, Macquarie University, Sydney, NSW, Australia 4 School of Medical Sciences, University of New South Wales, Sydney, NSW, Australia



Corresponding Author’s Email: [email protected].

80

Yousef AbouHashem, Manisha R. Dayal, Stephane Savanah et al.

ABSTRACT In recent years, anatomy education has been strongly influenced by new technologies. One of the most promising of these technologies is 3D printing. Previous studies indicate that good quality copies of human tissue can be printed and applied in anatomy education. However, detailed information is lacking on the usefulness of different equipment, technical details, cost and timing of printing, and print quality. This paper focuses on the 3D printing of bone models (vertebrae) from images made by a surface scanner (Artec Spider) using two consumer grade desktop printers (MakerBot and Mojo) followed by both qualitative and quantitative assessments of the print quality. Various strategies are available to lower the cost and time for 3D printing, and a moderate investment in equipment enabled printing of a number of copies of the model in durable plastic materials at a relatively minimal cost (due to the low price of printing material). The processes of scanning and printing were time consuming, but the resulting 3D images and prints could be used in a variety of teaching environments. The assessment of the accuracy of these 3D prints, as compared with the actual structures, suggested that the printed models were of good quality and suitable for anatomy education. This study supports the view that 3D printed bones can be an important resource in anatomy education.

Keywords: 3D printing, anatomy education, osteology

INTRODUCTION Recent developments in medical imaging have provided unprecedented insights into the structures of the human body. The field of osteology (science of bone and the skeleton) acquired a new dimension in recent years with the application of additive manufacturing techniques, enabling 3D printed models to greatly expand on the limited information offered by twodimensional images. Such 3D models have already found application in various medical and biomedical fields, including anatomy education. In general, 3D models are now regarded as a valuable resource in learning and teaching anatomy (Yammine and Violato, 2016).

Novel Use of 3D Bone Models in the Anatomical Sciences …

81

Additive manufacturing (otherwise known as 3D printing) originated in the mid-1980s as a digitally directed layer-by-layer material deposition manufacturing process (Lantada and Morgado, 2012). A computerized digital 3D image is captured of a chosen object, and then a replica or “prototype” is created of the complex geometrical 3D structure. In a biomedical application, the 3D digital image can be obtained from a number of different medical imaging modalities such as computed tomography (CT) and magnetic resonance imaging (MRI) scans, as well as scans produced from a surface 3D scanner (McMenamin et al., 2014; Negi et al., 2014; Adams et al., 2015). The use of 3D printed models in various medical and academic domains has been investigated, and a number of studies utilizing different imaging and printing techniques (including combining it with other, more traditional techniques of making anatomical models) suggested several ways in which 3D prints could be utilised in anatomy education (McMenamin et al., 2014; AbouHashem et al., 2015; Adams et al., 2015; Fredieu et al., 2015; O’Reilly et al., 2016, Sander et al., 2017). The advantages of using 3D prints are numerous and include the following: (1) the models are durable, (2) models can be reproduced in large numbers, (3) they can be used (and stored) in different learning and teaching environments, and (4) unlike commercial anatomical models, they preserve and reinforce the inherent biological variation (McMenamin et al., 2014; AbouHashem et al., 2015; Adams et al., 2015; Fredieu et al., 2015; Lim et al., 2015; Baskaran et al., 2016; O’Reilly et al., 2016, Sander et al., 2017; Langridge et al., 2018). Making the choice to use 3D printing for teaching anatomy depends on several factors including the costs, the time needed to complete the actual printing, and accepting the finished quality/accuracy of the prints (which in itself is dependable on a number of factors, including the quality/resolution of images sent to the printers) (McMenamin et al., 2014; Adams et al., 2015). Obtaining high quality original prints will most often result in higher quality 3D prints, but acquiring initial high-quality images often comes with a substantial investment in infrastructure and may require substantial more time (resulting in an expensive and time-consuming process). The use of 3D printing in anatomy is still in its early phases, therefore the infrastructure

82

Yousef AbouHashem, Manisha R. Dayal, Stephane Savanah et al.

needed to create high quality 3D prints is evolving at a very fast pace, resulting in constantly changing costs. Thus, providing as much information about the process of 3D printing and sharing our unique experiences with 3D printing can be very important to anatomists and anatomy educators, especially in this rapidly changing environment (Štrkalj and Dayal, 2014). The aim of this study was to throw more light on the activities involved in the preparation of 3D printed anatomy models and to further investigate the quality and accuracy of 3D printing as it applies to anatomy education. This study focused on bones printed with consumer-grade 3D printers, using average quality images captured by a 3D surface scanner. This particular equipment was chosen because these devices are both easily obtained and relatively inexpensive. These devices, based on their simple instructions and ease of use, can most likely be operated by the staff members from any anatomy department. Bones were chosen as models because by their very nature (consisting of hard tissue and mainly monochromatic) they seem to “lend themselves” better to high quality 3D printing (AbouHashem et al., 2015).

MATERIALS AND METHODS Sample Selection The focus in this study was on vertebrae as these are relatively small bones of irregular shape. It was assumed that if a good accuracy could be achieved in printing these bones it could also be achieved with bones of a simpler morphology. Vertebrae for the study were selected from the Macquarie University Skeletal Collection (Danilovic et al., 2013). The chosen samples were dry adult human vertebrae, currently used as part of a teaching collection in the anatomy program at the institution. The bones were symmetrical, pathology free and bearing of all identifiable anatomical features of a typical vertebra. The vertebrae included two samples each of the typical cervical, thoracic, and lumbar vertebrae totaling a sample size of six.

Novel Use of 3D Bone Models in the Anatomical Sciences …

83

Image Acquisition Three-dimensional surface images of the selected dry vertebrae were obtained using the hand-held Artec Spider (Artec Group, Luxembourg) 3D surface scanner. This scanner has a 3D resolution of up to 0.1 mm and 3D point accuracy of up to 0.05 mm. Each bone was scanned from several different angles. The Artec software recognized each scanned angle as a separate scan, and multiple angled scans were performed to ensure all sides and features of the bones were captured. The 3D printed model relies heavily on the quality of the obtained image, therefore these steps were required to retain all of the anatomical features needed to produce a high quality 3D rendering. The same software used to capture the scans was used for image processing. Post-hoc processing of the images was performed with the compatible Artec Studio 9.2 software, which was also provided with the scanner.

3D Printing A total of six 3D surface images (two of cervical, thoracic and lumbar vertebrae) in the .stl file format were acquired from 3D scanning and image processing. Two desktop 3D printers were used: the MakerBot Replicator 2 (MakerBot Industries, New York) and the Mojo (Stratasys Ltd, Minnesota). The MakerBot uses the fused deposition modelling (FDM) technique with biodegradable polylactic acid (PLA) filaments as its primary material. It has a large build volume (28.4 x 15.2 x 15.5 cm) and a high layer resolution of 0.1 mm. The Mojo also uses FDM technology but instead of PLA filaments uses an acrylonitrile butadiene styrene (ABS) thermoplastic polymer as its main material. The maximum size that can be produced on this printer is slightly smaller at 12.7 x 12.7 x 12.7 cm with a layer thickness at 0.178 mm. Both printers use materials (that belong to the same family of thermoplastics) that are moldable when heated and hard when cooled. However, a few differences are present: ABS is a strong and high

84

Yousef AbouHashem, Manisha R. Dayal, Stephane Savanah et al.

temperature resistant material, making it ideal for the production of robust durable models. PLA, on the other hand, exhibits these properties to a lesser extent. PLA’s main advantage is the smaller layer thickness (0.1 mm compared to 0.178 mm of ABS), which makes it more accurate and gives a sharper finish to the final geometry of the 3D printed model - suitable for showing bony features and landmarks. The .stl files were opened using the printers’ software. In preparation for printing, the scans were positioned as flat as possible on the virtual platform in the software. This allowed for the minimum use of ‘supports’ (easily removable support structures deposited under overhanging parts of the scan) and also reduced the printing time for each vertebra. The MakerBot supports were manually removed by hand (Figure 1). The Mojo uses a different type of material (SR-30 soluble support) for printing, therefore manual removal of the supports is not necessary because the supports are usually dissolved away in a dish washing tablet and water solution in the ultrasound bath (WaveWash 55). Since the process is automated, the 3D printed vertebrae were placed inside the WaveWash 55 where the support material was dissolved away.

Figure 1. 3D printed model of lumbar sample 2 on the MakerBot Replicator 2 build tray with support material still intact.

Accuracy Assessment A total of twelve vertebrae were printed using the original .stl files, six on each printer. Both qualitative and quantitative assessment of the printed

Novel Use of 3D Bone Models in the Anatomical Sciences …

85

vertebrae were made. Qualitative assessment was carried out by all four researchers and consisted of visual observation of the printed vertebra (which were compared) with the original bones and the subjective estimate of visibility of the key morphological features. A metric analysis was carried out for the quantitative assessment. This analysis consisted of a series of linear measurements on the selected dry vertebrae and each of the 3D printed models, by calculating the maximum distance between the two points using the following parameters (Gilad et al., 1985; Laporte et al., 2000; Ogden et al., 2014; Tan et al., 2014):  

  

 

Anterior vertebral body height (ABH): the most anterior distance between the superior and inferior endplates of the vertebral body, Spinous process length (SPL): the distance between the most anterior point of the spinous process (in the vertebral foramen) to the most posterior point, Vertebral foramen width (VFW): the inner measured distances of the vertebral foramen from lateral to lateral sides, Vertebral foramen breadth (VFB): the inner measured distances of the vertebral foramen from anterior to posterior sides, Pedicle height (right and left) (PHR and PHL): the maximum perpendicular distance of the pedicles in relation to the horizontal line passing through the body, Lamina height (right and left) (LLR and LLL): the maximum length of the laminae measured from superior to inferior, Bi-transverse process length (TVPL): the maximum vertebral width distance from the most lateral points of the right and left transverse processes.

Measurements were determined with a digital caliper (Mitutoyo Corporation, Japan) that has a precision of 0.01 mm and an accuracy of 0.02 mm. Each measurement was taken by one of the authors and repeated three times on three separate days, with a one-week interval between each day to minimize bias and ensure consistency and repeatability. Thus, a total of 21 measurements were collected for each vertebra. The mean was calculated

86

Yousef AbouHashem, Manisha R. Dayal, Stephane Savanah et al.

from the total of each of the three measurements. Paired sample t-test was used to compare the means of each measurement obtained from the dry vertebrae to the 3D printed vertebrae. Statistical analyses were performed using IBM SPSS Statistics, Version 22 (IBM Corporation, New York). A significance level (p-value) of 0.05 was used when multiple comparisons were made. Results were analyzed using the paired sample t-test (Student’s t-test) for each measurement parameter to compare the differences of the means between the dry vertebrae and each of the 3D printed vertebrae generated by each printer.

RESULTS The scanning and image processing of each bone took approximately 30 minutes. Table 1 shows the approximate time taken for 3D printing and the amount of material used for each vertebra on both printers. The samples were printed using the default high quality setting on the native software of each 3D printer. The overall appearances of the 3D printed vertebrae were of a high similarity with the dry vertebrae. There was an agreement among the researchers that the 3D printed models showed all morphological landmarks and features seen on the bones, and were deemed accurate enough to be used in anatomy learning and teaching at the undergraduate level (Figure 2). Table 1. Printing time and amount of material used for each of the vertebrae printed Sample Cervical 1 Cervical 2 Thoracic 1 Thoracic 2 Lumbar 1 Lumbar 2

MakerBot Replicator 2 Time (hr:min) Material (g) 1:01 8.45 1:00 8.31 1:58 13.86 2:15 15.73 2:44 21.82 3:27 27.89

Mojo Time (hr:min) 1:38 1:30 2:40 3:40 4:16 5:29

Material (g) 10.57 10.32 21.32 24.94 38.66 50.19

Novel Use of 3D Bone Models in the Anatomical Sciences …

87

(A)

(B)

(C) Figure 2. Comparison of the real vertebra and the two printed vertebrae. (A) Real, (B) MakerBot Replicator 2, (C) Mojo printed.

Relative assessment of the differences in the means of the 3D printed vertebrae compared to the dry vertebrae is presented in Table 2. The absolute dimensional error of the measurements means for the 3D printed vertebrae from both 3D printers is 0.188 mm (0.01%). Further analysis revealed that the Mojo 3D printed models showed less relative error (0.148 mm; 0.99%) compared to the MakerBot (0.227 mm; 1.49%). This difference is noteworthy considering the fact that the layer of the material produced by the MakerBot is thinner when compared to the Mojo.

% 0.280 0.670 0.810 0.400 1.440 0.180 0.630

% 1.720 0.640 1.890 2.230 1.430 2.100 1.670

SPL mm 0.043 0.110 0.300 0.173 0.343 0.043 0.169 SPL mm 0.270 0.103 0.697 0.973 0.347 0.503 0.482

% 0.550 1.070 1.050 0.060 0.330 1.280 0.720

% 1.290 0.760 1.280 1.080 1.160 0.560 1.020

VFW mm 0.073 0.023 0.100 0.290 0.087 0.127 0.117

VFW mm 0.013 0.023 0.077 0.087 0.093 0.107 0.067

% 0.330 0.100 0.610 1.800 0.450 0.540 0.640

% 0.060 0.100 0.460 0.540 0.480 0.460 0.350 VFB mm 0.021 0.077 0.157 0.113 0.070 0.153 0.099

VFB mm 0.133 0.060 0.023 0.057 0.097 0.097 0.078

% 0.200 0.570 1.100 0.810 0.500 1.120 0.720

% 1.020 0.450 0.160 0.410 0.700 0.710 0.580 PHR mm 0.173 0.643 0.120 0.203 0.037 0.113 0.215

PHR mm 0.090 0.103 0.013 0.120 0.207 0.130 0.111

% 3.210 13.200 1.360 2.080 0.310 0.860 3.500

% 1.700 2.170 0.150 1.240 1.710 0.990 1.330 PHL mm 0.157 0.227 0.387 0.207 0.407 0.137 0.254

PHL mm 0.037 0.080 0.060 0.007 0.113 0.310 0.101

% 2.180 3.950 4.250 2.180 3.460 1.010 2.840

% 0.650 1.380 0.650 0.070 0.970 2.240 0.990 LLR mm 0.197 0.080 0.203 0.177 0.033 0.197 0.148

LLR mm 0.357 0.187 0.190 0.787 0.337 0.260 0.353

% 2.180 0.820 1.200 1.000 0.230 1.060 1.080

% 3.800 1.870 1.110 4.460 2.310 1.380 2.490 LLL mm 0.013 0.143 0.380 0.663 0.063 0.157 0.237

LLL mm 0.022 0.050 0.357 0.413 0.240 0.160 0.207

% 0.150 1.400 2.090 3.600 0.500 0.820 1.430

% 2.310 0.490 2.000 2.290 1.840 0.830 1.630

TVPL mm 0.293 0.333 0.133 0.207 0.577 0.410 0.326

TVPL mm 0.167 0.123 0.133 0.117 0.210 0.010 0.127

Abbreviations: ABH = Anterior body height, SPL = Spinous process length, VFW = Vertebral foramen width, VFB = Vertebral foramen breadth, PHR = right pedicle height, PHL = left pedicle height, LLR = right lamina length, LLL = left lamina length, TVPL = bi-transverse process length.

ABH mm Cervical sample 1 0.060 Cervical sample 2 0.113 Thoracic sample 1 0.167 Thoracic sample 2 0.010 Lumbar sample 1 0.073 Lumbar sample 2 0.293 Overall 0.119 MakerBot Replicator 2 ABH mm Cervical sample 1 0.140 Cervical sample 2 0.080 Thoracic sample 1 0.200 Thoracic sample 2 0.193 Lumbar sample 1 0.260 Lumbar sample 2 0.130 Overall 0.167

Mojo

% 0.590 0.680 0.230 0.380 0.750 0.490 0.520

% 0.370 0.250 0.230 0.210 0.270 0.010 0.220

Table 2. Absolute means and percentages for the differences in linear dimensions of the 3D printed vertebrae compared to the dry vertebrae

Mojo™ MakerBot™ Replicator® 2 Mojo™ MakerBot™ Replicator® 2 Mojo™ MakerBot™ Replicator® 2 Mojo™ MakerBot™ Replicator® 2 Mojo™ MakerBot™ Replicator® 2 Mojo™ MakerBot™ Replicator® 2

Cervical Sample 1

Significance of Measurement (p-value)* ABH SPL VFW VFB PHR 0.438 0.023 0.889 0.059 0.381 0.667 0.567 0.705 0.133 0.154 0.342 0.792 0.893 0.753 0.061 0.335 0.751 0.539 0.222 0.037 0.563 0.011 0.730 0.644 0.716 0.241 0.304 0.149 0.247 0.390 0.828 0.305 0.096 0.173 0.095 0.273 0.026 0.084 0.144